R4 Validation Criteria for Health Promotion Study
Principal Investigators: Charles Drum, JD, PhD; Tom Seekins, PhD
This study will utilize a panel of health promotion and disability experts to develop criteria for evaluating health promotion intervention programs for people with disabilities are acceptable to health promotion researchers, disability advocates, and program implementers. These criteria will be disseminated and used in a subsequent study (R5).
Hypothesis: Criteria developed by expert panel process for evaluating
health promotion intervention programs for people with disabilities
are acceptable to health promotion researchers, disability advocates,
and program implementers.
Study Design: Expert panel process to develop consensus on criteria
for evaluating health intervention programs.
Samples: 1) Panel comprised of 12 experts nationally recognized
for their expertise in disability and health intervention and/or
measurement. 2) Criteria for evidence-based practice (internal
validity), translational generalizability of interventions (external
validity), and social
acceptability of interventions (social validity).
3) 6 external reviewers to assess the panel’s criteria from
the perspective of disability and health research (2), disability
advocacy (2), and health interventionists (2).
Data Collection and Measurement: Expert Panel Process: We will
use an expert panel process in four related studies: the present
studies, R4 (Interventions) and R6 (Measurement), and they will
form workgroups to guide the work of R5, R7 and R8. The Expert
Panel Process is used successfully in a range of service fields
to identify and improve practice guidelines, to evaluate and highlight
critical research findings, and to suggest priority directions
for the future. Examples are the committees of the Institute of
Medicine (IOM) and the Agency for Health Research Quality (AHRQ).
With roots in nominal group consensus of the 1960s, it involves
the recruitment and participation of informed and involved members
who are experts in components of the study topic (Delbeq, Van de
Ven A, & Gustafson, 1975; Fink, Kosecoff, Chassin, & Brook,
1984). Through the selection of the participants, the panel represents
a wealth of experience and data to inform the decision-making process
(Murphy et al., 1998). Experts recruited to serve on the panel
for Intervention include: Tom Seekins (Chair), Tamar Heller, Jeff
McCubbin, James Rimmer,
and Glen White. Panelists for Measurement include Elena Andresen,
Brad Cardinal, Glenn Fujiura, and Peg Nosek. Gloria Krahn and Charles
Drum will serve on the panel and both resulting work groups. June
Kailes will also serve on the panel and provide consultation to
the workgroups. We will also invite one panel member from NIDRR
and one from the Centers for Disease Control and Prevention, two
agencies already collaborating to reduce health disparities for
people with disabilities. Together, these members bring a wealth
of expertise in assessment and intervention in disabilities and
health to the panel process. Panelists will nominate consultants
who can share additional areas of expertise to the full panel or
the work groups. Areas in which we anticipate recruiting consultants
include health insurance plans, psychometrics, and disability organizations.
Data Analysis Methods: We will use the following procedures modified
from those recommended by Jones and Hunter (1995, 2000): 1) The
PI will present the charge to the panel and its intended outcomes;
2) panel experts will nominate relevant reading material and external
experts to present to the full panel; 3) staff will compile and
present all relevant literature to the panelists prior to the initial
meeting (evidence-based practice, translation generalizability
criteria, social validity recommendations); 4) the panel will discuss
and clarify the charge to the group and intended outcomes; 5) each
participant will write a brief description of their views about
the concepts and criteria; 6) each participant will present one
(new) idea to the group, rotating among members until all views
are presented; 7) similar ideas will be grouped together, using
group discussion and clarification; 8) each participant will privately
rank the ideas for appropriateness and applicability; 9) rankings
will be presented and discussed to a point of consensus if possible,
and to a 2/3 majority vote if consensus cannot reasonably be reached.
These procedures will be used to structure a 2 ½ day meeting
in the fall or winter of 2004. Experts will convene as one full
panel initially, with intervention and measurement specialists
sharing their experiences with the full panel for the first day.
We anticipate the panel dividing up into its two workgroups for
the second day of the meeting to begin their distinct work. The
process outlined above will be used to structure the face-to-face
meeting, with a committee structure being assumed for subsequent
monthly work group meetings. Annual face-to-face meetings are included
in the budget for subsequent years. Each workgroup will have one
Graduate Research Assistant (GRA) available to staff the workgroup
(one with Seekins for Intervention, one at OHSU with Krahn), and
will alternate having a postdoctoral fellow assigned to them. The
postdoctoral position may be posted for any of the panelists’ sites,
depending on expertise needed and availability of supervision of
the fellow. The criteria established by the panel will be distributed
to the 6 external reviewers for their comments. These will be reviewed
and incorporated by the panel. The panel’s modifications
will be reviewed by the external reviewers.
Anticipated Findings: A set of criteria for evaluating health
promotion and intervention as determined through consensus of the
Intervention Workgroup of the Expert Panel. The criteria will be
used in R5.