Image: Research
Link: Click here to skip navigational elements
Image: orange spacer
Link: Home Link: About Link: Research Link: Training Link: Dissemination/Technical Assistance
Link: What's New
Link: Health Information
Link: Library
Link: Search
Link: Tools
Link: Site Map Link: Archive
Link: Barriers and Strategies Validation Study
Link: Clinic Access Checklist Reliability Study
Link: Health Care Access Intervention Study
Link: Validation Criteria for Health Promotion Study
Link: Ratings of Health Interventions Programs
Link: Review of Health Status Measures Study
Link: Analysis of BRFSS HRQOL Study
Link: Health Status Tools Development Study
  Image: Research

R8 Health Status Tools Development Study

Principal Investigator: Gloria Krahn, PhD, MPH; Expert Panel on Measurement

The purpose of this study is to develop and test a health status measurement tool to reliably assess the health status of individuals regardless of functional ability. The tool will also be tested to determine if it is sufficiently sensitive to detect change due to interventions. Several approaches will be used to develop and test the items used in the tool as well as validate its use with people living with disabilities.

Hypothesis 1: A measurement tool can be developed to reliably assess the health status of individuals regardless of functional ability.

Hypothesis 2: A measurement tool can be developed to reliably assess self-reported health status and that is sufficiently sensitive to detect change due to interventions

Background to the Hypotheses: In its 30-year history, HRQOL and health status assessment has generated numerous measures and developed a knowledge base about health assessment. The origins of this field in the 1970’s preceded the transformation in the conceptualization of health and function as separate constructs. Consequently, many current measures of health assessment are rooted in the presumption that health and functional status are highly correlated, and measures like the SF-36 used the ability to discriminate between the scores of people with disabilities and the general population as proof of the validity of the measure. For disability and health researchers, it is critically important to disentangle this conceptual confounding. Without this distinction in assessment, researchers cannot use these measures to assess health disparities or to confirm effectiveness of health intervention efforts to reduce these disparities. The first step in developing a conceptualization of the meaning of health and its measurement requires thorough discussion about how to define it (Leplege & Hunt, 1997), with input from both researchers and persons who experience living with a disability (Jette & Keysor, 2002). Numerous QOL measures have been criticized for not adequately defining QOL or the meaning of health before measuring “it” (Gill & Feinstein, 1994). Our earlier self-definition focus group study on the meaning of health and wellness generated four general themes in addition to much valuable narrative to help in this process. These themes addressed self-determination, self-efficacy, physical and emotional well-being, and freedom from pain. Other definitional elements will no doubt be added. Conceptualization of health status should lead to item selection and generation for its assessment. Data from R7 will help determine how well the HRQOL items of the BRFSS function together and with what other elements they correlate highly. The element of “freedom from pain” may be a unique contribution of disability to a general conceptualization of health and wellness, and BRFSS data should again provide a beginning point for item selection. In 1995, the WHO developed an item bank of 200 quality of life items that serve as a second source of items. Fewer data, however, are available by which to assess their performance. McHorney (1999) warns against relying too heavily on established items to assess health status, cautioning against earlier biases and multiple concepts embedded in some earlier items. The next step in the process is scale construction. The expert panel deliberations on whether health status should be regarded as unidimensional or multi-dimensional will influence this part of the process. If the scale is intended to reflect multi-dimensionality of health status, a multi-attribute health status classification (MAHSC) system can ensure equal weighting across multiple axes or elements of a construct definition (Feeny et al., 1995; Wang et al., 1999). A hierarchical or graded concept approach represents a more targeted measurement design. Within this approach, health concepts that have the greatest clinical, policy, social or economic bearing on the intended outcomes (e.g. participation), should be measured with the greatest precision. These concepts (e.g., physical health, emotional health, pain) sit in the center of the target, with related concepts such as vitality, social or sexual functioning reside in circles surrounding the bullseye of the target (McHorney, 1999). While traditional scale construction using classical test theory focused largely on items in relation to a total score, other approaches to test construction also emphasize the item in relation to other items. Item Response Theory (IRT) represents a statistical approach to careful selection of items. IRT is both a theoretical framework and a collection of quantitative techniques that help provide quantitative parameters for individual items that are independent of population samples. Rasch modeling is a type of IRT. It constructs a line of measurement, with the items placed hierarchically along a continuum based on the probabilistic linear interaction between the person’s “ability” and the item’s “difficulty” (Prieto, Alonso, & Lamarca, 2003). IRT is based on a unidimensional understanding of the construct. A multi-attribute view of health status would require the construction of multiple dimensions to create a health profile. This would likely require a lengthier assessment tool. A final issue for health status assessment is that of a tool being sufficiently responsive to change that it documents effects of intervention efforts. The review by Andresen & Meyers (2000) notes that while the SF-36 and Sickness Inventory Profile (SIP) show some degree of responsivity, these measures are not highly useful for that purpose. Our own work with single health assessment items of the BRFSS indicates that in a non-longitudinal design, health is much more highly predicted by the longstanding composite variables of education and income than by concurrent health promoting behaviors (e.g., exercise, fruits and vegetables consumption) or health risk behaviors (e.g., alcohol use, smoking) (Krahn et al., 2004). A second tool may be needed that more responsively captures changes in health behaviors that may not have immediate self-perceived effect on health status. Examples of such tools for generic populations include the Health Promoting Lifestyle Profile (HPLP) and the Craig Hospital inventory for SCI. We are currently developing a self-health checklist for people experiencing a variety of disabling conditions that could be expanded into such a tool. We plan to collaborate with the Rehabilitation Institute of Chicago to refine this tool (see attached letter of collaboration).

Research Design: Expert panel process and MAHSC for conceptualization of measure, classical test theory and item response theory approaches for selection of test items and validation of the scale.

Sample: Independent samples of persons with disabilities for alpha and beta testing of the tool(s)

a) 10 adults with disabilities (AWD) of differing education levels for cognitive appraisal of test items

b) 100 AWD with at least 50 having lower education levels from a Medicaid population for pilot testing of the tools

c) 700 AWD stratified on education level; 100 of these AWD will also complete the SF-36. The sample will be recruited from the RRTC: Health & Wellness database on AWD, Independent Living Center (ILC) populations, and the CareOregon cohort

d) 100 AWD enrolled in intervention program(s)—pre and post test

Data Collection and Measurement: There are four stages to data collection: 1) item selection and/or generation; 2) cognitive appraisal testing of the items on health status; 3) initial testing of the items and scale construction for health status (alpha test); and 4) field testing of the health status tool and the health behaviors tool with several samples (beta test). The expert panel will provide oversight and consultation throughout this process. Initial item selection for health status will be determined based on the expert review panel recommendations from R6 regarding definition of health status and its component element(s), and from the panel’s recommendations for desired measurement characteristics. These recommendations will serve as the framework for the health status scale construction. Findings from the analyses of R7 will influence the potential inclusion of the BRFSS HRQOL items. The WHO item inventory will also be examined for potential items, and additional items will be generated as needed. Items will be examined for reading level, with the intent of keeping reading level to the 8th grade level or below. Once the initial set of questions has been drafted, it will undergo cognitive appraisal testing. At least 10 people with disabilities with varied educational backgrounds, disabilities and life experiences will be asked to read the items and be interviewed in a standard format to understand their interpretation of the question and the reasoning behind their responses. Items will be reworded for increased clarity and precision as needed. Additional respondents will be recruited if a second round of cognitive appraisal testing is necessary. The initial draft of the health status tool along with the health behavior tool will be pilot-tested with 100 adults with disabilities to identify potential difficulties with administration, respondent burden, and to provide initial data on item performance. At least half of this sample will be recruited through Care Oregon and be selected to have an educational level of some college or less to ensure utility with a range of educational backgrounds. Field-testing of the tools will occur through a series of samples. First, a sample of 700 adults with disabilities will be recruited that is stratified by educational level and gender. This sample size should be adequate to allow for factor analyses of a beta-test measure with as many as 60-70 items. Demographic information will also be collected that includes age, type and severity of disability. These variables will be examined later for their correlation with the scales and the items, but will not be used for stratification purposes. Administration will be conducted as a paper and pencil test or online. All respondents will be asked to complete all items from the health status tool and the health behavior tool. Respondents will be compensated for their participation (budgeted at $30 per respondent). A second sample of 50 adults who are participating in a health promotion intervention program will be asked to complete the tools as pre and post-test measures of their participation. This sample size of 50 will be used because it is a reasonable cohort size for an intervention program. If effect sizes cannot be detected with this sample size, the tool(s) are not likely to be useful as measures of intervention effectiveness. A third sample of 100 individuals with disabilities will be recruited for test-retest reliability. These may be drawn from the larger general sample with a retest administration occurring 2 weeks after first administration to examine short-term stability of the measure(s).

Data Analyses: Classical test theory indicators will be used for initial screening of potential items, and item response theory will be used for refinement of the unidimensional scale(s). For the pilot test data, the draft items of the heath status tool will be subjected to item analysis using standard statistical procedures. An index of discrimination will be calculated as an item-total correlation, with items with a correlation of less than .40 being considered for elimination. An endorsement index will be calculated as the proportion of people responding “yes” to an item; those with scores of less than .20 (too hard) or .80 (too easy) will be considered for elimination. Test-retest reliability indices will be calculated across the two test administrations, and items with less than a .50 will be considered for elimination. These results will inform the selection of items for the beta-test version. Analyses will be performed in either the Statistical Package for Social Sciences (SPSS) or Statistical Analysis Software (SAS) program. Exploratory factor analyses will be conducted on the larger sample to determine the degree to which a unidimensional measure is feasible. Based on these findings, item response theory analyses will proceed with the full set or subsets of the health status items. Log-odds units (logits) will be calculated for items. Item calibration (expressed as logits) will define the hierarchical order of severity of “difficulty” of the item along the health status continuum. Logits of greater magnitude indicate increasing item severity. In addition, the unidimensionality of the scale or subscales will be evaluated through a goodness-of-fit test (Chi-square fit statistics) and by a formal test of the assumption of local independence. Successive Rasch analyses will be performed until a final set of items satisfy the model fit requirements. These analyses will be conducted in BIGSTEPS or an equivalent program. Analyses for the health behavior tool will be more limited and follow classical test theory procedures. Unidimensionality is not necessarily a desired feature of this tool. For the tool to demonstrate responsiveness to a range of interventions (e.g, physical activity, stress management, nutrition), a “profile” measure may be more appropriate that includes several components that are measured separately. Analyses for this tool will examine item-subtotal correlations, item-endorsement indices, test-retest correlations, and pre-test/post-test comparisons. Analyses will be conducted in SPSS or SAS.

Anticipated Findings: A beta-tested measure of health status assessment for adults with disabilities that has been constructed to be nonbiased with respect to functional abilities. Psychometric characteristics will have been determined for internal consistency, hierarchical linearity, test-retest reliability, responsivity to intervention, and correlation with the SF-36. Despite problems with the SF-36, it remains an established standard in the field of health measurement. It is important to explore how a measure with a different philosophical approach will relate to the SF-36.







We want your feedback! Please tell us what you think of the information on this page.

Take the short survey!

Home | About | Research | Training | Dissemination/Technical Assistance | Health Info | Library
Search | Tools | Site Map | What's New

© 2004, RRTC Health & Wellness Consortium. All Rights Reserved. Contact webmaster.

Updated 4/15/05