Abstract
Objective
To assess differential item functioning (DIF) in an item pool measuring the mobility
of hospitalized patients across educational, age, and sex groups.
Design
Measurement evaluation cohort study. Content experts generated DIF hypotheses to guide
the interpretation. The graded response item response theory (IRT) model was used.
Primary DIF tests were Wald statistics; sensitivity analyses were conducted using
the IRT ordinal logistic regression procedure. Magnitude and impact were evaluated
by examining group differences in expected item and scale score functions.
Setting
Hospital-based rehabilitation.
Participants
Hospitalized patients (N=2216).
Interventions
Not applicable.
Main Outcome Measures
A total of 111 self-reported mobility items.
Results
Two linking items among those used to set the metric across forms evidenced DIF for
sex and age: “difficulty climbing stairs step-over-step without a handrail (alternating
feet)” and “difficulty climbing 3-5 steps without a handrail.” Conditional on the
mobility state, the items were more difficult for women and older people (aged ≥65y).
An additional 18 items were identified with DIF. Items with both high DIF magnitude
and hypotheses related to age were difficulty “crossing road at a 4-lane traffic light
with curbs,” “jumping/landing on one leg,” “strenuous activities,” and “descending
3-5 steps with no handrail.” Although DIF of higher magnitude was observed for several
items, the scale-level effect was relatively small and the exposure rate for the most
problematic items was low (0.35, 0.27, and 0.20).
Conclusions
This was the first study to evaluate measurement equivalence of the hospital-based
rehabilitation mobility item bank. Although 20 items evidenced high magnitude DIF,
5 of which were related to stairs, the scale-level effect was minimal; however, it
is recommended that such items be avoided in the development of short-form measures.
No items with salient DIF were removed from calibrations, supporting the use of the
item bank across groups differing in education, age, and sex. The bank may thus be
useful to assist clinical assessment and decision-making regarding risk for specific
mobility restrictions at discharge as well as identifying mobility-related functions
targeted for postdischarge interventions. Additionally, with the goal of avoiding
long and burdensome assessments for patients and clinical staff, these results could
be informative for those using the item bank to construct short forms.
Keywords
List of abbreviations:
CAT (computerized adaptive testing), DIF (differential item functioning), FKGL (Flesch-Kincaid grade level), FREI (Flesch-Kincaid Reading Ease Index), IRT (item response theory), LD (local dependency), NCDIF (noncompensatory DIF), OLR (ordinal logistic regression)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Archives of Physical Medicine and RehabilitationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Sex differences in the prevalence of mobility disability in old age: the dynamics of incidence, recovery, and mortality.J Gerontol B Psychol Sci Soc Sci. 2000; 55: S41-S50
- Mobility in human aging: a multidisciplinary life span conceptual framework.Annu Rev Gerontol Geriatr. 2013; 33: 171-192
- Disability rises gradually for a cohort of older Americans.J Gerontol B Psychol Sci Soc Sci. 2017; 72: 151-161
- Growing older with a physical disability: a special application of the successful aging paradigm.J Gerontol B Psychol Sci Soc Sci. 2017; 72: 290-299
- Can successful aging and chronic illness coexist in the same individual? A multidimensional concept of successful aging.J Am Med Dir Assoc. 2009; 10: 87-92
- Assessment of function and disability in longitudinal studies.J Am Geriatr Soc. 2010; 58: S308-S312
- Upper-extremity and mobility subdomains from the Patient-Reported Outcomes Measurement Information System (PROMIS) adult physical functioning item bank.Arch Phys Med Rehabil. 2013; 94: 2291-2296
- Assessing the building blocks of function: utilizing measures of functional limitation.Am J Prev Med. 2003; 25: 112-121
- Validation of a mobility item bank for older patients in primary care.Health Qual Life Outcomes. 2012; 10: 147
- Gait speed at usual pace as a predictor of adverse outcomes in community-dwelling older people an International Academy on Nutrition and Aging (IANA) Task Force.J Nutr Health Aging. 2009; 13: 881-889
- Improvement in usual gait speed predicts better survival in older adults.J Am Geriatr Soc. 2007; 55: 1727-1734
- Mobility difficulties and physical activity as predictors of mortality and loss of independence in the community living older population.J Am Geriatr Soc. 2000; 48: 493-498
- Association of long-distance corridor walk performance with mortality, cardiovascular disease, mobility limitation, and disability.JAMA. 2006; 295: 2018-2026
- Gait speed and survival in older adults.JAMA. 2011; 305: 50-58
- Evaluation of the Patient-Reported Outcomes Information System (PROMIS) Spanish physical functioning items.Qual Life Res. 2013; 22: 1819-1830
- Differential item functioning in Patient Reported Outcomes Measurement Information System® (PROMIS®) Physical Functioning short forms: analyses across ethnically diverse groups.Psychol Test Assess Model. 2016; 58: 371-402
- EuroQol-a new facility for the measurement of health-related quality of life.Health Policy. 1990; 16: 199-208
- A Rasch model analysis to test the cross-cultural validity of the EuroQoL-5D in the Schizophrenia Outpatient Health Outcomes Study.Acta Psychiatrica Scandanavica. 2003; 107: 24-29
- A differential item functioning analysis of the EQ-5D in cancer.Value Health. 2016; 19: 1063-1067
- The Sickness Impact Profile: development and final revision of a health status measure.Med Care. 1981; 19: 787-805
- Scaling the Sickness Impact Profile using item response theory: an exploration of linearity, adaptive use, and patient driven item weights.J Clin Epidemiol. 2004; 57: 66-74
- Nottingham Health Profile.in: Spilker B Quality of life and pharmacoeconomics in clinical trials. 3rd ed. Lippincott-Raven Publishers, Philadelphia1996: 281-286
- Structural validity and age-based differential item functioning of the French Nottingham Health Profile in a sample of surgery patients.Adv Psychol Study. 2012; 1: 14-21
- The Rivermead Mobility Index allows valid comparisons between subgroups of patients undergoing rehabilitation after stroke who differ with respect to age, sex, or side of lesion.Arch Phys Med Rehabil. 2012; 93: 1086-1090
- Occurrences and sources of differential item functioning (DIF) in patient-reported outcome measures: description of DIF methods, and review of measures of depression, quality of life and general health.Psychol Sci Q. 2008; 50: 538-612
- A new readability yardstick.J Appl Psychol. 1948; 32: 221-223
- Measurement invariance, factor analysis and factorial invariance.Psychometrika. 1993; 58: 525-543
- An essay on measurement and factorial invariance.Med Care. 2006; 44: S69-S77
- Inferential conditions in the in the statistical detection of measurement bias.Appl Psychol Meas. 1992; 16: 389-402
- Methods and data analyses for cross-cultural research.Sage Publications, Thousand Oaks1997
- Differential item functioning.Lawrence Erlbaum, Hillsdale1993
- Applications of item response theory to practical testing problems.Lawrence Erlbaum, Hillsdale1980
Wang & Weiss, this series, in press.
- Estimation of latent ability using a response pattern of graded scores.Psychometrika Monogr Suppl. 1969; 34: 100-114
- Identification of differential item functioning using item response theory and the likelihood-based model comparison approach: applications to the Mini-Mental State Examination.Med Care. 2006; 44: S134-S142
- IRTPRO: Flexible, multidimensional, multiple categorical IRT Modeling.Scientific Software International, Inc, Chicago2011
- A re-examination of Lord's Wald test for differential item functioning using item response theory and modern error estimation [dissertation].University of North Carolina at Chapel Hill, Chapel Hill, NC2008
- Modern psychometric methods for detection of differential item functioning: application to cognitive assessment measures.Stat Med. 2000; 19: 1651-1683
- The Langer-improved Wald test for DIF testing with multiple groups: evaluation and comparison to two-group IRT.Educ Psychol Meas. 2013; 73: 532-547
- Overview to the two-part series: measurement equivalence of the patient-reported outcomes measurement information system (PROMIS®) short-forms.Psychol Test Assess Model. 2016; 58: 31-35
Teresi JA, Wang C, Kleinman M, Jones RN, Weiss DJ. Differential item functioning analyses of the Patient Reported Outcomes Measurement Information System (PROMIS) measures: methods, challenges, advances and future directions. Psychometrika. 2021 Jul 12. [Epub ahead of print].
- Detecting differential item functioning using logistic regression procedures.J Educ Meas. 1990; 27: 361-370
- A handbook on the theory and methods of differential item functioning (DIF): logistic regression modeling as a unitary framework for binary and Likert-type (ordinal) item scores.Directorate of Human Resources Research and Evaluation, Department of National Defense, Ottawa, Canada1999
- DFITP5: a Fortran program for calculating dichotomous DIF/DTF.Illinois Institute of Technology, Chicago1999
- IRT-based internal measures of differential functioning of items and tests.Appl Psychol Meas. 1995; 19: 353-368
- A description and demonstration of the polytomous DFIT framework.Appl Psychol Meas. 1999; 23: 309-326
- DFIT for Windows user's manual: differential functioning of items and tests.St. Paul: Assessment Systems Corporation;. 2009;
- Test theory: a unified treatment.L. Erlbaum Associates, Mahwah1999
- Coefficient alpha and the internal structure of tests.Psychometrika. 1951; 16: 297-334
- Ordinal versions of coefficient alpha and theta for Likert rating scales.J Mod Appl Stat Methods. 2007; 6: 21-29
- Statistical theories of mental test scores.Addison Wesley, Reading1968
- Standard error of ability estimates and the classification accuracy and consistency of binary decisions.Psychometrika. 2015; 80: 645-664
- Comparison of reliability measures under factor analysis and item response theory.Ed Psych Meas. 2012; 72: 52-67
- DIF-CAT: doubly adaptive CAT using subgroup information to improve measurement precision.in: Paper presented at: 2017 International Association of Computerized Adaptive Testing, Niigata, Japan2017 (August 18-21,)
- Sample size requirements for estimation of item parameters in the multidimensional graded response model.Frontiers Psych. 2016; 7: 1-10
- Demographic variation in SF-12 scores: true differences or differential item functioning?.Med Care. 2003; 41 (III75-86)
- A comparison of three sets of criteria for determining the presence of differential item functioning using ordinal logistic regression.Qual Life Res. 2007; 16: 69-84
- Evaluating measurement equivalence using the item response theory log-likelihood ratio (IRTLR) method to assess differential item functioning (DIF): applications (with illustrations) to measures of physical functioning ability and general distress.Qual Life Res. 2007; 16: 43-68
- Assessment of differential item functioning for demographic comparisons in the MOS SF-36 health survey.Qual Life Res. 2006; 15: 331-348
- Using item banks to construct measures of patient reported outcomes in clinical trials: investigator perceptions.Clin Trials. 2008; 5: 575-586
- Item banking to improve, shorten and computerize self-reported fatigue: an illustration of steps to create a core item bank from the FACIT-Fatigue Scale.Qual Life Res. 2003; 12: 485-501
- More relevant, precise, and efficient items for assessment of physical function and disability: moving beyond the classic instruments.Ann Rheum Dis. 2006; 65 (iii16-21)
- Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS).Med Care. 2007; 45 (Suppl 1): S22-S31
- After differential item functioning is detected: IRT item calibration and scoring in the presence of DIF.Appl Psychol Meas. 2016; 40: 573-591
- Handbook of Item Response Theory: volume 3: applications.(editor.) Routledge, Milton Park, UK2018
Article info
Publication history
Published online: June 15, 2021
Accepted:
March 25,
2021
Received in revised form:
March 20,
2021
Received:
July 24,
2020
Footnotes
Supported by the National Institute of Child Health and Human Development (grant no. 1R01HD079439-01A1), the National Institute on Aging, Pepper Center at Mount Sinai (grant no. P30AG028741), and the Roybal Center at Weill Cornell (grant no. P30AG022845).
Disclosures: none.
This paper is part of a supplement from the American Congress of Rehabilitation Medicine.
Identification
Copyright
© 2021 Published by Elsevier Inc. on behalf of the American Congress of Rehabilitation Medicine
