Abstract
Objective
To develop and evaluate an efficient and precise variable-length functional assessment
of applied cognition, daily activity, and mobility to inform mobility preservation
and rehabilitation service delivery among hospitalized patients.
Design
A multidimensional item bank tapping into these dimensions was developed, with all
items calibrated using a multidimensional graded response model. The items were adaptively
selected from the item banks to maximize the test information, and the test ended
when a joint stopping rule was satisfied. A simulation study was conducted based on
the completed instrument, the Functional Assessment in Acute Care Multidimensional
Computerized Adaptive Test (FAMCAT), to compare its measurement precision and efficiency
capabilities relative to conventional unidimensional computerized adaptive testing.
Precision was measured by the bias and root mean squared error between the estimated
and true (ie, simulated) θ estimates, whereas efficiency was measured by average test
length. Data were collected by an interviewer reading questions from a tablet computer
and entering patients’ responses.
Setting
A large Midwestern hospital.
Participants
A total of 4143 patients hospitalized with medical diagnosis and/or surgical complications,
with 2060 in the calibration sample and 2083 in the validation cohort.
Intervention
Not applicable.
Results
Among the 2083 patients in the validation sample, FAMCAT administration required an
average of 6 (SD=3.11) minutes. Ninety-six percent had their tests terminated by the
standard error rule after responding to an average of 22.05 (SD=7.98) items, whereas
15 were terminated by the change in θ rule, with an average test length of 45.27 (SD=11.49).
The remaining 76 responded until reaching the maximum test length of 60 items.
Conclusions
The FAMCAT has the potential to satisfy the need for structured, frequent, and precise
assessment of functional domains among hospitalized patients with medical diagnosis
and/or surgical complications. The results are promising and may be informative for
others who wish to develop similar instruments when concurrent assessment of correlated
domains is required.
List of abbreviations:
CAT (computerized adaptive testing), CT-rule (change of θ rule), FAMCAT (Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test), IRT (item response theory), LD (local dependence), MAP (maximum a posteriori), MCAT (multidimensional computerized adaptive testing), MGRM (multidimensional graded response model), MIRT (multidimensional item response theory), PRO (patient-reported outcome), PROM (patient-reported outcome measure), PROMIS (Patient-Reported Outcome Measurement Information System), RMSE (root mean squared error), RMSEA (root mean square error of approximation), RT (response time), SE-rule (standard error rule), UCAT (unidimensional computerized adaptive testing)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Archives of Physical Medicine and RehabilitationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Modernizing quality of life assessment: development of a multidimensional computerized adaptive questionnaire for patients with schizophrenia.Qual Life Res. 2018; 27: 1041-1054
- Content-balancing strategy in bifactor computerized adaptive patient-reported outcome measurement.Qual Life Res. 2013; 22: 491-499
- Applying item response theory and computer adaptive testing: the challenges for health outcomes assessment.Qual Life Res. 2007; 16: 187-194
- Quality of life: the assessment, analysis and interpretation of patient-reported outcomes.John Wiley & Sons, Chichester, England2013
- Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): depression, anxiety, and anger.Assessment. 2011; 18: 263-283
- Improving measurement precision of hierarchical latent traits using adaptive testing.J Educ Behav Stat. 2014; 39: 452-477
- Predicting ADL disability in community-dwelling elderly people using physical frailty indicators: a systematic review.BMC Geriatr. 2011; 11: 1-11
- Measuring patient-reported outcomes adaptively: multidimensionality matters!.Appl Psychol Meas. 2018; 42: 327-342
- Measurement efficiency for fixed-precision multidimensional computerized adaptive tests: comparing health measurement and educational testing using example banks.Appl Psychol Meas. 2019; 43: 68-83
- Test equating, scaling, and linking: methods and practices.3rd ed. Springer Science + Business Media, New York2014
- Variable-length stopping rules for multidimensional computerized adaptive testing.Psychometrika. 2018; 84: 1-23
- Patient outcomes after hospital discharge to home with home health care vs to a skilled nursing facility.JAMA Intern Med. 2019; 179: 617-623
- Trends in post–acute care use among Medicare beneficiaries: 2000 to 2015.JAMA. 2018; 319: 1616-1617
- Improving the delivery of function-directed care during acute hosptilization: methods to develop and validate the functional assessment in acute care multidimensional computerized adaptive test (FAMCAT).Arch Rehabil Res Clin Transl. 2021; 3100112
- Adaptive measurement of change: a novel method to reduce respondent burden and detect significant individual-level change in patient-reported outcome measures.Arch Phys Med Rehabil. 2022; 103: S43-S52
- Sample size requirements for estimation of item parameters in the multidimensional graded response model.Front Psychol. 2016; 7: 109
- Linking scores with patient-reported health outcome instruments: a validation study and comparison of three linking methods.Psychometrika. 2021; 86: 1-30
Lee W-C, Lee G. IRT linking and equating. In: Irwing P, Booth T, Hughes DJ, Editors. The Wiley handbook of psychometric testing: a multidisciplinary reference on survey, scale and test development. Hoboken: Wiley Blackwell; 2018. p 639–73.
- flexMIRT® version 2: Flexible multilevel multidimensional item analysis and test scoring.Vector Psychometric Group, Chapel Hill, NC2013 ([Computer software])
Jiang S, Wang C. The different effects of collapsing categories on the graded response model and the generalized partial credit model. Paper presented at: American Educational Research Association Annual Meeting. April 5–9, Toronto, Canada; 2019.
- Deriving stopping rules for multidimensional computerized adaptive testing.Appl Psychol Meas. 2013; 37: 99-122
- Item selection in multidimensional computerized adaptive testing—gaining information from different angles.Psychometrika. 2011; 76: 363-384
- Multidimensional adaptive testing.Psychometrika. 1996; 61: 331-354
- Combining CAT with cognitive diagnosis: a weighted item selection approach.Behav Res Methods. 2012; 44: 95-109
- On latent trait estimation in multidimensional compensatory item response models.Psychometrika. 2015; 80: 428-449
- IRT model fit evaluation from theory to practice: progress and some unanswered questions.Measurement (Mahwah, NJ). 2013; 11: 102-106
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives.Struct Equ Modeling. 1999; 6: 1-55
- Application of the graded response model to a scale of empathie behavior.Int J Psychol Res. 2019; 12: 49-56
- Local dependence indexes for item pairs using item response theory.J Educ Behav Stat. 1997; 22: 265-289
- Likelihood-based item-fit indices for dichotomous item response theory models.Appl Psychol Meas. 2000; 24: 50-64
- Performance of the S− χ2 statistic for the multidimensional graded response model.Educ Psychol Meas. 2021; 81: 491-522
- State of the psychometric methods: patient-reported outcome measure development and refinement using item response theory.J Patient Rep Outcomes. 2019; 3: 1-16
- Using self-reported data to segment older adult populations with complex care needs.eGEMs. 2019; 7: 1-11
- Correlation and crosswalks between patient-reported functional outcomes and PROMIS physical function among medically ill patients.Arch Phys Med Rehabil. 2022; 103: S15-S23
Keeney T, Weiss DJ, Ni P, Wang C, Cheville AL. Ability of the Functional Assessment in Acute Care Multidimensional Computer Adaptive Test (FAMCAT) to predict hospitalization outcomes: 30-day hospital readmission and discharge to institutional post-acute care. Arch Phys Med Rehabil. 2021 Oct 17. [Epub ahead of print].
- Modeling response time and responses in multidimensional health measurement.Front Psychol. 2019; 10: 51
Lu J, Wang C, Weiss DJ. Using response time to improve precision and efficiency of computerized adaptive testing. Paper presented at: Biannual Meeting of the International Association of Computerized Adaptive Testing (IACAT). June 10–13, 2019; Minneapolis, MN.
- A theory of cognitive control, aging cognition, and neuromodulation.Neurosci Biobehav Rev. 2002; 26: 809-817
- Cognitive, sensory and physical factors enabling driving safety in older adults.Clin Psychol Rev. 2005; 25: 45-65
- Age and sex differences in reaction time in adulthood: results from the United Kingdom Health and Lifestyle Survey.Psychol Aging. 2006; 21: 62-73
- Utilizing response time distributions for item selection in CAT.J Educ Behav Stat. 2012; 37: 655-670
- Stopping rules for multi-category computerized classification testing.Br J Math Stat Psychol. 2021; 74: 184-202
- A zero-inflated Box-Cox normal unipolar item response model for measuring constructs of psychopathology.Appl Psychol Meas. 2018; 42: 571-589
- Item response theory and clinical measurement.Annu Rev Clin Psychol. 2009; 5: 27-48
- Item response theory and the measurement of psychiatric constructs: some empirical and conceptual issues and challenges.Psychol Med. 2016; 46: 2025-2039
- Robustness of parameter estimation to assumptions of normality in the multidimensional graded response model.Multivariate Behav Res. 2018; 53: 403-418
Lucke JF. Unipolar item response models. In: Reise SP, Revicki DA, editors. Handbook of item response theory modeling. Oxfordshire, England, UK: Routledge; 2014. p 290–302.
- Physical therapy in acute care: a clinician’s guide.Slack Inc, West Deptford2006
- Identifying clinically relevant functional strata to direct mobility preservation among patients hospitalized with medical conditions.Arch Phys Med Rehabil. 2022; 103: S78-S83
Article info
Publication history
Published online: January 24, 2022
Accepted:
January 12,
2022
Received in revised form:
December 20,
2021
Received:
August 8,
2021
Footnotes
Supported by the Eunice Kennedy Shriver National Institutes of Child Health and Human Development of the National Institutes of Health under Award Number R01HD079439 to the Mayo Clinic in Rochester Minnesota through a subcontract to the University of Minnesota and the University of Washington.
Disclosures: none.
This paper is part of a supplement from the American Congress of Rehabilitation Medicine.
Identification
Copyright
© 2022 by the American Congress of Rehabilitation Medicine.
