Abstract
Objective
To describe the adaptive measurement of change (AMC) as a means to identify psychometrically
significant change in reported function of hospitalized patients and to reduce respondent
burden on follow-up assessments.
Design
The AMC method uses multivariate computerized adaptive testing (CAT) and psychometric
hypothesis tests based in item response theory to more efficiently measure intra-individual
change using the responses of a single patient over 2 or more testing occasions. Illustrations
of the utility of AMC in clinical care and estimates of AMC-based item reduction are
provided using the Functional Assessment in Acute Care Multidimensional Computerized
Adaptive Test (FAMCAT), a newly developed functional multidimensional CAT-based measurement
of basic mobility, daily activities, and applied cognition.
Setting
Two quaternary hospitals in the Upper Midwest.
Participants
Four hundred ninety-five hospitalized patients who completed the FAMCAT on 2 to 4
occasions during their hospital stay.
Intervention
N/A.
Results
Of the 495 patients who completed more than 1 FAMCAT, 72% completed 2 sessions, 13%
completed 3, and 15% completed 4, with 22.1%, 23.4%, and 23.0%, respectively, exhibiting
significant multivariate change. Use of the AMC in conjunction with the FAMCAT reduced
respondent burden from that of the FAMCAT alone for follow-up assessments. On average,
when used without the AMC, 22.7 items (range, 20.4-24.4) were administered during
FAMCAT sessions. Post hoc analyses determined that when the AMC was used with the
FAMCAT a mean±standard deviation reduction in FAMCAT number of items of 13.6 (11.1),
13.1 (9.8), and 18.1 (10.8) would occur during the second, third, and fourth sessions,
respectively, which corresponded to a reduction in test duration of 3.0 (2.4), 3.0
(2.8), and 4.7 (2.6) minutes. Analysis showed that the AMC requires no assumptions
about the nature of change and provides data that are potentially actionable for patient
care. Various patterns of significant univariate and multivariate change are illustrated.
Conclusions
The AMC method is an effective and parsimonious approach to identifying significant
change in patients’ measured CAT scores. The AMC approach reduced FAMCAT sessions
by an average of 12.6 items (55%) and 2.9 minutes (53%) among patients with psychometrically
significant score changes.
Keywords
List of abbreviations:
AMC (adaptive measurement of change), CAT (computerized adaptive testing), ePROMs (electronic PROMs), FAMCAT (Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test), IRT (item response theory), PROM (patient-reported outcome measure), SEM (standard error of measurement)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Archives of Physical Medicine and RehabilitationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial.J Clin Oncol. 2016; 34: 557-565
- A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.BMC Health Serv Res. 2013; 13: 211
- Patient reported outcome measures could help transform healthcare.Bmj. 2013; 346: f167
- Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. Patient-reported outcomes: instrument development and selection issues.Value Health. 2007; 10: S86-S93
- Response burden and questionnaire length: is shorter better? A review and meta-analysis.Value Health. 2011; 14: 1101-1108
- Maximising response to postal questionnaires–a systematic review of randomised trials in health research.BMC Med Res Methodol. 2006; 6: 5
- Perceived burden of completion of patient-reported outcome measures in clinical trials: results of a preliminary study.Ther Innov Regul Sci. 2019; 53: 318-323
- Follow-up by mail in clinical trials: does questionnaire length matter?.Control Clin Trials. 2004; 25: 31-52
- Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes.J Clin Epidemiol. 2008; 61: 102-109
- Methods for assessing responsiveness: a critical review and recommendations.J Clin Epidemiol. 2000; 53: 459-468
- Minimum clinically important differences in chronic pain vary considerably by baseline pain and methodological factors: systematic review of empirical studies.J Clin Epidemiol. 2018; 101: 87-106.e2
- Depression outcomes in smokers and nonsmokers: comparison of collaborative care management versus usual care.J Prim Care Community Health. 2019; 102150132719861265
- Performance of an item response theory–based computer adaptive test in identifying functional decline.Arch Phys Med Rehabil. 2012; 93: 1153-1160
- Growth modeling: structural equation and multilevel modeling approaches.2016 (New York: Guilford; 2017)
- Latent curve analysis.Psychometrika. 1990; 55: 107-122
- Using covariance structure analysis to detect correlates and predictors of individual change over time.Psychol Bull. 1994; 116: 363-381
- Application of hierarchical linear models to assessing change.Psychol Bull. 1987; 101: 147-158
Cheville AL, Wang C, Yost, KJ, et al. Improving the delivery of function-directed care during acute hospitalizations: methods to develop and validate the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT). Arch Rehabil Res Clin Transl 2021;3:1–16.
Wang C, Weiss D, Cheville A. Multidimensional computerized adaptive testing for efficient and precise assessment of applied cognition, daily activity, and mobility for hospitalized patients. Arch Phys Med Rehabil; in press.
Weiss DJ, Wang C, DeWeese J, Basford J, Cheville A. Can proxy ratings supplement patient report to assess functional domains among hospitalized patients? Arch Phys Med Rehabil 2022;103:S34–42.
- Application of computerized adaptive testing to educational problems.J Educ Meas. 1984; 21: 361-375
- Item selection and hypothesis testing for the adaptive measurement of change.Appl Psychol Meas. 2010; 34: 238-254
- Hypothesis testing for adaptive measurement of individual change [dissertation].University of Minnesota, Minneapolis2015
- Measuring intra-individual change at two or more occasions with hypothesis testing methods [dissertation].University of Minnesota, Minneapolis2017
- Multivariate hypothesis testing methods for evaluating significant individual change.Appl Psychol Meas. 2018; 42: 221-239
- Hypothesis testing methods for multivariate multi-occasion intra-individual change.Multivariate Behav Res. 2020; 56: 1-17
Article info
Publication history
Published online: October 01, 2021
Accepted:
July 2,
2021
Received in revised form:
June 21,
2021
Received:
April 28,
2021
Footnotes
This research was supported by the Eunice Kennedy Shriver National Institutes of Child Health and Human Development of the National Institutes of Health under Award Number R01HD079439 to the Mayo Clinic in Rochester Minnesota through subcontracts to the University of Minnesota and the University of Washington. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Disclosures: none.
This paper is part of a supplement from the American Congress of Rehabilitation Medicine.
Identification
Copyright
© 2021 by the American Congress of Rehabilitation Medicine.
