Advertisement

Adaptive Measurement of Change: A Novel Method to Reduce Respondent Burden and Detect Significant Individual-Level Change in Patient-Reported Outcome Measures

Published:October 01, 2021DOI:https://doi.org/10.1016/j.apmr.2021.07.814

      Abstract

      Objective

      To describe the adaptive measurement of change (AMC) as a means to identify psychometrically significant change in reported function of hospitalized patients and to reduce respondent burden on follow-up assessments.

      Design

      The AMC method uses multivariate computerized adaptive testing (CAT) and psychometric hypothesis tests based in item response theory to more efficiently measure intra-individual change using the responses of a single patient over 2 or more testing occasions. Illustrations of the utility of AMC in clinical care and estimates of AMC-based item reduction are provided using the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT), a newly developed functional multidimensional CAT-based measurement of basic mobility, daily activities, and applied cognition.

      Setting

      Two quaternary hospitals in the Upper Midwest.

      Participants

      Four hundred ninety-five hospitalized patients who completed the FAMCAT on 2 to 4 occasions during their hospital stay.

      Intervention

      N/A.

      Results

      Of the 495 patients who completed more than 1 FAMCAT, 72% completed 2 sessions, 13% completed 3, and 15% completed 4, with 22.1%, 23.4%, and 23.0%, respectively, exhibiting significant multivariate change. Use of the AMC in conjunction with the FAMCAT reduced respondent burden from that of the FAMCAT alone for follow-up assessments. On average, when used without the AMC, 22.7 items (range, 20.4-24.4) were administered during FAMCAT sessions. Post hoc analyses determined that when the AMC was used with the FAMCAT a mean±standard deviation reduction in FAMCAT number of items of 13.6 (11.1), 13.1 (9.8), and 18.1 (10.8) would occur during the second, third, and fourth sessions, respectively, which corresponded to a reduction in test duration of 3.0 (2.4), 3.0 (2.8), and 4.7 (2.6) minutes. Analysis showed that the AMC requires no assumptions about the nature of change and provides data that are potentially actionable for patient care. Various patterns of significant univariate and multivariate change are illustrated.

      Conclusions

      The AMC method is an effective and parsimonious approach to identifying significant change in patients’ measured CAT scores. The AMC approach reduced FAMCAT sessions by an average of 12.6 items (55%) and 2.9 minutes (53%) among patients with psychometrically significant score changes.

      Keywords

      List of abbreviations:

      AMC (adaptive measurement of change), CAT (computerized adaptive testing), ePROMs (electronic PROMs), FAMCAT (Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test), IRT (item response theory), PROM (patient-reported outcome measure), SEM (standard error of measurement)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Basch E
        • Deal AM
        • Kris MG
        • et al.
        Symptom monitoring with patient-reported outcomes during routine cancer treatment: a randomized controlled trial.
        J Clin Oncol. 2016; 34: 557-565
        • Chen J
        • Ou L
        • Hollis SJ.
        A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting.
        BMC Health Serv Res. 2013; 13: 211
        • Black N.
        Patient reported outcome measures could help transform healthcare.
        Bmj. 2013; 346: f167
        • Turner RR
        • Quittner AL
        • Parasuraman BM
        • Kallich JD
        • Cleeland CS
        Mayo/FDA Patient-Reported Outcomes Consensus Meeting Group. Patient-reported outcomes: instrument development and selection issues.
        Value Health. 2007; 10: S86-S93
        • Rolstad S
        • Adler J
        • Ryden A.
        Response burden and questionnaire length: is shorter better? A review and meta-analysis.
        Value Health. 2011; 14: 1101-1108
        • Nakash RA
        • Hutton JL
        • Jorstad-Stein EC
        • Gates S
        • Lamb SE.
        Maximising response to postal questionnaires–a systematic review of randomised trials in health research.
        BMC Med Res Methodol. 2006; 6: 5
        • Bodart S
        • Byrom B
        • Crescioni M
        • Eremenco S
        • Flood E.
        Perceived burden of completion of patient-reported outcome measures in clinical trials: results of a preliminary study.
        Ther Innov Regul Sci. 2019; 53: 318-323
        • Edwards P
        • Roberts I
        • Sandercock P
        • Frost C.
        Follow-up by mail in clinical trials: does questionnaire length matter?.
        Control Clin Trials. 2004; 25: 31-52
        • Revicki D
        • Hays RD
        • Cella D
        • Sloan J.
        Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes.
        J Clin Epidemiol. 2008; 61: 102-109
        • Husted JA
        • Cook RJ
        • Farewell VT
        • Gladman DD.
        Methods for assessing responsiveness: a critical review and recommendations.
        J Clin Epidemiol. 2000; 53: 459-468
        • Olsen MF
        • Bjerre E
        • Hansen MD
        • Tendal B
        • Hilden J
        • Hrobjartsson A.
        Minimum clinically important differences in chronic pain vary considerably by baseline pain and methodological factors: systematic review of empirical studies.
        J Clin Epidemiol. 2018; 101: 87-106.e2
        • Akambase JA
        • Miller NE
        • Garrison GM
        • Stadem P
        • Talley H
        • Angstman KB.
        Depression outcomes in smokers and nonsmokers: comparison of collaborative care management versus usual care.
        J Prim Care Community Health. 2019; 102150132719861265
        • Cheville AL
        • Yost KJ
        • Larson DR
        • et al.
        Performance of an item response theory–based computer adaptive test in identifying functional decline.
        Arch Phys Med Rehabil. 2012; 93: 1153-1160
        • Grimm KJ
        • Ram N
        • Estabrook R.
        Growth modeling: structural equation and multilevel modeling approaches.
        2016 (New York: Guilford; 2017)
        • Meredith W
        • Tisak J.
        Latent curve analysis.
        Psychometrika. 1990; 55: 107-122
        • Willett JB
        • Sayer AG.
        Using covariance structure analysis to detect correlates and predictors of individual change over time.
        Psychol Bull. 1994; 116: 363-381
        • Bryk AS
        • Raudenbush SW.
        Application of hierarchical linear models to assessing change.
        Psychol Bull. 1987; 101: 147-158
      1. Cheville AL, Wang C, Yost, KJ, et al. Improving the delivery of function-directed care during acute hospitalizations: methods to develop and validate the Functional Assessment in Acute Care Multidimensional Computerized Adaptive Test (FAMCAT). Arch Rehabil Res Clin Transl 2021;3:1–16.

      2. Wang C, Weiss D, Cheville A. Multidimensional computerized adaptive testing for efficient and precise assessment of applied cognition, daily activity, and mobility for hospitalized patients. Arch Phys Med Rehabil; in press.

      3. Weiss DJ, Wang C, DeWeese J, Basford J, Cheville A. Can proxy ratings supplement patient report to assess functional domains among hospitalized patients? Arch Phys Med Rehabil 2022;103:S34–42.

        • Weiss DJ
        • Kingsbury GG.
        Application of computerized adaptive testing to educational problems.
        J Educ Meas. 1984; 21: 361-375
        • Finkelman MD
        • Weiss DJ
        • Kim-Kang G.
        Item selection and hypothesis testing for the adaptive measurement of change.
        Appl Psychol Meas. 2010; 34: 238-254
        • Lee JE.
        Hypothesis testing for adaptive measurement of individual change [dissertation].
        University of Minnesota, Minneapolis2015
        • Phadke C.
        Measuring intra-individual change at two or more occasions with hypothesis testing methods [dissertation].
        University of Minnesota, Minneapolis2017
        • Wang C
        • Weiss DJ.
        Multivariate hypothesis testing methods for evaluating significant individual change.
        Appl Psychol Meas. 2018; 42: 221-239
        • Wang C
        • Weiss DJ
        • Suen KY.
        Hypothesis testing methods for multivariate multi-occasion intra-individual change.
        Multivariate Behav Res. 2020; 56: 1-17