Archives of Physical Medicine and Rehabilitation
Volume 89, Issue 2 , Pages 275-283 , February 2008

Computerized Adaptive Testing for Follow-Up After Discharge From Inpatient Rehabilitation: II. Participation Outcomes

  • Stephen M. Haley, PhD

      Affiliations

    • Health and Disability Research Institute, School of Public Health, Boston University Medical Center, Boston, MA
    • Corresponding Author InformationReprint requests to Stephen Haley, PhD, Health and Disability Research Institute, Boston University School of Public Health, Boston University Medical Center, 580 Harrison Ave, 4th Fl, Boston, MA 02118-2639
    • ,
  • Barbara Gandek, MS

      Affiliations

    • Health Assessment Lab, Waltham, MA
    • ,
  • Hilary Siebens, MD

      Affiliations

    • Department of Physical Medicine and Rehabilitation, University of Virginia at Charlottesville, Charlottesville, VA
    • ,
  • Randie M. Black-Schaffer, MD

      Affiliations

    • Spaulding Rehabilitation Hospital and Department of Physical Medicine and Rehabilitation, Harvard Medical School, Boston, MA.
    • ,
  • Samuel J. Sinclair, PhD

      Affiliations

    • Health Assessment Lab, Waltham, MA
    • ,
  • Wei Tao, BS

      Affiliations

    • Health and Disability Research Institute, School of Public Health, Boston University Medical Center, Boston, MA
    • ,
  • Wendy J. Coster, PhD

      Affiliations

    • Department of Occupational Therapy and Rehabilitation Counseling, Sargent College of Health and Rehabilitation Sciences, Boston University Medical Center, Boston, MA
    • ,
  • Pengsheng Ni, MD

      Affiliations

    • Health and Disability Research Institute, School of Public Health, Boston University Medical Center, Boston, MA
    • ,
  • Alan M. Jette, PhD

      Affiliations

    • Health and Disability Research Institute, School of Public Health, Boston University Medical Center, Boston, MA

References 

  1. Gandek B, Sinclair J, Jette A, Ware JE. Development and initial testing of the Participation Measure for Post-Acute Care (PM-PAC). Am J Phys Med Rehabil. 2007;86:57–71
  2. Gray D, Hollingsworth H, Stark S, Morgan K. Participation survey/mobility: psychometric properties of a measure of participation for people with mobility impairments and limitations. Arch Phys Med Rehabil. 2006;87:189–197
  3. Jette AM, Keysor J, Coster W, Ni P, Haley S. Beyond function: predicting participation in a rehabilitation cohort. Arch Phys Med Rehabil. 2005;86:2087–2094
  4. Noreau L, Desrosiers J, Robichaud L, Fouqeyrollas P, Rochette A, Viscoqliosi C. Measuring social participation: reliability of the LIFE-H in older adults with disabilities. Disabil Rehabil. 2004;26:346–352
  5. Dijkers MP, Whiteneck G, El-Jaroudi R. Measures of social outcomes in disability research. Arch Phys Med Rehabil. 2000;81(12 Suppl 2):S63–S80
  6. Whiteneck GC, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: a new measure of long-term rehabilitation outcomes. Arch Phys Med Rehabil. 1992;73:519–526
  7. Hearing before the Subcommittee on Health of the House Committee on Ways and Means on Standardized Payment and Patient Assessments in Post-Acute Care, 109th Cong, 1st Sess (testimony of Herb Kuhn, director, Center for Medicare Management, Centers for Medicare Medicaid Services).
  8. Nagi S. A study in the evaluation of disability and rehabilitation potential: concepts, methods, and procedures. Am J Public Health. 1964;54:1568–1579
  9. Pope AM, Tarlov A. Disability in America: toward a national agenda for prevention. Washington (DC): Natl Acad Pr; 1991;
  10. World Health Organization. International classification of functioning, disability, and health. Geneva: WHO; 2001;
  11. Stucki G, Ewert T, Cieza A. Value and application of the ICF in rehabilitation medicine. Disabil Rehabil. 2003;25:628–634
  12. World Health Organization. International classification of functioning, disability and handicap (ICF). Geneva: WHO; 2001;
  13. Stucki G. International classification of functioning, disability, and health (ICF): a promising framework and classification for rehabilitation medicine. Am J Phys Med Rehabil. 2005;84:733–740
  14. Weigl M, Cieza A, Andersen C, Kollerits B, Amann E, Stucki G. Identifications of relevant ICF categories in patients with chronic health conditions: a Delphi exercise. J Rehabil Med. 2004;36:12–21
  15. Brown M, Dijkers M, Gordon W, Ashman T, Charatz H, Cheng Z. Participation objective, participation subjective: a measure of participation combining outsider and insider perspectives. J Head Trauma Rehabil. 2004;19:459–481
  16. Ware JE, Gandek B, Sinclair SJ, Bjorner B. Item response theory in computer adaptive testing: implications for outcomes measurement in rehabilitation. Rehabil Psychol. 2005;50:71–78
  17. Jette AM, Haley SM. Contemporary measurement techniques for rehabilitation outcomes assessment. J Rehabil Med. 2005;37:339–345
  18. Cella D, Gershon R, Lai JS, Choi S. The future of outcomes measurement:item banking, tailored short forms, and computerized adaptive assessment. Qual Life Res. 2007;16(Suppl 1):133–141
  19. Fries J, Bruce B, Cella D. The promise of PROMIS: using item response theory to improve assessment of patient-reported outcomes. Clin Exp Rheumatol. 2005;23:S53–S57
  20. Cella D, Young S, Rothrock N, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45:S3–S11
  21. Hambleton RK. Applications of item response theory to improve health outcomes assessment: developing item banks, linking instruments, and computer-adaptive testing. In:  Lipscomb J,  Gotay CC,  Snyder C editor. Outcomes assessment in cancer. Cambridge: Cambridge Univ Pr; 2005;p. 445–464
  22. Fayers P. Applying item response theory and computer adaptive testing: the challenges for health outcomes assessment. Qual Life Res. 2007;16(Suppl 1):187–194
  23. Wainer H. Computerized adaptive testing: a primer. Mahwah: Lawrence Erlbaum Assoc; 2000;
  24. Dijkers MP. A computer adaptive testing simulation applied to the FIM instrument motor component. Arch Phys Med Rehabil. 2003;84:384–393
  25. Revicki DA, Cella DF. Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res. 1997;6:595–600
  26. Haley SM, Ni P, Hambleton RK, Slavin MD, Jette AM. Computer adaptive testing improves accuracy and precision of scores over random item selection in a physical functioning item bank. J Clin Epidemiol. 2006;59:1174–1182
  27. Haley SM, Coster WJ, Andres PL, Kosinski M, Ni P. Score comparability of short-forms and computerized adaptive testing: simulation study with the activity measure for post-acute care. Arch Phys Med Rehabil. 2004;85:661–666
  28. Andres PL, Black-Schaffer RM, Ni PS, Haley SM. Computer adaptive testing: a strategy for monitoring stroke rehabilitation across settings. Top Stroke Rehabil. 2004;11:33–39
  29. Siebens H, Andres PL, Ni P, Coster WJ, Haley SM. Measuring physical function in patients with complex medical and postsurgical conditions: a computer adaptive approach. Am J Phys Med Rehabil. 2005;84:741–748
  30. Hart D, Mioduski J, Werenke M, Stratford P. Simulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:947–956
  31. Hart DL, Cook KF, Mioduski JE, Teal CR, Crane PK. Simulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:290–298
  32. Hart DL, Mioduski JE, Stratford PW. Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments. J Clin Epidemiol. 2005;58:629–638
  33. Kosinski M, Bjorner JB, Ware JE, Sullivan E, Straus WL. An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. J Clin Epidemiol. 2006;59:715–723
  34. Haley SM, Raczek AE, Coster WJ, Dumas HM, Fragala-Pinkham MA. Assessing mobility in children using a computer adaptive testing version of the Pediatric Evaluation of Disability Inventory. Arch Phys Med Rehabil. 2005;86:932–939
  35. Haley SM, Fragala-Pinkham MA, Ni P. Sensitivity of a computer adaptive assessment for measuring functional mobility changes in children enrolled in a community fitness program. Clin Rehabil. 2006;20:616–622
  36. Jette A, Haley S, Tao W, et al. Prospective evaluation of the AM-PAC-CAT in outpatient rehabilitation settings. Phys Ther. 2007;87:385–398
  37. Haley S, Siebens H, Coster W, et al. Computerized adaptive testing for follow-up after discharge from inpatient rehabilitation: I. Activity outcomes. Arch Phys Med Rehabil. 2006;87:1033–1042
  38. Jette AM. Toward a common language for function, disability, and health. Phys Ther. 2006;86:726–734
  39. van Swieten JC, Koudstaal PJ, Visser MC, Schouten HJ, van Gijn J. Interobserver agreement for the assessment of handicap in stroke patients. Stroke. 1988;19:604–607
  40. Ware J, Kosinski M, Dewey J, Gandek B. How to score and interpret single-item health status measures: a manual for users of the SF-8 health survey. Lincoln: QualityMetric; 1999;
  41. Willer B, Ottenbacher KJ, Coad ML. The community integration questionnaire: a comparative examination. Am J Phys Med Rehabil. 1994;73:10–11
  42. Jette AM, Davies AR, Cleary PD, et al. The Functional Status Questionnaire: reliability and validity when used in primary care. J Gen Intern Med. 1986;1:143–149
  43. Cardol M, de Haan RJ, de Jong BA, van den Bos GA, de Groot IJ. Psychometric properties of the Impact on Participation and Autonomy Questionnaire. Am J Phys Med Rehabil. 2001;82:210–216
  44. In:  Stewart AL,  Ware JE editor. Measuring function and well-being: the measuring outcomes study approach. Durham: Duke Univ Pr; 1992;
  45. U.S. Department of Health and Human ServicesNational Center for Health Statistics. National Health Interview Survey, 1994: Second Longitudinal Study on Aging, Wave 3. [computer file] Hyattsville: DHHS, NCHS; 2003;
  46. U.S. Dept. of Health and Human ServicesNational Center for Health Statistics. National Health Interview Survey on Disability, 1994: Phase I, disability outcome supplement. Hyattsville: DHHS, NCHS; 1997;
  47. Wood-Dauphinee SL, Opzoomer MA, Williams JI, Marchand B, Spitzer WO. Assessment of global function: the Reintegration to Normal Living Index. Arch Phys Med Rehabil. 1988;69:583–590
  48. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health status measure. Med Care. 1981;19:787–805
  49. U.S. Census Bureau. United States Census 2000: Demographic profiles: 100-percent and sample data. http://www.census.gov/Press-Release/www/2002/demoprofiles.htmlAccessed October 16, 2007
  50. Meenan RF, Gertman PM, Mason JH. Measuring health status in arthritis (The arthritis impact measurement scales). Arthritis Rheum. 1980;23:146–152
  51. Wiklund I. The Nottingham Health Profile—a measure of health-related quality of life. Scand J Prim Health Care Suppl. 1990;1:15–18
  52. Duncan PW, Wallace D, Lai SM, Johnson D, Embretson S, Laster LJ. The stroke impact scale version 2.0 (Evaluation of reliability, validity, and sensitivity to change). Stroke. 1999;10:2131–2140
  53. Muthen BO, Muthen L. MPius user’s guide. Los Angeles: Muthen & Muthen; 1998;
  54. Ware JE, Gandek B. Methods for testing data quality, scaling assumptions, and reliability: the IQOLA Project approach (International Quality of Life Assessment). J Clin Epidemiol. 1998;51:945–952
  55. Ware JE, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used headache impact scales. Med Care. 2000;38(9 Suppl):II73–II82
  56. Ramsay JO. TestGraf—a program for the graphicalanalysis of multiple choice test and questionnaire data. Montreal: McGill Univ; 1995;
  57. Muraki E. A generalized partial credit model. In:  van der Linden WJ,  Hambleton RK editor. Handbook of modern item response theory. Berlin: Springer; 1997;p. 153–164
  58. Muraki E, Bock RD. PARSCALE: IRT item analysis and test scoring for rating—scale data. Chicago: Scientific Software; 1997;
  59. Muraki E, Bock RD. PARSCALE: IRT based test scoring and item analysis for graded open-ended exercises and performance tasks. Chicago: Scientific Software; 1996;
  60. Bock RD, Aitkin M. Marginal maximum likelihood estimation of item parameters: application of an EM algorithm. Psychometrika. 1981;46:443–459
  61. Orlando M, Thissen D. Likelihood-based item-fit indices for dichotomous item response theory models. Appl Psychol Meas. 2000;24:5044
  62. Ware J, Kosinski , Bjorner J, et al. Applications of computerized adaptive testing of headache impact. Qual Life Res. 2003;12:935–952
  63. Jaeschke R, Singer J, Guyatt GH. Measurement of health status: ascertaining the minimal clinically important difference. Control Clin Trials. 1989;10:407–415
  64. Osoba D, Rodrigues G, Myles J, Zee B, Pater J. Interpreting the significance of changes in health-related quality-of-life scores. J Clin Oncol. 1998;16:139–144
  65. Wiebe S, Matijevic S, Eliasziw M, Derry PA. Clinically important change in quality of life in epilepsy. J Neurol Neurosurg Psychiatry. 2002;73:116–120
  66. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428
  67. de Haan R, Horn J, Limburg M, Van Der Meulen J, Bossuyt P. A comparison of five stroke scales with measures of disability, handicap, and quality of life. Stroke. 1993;24:1178–1181
  68. Kazis LE, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care. 1989;27(3 Suppl):S178–S189
  69. Rosenthal R, Rosnow R. Essentials of behavioral research: methods and data analysis. 2nd ed.. New York: McGraw-Hill; 1991;
  70. Dunlap W, Cortina J, Vaslow J, Burke M. Meta-analysis of experiments with matched groups or repeated measures designs. Psychol Methods. 1996;1:170–177
  71. Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology. 1983;148:839–843
  72. Weinstein MC, Berwick DM, Goldman PA, Murphy JM, Barsky A. A comparison of three psychiatric screening tests using receiver operating characteristic (ROC) analysis. Med Care. 1989;27:593–607
  73. Delong E, DeLong D, Clarke-Pearson D. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44:837–845
  74. Keysor J, Jette A, Coster W, et al. Association of environmental factors with levels of home and community participation. Arch Phys Med Rehabil. 2006;87:1566–1575
  75. Cella D, Eton DT, Lai JS, Peterman AH, Merkel DE. Combining anchor and distribution-based methods to derive minimal clinically important differences on the Functional Assessment of Cancer Therapy (FACT) anemia and fatigue scales. J Pain Symptom Manage. 2002;24:547–561

 Supported in part by the National Institute of Child Health and Human Development and the Agency for Healthcare Research and Quality (grant no. R01 HD043568), and an Independent Scientist Award (grant no. K02 HD45354-01).A commercial party having a direct financial interest in the results of the research supporting this article has conferred or will confer a financial benefit on the author or 1 or more of the authors. Haley and Jette have stock interests in CRE Care LLC, which distributes the AM-PAC products discussed in this study.

PII: S0003-9993(07)01690-5

doi: 10.1016/j.apmr.2007.08.150

Archives of Physical Medicine and Rehabilitation
Volume 89, Issue 2 , Pages 275-283 , February 2008

Article Tools

* Image Usage & Resolution