Archives of Physical Medicine and Rehabilitation
Volume 89, Issue 11 , Pages 2129-2139 , November 2008

A Computerized Adaptive Test for Patients With Hip Impairments Produced Valid and Responsive Measures of Function

  • Dennis L. Hart, PT, PhD

      Affiliations

    • Department of Consulting and Research, Focus On Therapeutic Outcomes, Inc, Knoxville, TN
    • Corresponding Author InformationReprint requests to Dennis L. Hart, PT, PhD, Director of Consulting and Research, Focus On Therapeutic Outcomes, Inc, 551 Yopps Cove Rd, White Stone, VA, 22578-2403
    • ,
  • Ying-Chih Wang, OT, PhD

      Affiliations

    • Department of Consulting and Research, Focus On Therapeutic Outcomes, Inc, Knoxville, TN
    • Sensory Motor Performance Program, Rehabilitation Institute of Chicago, Chicago, IL
    • ,
  • Paul W. Stratford, PT, MS

      Affiliations

    • Department of Clinical Epidemiology and Biostatistics, School of Rehabilitation Science, McMaster University, Toronto, ON, Canada
    • ,
  • Jerome E. Mioduski, MS

      Affiliations

    • Department of Information Technology, Focus On Therapeutic Outcomes, Inc, Knoxville, TN

References 

  1. Fliege H, Becker J, Walter OB, Bjorner JB, Klapp BF, Rose M. Development of a computer-adaptive test for depression (D-CAT). Qual Life Res. 2005;14:2277–2291
  2. Kosinski M, Bjorner JB, Ware JE, Sullivan E, Straus WL. An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. J Clin Epidemiol. 2006;59:715–723
  3. Ware JE, Bjorner JB, Kosinski M. Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used headache impact scales. Med Care. 2000;38(9 Suppl):II73–II82
  4. Mills CN, Potenza MT, Fremer JJ, Ward WC. Computer-based testing (Building the foundation for future assessments). Mahwah: Lawrence Erlbaum Associates; 2002;
  5. Sands WA, Waters BK, McBride JR. Computerized adaptive testing (From inquiry to operation). Washington (DC): American Psychological Assoc; 1997;
  6. Wainer H. Computerized adaptive testing (A primer). 2nd ed.. Mahway: Lawrence Erlbaum Associates; 2000;
  7. Hambleton RK, Swaminathan H, Rogers HJ. Fundamentals of item response theory. Newbury Park: Sage; 1991;
  8. Hays RD, Morales LS, Reise SP. Item response theory and health outcomes measurement in the 21st century. Med Care. 2000;38(9 Suppl):II28–II42
  9. Bjorner JB, Kosinski M, Ware JE. The feasibility of applying item response theory to measures of migraine impact: a re-analysis of three clinical studies. Qual Life Res. 2003;12:887–902
  10. Cella D, Yount S, Rothrock N, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years. Med Care. 2007;45(5 Suppl 1):S3–S11
  11. Hambleton RK. Emergence of item response modeling in instrument development and data analysis. Med Care. 2000;38(9 Suppl):II60–II65
  12. Institute of Medicine. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): National Academy Pr; 2001;
  13. Institute of Medicine. Rewarding provider performance: aligning incentives in Medicare. Washington (DC): National Academies Pr; 2006;
  14. Porter ME, Teisberg EO. Redefining health care (Creating value-based competition on results). Boston: Havard Business School Pr; 2006;
  15. Lord FM, Novick MR. Statistical theories of mental test scores. Reading: Addison-Wesley; 1968;
  16. Lord F. Some test theory for tailored testing. In:  Holtzman W editors. Computer-assisted instruction, testing, and guidance. New York: Harper and Row; 1970;p. 139–183
  17. Ware JE, Kosinski M, Bjorner JB, et al. Applications of computerized adaptive testing (CAT) to the assessment of headache impact. Qual Life Res. 2003;12:935–952
  18. Jette AM, Haley SM, Tao W, et al. Prospective evaluation of the AM-PAC-CAT in outpatient rehabilitation settings. [published erratum appears in Phys Ther 2007;87:617] Phys Ther. 2007;87:385–398
  19. McHorney CA. Generic health measurement: past accomplishments and a measurement paradigm for the 21st century. Ann Intern Med. 1997;127(8 Pt 2):743–750
  20. Patrick DL, Chiang YP. Convening health outcomes methodologists. Med Care. 2000;38(9 Suppl):II3–II6
  21. Revicki DA, Cella DF. Health status assessment for the twenty-first century: item response theory, item banking and computer adaptive testing. Qual Life Res. 1997;6:595–600
  22. Dijkers MP. A computer adaptive testing simulation applied to the FIM instrument motor component. Arch Phys Med Rehabil. 2003;84:384–393
  23. Gardner W, Kelleher KJ, Pajer KA. Multidimensional adaptive testing for mental health problems in primary care. Med Care. 2002;40:812–823
  24. Haley SM, Ni P, Hambleton RK, Slavin MD, Jette AM. Computer adaptive testing improved accuracy and precision of scores over random item selection in a physical functioning item bank. J Clin Epidemiol. 2006;59:1174–1182
  25. Hart DL, Cook KF, Mioduski JE, Teal CR, Crane PK. Simulated computerized adaptive test for patients with shoulder impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:290–298
  26. Hart DL, Mioduski JE, Stratford PW. Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments. J Clin Epidemiol. 2005;58:629–638
  27. Hart DL, Mioduski JE, Werneke MW, Stratford PW. Simulated computerized adaptive test for patients with lumbar spine impairments was efficient and produced valid measures of function. J Clin Epidemiol. 2006;59:947–956
  28. Haley SM, Fragala-Pinkham M, Ni P. Sensitivity of a computer adaptive assessment for measuring functional mobility changes in children enrolled in a community fitness programme. Clin Rehabil. 2006;20:616–622
  29. Ware JE, Gandek B, Sinclair SJ, Bjorner J. Item response theory in computer adaptive testing: implications for outcomes measurement in rehabilitation. Rehabil Psychol. 2005;50:71–78
  30. Deutscher D, Hart DL, Dickstein R, Horn SD, Gutvirtz M. Implementing an integrated electronic outcomes and electronic health record process to create a foundation for clinical practice improvement. Phys Ther. 2008;88:270–285
  31. Hart DL, Wang YC, Stratford PW, Mioduski JE. Computerized adaptive test for patients with knee impairments produced valid and responsive measures of function. J Clin Epidemiol. 2008;Jul 9. [Epub ahead of print]
  32. Hart DL, Connolly JB. Pay-for-Performance for Physical Therapy and Occupational Therapy: Medicare Part B Services (Health & Human Services/Centers for Medicare & Medicaid Services; 2006). www.cms.hhs.gov/TherapyServices/downloads/P4PFinalReport06-01-06.pdfAccessed August 27, 2008
  33. Ader DN. Developing the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5 Suppl 1):S1–S2
  34. Haley SM, Coster WJ, Andres PL, Kosinski M, Ni P. Score comparability of short forms and computerized adaptive testing: simulation study with the activity measure for post-acute care. Arch Phys Med Rehabil. 2004;85:661–666
  35. Haley SM, Coster WJ, Andres PL, et al. Activity outcome measurement for postacute care. Med Care. 2004;42(1 Suppl):I49–I61
  36. American Physical Therapy Association. Guide to physical therapist practice. Phys Ther. 2001;81:1–768
  37. Resnik L, Hart DL. Using clinical outcomes to identify expert physical therapists. Phys Ther. 2003;83:990–1002
  38. Werneke MW, Hart DL. Centralization phenomenon as a prognostic factor for chronic low back pain and disability. Spine. 2001;26:758–764
  39. 2007 Physician Quality Reporting Initiative (PQRI). Physician Quality Measures (Centers for Medicare and Medicaid Services; 2007). http://www.cms.hhs.gov/PQRI/EmailUpdates/list.asp#TopOfPageAccessed August 27, 2008
  40. Dobrzykowski EA, Nance T. The Focus On Therapeutic Outcomes (FOTO) Outpatient Orthopedic Rehabilitation Database: results of 1994–1996. J Rehabil Outcomes Meas. 1997;1:56–60
  41. Swinkels IC, van den Ende CH, de Bakker D, et al. Clinical databases in physical therapy. Physiother Theory Pract. 2007;23:153–167
  42. Binkley JM, Stratford PW, Lott SA, Riddle DL. The Lower Extremity Functional Scale (LEFS): scale development, measurement properties, and clinical application (North American Orthopaedic Rehabilitation Research Network). Phys Ther. 1999;79:371–383
  43. Alcock GK, Stratford PW. Validation of the Lower Extremity Functional Scale on athletic subjects with ankle sprains. Physiother Can. 2002;54:233–240
  44. Stratford PW. Getting more from the literature: estimating the standard error of measurement from reliability studies. Physiother Can. 2004;56:27–30
  45. Stratford PW, Binkley JM, Watson J, Heath-Jones T. Validation of the LEFS on patients with total joint arthroplasty. Physiother Can. 2000;52:97–205
  46. Stratford PW, Hart DL, Binkley JM, Kennedy DM, Alcock GK, Hanna SE. Interpreting lower extremity functional status scores. Physiother Can. 2005;57:154–162
  47. World Health Organization. International Classification of Functioning, Disability and Health. Geneva: World Health Organization; 2001;
  48. Hart AC, Stegman MS. ICD-9-CM 2008 expert. 6th ed.. Salt Lake City: Ingenix; 2007;
  49. Andrich D. A rating formulation for ordered response categories. Psychometrika. 1978;43:561–573
  50. Millsap RE, Everson HT. Methodology review: statistical approaches for assessing measurement bias. Appl Psychol Meas. 1993;17:287–334
  51. Crane PK, Hart DL, Gibbons LE, Cook KF. A 37-item shoulder functional status item pool had negligible differential item functioning. J Clin Epidemiol. 2006;59:478–484
  52. Thissen D, Mislevy RJ. Testing algorithms. In:  Wainer H editors. Computerized adaptive testing: a primer. 2nd ed.. Mahwah: Lawrence Erlbaum Associates; 2000;p. 101–134
  53. Hart DL, Mioduski JE. CAT development and testing software user's guide. Knoxville: FOTO Inc; 2006;
  54. Lord FM. Applications of item response theory to practical testing problems. Hillsdale: Lawrence Erlbaum Associates; 1980;
  55. Linacre JM. Estimating measures with known polytomous item difficulties. Rasch Meas Trans. 1998;12:638
  56. Folk VG, Smith RL. Models for delivery of CBTs. In:  Mills CN,  Potenza MT,  Fremer JJ,  Ward WC editor. Computer-based testing building the foundation for future assessments. Mahwah: Lawrence Erlbaum Associates; 2002;p. 41–66
  57. Groll DL, To T, Bombardier C, Wright JG. The development of a comorbidity index with physical function as the outcome. J Clin Epidemiol. 2005;58:595–602
  58. Shapiro S, Wilk MB. An analysis of variance test for normality. Biometrika. 1965;52:591–611
  59. Jette DU, Jette AM. Physical therapy and health outcomes in patients with spinal impairments. Phys Ther. 1996;76:930–945
  60. Jette DU, Jette AM. Physical therapy and health outcomes in patients with knee impairments. Phys Ther. 1996;76:1178–1187
  61. Resnik L, Feng Z, Hart DL. State regulation and the delivery of physical therapy services. [published erratum appears in Phys Ther 1997;77:113] Health Serv Res. 2006;41(4 Pt 1):1296–1316
  62. Hart DL. The power of outcomes: FOTO Industrial Outcomes Tool-Initial Assessment. Work. 2001;16:39–51
  63. Hart DL, Dobrzykowski EA. Influence of orthopaedic clinical specialist certification on clinical outcomes. J Orthop Sports Phys Ther. 2000;30:183–193
  64. Hart DL, Dobrzykowski EA. Impact of exercise history on health status outcomes in patients with musculoskeletal impairments. Orthop Phys Ther Clin N Am. 2000;9:1–16
  65. Beaton DE, Bombardier C, Katz JN, Wright JG. A taxonomy for responsiveness. J Clin Epidemiol. 2001;54:1204–1217
  66. Liang MH. Longitudinal construct validity: establishment of clinical meaning in patient evaluative instruments. Med Care. 2000;38(9 Suppl):II84–II90
  67. Wright JG, Young NL. A comparison of different indices of responsiveness. J Clin Epidemiol. 1997;50:239–246
  68. Kirshner B, Guyatt G. A methodological framework for assessing health indices. J Chronic Dis. 1985;38:27–36
  69. Riddle DL, Stratford PW, Binkley JM. Sensitivity to change of the Roland-Morris Back Pain Questionnaire: part 2. Phys Ther. 1998;78:1197–1207
  70. Beaton DE, Hogg-Johnson S, Bombardier C. Evaluating changes in health status: reliability and responsiveness of five generic health status measures in workers with musculoskeletal disorders. J Clin Epidemiol. 1997;50:79–93
  71. Liang MH. Evaluating measurement responsiveness. J Rheumatol. 1995;22:1191–1192
  72. Stratford PW, Binkley FM, Riddle DL. Health status measures: strategies and analytic methods for assessing change scores. Phys Ther. 1996;76:1109–1123
  73. Hsieh YW, Wang CH, Wu SC, Chen PC, Sheu CF, Hsieh CL. Establishing the minimal clinically important difference of the Barthel Index in stroke patients. Neurorehabil Neural Repair. 2007;21:233–238
  74. Stratford PW, Riddle DL. Assessing sensitivity to change: choosing the appropriate change coefficient. Health Qual Life Outcomes. 2005;3:23
  75. Crosby RD, Kolotkin RL, Williams GR. Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol. 2003;56:395–407
  76. Jaeschke R, Singer J, Guyatt GH. Measurement of health status (Ascertaining the minimal clinically important difference). Control Clin Trials. 1989;10:407–415
  77. Lingard EA, Riddle DL. Impact of psychological distress on pain and function following knee arthroplasty. J Bone Joint Surg Am. 2007;89:1161–1169
  78. Tubach F, Ravaud P, Baron G, et al. Evaluation of clinically relevant changes in patient reported outcomes in knee and hip osteoarthritis: the minimal clinically important improvement. Ann Rheum Dis. 2005;64:29–33
  79. Tubach F, Ravaud P, Beaton D, et al. Minimal clinically important improvement and patient acceptable symptom state for subjective outcome measures in rheumatic disorders. J Rheumatol. 2007;34:1188–1193
  80. Deyo RA, Centor RM. Assessing the responsiveness of functional scales to clinical change: an analogy to diagnostic test performance. J Chronic Dis. 1986;39:897–906
  81. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36
  82. George SZ, Fritz JM, Bialosky JE, Donald DA. The effect of a fear-avoidance-based physical therapy intervention for patients with acute low back pain: results of a randomized clinical trial. Spine. 2003;28:2551–2560
  83. Chakravarty EF, Bjorner JB, Fries JF. Improving patient reported outcomes using item response theory and computerized adaptive testing. J Rheumatol. 2007;34:1426–1431
  84. Dodd BG, Koch WR, De Ayala RJ. Operational characteristics of adaptive testing procedures using the Graded Response Model. Appl Psychol Meas. 1989;13:129–143
  85. Martin M, Kosinski M, Bjorner JB, Ware JE, Maclean R, Li T. Item response theory methods can improve the measurement of physical function by combining the modified health assessment questionnaire and the SF-36 physical function scale. Qual Life Res. 2007;16:647–660
  86. Ware JE, Sherbourne CD. The MOS 36-item short-form health survey (SF-36) (I. Conceptual framework and item selection). Med Care. 1992;30:473–483
  87. Hart DL, Wright BD. Development of an index of physical functional health status in rehabilitation. Arch Phys Med Rehabil. 2002;83:655–665
  88. Jette AM, Haley SM, Ni P. Comparison of functional status tools used in post-acute care. Health Care Financ Rev. 2003;24:13–24
  89. McHorney CA, Cohen AS. Equating health status measures with item response theory: illustrations with functional status items. Med Care. 2000;38(9 Suppl):II43–II59
  90. Steinberg L, Thissen D, Wainer H. Validity. In:  Wainer H editors. Computerized adaptive testing: a primer. 2nd ed.. Mahwah: Lawerence Erlbaum Associates; 2000;p. 185–229
  91. Hart DL. Assessment of unidimensionality of physical functioning in patients receiving therapy in acute, orthopedic outpatient centers. J Outcome Meas. 2000;4:413–430
  92. Reeve BB, Hays RD, Bjorner JB, et al. Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS). Med Care. 2007;45(5 Suppl 1):S22–S31
  93. American Medical Association. Current Procedural Terminology (CPT) 2007, Professional Edition. Chicago: American Medical Assoc; 2006;

 Supported by Focus On Therapeutic Outcomes, Inc.

 A commercial party having a direct financial interest in the results of the research supporting this article has conferred or will confer a financial benefit on the author or 1 or more of the authors. Hart, Wang, and Mioduski are employees of and Hart is an investor in Focus On Therapeutic Outcomes, Inc, which distributes the hip CAT discussed in this study.

PII: S0003-9993(08)00829-0

doi: 10.1016/j.apmr.2008.04.026

Archives of Physical Medicine and Rehabilitation
Volume 89, Issue 11 , Pages 2129-2139 , November 2008

Article Tools

* Image Usage & Resolution