Abstract
Velozo CA, Seel RT, Magasi S, Heinemann AW, Romero S. Improving measurement methods
in rehabilitation: core concepts and recommendations for scale development.
Validated measurement scales are essential to evaluating clinical outcomes and conducting
meaningful and reliable research. The purpose of this article is to present the clinician
and researcher with a contemporary 8-stage framework for measurement scale development
based on a mixed-methods qualitative and quantitative approach. Core concepts related
to item response theory are presented. Qualitative methods are described to conceptualize
scale constructs; obtain patient, family, and other stakeholder perspectives; and
develop item pools. Item response theory statistical methodologies are presented,
including approaches for testing the assumptions of unidimensionality, local independence,
monotonicity, and indices of model fit. Lastly, challenges faced by scale developers
in implementing these methodologies are discussed. While rehabilitation research has
recently started to apply mixed-methods qualitative and quantitative methodologies
to scale development, these approaches show considerable promise in advancing rehabilitation
measurement.
Key Words
List of Abbreviations:
CAT (computerized adaptive testing), CFA (confirmatory factor analysis), DIF (differential item functioning), ICF (International Classification of Functioning, Disability and Health), IRT (item response theory), logit (log odds unit), NIH (National Institutes of Health), PROMIS (Patient-Reported Outcomes Measurement Information System)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Archives of Physical Medicine and RehabilitationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
U.S. Department of Health and Human Services. Research plan for the National Center for Medical Rehabilitation Research. NIH Publication No. 93–3509 1993. Bethesda: National Institutes of Health.
- Disability and rehabilitation research projects and centers program—Rehabilitation Research and Training Centers (RRTCs)—employment policy and measurement.Federal Register. 2010; 75
- Applying psychometric criteria to functional assessment in medical rehabilitation: II.Arch Phys Med Rehabil. 1992; 73: 507-518
- The structure and stability of the Functional Independence Measure.Arch Phys Med Rehabil. 1994; 75: 127-132
- Pediatric Evaluation of Disability Inventory: clinical interpretation of summary scores using Rasch rating scale methodology.Phys Med Rehabil Clin N Am. 1993; 4: 529-540
- The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years.Med Care. 2007; 45: S3-S11
- AHRQ Patient Centered Outcomes Research Institutional Award (K12) 2011.(Accessed March 4, 2012)
- PCORI funding announcement pilot projects grants 2011.(Accessed March 3, 2012)
- Guidance for industry: patient-reported outcome measures: use in medical product development to support labeling claims.(Accessed March 4, 2012.)
- Qualitative research in occupational therapy: from the first to the second generation.OTJR (Thorofare N J). 2010; 30: 51-57
- Qualitative methodology for rehabilitation research.J Rehabil Med. 2005; 37: 273-280
- PRO development: rigorous qualitative research as the crucial foundation.Qual Life Res. 2010; 19: 1087-1096
- Mixed research as a tool for developing quantitative instruments.J Mix Methods Res. 2010; 4: 56-77
- Evaluation of item candidates: the PROMIS qualitative item review.Med Care. 2007; 45: S12-S21
- Applying item response theory modeling for evaluating questionnaire item and scale properties.in: Fayers P. Hays R. 2nd ed. Oxford University Pr, Oxford2005: 55-73
- Observations are always ordinal; measurements, however, must be interval.Arch Phys Med Rehabil. 1989; 70: 857-860
- Rasch and Wright: the early years.Rasch Meas Trans. 1987; : 1-4
- Estimation of latent ability using a response pattern of graded scores.Psychometrika Monogr. 1969; 17: 1-100
- Applying the Rasch model: fundamental measurement in the human sciences.Lawrence Erlbaum Associates, Mahwah2007
- Handbook of modern item response theory.Springer, New York1997
- Principles and selected applications of item response theory.in: Linn R.L. Educational measurement. 3rd ed. Macmillan, New York1989: 147-200
- Controversy and the Rasch model: a characteristic of incompatible paradigms?.Med Care. 2004; 42: I7-I16
- Computerized adaptive testing: a primer.Qual Life Res. 2001; 10: 733-734
- The use of concept mapping for scale development and validation in evaluation.Eval Program Plann. 2007; 30: 125-135
- Concept mapping for planning and evaluation.Sage Publications, Thousand Oaks2007
- The Delphi technique: making sense of consensus.Practical Assess Res Eval. 2007; 12: 1-8
- Basics of qualitative research: techniques and procedures for developing grounded theory.2nd ed. Sage Publications, Thousand Oaks1998
- Qualitative research and content validity: developing best practices based on science and experience.Qual Life Res. 2009; 18: 1263-1278
- Content validity—establishing and reporting the evidence in newly developed patient-reported outcomes (PRO) instruments for medical product evaluation: ISPOR PRO Good Research Practices Task Force report: part 2—assessing respondent understanding.Value Health. 2011; 14: 978-988
- Designing social research: the logic of anticipation.Polity Pr, Malden2000
- Naturalistic inquiry and the saturation concept: a research note.Qual Res. 2008; 8: 137-152
- Constructing ground theory: a practical guide through qualitative analysis.Sage, Thousand Oaks2006
- Steps in item construction.Rasch Meas Trans. 1977; 11: 559
- Utilizing Rasch measurement models to develop a computer adaptive self-report of walking, climbing, and running.Disabil Rehabil. 2008; 30: 458-467
- Cognitive interviewing techniques in the lab and in the field.in: Schwarz N. Sudman S. Answering questions: methodology for determining cognitive and communicative processes in survey research. 1st ed. Jossey-Bass, San Francisco1996: 177-195
- Cognitive interviewing: a tool for improving questionnaire design.Sage Publications, Thousand Oaks2005
- Item parameter recovery, standard error estimates, and fit statistics of the Winsteps program for the family of Rasch models.Educ Psychol Meas. 2005; 65: 376-404
- Parameter recovery in the graded response model using multilog.J Educ Meas. 1990; 27: 133-144
- Multivariate analysis: the need for data and other problems.Br J Psychiatry. 1975; 126: 237-240
- Psychometric evaluation and calibration of health-related quality of life item banks: plans for the Patient-Reported Outcomes Measurement Information System (PROMIS).Med Care. 2007; 45: S22-S31
- Psychometric theory.3rd ed. McGraw Hill, New York1994
- Detecting multidimensionality: which residual data-type works best?.J Outcome Meas. 1998; 2: 266-283
- Optimizing rating scale category effectiveness.J Appl Meas. 2002; 3: 85-106
- Best test design.MESA Pr, Chicago1979
- An overview of statistical approaches for assessing model fit.Int J Educ Psychol Assess. 2011; 8: 69-87
- Dimensionality and construct validity of the Fugl-Meyer Assessment of the upper extremity.Arch Phys Med Rehabil. 2007; 88: 715-723
- The structure of functional and community outcomes following traumatic brain injury.Brain Inj. 2006; 20: 391-407
- Rasch analysis of the Gross Motor Function Measure: validating the assumptions of the Rasch model to create an interval-level measure.Arch Phys Med Rehabil. 2003; 84: 697-705
- Rating scale analysis.Mesa Pr, Chicago1982
- Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS).J Clin Epidemiol. 2008; 61: 17-33
- Item response theory and health outcomes measurement in the 21st century.Med Care. 2000; 38: II28-II42
- Differential item functioning of the Functional Independence Measure in higher performing neurological patients.J Rehabil Med. 2005; 37: 346-352
- Prospective evaluation of the AM-PAC-CAT in outpatient rehabilitation settings.Phys Ther. 2007; 87: 385-398
- Computerized adaptive test for patients with knee impairments produced valid and responsive measures of function.J Clin Epidemiol. 2008; 61: 1113-1124
- Computerized adaptive test for patients with foot or ankle impairments produced valid and responsive measures of function.Qual Life Res. 2008; 17: 1081-1091
- A computerized adaptive test for patients with shoulder impairments produced responsive measures of function.Phys Ther. 2010; 90: 928-938
- Measurement precision and efficiency of multidimensional computer adaptive testing of physical functioning using the Pediatric Evaluation of Disability Inventory.Arch Phys Med Rehabil. 2006; 87: 1223-1229
- Development of a short form of the Boston naming test for individuals with aphasia.J Speech Lang Hear Res. 2011; 54: 1089-1100
- Headache Impact Test.(Accessed March 5, 2012)
- Translating measurement findings into rehabilitation practice: an example using Fugl-Meyer Assessment-Upper Extremity with patients following stroke.J Rehabil Res Dev. 2011; 48: 1211-1222
- Rasch analysis of items from two self-report measures of motor function: determination of item difficulty and relationships with children's ability levels.Dev Med Child Neurol. 2012; 54: 443-450
- Does the Rasch model really work for multiple-choice items—not if you look closely.J Educ Meas. 1986; 23: 283-298
- Improved responsiveness and reduced sample size requirements of PROMIS physical function scales with item response theory.Arthritis Res Ther. 2011; 13: R147
Article info
Footnotes
Supported in part by the National Institute on Disability and Rehabilitation Research, U.S. Department of Education (grant no. H133G080153).
No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.
Reprints are not available from the author.
Identification
Copyright
© 2012 American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.
