Special communication| Volume 93, ISSUE 8, SUPPLEMENT , S127-S137, August 2012

Download started.


Practice-Based Evidence Research in Rehabilitation: An Alternative to Randomized Controlled Trials and Traditional Observational Studies


      Horn SD, DeJong G, Deutscher D. Practice-based evidence research in rehabilitation: an alternative to randomized controlled trials and traditional observational studies.
      Sound rigorous methods are needed by researchers and providers to address practical questions about risks, benefits, and costs of interventions as they occur in routine clinical practice such as: Are treatments used in daily practice associated with intended outcomes? For whom does an intervention work best? With limited clinical resources, what are the best interventions to use for specific types of patients? Answers to such questions can help clinicians, patients, researchers, and health care administrators learn from, and improve, real-world everyday clinical practice. In this article, we describe existing research designs to demonstrate clinical usefulness and comparative effectiveness of rehabilitation treatments. We compare randomized controlled trials and observational cohort studies of various types, including those that use instrumental variables or propensity scores to control for potential patient or treatment selection effects. We argue that practice-based evidence (PBE) study designs include features that address limitations inherent in both randomized trials and traditional observational studies, and also reduce the need for instrumental variables and propensity scores methods. We give examples of how PBE designs have been used in various rehabilitation areas to determine better treatments for specific types of patients.

      Key Words

      List of Abbreviations:

      CER (comparative effectiveness research), CSI (Comprehensive Severity Index), EBP (evidence-based practice), EHR (electronic health record), PBE (practice-based evidence), RCT (randomized controlled trial)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Tunis S.R.
        • Stryer D.B.
        • Clancy C.M.
        Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy.
        JAMA. 2003; 290: 1624-1632
        • Berguer R.
        The evidence thing.
        Ann Vasc Surg. 2004; 18: 265-270
        • Horn S.D.
        • DeJong G.
        • Ryser D.K.
        • Veazie P.J.
        • Teraoka J.
        Another look at observational studies in rehabilitation research: going beyond the holy grail of the randomized controlled trial.
        Arch Phys Med Rehabil. Dec 2005; 86: S8-S15
        • Benson K.
        • Hartz A.J.
        A comparison of observational studies and randomized, controlled trials.
        N Eng J Med. 2000; 342: 1878-1886
        • Concato J.
        • Shah N.
        • Horwitz R.I.
        Randomized, controlled trials, observational studies, and the hierarchy of research designs.
        N Engl J Med. 2000; 342: 1887-1892
        • Ioannidis J.P.
        • Haidich A.B.
        • Pappa M.
        • et al.
        Comparison of evidence of treatment effects in randomized and nonrandomized studies.
        JAMA. 2001; 286: 821-830
        • Horn S.
        • Gassaway J.
        Practice-based evidence study design for comparative effectiveness research.
        Med Care. 2007; 45: S50-S57
        • Horn S.D.
        • Gassaway J.
        Practice-based evidence: incorporating clinical heterogeneity and patient-reported outcomes for comparative effectiveness research.
        Med Care. 2010; 48: S17-S22
        • Benech I.
        • Wilson A.E.
        • Dowell A.C.
        Evidence-based practice in primary care: past, present and future.
        J Eval Clin Pract. 1996; 2: 249-263
        • Deutscher D.
        • Hart D.L.
        • Dickstein R.
        • Horn S.D.
        • Gutvirtz M.
        Implementing an integrated electronic outcomes and electronic health record process to create a foundation for clinical practice improvement.
        Phys Ther. 2008; 88: 270-285
        • Pearl J.
        Causal inference in the health sciences: a conceptual introduction.
        Health Serv Outcomes Res Methodol. 2002; 2: 189-220
        • Pearl J.
        Statistics and causal inference: a review.
        Test. 2003; 12: 281-345
        • Lee P.Y.
        • Alexander K.P.
        • Hammill B.G.
        • Pasquali S.K.
        • Peterson E.D.
        Representation of elderly persons and women in published randomized trials of acute coronary syndromes.
        JAMA. 2001; 286: 708-713
        • Gurwitz J.H.
        • Col N.F.
        • Avorn J.
        The exclusion of the elderly and women from clinical trials in acute myocardial infarction.
        JAMA. 1992; 268: 1417-1422
        • Kotwall C.A.
        • Mahoney L.J.
        • Myers R.E.
        • DeCoste L.
        Reasons for non-entry in randomized clinical trials for breast cancer: a single institutional study.
        J Surg Oncol. 1992; 50: 125-129
        • Blanch D.C.
        • Rudd R.E.
        • Wright E.
        • Gall V.
        • Katz J.N.
        Predictors of refusal during a multi-step recruitment process for a randomized controlled trial of arthritis education.
        Patient Educ Couns. 2008; 73: 280-285
        • van Heuvelen M.J.
        • Hochstenbach J.B.
        • Brouwer W.H.
        • et al.
        Differences between participants and non-participants in an RCT on physical activity and psychological interventions for older persons.
        Aging Clin Exp Res. 2005; 17: 236-245
        • Solomon M.J.
        • Pager C.K.
        • Young J.M.
        • Roberts R.
        • Butow P.
        Patient entry into randomized controlled trials of colorectal cancer treatment: factors influencing participation.
        Surgery. 2003; 133: 608-613
        • Bjoernshave B.
        • Korsgaard J.
        • Nielsen C.V.
        Does pulmonary rehabilitation work in clinical practice?.
        Clin Epidemiol. 2010; 2: 73-83
        • de Jong Z.
        • Munneke M.
        • Jansen L.M.
        • et al.
        Differences between participants and nonparticipants in an exercise trial for adults with rheumatoid arthritis.
        Arthritis Rheum. 2004; 51: 593-600
        • Carter M.J.
        • Fife C.E.
        • Walker D.
        • Thomson B.
        Estimating the applicability of wound care randomized controlled trials to general wound-care populations by estimating the percentage of individuals excluded from a typical wound-care population in such trials.
        Adv Skin Wound Care. 2009; 22: 316-324
        • Fujita T.
        Generalizability in rigorous surgical randomized controlled trials.
        J Am Coll Surg. 2009; 209 (author reply 151-2): 151
        • Rosenbaum P.R.
        Observational studies.
        Springer, New York2002
        • Heckman J.J.
        Micro data, heterogeneity, and the evaluation of public policy: Nobel lecture.
        J Political Econ. 2001; 109: 673-748
        • Little R.J.
        • Rubin D.B.
        Causal effects in clinical and epidemiological studies via potential outcomes: concepts and analytical approaches.
        Annu Rev Public Health. 2000; 21: 121-145
        • Winship C.
        • Morgan S.L.
        The estimation of causal effects from observational data.
        Annu Rev Sociol. 1999; 25: 659-706
        • Winship C.
        • Mare R.D.
        Models for sample selection bias.
        Annu Rev Sociol. 1992; 18: 327-350
        • Newhouse J.P.
        • McClellan M.
        Econometrics in outcomes research: the use of instrumental variables.
        Annu Rev Public Health. 1998; 19: 17-34
        • Heckman J.
        • Navarro-Lozano S.
        Using matching, instrumental variables, and control functions to estimate economic choice models.
        Rev Econ Stat. 2004; 86: 30-57
        • Martens E.P.
        • Pestman W.R.
        • de Boer A.
        • Belitser S.V.
        • Klungel O.H.
        Instrumental variables: application and limitations.
        Epidemiology. 2006; 17: 260-267
        • Hernán M.A.
        • Robins J.M.
        Instruments for causal inference: an epidemiologist's dream?.
        Epidemiology. 2006; 17: 360-372
        • Rosenbaum P.R.
        • Rubin D.B.
        The central role of the propensity score in observational studies for causal effects.
        Biometrika. 1983; 70: 41-55
        • Rosenbaum P.R.
        • Rubin D.B.
        Reducing bias in observational studies using sub-classification on the propensity score.
        J Amer Statist Assoc. 1984; 79: 516-524
        • Rosenbaum P.R.
        • Rubin D.B.
        The bias due to incomplete matching.
        Biometrics. 1985; 41: 103-116
        • Rosenbaum P.R.
        • Rubin D.B.
        Difficulties with regression analyses of age-adjusted rates.
        Biometrics. 1984; 40: 437-443
        • Mitchell M.
        • Jolley J.
        Research design explained.
        4th ed. Harcourt, New York2001
        • Horn S.
        Real-world efficacy: alternative evidence-based study designs as measures of comparative effectiveness.
        Drug Discov Dev. 2007; 10: 30-32
        • Deutscher D.
        • Horn S.D.
        • Dickstein R.
        • et al.
        Associations between treatment processes, patient characteristics, and outcomes in outpatient physical therapy practice.
        Arch Phys Med Rehabil. 2009; 90: 1349-1363
        • Deutscher D.
        • Hart D.L.
        • Stratford P.W.
        • Dickstein R.
        Construct validation of a knee-specific functional status measure: a comparative study between the United States and Israel.
        Phys Ther. 2011; 91: 1072-1084
        • Horn S.D.
        • Sharkey P.D.
        • Kelly H.W.
        • Uden D.L.
        Newness of drugs and use of HMO services by asthma patients.
        Ann Pharmacother. 2001; 35: 990-996
        • Averill R.F.
        • McGuire T.E.
        • Manning B.E.
        • et al.
        A study of the relationship between severity of illness and hospital cost in New Jersey hospitals.
        Health Serv Res. 1992; 27 (discussion 607-12): 587-606
        • Clemmer T.P.
        • Spuhler V.J.
        • Oniki T.A.
        • Horn S.D.
        Results of a collaborative quality improvement program on outcomes and costs in a tertiary critical care unit.
        Crit Care Med. 1999; 27: 1768-1774
        • Horn S.D.
        • Torres Jr, A.
        • Willson D.
        • Dean J.M.
        • Gassaway J.
        • Smout R.
        Development of a pediatric age- and disease-specific severity measure.
        J Pediatrics. 2002; 141: 496-503
        • Willson D.F.
        • Horn S.D.
        • Smout R.
        • Gassaway J.
        • Torres A.
        Severity assessment in children hospitalized with bronchiolitis using the pediatric component of the Comprehensive Severity Index.
        Pediatr Crit Care Med. 2000; 1: 127-132
        • Ryser D.K.
        • Egger M.J.
        • Horn S.D.
        • Handrahan D.
        • Gandhi P.
        • Bigler E.D.
        Measuring medical complexity during inpatient rehabilitation after traumatic brain injury.
        Arch Phys Med Rehabil. 2005; 86: 1108-1117
        • Horn S.D.
        • Sharkey P.D.
        • Buckle J.M.
        • Backofen J.E.
        • Averill R.F.
        • Horn R.A.
        The relationship between severity of illness and hospital length of stay and mortality.
        Med Care. 1991; 29: 305-317
        • Gassaway J.V.
        • Horn S.D.
        • DeJong G.
        • Smout R.J.
        • Clark C.
        Applying the clinical practice improvement approach to stroke rehabilitation: methods used and baseline results.
        Arch Phys Med Rehabil. 2005; 86: S16-S33
        • DeJong G.
        • Hsieh C.
        • Gassaway J.
        • et al.
        Characterizing rehabilitation services for patients with knee and hip replacement in skilled nursing and inpatient rehabilitation facilities.
        Arch Phys Med Rehabil. 2009; 90: 1269-1283
        • Horn S.
        • DeJong G.
        • Smout R.
        • Gassaway J.
        • James R.
        • Conroy B.
        Stroke rehabilitation patients, practice, and outcomes: is earlier and more aggressive therapy better?.
        Arch Phys Med Rehabil. 2005; 86: S101-S114
        • Deutscher D.
        • Horn S.D.
        • Smout R.J.
        • DeJong G.
        • Putman K.
        Black-white disparities in motor function outcomes taking into account patient characteristics, non-therapy ancillaries, therapy activities, and therapy interventions.
        Arch Phys Med Rehabil. 2010; 91: 1722-1730
        • Heinemann A.
        • Hamilton B.
        • Linacre J.
        • Wright B.D.
        • Granger C.
        Functional status and therapeutic intensity during inpatient rehabilitation.
        Am J Phys Med Rehabil. 1995; 74: 315-326
        • Fielder R.
        • Granger C.
        Functional Independence Measure: a measurement of disability and medical rehabilitation.
        in: Springer-Verlag, Tokyo1996: 75-92
        • Hamilton B.B.
        • Laughlin J.A.
        • Fiedler R.C.
        • Granger C.V.
        Interrater reliability of the 7-level functional independence measure (FIM).
        Scand J Rehabil Med. 1994; 26: 115-119
        • Heinemann A.W.
        • Linacre J.M.
        • Wright B.D.
        • Hamilton B.B.
        • Granger C.
        Prediction of rehabilitation outcomes with disability measures.
        Arch Phys Med Rehabil. 1994; 75: 133-143
        • Horn S.
        • Bender S.
        • Ferguson M.
        • et al.
        The National Pressure Ulcer Long-term Care Study (NPULS): pressure ulcer development in long-term care residents.
        J Amer Geriatr Soc. 2004; 52: 359-367
        • Bergstrom N.
        • Horn S.
        • Smout R.
        • et al.
        The National Pressure Ulcer Long-Term Care Study (NPULS): outcomes of pressure ulcer treatments in long-term care.
        J Amer Geriatr Soc. 2005; 53: 1721-1729
        • Bergstrom N.
        • Smout R.
        • Horn S.
        • Spector W.
        • Hartz A.
        • Limcangco M.R.
        Stage 2 pressure ulcer healing in nursing homes.
        J Amer Geriatr Soc. 2008; 56: 1252-1258
        • Hart D.L.
        • Deutscher D.
        • Werneke M.
        • Holder J.
        • Wang Y.C.
        Implementing computerized adaptive tests in routine clinical practice: experience implementing CATs.
        J Appl Meas. 2010; 11: 288-303
      1. Cohen J. Statistical power analysis for the behavioral sciences. 2nd ed. Lawrence Erlbaum Associates, Inc, Hillsdale1998
        • Greenfield S.
        • Kravitz R.
        • Duan N.
        • Kaplan S.H.
        Heterogeneity of treatment effects: implications for guidelines, payment, and quality assessment.
        Am J Med. 2007; 120: S3-S9