Implementing Patient-Reported Outcome Measures in Inpatient Rehabilitation: Challenges and Solutions


      A project with the goal of implementing electronic health record (EHR)–based patient-reported outcome measures (PROMs) into a large inpatient spinal cord injury (SCI) rehabilitation program took twice as long as expected. This report details the lessons learned from the barriers, successes, and unexpected issues that arose during this prolonged, but now successful, project. The goals of this implementation project were to (1) identify barriers and supports to the use of PROMs; (2) develop an implementation strategy to incorporate the use of PROMs into inpatient rehabilitation; and (3) implement the strategy and evaluate its effects on team communication.
      In brief, we conducted an initial pilot phase outside of the EHR and used our findings to guide procedural and EHR incorporation during a demonstration phase. We encountered multiple barriers. Procedural issues were significant; although grant funding covered the cost of writing the code for integration of the PROMs into the EHR, our institution's competing priorities slowed progress. Institutional inertia was reflected in the reluctance of some clinical staff members to assume new duties that would take away from direct patient care responsibilities. Therefore, we needed to obtain additional staffing. Detailed planning upfront, guided by changes when necessary; cooperation and interaction with our institution's Information Systems department; and identification of key players and Implementation Champions proved essential to our success. We now have an up-and-running system and are sharing our experience, observations, and recommendations to assist other health care organizations incorporate PROMs into their EHRs.


      List of abbreviations:

      CAT (computer adaptive test), EHR (electronic health record), KTA (Knowledge-to-Action Framework), NQF (National Quality Forum), PROM (patient-reported outcome measure), PROMIS (Patient-Reported Outcomes Measurement Information System), QOL (quality of life), SCI (spinal cord injury), SCI-QOL (Spinal Cord Injury-Quality of Life measurement system)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Melnyk BM
        • Fineout-Overholt E
        Evidence-based practice in nursing & healthcare: a guide to best practice.
        Lippincott Williams & Wilkins, Philadelphia2011
        • Law MC
        • MacDermid J
        Evidence-based rehabilitation: a guide to practice.
        West DeptfordSlack Incorporated, 2008
        • Sudsawad P
        A conceptual framework to increase usability of outcome research for evidence-based practice.
        Am J Occup The. 2005; 59: 351-355
      1. (Introducing evidence-based policy and practice in public services)
        • Davies H
        • Nutley S
        • Smith P
        • Davies HTO
        • Nutley SM
        • Smith PC
        What works? Evidence-based policy and practice in public services.
        The Policy Press, Bristol, UK2000: 1-12
        • Jette DU
        • Halbert J
        • Iverson C
        • Miceli E
        • Shah P
        Use of standardized outcome measures in physical therapist practice: perceptions and applications.
        Phys Ther. 2009; 89: 125-135
        • Snyder CF
        • Aaronson NK
        • Choucair AK
        • et al.
        Implementing patient-reported outcomes assessment in clinical practice: a review of the options and considerations.
        Qual Life Res. 2012; 21: 1305-1314
      2. Centers for Medicare and Medicaid Services. Final IRF-PAI Version 4.0. Available at: Accessed January 24, 2020.

        • Scivoletto G
        • Galli G
        • Torre M
        • Molinari M
        • Pazzaglia M
        The overlooked outcome measure for spinal cord injury: use of assistive devices.
        Front Neurol. 2019; 10: 272
        • Howe TE
        • Skelton DA
        Consensus on core outcome measures of function are needed to progress our knowledge of 'best practice' exercise components for older people.
        Age Ageing. 2011; 40: 532-533
        • Dawson J
        • Doll H
        • Fitzpatrick R
        • Jenkinson C
        • Carr AJ
        The routine use of patient reported outcome measures in healthcare settings.
        Brit Med J. 2010; 340: c186
        • Salbach NM
        • Jaglal SB
        • Korner-Bitensky N
        • Rappolt S
        • Davis D
        Practitioner and organizational barriers to evidence-based practice of physical therapists for people with stroke.
        Phys Ther. 2007; 87: 1284-1303
        • Gerrish K
        • Clayton J
        Promoting evidence-based practice: an organizational approach.
        J Nurs Manag. 2004; 12: 114-123
        • Lewin SA
        • Skea ZC
        • Entwistle V
        • Zwarenstein M
        • Dick J
        Interventions for providers to promote a patient-centred approach in clinical consultations.
        Cochrane Database Syst Rev. 2001; 4CD003267
        • Velikova G
        • Booth L
        • Smith AB
        • et al.
        Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial.
        J Clin Oncol. 2004; 22: 714-724
        • Senders A
        • Hanes D
        • Bourdette D
        • Whitham R
        • Shinto L
        Reducing survey burden: feasibility and validity of PROMIS measures in multiple sclerosis.
        Mult Scler. 2014; 20: 1102-1111
        • Jette AM
        • Tulsky DS
        • Ni P
        • et al.
        Development and initial evaluation of the spinal cord injury-functional index.
        Arch Phys Med Rehabil. 2012; 93: 1733-1750
        • Tulsky DS
        • Kisala PA
        • Victorson D
        • et al.
        Developing a contemporary patient-reported outcomes measure for spinal cord injury.
        Arch Phys Med Rehabil. 2011; 92: S44-S51
        • Wilson IB
        • Cleary PD
        Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes.
        JAMA. 1995; 273: 59-65
        • Graham ID
        • Logan J
        • Harrison MB
        • et al.
        Lost in knowledge translation: time for a map?.
        J Contin Educ Health Prof. 2006; 26: 13-24
        • Cella D
        • Yount S
        • Rothrock N
        • et al.
        The Patient-Reported Outcomes Measurement Information System (PROMIS): progress of an NIH Roadmap cooperative group during its first two years.
        Med Care. 2007; 45: S3-11
        • Ader DN
        Developing the Patient-Reported Outcomes Measurement Information System (PROMIS).
        Med Care. 2007; 45: S1-S2
        • Tulsky DS
        • Kisala PA
        • Victorson D
        • et al.
        Methodology for the development and calibration of the SCI-QOL item banks.
        J Spinal Cord Med. 2015; 38: 270-287
        • Tulsky DS
        • Kisala PA
        • Victorson D
        • et al.
        Overview of the Spinal Cord Injury–Quality of Life (SCI-QOL) measurement system.
        J Spinal Cord Med. 2015; 38: 257-269
        • Tulsky DS
        • Jette AM
        • Kisala PA
        • et al.
        Spinal cord injury-functional index: item banks to measure physical functioning in individuals with spinal cord injury.
        Arch Phys Med Rehabil. 2012; 93: 1722-1732
      3. Aaronson NK, Elliott T, Greenhalgh J, et al. User's guide to implementing patient-reported outcomes assessment in clinical practice version 2. International Society of Quality of Life Research. Available at: Accessed December 19, 2020.

      4. Chan EKH, Edwards TC, Haywood K, Mikles S, Newton L. Implementing patient-reported outcome measures in clinical practice: a companion guide to the ISOQOL user's guide version 1. International Society for Quality of Life Research. Available at: Accessed July 15, 2021.

      5. National Quality Forum. Patient-reported outcomes: approaches and challenges in selection and data collection - an environmental scan. National Quality Forum. Available at: Accessed January 20, 2020.

        • Harrison MB
        • van den Hoek J
        in: Harrison MB van den Hoek J CAN-IMPLEMENT©: a guideline adaptation and implementation planning resource. Queen's University School of Nursing and Canadian Partnership Against Cancer, Kingston, Ontario, Canada2012
        • Field B
        • Booth A
        • Ilott I
        • Gerrish K
        Using the Knowledge to Action Framework in practice: a citation analysis and systematic review.
        Implement Sci. 2014; 9: 172
        • Harrison MB
        • Legare F
        • Graham ID
        • Fervers B
        Adapting clinical practice guidelines to local context and assessing barriers to their use.
        CMAJ. 2010; 182: E78-E84
        • Fervers B
        • Burgers JS
        • Voellinger R
        • et al.
        Guideline adaptation: an approach to enhance efficiency in guideline development and improve utilisation.
        BMJ Qual Saf. 2011; 20: 228-236
      6. The ADAPTE Collaboration. The ADAPTE process: resource toolkit for guideline adaptation. Version 2.0. Available at: Accessed July 15, 2021.

        • Nitsch KP
        • Stipp K
        • Gracz K
        • Ehrlich-Jones L
        • Graham ID
        • Heinemann AW
        Integrating Spinal Cord Injury-Quality of Life instruments into rehabilitation: implementation science to guide adoption of patient-reported outcome measures.
        J Spinal Cord Med. 2020 Jan 23; ([Epub ahead of print])
        • Brown CH
        • Wyman PA
        • Guo J
        • Pena J
        Dynamic wait-listed designs for randomized trials: new designs for prevention of youth suicide.
        Clin Trials. 2006; 3: 259-271
        • Heinemann AW
        • Nitsch KP
        • Ehrlich-Jones L
        • et al.
        Effects of an implementation intervention to promote use of patient-reported outcome measures on clinicians' perceptions of evidence-based practice, implementation leadership, and team functioning.
        J Contin Educ Health Prof. 2019; 39: 103-111
      7. Ehrlich-Jones L, Nitsch KP, Engel E, Grasz K, Graham ID, Heinemann AW. Development of an education resource to support the implementation of the Spinal Cord Injury-Quality of Life instruments in inpatient rehabilitation. Arch Phys Med Rehabil; in press.

      8. Glinberg and Associates. NIH toolbox. Available at: Accessed December 6, 2020.

        • Kisala PA
        • Fyffe D
        • Heinemann A
        • Shermeyer A
        • Victorson D
        • Tulsky D
        Developing clinically relevant scoring standards for the SCI-QOL measurement system.
        Arch Phys Med Rehabil. 2018; 99: e65
        • Schwartz CE
        • Andresen EM
        • Nosek MA
        • Krahn GL
        • REPoHS Measurement
        Response shift theory: important implications for measuring quality of life in people with disability.
        Arch Phys Med Rehabil. 2007; 88: 529-536
        • Institute of Medicine
        Best care at lower cost: the path to continuously learning health care in America.
        The National Academies Press, Washington (DC)2013