Original research| Volume 100, ISSUE 1, P39-44, January 2019

Building a Rehabilitative Care Measurement Instrument to Improve the Patient Experience


      • We developed the WatLX patient experience measure for the rehabilitative care sector.
      • The measurement tool includes a parsimonious set of 10 patient experience questions that underwent face and content validation through expert consultations and a card sort activity, and usability testing through cognitive interviews.
      • The WatLX takes patients approximately 10 minutes to complete, which may contribute to higher completion rates.
      • Psychometric and feasibility testing is reported elsewhere. Future development includes releasing multi-lingual formats, developing a version for clients with cognitive deficits, and exploring administration via web-based and smartphone applications.



      To develop and test face and content validity, and user interface design of a rehabilitative care patient experience measure.


      Mixed methods, cross-sectional validation study that included subject matter expert input. Cognitive interviewing tested user interface and design.


      Outpatient rehabilitative care settings.


      Subject matter experts (n=3), health care providers (n=137), and patients and caregivers (n=5) contributed to the question development. Convenience and snowball sampling were used to recruit rehabilitative care patients postdischarge (n=9) for cognitive interviews to optimize survey design and user interface (N=154).


      Not applicable.

      Main Outcome Measure

      This novel survey instrument measures 6 concepts previously identified as key to outpatient rehabilitative care patients’ experience: ecosystem issues, client and informal caregiver engagement, patient and health care provider relations, pain and functional status, group and individual identity, and open-ended feedback.


      502 survey questions from psychometrically tested instruments, secondary data from a related ethnographic study, and consultations with health care providers, patients, caregivers, and subject matter experts, were analyzed to create a 10-item questionnaire representing 6 key constructs that influence patient experience quality. Cognitive interviewing with 9 patients (3 rounds of 3 participants each), produced 3 progressively edited versions of the survey instrument. A final version required no further modifications.


      Rehabilitative care clients have characteristics that differentiate their experience from that of other sectors and patient groups, warranting a distinct experience measure. The survey instrument includes a parsimonious set of questions that address strategic issues in the ongoing improvement of care delivery and the patient experience in the rehabilitative care sector.


      The rehabilitative care patient experience survey instrument developed has an acceptable user interface, and content and face validity. Psychometric testing of the survey instrument is reported elsewhere.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Jesus T.S.
        • Bright F.
        • Kayes N.
        • Cott C.A.
        Person-centred rehabilitation: what exactly does it mean? Protocol for a scoping review with thematic analysis towards framing the concept and practice of person-centred rehabilitation.
        BMJ Open. 2016; 6e011959
        • Browne K.
        • Roseman D.
        • Shaller D.
        • Edgman-Levitan S.
        Analysis & commentary. Measuring patient experience as a strategy for improving primary care.
        Health Aff (Millwood). 2010; 29: 921-925
        • Beattie M.
        • Murphy D.J.
        • Atherton I.
        • Lauder W.
        Instruments to measure patient experience of health care quality in hospitals: a systematic review.
        Syst Rev. 2015; 4: 97
        • McMurray J.
        • McNeil H.
        • Lafortune C.
        • Black S.
        • Prorok J.
        • Stolee P.
        Measuring patients’ experience of rehabilitation services across the care continuum. Part II: key dimensions.
        Arch Phys Med Rehabil. 2016; 97: 121-130
        • Kneebone I.I.
        • Hull S.L.
        • McGurk R.
        • Cropley M.
        Reliability and validity of the neurorehabilitation experience questionnaire for inpatients.
        Neurorehabil Neural Repair. 2012; 26: 834-841
        • Tsianakas V.
        • Maben J.
        • Wiseman T.
        • et al.
        Using patients’ experiences to identify priorities for quality improvement in breast cancer care: patient narratives, surveys or both?.
        BMC Health Serv Res. 2012; 12: 271
        • Anhang Price R.
        • Elliott M.N.
        • Zaslavsky A.M.
        • et al.
        Examining the role of patient experience surveys in measuring health care quality.
        Med Care Res Rev. 2014; 71: 522-554
        • McMurray J.
        • McNeil H.
        • Lafortune C.
        • Black S.
        • Prorok J.
        • Stolee P.
        Measuring patients’ experience of rehabilitation services across the care continuum. Part I: a systematic review of the literature.
        Arch Phys Med Rehabil. 2016; 97: 104-120
        • Glenny C.
        • Stolee P.
        • Sheiban L.
        • Jaglal S.
        Communicating during care transitions for older hip fracture patients: family caregiver and health care provider’s perspectives.
        Int J Integr Care. 2013; 13: e044
        • McColl M.A.
        • Shortt S.
        • Godwin M.
        • et al.
        Models for integrating rehabilitation and primary care: a scoping study.
        Arch Phys Med Rehabil. 2009; 90: 1523-1531
        • Borrie M.J.
        • Stolee P.
        • Knoefel F.D.
        • Wells J.L.
        • Seabrook J.A.
        Current best practices in geriatric rehabilitation in Canada.
        Geriatr Today Can J Geriatr Med Geriatr Psychiatry. 2005; 8: 148-153
        • Knoefel F.
        • Helliwell B.
        • Seabrook J.A.
        • Borrie M.J.
        • Stolee P.
        • Wells J.L.
        A comparison of functional independence and medical complexity in geriatric and physical medicine rehabilitation inpatients.
        Geriatr Today J Can Geriatr Soc. 2003; 6: 90-94
        • Wells J.L.
        • Seabrook J.A.
        • Stolee P.
        • Borrie M.J.
        • Knoefel F.
        State of the art in geriatric rehabilitation. Part I: review of frailty and comprehensive geriatric assessment.
        Arch Phys Med Rehabil. 2003; 84: 890-897
        • Wells J.L.
        • Seabrook J.A.
        • Stolee P.
        • Borrie M.J.
        • Knoefel F.
        State of the art in geriatric rehabilitation. Part II: clinical challenges.
        Arch Phys Med Rehabil. 2003; 84: 898-903
        • Heniford B.T.
        • Walters A.L.
        • Lincourt A.E.
        • Novitsky Y.W.
        • Hope W.W.
        • Kercher K.W.
        Comparison of generic versus specific quality-of-life scales for mesh hernia repairs.
        J Am Coll Surg. 2008; 206: 638-644
      1. OECD. Health ministerial statement: the next generation of health reforms. Paris, France; 2017.

        • Coulter A.
        Measuring what matters to patients.
        BMJ. 2017; 356: j816
        • Rolstad S.
        • Adler J.
        • Rydén A.
        Response burden and questionnaire length: is shorter better? A review and meta-analysis.
        Value Health. 2011; 14: 1101-1108
        • Hassan S.J.
        • Weymuller E.A.
        Assessment of quality of life in head and neck cancer patients.
        Head Neck. 1993; 15: 485-496
        • Moore F.
        • Wolfson C.
        • Alexandrov L.
        • Lapierre Y.
        Do general and multiple sclerosis-specific quality of life instruments differ?.
        Can J Neurol Sci. 2004; 31: 64-71
      2. McMurray J, McNeil H, Gordon A, Elliott J, Stolee P. Psychometric testing of a rehabilitative care patient experience instrument. Arch Phys Med Rehabil; in press.

        • Hsieh H.F.
        • Shannon S.E.
        Three approaches to qualitative content analysis.
        Qual Health Res. 2005; 15: 1277-1288
        • Choi B.C.
        • Pak A.W.
        A catalog of biases in questionnaires.
        Prev Chronic Dis. 2005; 2: A13
        • Rosenberg S.
        • Kim M.P.
        The method of sorting as a data-gathering procedure in multivariate research.
        Multivariate Behav Res. 1975; 10: 489-502
        • Collins D.
        Pretesting survey instruments: an overview of cognitive methods.
        Qual Life Res. 2003; 12: 229-238
        • Willis G.B.
        Cognitive interviewing: a tool for improving questionnaire design.
        SAGE Publications, Thousand Oaks, CA2005
        • Willis G.B.
        Analysis of the cognitive interview in questionnaire design: understanding qualitative research.
        Oxford University Press, New York, NY2015
      3. Hess J, Singer E. The Role of Respondent Debriefing Questions in Questionnaire Development. Washington, D.C.; 1995.

        • Creswell J.W.
        Research design: qualitative, quantitative and mixed methods approaches.
        2nd ed. Sage Publications Inc., Thousand Oaks, CA2003
        • Streiner D.L.
        • Norman G.R.
        Health measurement scales: a practical guide to their development and use.
        4th ed. Oxford University Press, Oxford, U.K.2008
        • Reichheld F.F.
        The one number you need to grow.
        Harv Bus Rev. 2003; 81: 46-54
        • Keiningham T.L.
        • Cooil B.
        • Aksoy L.
        • Andreassen T.W.
        • Weiner J.
        The value of different customer satisfaction and loyalty metrics in predicting customer retention, recommendation, and share-of-wallet.
        Manag Serv Qual An Int J. 2007; 17: 361-384
        • Watson D.E.
        • Peterson S.
        • Black C.
        In pursuit of quality: opportunities to improve patient experiences in British Columbian emergency departments.
        (Available at) (Accessed July 20, 2018)
        • Curry A.
        • Sinclair E.
        Assessing the quality of physiotherapy services using Servqual.
        Int J Health Care Qual Assur. 2002; 15: 197-205
        • Kuspinar A.
        • Bouchard V.
        • Moriello C.
        • Mayo N.E.
        Development of a bilingual MS-specific health classification system: the preference-based multiple sclerosis index.
        Int J MS Care. 2016; 18: 63-70
        • McNeil H.
        Engaging Canada’s older adults in health TECHnology innovation ecosystems: the ECOTECH Project.
        ([thesis]) University of Waterloo, Waterloo, ON2017
        • McLeod J.
        • McMurray J.
        • Walker J.D.
        • Heckman G.A.
        • Stolee P.
        Care transitions for older patients with musculoskeletal disorders: continuity from the providers’ perspective.
        Int J Integr Care. 2011; 11: e014
        • McMurray J.
        • Hicks E.
        • Johnson H.
        • Elliott J.
        • Byrne K.
        • Stolee P.
        “Trying to find information is like hating yourself every day”: the collision of electronic information systems in transition with patients in transition.
        Health Informatics J. 2013; 19: 218-232
        • Toscan J.
        • Mairs K.
        • Hinton S.
        • Stolee P.
        Integrated transitional care: patient, informal caregiver and health care provider perspectives on care transitions for older persons with hip fracture.
        Int J Integr Care. 2012; 12: e13