Original research| Volume 99, ISSUE 9, P1840-1847, September 2018

Download started.


Psychometric Testing of a Rehabilitative Care Patient Experience Instrument


      • The WatLX measures the experience of patients in ambulatory rehabilitative care.
      • Psychometric testing of the 10-question WatLX, that measures 6 key constructs, was positive.
      • A trial across regional borders confirmed the feasibility of using the WatLX on a larger scale.



      To evaluate the internal consistency and test-retest reliability, construct validity, and feasibility of the WatLX, a measure of the experience of patients in rehabilitative care.


      Multisite, cross-sectional, and test-retest self-report study.


      Outpatient rehabilitative care settings.


      The WatLX was administered to English-speaking, cognitively intact outpatients (N=1174) over 18 years old who had completed a program of cardiac, musculoskeletal, neurologic, stroke, pulmonary, or speech language rehabilitative care, at 2 separate time points: (1) immediately following completion of their rehabilitation program, and (2) 2 weeks later (n=29). A subsequent feasibility study was conducted with 1013 patients from 19 clinics.


      Not applicable.

      Main Outcome Measures

      The WatLX measures 6 concepts, previously identified as key to outpatient rehabilitative care patients’ experience: (1) ecosystem issues, (2) client and informal caregiver engagement, (3) patient and health care provider relations, (4) pain and functional status, (5) group and individual identity, and (6) open-ended feedback.


      Reliability analyses were conducted on 2 versions of the WatLX. Using a 7-point versus a 5-point Likert scale resulted in higher internal consistency and reliability scores. Cronbach’s alpha coefficients were .863 and .957 for the 5- and 7-point scale, respectively, and the ICC scores were .827 and .880, respectively. The proof of concept study recruited 1013 patients with little interruption of workflow; results displayed strong internal consistency (Cronbach’s alpha coefficient =.906). There is evidence of ceiling effects.


      The WatLX is a parsimonious question set that is feasible for administration in ambulatory rehabilitative care settings, and which shows promising psychometric properties.


      List of abbreviations:

      CI (confidence interval), ICC (intraclass correlation coefficient)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Manary J.P.
        • Boulding W.
        • Staelin R.
        • Glickman S.W.
        The patient experience and health outcomes.
        N Engl J Med. 2013; 368: 199-201
        • Tsai T.C.
        • Orav J.
        • Jha A.K.
        Patient satisfaction and quality of surgical care in U.S. hospitals.
        Ann Surg. 2015; 261: 2-8
        • Ali S.
        • Chessex C.
        • Bassett-Gunter R.
        • Grace S.L.
        Patient satisfaction with cardiac rehabilitation: association with utilization, functional capacity, and heart-health behaviors.
        Patient Prefer Adherence. 2017; 11: 821-830
        • Doyle C.
        • Lennox L.
        • Bell D.
        A systematic review of evidence on the links between patient experience and clinical safety and effectiveness.
        BMJ Open. 2013; 3e001570
        • Anhang Price R.
        • Elliott M.N.
        • Zaslavsky A.M.
        • et al.
        Examining the role of patient experience surveys in measuring health care quality.
        Med Care Res Rev. 2014; 71: 522-554
        • Conley D.M.
        • Singer S.J.
        • Edmondson L.
        • Berry W.R.
        • Gawande A.
        Effective surgical safety checklist implementation.
        J Am Coll Surg. 2011; 212: 873-879
        • Browne K.
        • Roseman D.
        • Shaller D.
        • Edgman-Levitan S.
        Analysis & commentary. Measuring patient experience as a strategy for improving primary care.
        Health Aff. 2010; 29: 921-925
        • McMurray J.
        • McNeil H.
        • Lafortune C.
        • Black S.
        • Prorok J.
        • Stolee P.
        Measuring patients' experience of rehabilitation services across the carecontinuum. Part I: A systematic review of the literature.
        Arch Phys Med Rehabil. 2016; 97: 104-120
        • Jesus T.S.
        • Bright F.
        • Kayes N.
        • Cott C.A.
        Person-centred rehabilitation: what exactly does it mean? Protocol for a scoping review with thematic analysis towards framing the concept and practice of person-centred rehabilitation.
        BMJ Open. 2016; 6e011959
        • Weeks C.
        Why the future of health care may depend on tearing down the hospital. The Globe and Mail. February 21, 2014.
        (Available at:) (Accessed June 16, 2014)
        • Arena R.
        • Williams M.
        • Forman D.E.
        • et al.
        Increasing referral and participation rates to outpatient cardiac rehabilitation: the valuable role of healthcare professionals in the inpatient and home health settings. A science advisory from the American Heart Association.
        Circulation. 2012; 125: 1321-1329
        • McMurray J.
        • McNeil H.
        • Lafortune C.
        • Black S.
        • Prorok J.
        • Stolee P.
        Measuring patients' experience of rehabilitation services across the carecontinuum. Part II: Key dimensions.
        Arch Phys Med Rehabil. 2016; 97: 121-130
        • McMurray J.
        • McNeil H.
        • Gordon A.
        • Elliott J.
        • Stolee P.
        WatLX™: Development and validation of a rehabilitative care patient experience instrument.
        Arch Phys Med Rehabil, [in press]. 2018;
        • Rosenberg S.
        • Kim M.P.
        The method of sorting as a data-gathering procedure in multivariate research.
        Multivar. 1975; 10: 489-502
        • Jick T.D.
        Mixing qualitative and quantitative methods: Triangulation in action.
        Adm Sci Q. 1979; 24: 602-611
        • Collins D.
        Pretesting survey instruments: an overview of cognitive methods.
        Qual Life Res. 2003; 12: 229-238
        • Moret L.
        • Nguyen J.-M.
        • Pillet N.
        • Falissard B.
        • Lombrail P.
        • Gasquet I.
        Improvement of psychometric properties of a scale measuring inpatient satisfaction with care: a better response rate and a reduction of the ceiling effect.
        BMC Health Serv Res. 2007; 7: 197
        • Streiner D.L.
        • Norman G.R.
        Health measurement scales: a practical guide to their development and use.
        4th ed. Oxford University Press, Oxford, U.K.2008
        • Marx R.G.
        • Menezes A.
        • Horovitz L.
        • Jones E.C.
        • Warren R.F.
        A comparison of two time intervals for test-retest reliability of health status instruments.
        J Clin Epidemiol. 2003; 56: 730-735
        • Paiva C.E.
        • Barroso E.M.
        • Carneseca E.C.
        • et al.
        A critical analysis of test-retest reliability in instrument validation studies of cancer patients under palliative care: a systematic review.
        BMC Med Res Methodol. 2014; 14: 8
        • Guyatt G.H.
        Measuring health status: what are the necessary measurement properties?.
        J Clin Epidemiol. 1992; 45: 1341-1345
        • Streiner D.L.
        Research methods in psychiatry: a checklist for evaluating the usefulness of rating scales.
        Can J Psychiatry. 1993; 38: 140-148
        • Grad B.H.
        • Williams G.
        • Olver J.
        • Ferris S.
        • Bialocerkowski A.
        Psychometric evaluation of the brachial assessment tool part 1: reproducibility.
        Arch Phys Med Rehabil. 2017; 99: 629-634
        • McGraw K.O.
        • Wong S.P.
        Forming inferences about some intraclass correlation coefficients.
        Psychol Methods. 1996; 1: 30-46
        • Spitzer W.O.
        • Dobson A.J.
        • Hall J.
        • Chesterman E.
        • Levi J.
        • Shepherd R.
        • Battista R.N.
        • Catchlove B.R.
        Measuring the quality of life of cancer patients: a concise QL-index for use by physicians.
        J Chron Dis. 1981; 34: 585-597
        • Goldstein M.S.
        • Elliott S.D.
        • Guccione A.A.
        The development of an instrument to measure satisfaction with physical therapy.
        Phys Ther. 2000; 80: 853-863
        • Cohen J.
        A power primer.
        Psychol Bull. 1992; 112: 155-159
        • Donner A.
        • Eliasziw M.
        Sample size requirements for reliability studies.
        Stat Med. 1987; 6: 441-448
        • Kraemer H.C.
        • Thiemann S.
        How many subjects? Statistical power analysis in research.
        Sage Publications, Newbury Park1987
        • The Health Foundation
        Measuring patient experience. Vol. 18. the Health Foundation, London, U.K.2013
        • Newman D.A.
        Missing data.
        Organ Res Methods. 2014; 17: 372-411
        • Bartlett J.E.
        • Kortlik J.W.
        • Higgins C.C.
        Organizational research: determining appropriate sample size in survey research.
        Inf Technol Learn Perform J. 2001; 19: 43-50
        • Slade S.C.
        • Keating J.L.
        Measurement of participant experience and satisfaction of exercise programs for low back pain: a structured literature review.
        Pain Med. 2010; 11: 1489-1499
        • Sitzia J.
        How valid and reliable are patient satisfaction data? An analysis of 195 studies.
        Int J Qual Heal Care. 1999; 11: 319-328
        • Casserley-Feeney S.N.
        • Phelan M.
        • Duffy F.
        • Roush S.
        • Cairns M.C.
        • Hurley D.
        Patient satisfaction with private physiotherapy for musculoskeletal pain.
        BMC Musculoskelet Disord. 2008; 9: 50
        • Normann B.
        • Moe S.
        • Salvesen R.
        • Sørgaard K.W.
        Patient satisfaction and perception of change following single physiotherapy consultations in a hospital’s outpatient clinic for people with multiple sclerosis.
        Physiother Theory Pract. 2012; 28: 108-118
        • Uijen A.
        • Schers H.J.
        • van Weel C.
        Continuity of care preferably measured from the patients’ perspective.
        J Clin Epidemiol. 2010; 63: 998-999
        • Hodson M.
        • Andrew S.
        • Roberts C.M.
        Towards an understanding of PREMS and PROMS in COPD.
        Breathe. 2013; 9: 358-364
        • Van der Linde H.
        • Hofstad C.J.
        • Geertzen J.H.B.
        • Postema K.
        • Van Limbeek J.
        From satisfaction to expectation: the patient’s perspective in lower limb prosthetic care.
        Disabil Rehabil. 2007; 29: 1049-1055
        • Beattie P.
        • Turner C.
        • Dowda M.
        • Michener L.
        • Nelson R.
        The MedRisk instrument for measuring care: a psychometric analysis.
        J Otrhopaedic Sport Phys Ther. 2005; 35: 24-32
        • Cott C.
        • Teare G.
        • McGilton K.S.
        • Lineker S.
        Reliability and construct validity of the client-centred rehabilitation questionnaire.
        Disabil Rehabil. 2006; 28: 1387-1397
        • Curry A.
        • Sinclair E.
        Assessing the quality of physiotherapy services using Servqual.
        Int J Health Care Qual Assur. 2002; 15: 197-205
        • Presser S.
        • Rothgeb J.M.
        • Couper M.P.
        • et al.
        Methods for testing and evaluating survey questionnaires. Vol. 546. John Wiley & Sons, Hoboken2004
        • Kanuk L.
        • Berenson C.
        Mail surveys and response rates: a literature review.
        J Mark Res. 1975; 12: 440-453
      1. Life in the fast lane: how are Canadians managing?, 2016. Statistics Canada (Government of Canada). The Daily, Ottawa, CanadaNovember 2017: 1-3 (Available at:) (Accessed March 7, 2018)