Advertisement

Determining the Interrater Reliability of the SOFMER Activity Score (version 2) for Individuals in Rehabilitation Centers

  • Lorraine Charvolin
    Correspondence
    Corresponding author Lorraine Charvolin, MD, Hôpital Femme–Mère–Enfant, L'Escale, Service Central de Rééducation, 59 Blvd Pinel, Bron, France.
    Affiliations
    Service de Médecine Physique et de Réadaptation Pédiatrique (L'Escale), Hôpital Femme–Mère–Enfant, Hospices Civils de Lyon, Bron
    Search for articles by this author
  • Pascal Rippert
    Affiliations
    Service Recherche et Épidémiologie Clinique, Pôle Santé Publique, Hospices Civils de Lyon, Lyon
    Search for articles by this author
  • Sylvain Roche
    Affiliations
    Université de Lyon, Lyon

    Université Lyon 1, Villeurbanne

    Service de Biostatistique-Bioinformatique, Hospices Civils de Lyon, Pôle Santé Publique, Lyon

    Laboratoire de Biométrie et Biologie Évolutive, Équipe Biostatistique-Santé, Villeurbanne
    Search for articles by this author
  • Muriel Rabilloud
    Affiliations
    Université de Lyon, Lyon

    Université Lyon 1, Villeurbanne

    Service de Biostatistique-Bioinformatique, Hospices Civils de Lyon, Pôle Santé Publique, Lyon

    Laboratoire de Biométrie et Biologie Évolutive, Équipe Biostatistique-Santé, Villeurbanne
    Search for articles by this author
  • Marie-Doriane Morard
    Affiliations
    Service de Médecine Physique et de Réadaptation Pédiatrique (L'Escale), Hôpital Femme–Mère–Enfant, Hospices Civils de Lyon, Bron
    Search for articles by this author
  • Julie Di Marco
    Affiliations
    Service de Médecine Physique et Réadaptation, Hôpital Henry-Gabrielle, Hospices Civils de Lyon, Saint-Genis-Laval
    Search for articles by this author
  • Mickael Dinomais
    Affiliations
    Département de Médecine Physique et Rééducation, Centre Hospitalier Universitaire, Angers
    Search for articles by this author
  • Margaux Pouyfaucon
    Affiliations
    Service Médecine Physique et Rééducation Fonctionnelle, Centre Hospitalier Universitaire d'Angers, Angers

    Centre de Rééducation et de Réadaptation Fonctionnelles Les Capucins, Angers

    Service de Rééducation, Centre Hospitaliser de Cholet, Cholet
    Search for articles by this author
  • Rémi Gimat
    Affiliations
    Service Rééducation Neurologique, Hôpital Sud Centre Hospitalier Universitaire de Grenoble-Alpes, Echirolles

    Laboratoire de Psychologie et Neurocognition (LPNC), Université Grenoble-Alpes, Grenoble
    Search for articles by this author
  • Dominique Perennou
    Affiliations
    Service Rééducation Neurologique, Hôpital Sud Centre Hospitalier Universitaire de Grenoble-Alpes, Echirolles

    Laboratoire de Psychologie et Neurocognition (LPNC), Université Grenoble-Alpes, Grenoble
    Search for articles by this author
  • Laetitia Houx
    Affiliations
    Service de Médecine Physique et de Réadaptation, Centre Hospitalier Régional et Universitaire de Brest, Brest

    Laboratoire de Traitement de l'Information Médicale (LaTIM), Brest

    Service de Médecine Physique et de Réadaptation Pédiatrique, Fondation Ildys, Brest
    Search for articles by this author
  • Jean Iwaz
    Affiliations
    Université de Lyon, Lyon

    Université Lyon 1, Villeurbanne

    Service de Biostatistique-Bioinformatique, Hospices Civils de Lyon, Pôle Santé Publique, Lyon

    Laboratoire de Biométrie et Biologie Évolutive, Équipe Biostatistique-Santé, Villeurbanne
    Search for articles by this author
  • Gilles Rode
    Affiliations
    Université de Lyon, Lyon

    Service de Médecine Physique et Réadaptation, Hôpital Henry-Gabrielle, Hospices Civils de Lyon, Saint-Genis-Laval

    Integrative, Multisensory, Perception, Action, and Cognition Team (IMPACT), Centre de Recherche en Neurosciences de Lyon, Université Saint-Etienne, Bron
    Search for articles by this author
  • Carole Vuillerot
    Affiliations
    Service de Médecine Physique et de Réadaptation Pédiatrique (L'Escale), Hôpital Femme–Mère–Enfant, Hospices Civils de Lyon, Bron

    Université de Lyon, Lyon

    Institut Neuromyogène, Lyon, France
    Search for articles by this author
Published:December 06, 2021DOI:https://doi.org/10.1016/j.apmr.2021.11.005

      Highlights

      • SOFEMER Activity Score (SAS) is devised to quantify patient activity limitation in rehabilitation centers.
      • SAS content validity and feasibility have been already established.
      • The SAS interrater reliability is assessed and found “good” to “excellent.”
      • Analyses of score agreements and disagreements helped tuning scoring instructions.
      • The SAS good interrater reliability paves the way for further validation steps.

      Abstract

      Objective

      To assess the interrater reliability of the SOFMER Activity Score (SAS) (version 2 [v2], an 8-item [4 motor and 4 cognitive] and 5-level scale) and improve its scoring system before conducting further validation steps.

      Design

      Cross-sectional, prospective, observational, noninterventional, and multicentric study.

      Setting

      The study was conducted between November 2018 and September 2019 in 4 French rehabilitation centers (2 public university hospitals for adults and 2 private not-for-profit rehabilitation centers for children).

      Participants

      The study included 101 participants (N=101; mean age, 44.5±25.4 years; 28.7% younger than 18 and 18.8% older than 65 years). The female/male sex ratio was 0.6. The causes for admission to the center were mainly neurologic (65%) or orthopedic (24%).

      Interventions

      None.

      Main Outcome Measures

      Activity limitation was rated with the SAS the same day by 2 independent multidisciplinary teams. The interrater reliabilities of the score items were assessed using weighted kappa coefficients.

      Results

      All weighted kappa coefficients ranged between 0.83 and 0.92, indicating “good” to “excellent” interrater reliability. Interteam score disagreements occurred in 227 of 808 scores (28%). The reason for most disagreements was unnoticed human or material aid during the observation period.

      Conclusions

      The results demonstrate the high interrater reliability of the SASv2 and allow carrying out further validation steps after minor changes to item scoring instructions and clearer definitions of some items that help improving scoring standardization. The SASv2 may then become a consistent measure of activity level for clinical research or burden of care investigations.

      Keywords

      List of abbreviations:

      HCP (health care provider), ICC (intraclass correlation coefficient), ICF (International Classification of Functioning, Disability, and Health), IRR (interrater reliability), κw (weighted kappa coefficient), RC (rehabilitation center), SAS (SOFEMER Activity Score), v2 (version 2)
      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to Archives of Physical Medicine and Rehabilitation
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

      1. World Health Organziation. International Classification of Functioning, Disability and Health (ICF). Available at: http://www.who.int/classifications/icf/en. Accessed January 11, 2022.

        • Stucki G
        • Cieza A
        • Melvin J.
        The International Classification of Functioning, Disability and Health (ICF): a unifying model for the conceptual description of the rehabilitation strategy.
        J Rehabil Med. 2007; 39: 279-285
        • Holden MK
        • Gill KM
        • Magliozzi MR
        • Nathan J
        • Piehl-Baker L.
        Clinical gait assessment in the neurologically impaired. Reliability and meaningfulness.
        Phys Ther. 1984; 64: 35-40
        • Wilson JTL
        • Hareendran A
        • Grant M
        • et al.
        Improving the assessment of outcomes in stroke: use of a structured interview to assign grades on the modified Rankin Scale.
        Stroke. 2002; 33: 2243-2246
        • Lawton MP
        • Brody EM.
        Assessment of older people: self-maintaining and instrumental activities of daily living.
        Gerontologist. 1969; 9: 179-186
        • Lam SC
        • Lee DTF
        • Yu DSF.
        Establishing CUTOFF values for the Simplified Barthel Index in elderly adults in residential care homes.
        J Am Geriatr Soc. 2014; 62: 575-577
        • Bouwstra H
        • Smit EB
        • Wattel EM
        • et al.
        Measurement properties of the Barthel Index in geriatric rehabilitation.
        J Am Med Dir Assoc. 2019; 20: 420-425
        • Smit EB
        • Bouwstra H
        • van der Wouden JC
        • et al.
        Development of a Patient-Reported Outcomes Measurement Information System (PROMIS®) short form for measuring physical function in geriatric rehabilitation patients.
        Qual Life Res. 2020; 29: 2563-2572
        • Mayoral AP
        • Ibarz E
        • Gracia L
        • Mateo J
        • Herrera A.
        The use of Barthel Index for the assessment of the functional recovery after osteoporotic hip fracture: one year follow-up.
        PLoS One. 2019; 14e0212000
        • Duffy L
        • Gajree S
        • Langhorne P
        • Stott DJ
        • Quinn TJ.
        Reliability (inter-rater agreement) of the Barthel Index for assessment of stroke survivors: systematic review and meta-analysis.
        Stroke. 2013; 44: 462-468
        • Dodds TA
        • Martin DP
        • Stolov WC
        • Deyo RA.
        A validation of the functional independence measurement and its performance among rehabilitation inpatients.
        Arch Phys Med Rehabil. 1993; 74: 531-536
        • Granger CV
        • Hamilton BB
        • Keith RA
        • Zielezny M
        • Sherwin FS.
        Advances in functional assessment for medical rehabilitation.
        Top Geriatr Rehabil. 1986; 1: 59-74
      2. Health measurement scales: a practical guide to their development and use (5th edition).
        Aust N Z J Public Health. 2016; 40: 294-295
        • Morard MD
        • Gonzalez-Monge S
        • Rippert P
        • et al.
        Construction and feasibility study of the SOFMER Activity Score (SAS), a new assessment of physical and cognitive activity.
        Ann Phys Rehabil Med. 2018; 61: 315-322
        • Walton MK
        • Powers JH
        • Hobart J
        • et al.
        Clinical outcome assessments: conceptual foundation–report of the ISPOR Clinical Outcomes Assessment – Emerging Good Practices for Outcomes Research Task Force.
        Value Health. 2015; 18: 741-752
      3. COSMIN. COSMIN risk of bias checklist. Available at: https://www.cosmin.nl/wp-content/uploads/COSMIN-RoB-checklist-V2-0-v17_rev3.pdf. Accessed January 11, 2022.

        • Fleiss JL
        • Cohen J.
        The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability.
        Educ Psychol Meas. 1973; 33: 613-619
        • Landis JR
        • Koch GG.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Bangdiwala SI
        • Shankar V.
        The agreement chart.
        BMC Med Res Methodol. 2013; 13: 97
        • Everitt BS.
        The Cambridge dictionary of statistics.
        2nd ed. Cambridge University Press, Cambridge, UK2002
      4. Mount Sinai Rehabilitation Center. 2017 Scorecard inpatient rehabilitation. Available at: https://www.mountsinai.org/files/MSHealth/Assets/HS/Care/Rehab-Medicine/Inpatient-Services/2017Overall-Rehabilitation-Center-Outcomes.pdf. Accessed January 11, 2022.

        • Velozo CA
        • Byers KL
        • Wang YC
        • Joseph BR.
        Translating measures across the continuum of care: using Rasch analysis to create a crosswalk between the Functional Independence Measure and the Minimum Data Set.
        J Rehab Res Dev. 2007; 44: 467-478
        • Ottenbacher KJ
        • Hsu Y
        • Granger CV
        • Fiedler RC.
        The reliability of the functional independence measure: a quantitative review.
        Arch Phys Med Rehabil. 1996; 77: 1226-1232
        • Tarvonen-Schröder S
        • Laimi K
        • Kauko T
        • Saltychev M.
        Concepts of capacity and performance in assessment of functioning amongst stroke survivors: a comparison of the Functional Independence Measure and the International Classification of Functioning, Disability and Health.
        J Rehabil Med. 2015; 47: 662-664
        • Saltychev M
        • Tarvonen-Schröder S
        • Bärlund E
        • Laimi K.
        Differences between rehabilitation team, rehabilitants, and significant others in opinions on functioning of subacute stroke survivors: Turku ICF study.
        Int J Rehabil Res. 2014; 37: 229-235
        • Shrout PE
        • Fleiss JL.
        Intraclass correlations: uses in assessing rater reliability.
        Psychol Bull. 1979; 86: 420-428
        • Sainsbury A
        • Seebass G
        • Bansal A
        • Young JB.
        Reliability of the Barthel Index when used with older people.
        Age Ageing. 2005; 34: 228-232
        • Quinn TJ
        • Dawson J
        • Walters MR
        • Lees KR.
        Reliability of the modified Rankin Scale: a systematic review.
        Stroke. 2009; 40: 3393-3395