Archives of Physical Medicine and Rehabilitation
Volume 89, Issue 9 , Pages 1720-1723, September 2008

A Comparison Between Self-Reported and Observed Activity Limitations in Adults With Neuromuscular Disorders

  • Laure Vandervelde, PhD

      Affiliations

    • Rehabilitation and Physical Medicine Unit, Université catholique de Louvain, Brussels, Belgium
    • ,
  • Delphine Dispa, PT

      Affiliations

    • Rehabilitation and Physical Medicine Unit, Université catholique de Louvain, Brussels, Belgium
    • ,
  • Peter Y. Van den Bergh, MD

      Affiliations

    • Neuromuscular Center & Neurology Department, Cliniques Universitaires Saint-Luc, Brussels, Belgium
    • ,
  • Jean-Louis Thonnard, PhD

      Affiliations

    • Rehabilitation and Physical Medicine Unit, Université catholique de Louvain, Brussels, Belgium
    • Corresponding Author InformationReprint requests to Jean-Louis Thonnard, PhD, Université catholique de Louvain, Unité de Réadaptation et de Médecine Physique 53, Av Mounier (5375), B-1200 Brussels, Belgium

Article Outline

Abstract 

Vandervelde L, Dispa D, Van den Bergh PY, Thonnard J-L. A comparison between self-reported and observed activity limitations in adults with neuromuscular disorders.

Objective

To investigate the agreement between the self-reported and examiner-reported difficulties of patients with neuromuscular disorders (NMDs) in performing daily activities at home.

Design

A comparison between 2 methods of administering a measurement instrument.

Setting

Neuromuscular reference center in a university hospital.

Participants

Adult patients (N=57) with diagnosed NMDs living at home.

Interventions

Not applicable.

Main Outcome Measure

The ACTIVLIM questionnaire.

Results

The intraclass correlation coefficient, model 2,1 (ICC2,1), between the measures was very good (ICC2,1=.87), indicating a good agreement between self-perceived and observed measures.

Conclusions

The use of ACTIVLIM as a self-reporting questionnaire is a valid method for assessing activity limitations in patients with NMD.

Key Words: Neuromuscular diseases, Outcome assessment (health care), Questionnaires, Rehabilitation, Task performance and analysis

List of Abbreviations: DIF, differential item functioning, ICC, intraclass correlation coefficient, NMD, neuromuscular disorder

 

NEUROMUSCULAR DISORDERS are hereditary or acquired diseases of the motor unit1 and sometimes involve parts of the central nervous system. Most of these diseases have a progressive clinical course characterized by decreased muscle strength leading to impaired motor function.2 Consequences of these diseases include fatigue, problems with locomotion, and loss of functionality in activities of daily living. Difficulties in performing daily activities are defined by the World Health Organization as activity limitations.3 Activity limitations in patients with NMDs can be measured with a new scale, the ACTIVLIM questionnaire, which was developed and validated by using the Rasch model.4 It presents very good psychometric qualities including reliability, construct validity, reproducibility, linearity, and unidimensionality. Moreover, it offers the benefits of being inexpensive and easy to administer. This questionnaire is comprised of self-reported measures focused on the patients' perceptions of their activity limitations and informs clinicians on how well patients manage in their home environments.5 Nevertheless, self-reported measures are subjective evaluations that could be influenced by factors such as culture, level of education, or emotional state.6 For these reasons, performance-based measures have been developed because they are considered to be more objective and standardized for the assessment of the functional status of patients.5, 7 The relationships between self-reported and performance-based measures have been studied in various pathologies,8, 9, 10, 11, 12, 13, 14, 15 but different items and scoring procedures were used for both measures. Indeed, most of the self-reported items correspond to usual daily activities assessed with rating scales determining the degree of difficulty or assistance required to complete these activities, whereas the performance-based items are often timed and correspond to standardized tasks.16 Scoring the performance by direct observation with rating scales is also used as a method of functional assessment.17 Nevertheless, comparing scores observed on rating scales by clinicians with self-reported scores was never performed in patients with NMDs and certainly not with the aim of validating the use of a self-reporting questionnaire.

The purpose of this study was to compare self-reported and observed measures by using the ACTIVLIM questionnaire to endorse or question the use of a self-reported questionnaire in future clinical and research settings to assess activity limitations in patients with NMDs.

Back to Article Outline

Methods 

Participants 

This study was approved by the medical ethics committee of the Université catholique de Louvain. The patients were recruited through the Neuromuscular Reference Center of the Cliniques Universitaires Saint-Luc in Brussels. The patients gave written informed consent before the evaluation. Fifty-seven adult patients with a diagnosed NMD were assessed by 4 physical therapists. Patient descriptions are given in table 1.

Table 1. Patient Sample Description (N=57)
CharacteristicsValues
Age (y)⁎50(19–80)
Sex (n)
Male25
Female32
Diagnosis (n)
DMD/BMD or LGMD7
Hereditary neuropathy12
Myotonic dystrophy9
SMA4
FSHD8
Others (eg, ALS, PPS)17
Mobility level (n)
Walking48
Wheelchair-bound9

Abbreviations: ALS, amyotrophic lateral sclerosis; BMD, Becker's muscular dystrophy; DMD, Duchenne's muscular dystrophy; FSHD, facioscapulohumeral dystrophy; LGMD, limb-girdle muscular dystrophy; PPS, postpoliomyelitis syndrome; SMA, spinal muscular atrophy.

⁎Values are mean (range).

Self-Reported and Observed Measures 

Both self-reported and observed measures were obtained with the ACTIVLIM questionnaire so that they were standardized in the item and scoring procedure. This questionnaire consists of 22 daily activities presented in the questionnaire development study4 and was originally developed by using the Rasch model, which allows the conversion of ordinal scores into linear measures located on a unidimensional scale.18 These linear measures are expressed in logits (ie, log-odd units), the constant measurement unit of activity scale; the higher the value is in logits, the higher the activity level is of the patient.

The self-reported measures were obtained by asking patients to provide their perceived difficulty in performing each activity on the ACTIVLIM questionnaire, without using technical or human help, on a 3-level ordinal scale (0=impossible, 1=difficult, 2=easy). The patients were observed at their homes so that they could be in their usual environment when performing the activities to be assessed.19 The physical therapists observed the patients performing the activities and assessed the difficulty experienced by the patients in completing these activities. The scoring procedure of the observed measures was identical to the self-reported measures (0=impossible [the patient is unable to achieve the activity without using technical or human help]; 1=difficult [the patient is able to achieve the activity without any help but experiences some difficulty]; 2=easy [the patient is able to achieve the activity without any help and experiences no difficulty]).

Procedure 

The evaluation comprised 3 stages. The first self-reported measure was collected by mail about 2 weeks before a home visit by the physical therapists. The patients' responses were not disclosed to the therapists to prevent influencing their observations. Second, the patients performed each activity of the ACTIVLIM questionnaire within the limits of their abilities during the evaluation at home. The physical therapists observed the patients performing the activities and rated each of them on the ACTIVLIM ordinal scale. For ethical reasons, some activities such as taking a bath, taking a shower, or washing one's upper body were mimicked, and the patients explained how they usually performed them. Finally, a second self-reported measure was collected about a month after the home visit. In each questionnaire, the activities were randomly presented to avoid any effects caused by the item order. Moreover, the objective of the study was not revealed to the patients to prevent any influence on their responses.

The 57 patients were evaluated by 4 examiners (E1, E2, E3, E4). First, 26 patients were simultaneously assessed by E1 and E2, with an excellent interrater reliability (ICC=.98). Second, 5 patients were assessed by E1, E3, and E4, also with an excellent interrater reliability (ICC=.99). Hence, the last 26 patients were evaluated by 1 examiner (E3 or E4). One observed measure was retained for the 31 patients evaluated by 2 or 3 examiners, and these measures were put together with the 26 measures observed by E3 or E4. As a result, each patient had only 1 observed measure for further data analyses.

Statistical Analysis 

The self-reported and observed responses were first fitted to the Rasch model by using the Rasch Unidimensional Measurement Models computer program.a This software reported overall fit statistics that were close to standardized normative distribution for item and person fit residuals (mean ± SD, −.036±.400 and −.266±.937, respectively, for the first self-reported measure; mean ± SD, −.322±.863 and −.177±.66, respectively, for observed measure; mean ± SD, −.031±.566 and −.144±.946, respectively, for the second self-reported measure) and were not significant for the chi-square item-trait interaction (P=.56, P=.28, and P=.42, respectively, for first self-reported measure, observed measure, and the second self-reported measure). As a result, the ordinal total scores obtained from the self-reported and observed ACTIVLIM questionnaires could be transformed into interval-level measures of activity limitations. Three linear measures of the patients' activity level (first self-reported measure, observed measure, second self-reported measure) were reported in such a way that they could be quantitatively compared and treated as a continuous variable.

The invariance of item hierarchy across the 3 evaluations was checked by using DIF test. A DIF can be detected by a 2-way analysis of variance for each item by comparing scores across each level of patient-related factor (first self-reported measure, observed measure, and second self-reported measure in this case) and across levels of the activity construct (eg, the class intervals).20 A significant main effect for the patient-related factor shows the presence of a uniform DIF, whereas a nonuniform DIF corresponds to a significant interaction effect (patient-related factor by class interval).

The agreement between the self-reported and observed measures was calculated by using the ICC model 2,1 (ICC2,1),21 The ICC2,1 was computed as a 2-way analysis of variance in which the targets (ie, the patients) were 1 source of variability and the raters or measures (the self-perceived and observed activity levels) were the second source of variability.22 This coefficient is preferred to the Pearson correlation coefficient because it takes into account the extent of the relative discrepancies between the evaluations, whereas the Pearson coefficient measures the degree of linearity between 2 measures.22, 23

The κ coefficient was used to quantify item-by-item agreement between both administration methods of the ACTIVLIM questionnaire.24

Back to Article Outline

Results 

No significant DIF, uniform or nonuniform, was found across the 3 evaluations (P≥0.1), indicating that the item hierarchy is invariant if the item difficulty is evaluated by the patients or by the external examiners.

The ICCs between first self-reported measure and observed measure and between first self-reported measure and second self-reported measure were equal to .87 and .93, respectively, indicating good agreement between the measures. Figure 1 shows the relationships between the different measures obtained by self-reporting or observation in relation to the line x equal to y. For example, the score of patient “a” lies on this identity line, indicating a perfect agreement between his activity level obtained by the self-reported questionnaire and the one obtained by observation. For patient “b,” the self-reported score was lower than the score obtained by observation.

  • View full-size image.
  • Fig 1. 

    The relationships between the activity measures of the patients obtained (A) by the first self-report (SR-1) and by observation (O1) and (B) by the first (SR-1) and the second (SR-2) self-reports. Patients (dots) with the highest activity levels are plotted in the top right part of the figure. The dashed line represents the identity line.

Most of the items showed κ coefficients between .44 and .99 for their response agreement between the first self-reported measure and observed measure. Only the item “carrying a heavy load” had a κ coefficient of .25.

Back to Article Outline

Discussion 

The self-perceived difficulty of 57 adult patients with NMDs in performing daily activities was compared with the difficulty in performing the same activities as observed by 4 physical therapists. Both perceived and observed measures were obtained by using the ACTIVLIM questionnaire, allowing the use of the same items and the same scoring procedure for each measure. The high ICC between self-perceived and observed activity limitations (ICC=.87) indicates a good agreement between these measures in patients with NMDs. Therefore, the use of a self-reported questionnaire is a valid method for assessing activity limitations in patients with NMDs. Additionally, self-reporting presents considerable advantages over observation. The self-reported questionnaire is low in cost and extremely easy to administer because it can be completed in 5 minutes in the waiting room by the patient, whereas the observation of daily activities at home took about 45 minutes. Therefore, the self-reported ACTIVLIM questionnaire could be integrated into any research protocols to evaluate activity limitations in patients with NMDs. Moreover, such a questionnaire is really easy to use for clinicians who want to assess and characterize evolution of these patients in neuromuscular clinics for example.

The values of the κ coefficients showed moderate to excellent agreement between the first self-reported and observed responses of each item except for the item “carrying a heavy load.” Indeed, many patients evaluated this task as impossible, whereas the examiners evaluated it as difficult. This is probably because of the fact that, during the observation, the load was standardized for all patients and was smaller or lighter than what some of them imagined when they filled in the questionnaire.

The patients' perceptions were stable and reproducible after a delay of approximately 45 days, as shown by the ICC of .93 between the first self-reported measure and the second self-reported measure, despite the fact that after the delay the patients better understood some items of the questionnaire. Indeed, performing daily activities in front of physical therapists could influence the patients' perceptions of the difficulty of these activities and, therefore, the second self-reported measures. The reproducibility of the patients' measures was previously studied during the development of the ACTIVLIM questionnaire, and the 369 patients with NMDs evaluated their activity levels consistently over a period of about 1 month (ICC=.93).4 The present study confirms this previous result and shows that receiving further information about these activities has no effect on the patients' perceptions of their activity levels.

Some differences could be observed between self-perceived and observed activity limitations, especially for patients with an activity level higher than 2 logits (see fig 1 A). The observed activity levels of these patients are higher than the self-reported ones, indicating that either the external examiners overestimated the patients' activity levels or the patients underestimated them. On the one hand, this could be because of the fact, as previously suggested, that patients with a lack of self-confidence tend to report a lower level of functioning as compared with their performance-based functioning.10, 25 On the other hand, some patients show motivation that could have improved their performance during observation, possibly because they were conscious of being part of an experimental study26 or because they were aware of the presence of the examiners.27 However, personality, motivation and affective functioning were not evaluated in this study, and, therefore, their influence on the first self-reported measure and the observed measure cannot be determined. Nevertheless, these slight divergences did not affect the good concordance between the self-reported and observed activity levels expressed by the ICC of .87.

Study Limitations 

The present results only apply to adult patients with NMDs. Self-reported measures could be overestimated or underestimated in other pathologies such as those with cognitive impact. Further researches are, however, required to verify this assumption by adding personality and depression tests to know the affective and personality characteristics of the patients at the time of the different evaluations. Moreover, because the ACTIVLIM questionnaire is designed for adults as well as for children, this study could also be performed in children with NMDs by comparing self-reported measures reported by their parents with observed measures by external examiners.

Back to Article Outline

Conclusions 

The ICC between the 2 measures shows very good agreement between self-reported and observed measures, which indicates that the questionnaire is a valid method for assessing activity limitations in patients with NMDs. Moreover, the patients' perceptions were reproducible after a delay of 45 days, even though they had performed the activities in front of physical therapists.

Supplier

Back to Article Outline

Acknowledgments 

We thank physical therapists Aude Nécandre, Emmanuelle Pernot, and Floriane Collet for their assistance in data acquisition.

The ACTIVLIM questionnaire and its administration instructions can be downloaded from http://www.rehab-scales.org in English, French, and Dutch. The website also allows total raw scores for the ACTIVLIM questionnaire to be converted into a linear measure of activity limitations, according to the Rasch model.

Back to Article Outline

References 

  1. McDonald CM. Physical activity, health impairments, and disability in neuromuscular disease. Am J Phys Med Rehabil. 2002;81(11 Suppl):S108–S120
  2. Piccininni M, Falsini C, Pizzi A. Quality of life in hereditary neuromuscular diseases. Acta Neurol Scand. 2004;109:113–119
  3. World Health Organization. The International classification of functioning, disability and health: ICF. Geneva: WHO; 2001;
  4. Vandervelde L, Van den Bergh PY, Goemans N, Thonnard JL. ACTIVLIM: a Rasch-built measure of activity limitations in children and adults with neuromuscular disorders. Neuromuscul Disord. 2007;17:459–469
  5. Owens PL, Bradley EH, Horwitz SM, et al. Clinical assessment of function among women with a recent cerebrovascular event: a self-reported versus performance-based measure. Ann Intern Med. 2002;136:802–811
  6. Kivinen P, Sulkava R, Halonen P, Nissinen A. Self-reported and performance-based functional status and associated factors among elderly men: the Finnish cohorts of the Seven Countries Study. J Clin Epidemiol. 1998;51:1243–1252
  7. Ferrer M, Lamarca R, Orfila F, Alonso J. Comparison of performance-based and self-rated functional capacity in Spanish elderly. Am J Epidemiol. 1999;149:228–235
  8. Cress ME, Schechtman KB, Mulrow CD, Fiatarone MA, Gerety MB, Buchner DM. Relationship between physical performance and self-perceived physical function. J Am Geriatr Soc. 1995;43:93–101
  9. Hoeymans N, Feskens EJ, van den Bos GA, Kromhout D. Measuring functional status: cross-sectional and longitudinal associations between performance and self-report (Zutphen Elderly Study 1990-1993). J Clin Epidemiol. 1996;49:1103–1110
  10. Kempen GI, Steverink N, Ormel J, Deeg DJ. The assessment of ADL among frail elderly in an interview survey: self-report versus performance-based tests and determinants of discrepancies. J Gerontol B Psychol Sci Soc Sci. 1996;51:P254–P260
  11. Wijlhuizen GJ, Ooijendijk W. Measuring disability, the agreement between self evaluation and observation of performance. Disabil Rehabil. 1999;21:61–67
  12. Reuben DB, Valle LA, Hays RD, Siu AL. Measuring physical function in community-dwelling older persons: a comparison of self-administered, interviewer-administered, and performance-based measures. J Am Geriatr Soc. 1995;43:17–23
  13. van den Ende CH, Hazes JM, Le Cessie S, Breedveld FC, Dijkmans BA. Discordance between objective and subjective assessment of functional ability of patients with rheumatoid arthritis. Br J Rheumatol. 1995;34:951–955
  14. Mannerkorpi K, Svantesson U, Broberg C. Relationships between performance-based tests and patients' ratings of activity limitations, self-efficacy, and pain in fibromyalgia. Arch Phys Med Rehabil. 2006;87:259–264
  15. Itzkovich M, Tamir A, Philo O, et al. Reliability of the Catz-Itzkovich Spinal Cord Independence Measure assessment by interview and comparison with observation. Am J Phys Med Rehabil. 2003;82:267–272
  16. Coman L, Richardson J. Relationship between self-report and performance measures of function: a systematic review. Can J Aging. 2006;25:253–270
  17. Salter K, Jutai JW, Teasell R, Foley NC, Bitensky J, Bayley M. Issues for selection of outcome measures in stroke rehabilitation: ICF activity. Disabil Rehabil. 2005;27:315–340
  18. Rasch G. Probabilistic models for some intelligence and attainment tests. Chicago: Mesa Pr; 1980;
  19. Rogers JC, Holm MB, Beach S, et al. Concordance of four methods of disability assessment using performance in the home as the criterion method. Arthritis Rheum. 2003;49:640–647
  20. Pallant JF, Tennant A. An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS). Br J Clin Psychol. 2007;46(Pt 1):1–18
  21. Shrout P, Fleiss J. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–428
  22. McGraw K, Wong S. Forming inferences about some intraclass correlation coefficients. Psychol Methods. 1996;1:30–46
  23. Hamilton BB, Laughlin JA, Fiedler RC, Granger CV. Interrater reliability of the 7-level functional independence measure (FIM). Scand J Rehabil Med. 1994;26:115–119
  24. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas. 1960;20:27–46
  25. Smith TW, Williams PG. Personality and health: advantages and limitations of the five-factor model. J Pers. 1992;60:395–423
  26. Campbell JP, Maxey VA, Watson WA. Hawthorne effect: implications for prehospital research. Ann Emerg Med. 1995;26:590–594
  27. Karamehmetoğlu SS, Karacan I, Elbaşi N, Demirel G, Koyuncu H, Dösoğlu M. The functional independence measure in spinal cord injured patients: comparison of questioning with observational rating. Spinal Cord. 1997;35:22–25
  • a RUMM2020; RUMM Laboratory Pty Ltd, 14 Dodonaea Ct, Duncraig, WA 6023, Australia.

 Supported by the Association Belge contre les Maladies NeuroMusculaires, the Fonds Spéciaux de Recherche of the Université catholique de Louvain, the Association Nationale d'Aide aux Handicapés, and the Fondation Saint-Luc.

 No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.

PII: S0003-9993(08)00398-5

doi:10.1016/j.apmr.2008.01.024

Archives of Physical Medicine and Rehabilitation
Volume 89, Issue 9 , Pages 1720-1723, September 2008

Article Tools

* Image Usage & Resolution