If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Rehabilitation Sciences Graduate Program, University of British Columbia, Vancouver, British Columbia, CanadaG.F. Strong Rehabilitation Centre, Vancouver, British Columbia, Canada
Rushton PW, Kirby RL, Miller WC. Manual wheelchair skills: objective testing versus subjective questionnaire.
Objectives
To test the hypothesis that the total scores of the Wheelchair Skills Test (WST) version 4.1, an observer-rated scale of wheelchair performance, and the Wheelchair Skills Test Questionnaire (WST-Q) version 4.1, a self-report of wheelchair skills, are highly correlated. We also anticipate that the WST-Q scores will be slightly higher, indicating an overestimation of capacity to perform wheelchair skills as compared with actual capacity.
Design
A cross-sectional, within-subjects comparison design.
Setting
Three Canadian cities.
Participants
Convenience sample of community-dwelling, experienced manual wheelchair users (N=89) ranging in age from 21 to 94 years.
Intervention
Not applicable.
Main Outcome Measures
Participants completed the subjective WST-Q version 4.1 followed by the objective WST version 4.1 in 1 testing session.
Results
The mean ± SD total percentage scores for WST and WST-Q were 79.5%±14.4% and 83.0%±12.1% for capacity and 99.4%±1.5% and 98.9%±2.5% for safety, respectively. The correlations between the WST and WST-Q scores were ρ=.89 (P=.000) for capacity and ρ=.12 (P=.251) for safety. WST-Q total score mean differences were an average of 3.5%±6.5% higher than WST scores for capacity (P=.000) and .52%±2.8% lower for safety (P=.343). For the 32 individual skills, the percentage agreement between the WST and WST-Q scores ranged from 82% to 100% for capacity and from 90% to 100% for safety.
Conclusion
WST and WST-Q version 4.1 capacity scores are highly correlated although the WST-Q scores are slightly higher. Decisions on which of these assessments to use can safely be based on the circumstances and objectives of the evaluation.
highly. The WST is an objective test of a set of skills, and a questionnaire version (the Wheelchair Skills Test Questionnaire [WST-Q]) is also available. Previous research has documented the psychometric properties of the WST,
Effectiveness and retention of the French-Canadian version of the Wheelchair Skills Training Program for manual wheelchair users: a randomized controlled trial.
The objective, performance-based WST and the subjective, self-report WST-Q both have merits and limitations. The main advantage of the WST is that the tester can see how the wheelchair user carries out the skill, permitting problems due to the wheelchair or the technique used to be identified and addressed. However, the WST requires time to perform (∼30min), space and equipment, or the availability of the environmental obstacles in the wheelchair users' home or community environments. By virtue of providing an opportunity to attempt skills, it can cause an unintended training effect that can be seen at subsequent assessments. As well, it captures an individual's ability to perform wheelchair skills on a particular day at a particular time, which may be influenced by a variety of factors, such as fatigue and anxiety.
The WST-Q, on the other hand, can be administered in any location. It enables assessment of wheelchair skills in situations in which it is not feasible to assess them objectively, such as when the wheelchair user is confined to a bed. The WST-Q can also be used as a screening assessment, for example, in a busy outpatient clinic to determine the need for referral for wheelchair skills training. From a research perspective, the WST-Q can be used when objective testing is not feasible because of a lack of human or space and equipment resources, or to reduce participant burden. Depending on how the questions are posed, the WST-Q can be used as a measure of performance (what the individual does do) or capacity (what the subject can do).
The WST-Q is advantageous in that it requires less time (only ∼10min), expense, equipment, and space to administer. However, there are drawbacks to using the WST-Q. For instance, the accuracy of responses can be influenced by cognitive or communication deficits. As well, describing rather than executing a task removes the environmental and motor-memory cues that may be relied upon to perform the task.
In an exploratory study designed to determine whether objective testing of wheelchair skills is necessary or whether subjective estimates would suffice, Newton et al
found a high correlation between the WST version 2.4 and 21 wheelchair users' subjective perceptions of their ability to perform wheelchair skills (ρ=.95), but wheelchair users overestimated their abilities by an average of 18% (P=.000). Participants in this study were asked to respond categorically whether they could or could not perform each skill. Mountain et al
used a semistructured interview for WST-Q version 2.4 in a similar study of 20 wheelchair users. The interviewer probed beyond the initial responses to better determine whether the evaluation criteria were met. They found a positive correlation between the objective and questionnaire versions (r=.91), but the wheelchair users overestimated their objective capacities by only 7%. No such comparison has been carried out for the latest version of the WST (version 4.1), which includes scores for both capacity and safety.
The primary objectives of this study were to test the hypotheses that the total scores of the WST and the WST-Q, version 4.1, are highly correlated but that the WST-Q scores are slightly higher, indicating an overestimation of capacity to perform the wheelchair skills as compared with actual capacity. Our secondary objective was to assess the agreement for individual test items (skills) between the WST-Q and the WST.
Methods
Study Design
This study used a cross-sectional, within-subjects comparison design. The data were collected as part of a larger study on wheelchair confidence that has been reported elsewhere.
Participants were recruited from 3 Canadian cities: Vancouver, British Columbia; Hamilton, Ontario; and Halifax, Nova Scotia. Letters of information were provided to potential participants on a rehabilitation research volunteer database and to those who attended rehabilitation services at local rehabilitation centers and a university-based rehabilitation gymnasium. Advertisements were posted in facilities that manual wheelchair users frequent (eg, fitness facilities and community centers), as well as online via e-bulletins and community organization websites, such as the Canadian Paraplegic Association. To be included in the study, participants had to be at least 19 years of age, use a manual wheelchair as their primary means of mobility (at least 4h per day), have at least 6 months of manual wheelchair experience, and live in the community. Participants were excluded if they were unable to read and write in English. Folstein et al's
Both tests evaluate 32 wheelchair skills (listed later) ranging in difficulty from wheeling forward for 10 meters to negotiating stairs. The tests were administered according to the WST 4.1 Manual.
All research assistants were trained by the primary investigator (P.W.R.). A spotter strap was used during the WST to prevent acute injury to the wheelchair user while performing skills for which there was a known risk of tips or falls from the wheelchair. The WST-Q 4.1 was administered using a semistructured interview, whereby the individual was asked whether he/she believed himself/herself to be capable of performing each wheelchair skill and, if so, how he/she would perform the skill. In this study, the WST-Q questions were posed to assess capacity (rather than performance)
because capacity is what is assessed by the WST. For both the WST and the WST-Q, dichotomous response formats were used to score the capacity (pass/fail) and safety (safe/unsafe) of each wheelchair skill using explicit criteria.
Percentage scores were calculated (number of passed or safe skills/number of possible skills × 100%).
Procedure
After recruitment, screening, and informed consent, a demographic questionnaire was completed to determine the demographic, clinical, and wheelchair-use characteristics of the sample. Administration of the WST-Q and the WST was completed in 1 testing session. The WST-Q preceded the WST to ensure that subjective perception of wheelchair skill was not influenced by objective measurement. Participants used their own manual wheelchairs.
Data Analysis
We used the Statistical Package for the Social Sciences version 16.0 for the statistical analysis.a Descriptive statistics and total percentages were calculated for the total WST and WST-Q capacity and safety scores. Normal distribution of the data was tested with the Kolmogorov-Smirnov test, and Bland-Altman limits-of-agreement plots were used to identify bias and outliers. A Spearman correlation coefficient was used for the total WST and WST-Q capacity and safety scores. The paired Wilcoxon signed-rank test was used to assess the extent of differences between the total WST and WST-Q scores. For the statistical analyses, we used an alpha level of .05. We compared individual wheelchair skills between the WST and WST-Q performance and safety scores using percentage agreements. A percentage agreement of 75% was defined as clinically significant.
The participants' demographic, clinical, and wheelchair-use characteristics are presented in table 1. The sample was composed mostly of men who had a mean age of 50.5±14.7 years and who ranged in wheelchair experience from 0.5 to 53.0 years. The primary diagnosis was spinal cord injury.
Table 1Participants' Demographic, Clinical, and Wheelchair-Use Characteristics
The mean ± SD, median, and range for the WST and WST-Q capacity and safety total percentage scores are presented in table 2. Both the WST and WST-Q capacity and safety scores demonstrated negatively skewed distributions, which were confirmed by Kolmogorov-Smirnov tests (P<.05). Bland-Altman plots (Fig 1, Fig 2) showed a fairly equal distribution of values above and below the mean difference, illustrating an unsystematic variability in scores between both the WST and WST-Q capacity and safety scores. A ceiling effect was found for both capacity and safety. For the capacity scores, there were 6 outliers, so nearly 95% of the capacity differences fell between ±2 SDs. For the safety scores, 5 outliers were identified, so 95% of the safety differences fell between ±2 SDs. Using the Mini-Mental State Examination scores, it was determined that none of the outliers had cognitive impairment.
Table 2Comparison of the Total Percentage Scores for the WST and WST-Q Capacity and Safety Scores
Fig 1Bland-Altman plot of the mean versus the difference in the WST and WST-Q 4.1 capacity scores. This figure shows a comparison of the WST and the WST-Q 4.1. The mean difference between the WST and the WST-Q was 3.5, with an upper limit of agreement of 16.5 (mean difference + 2 SD) and a lower limit of −9.5 (mean difference – 2 SDs). There were 6 outliers, so nearly 95% of the capacity differences fell between the 2 SDs.
Fig 2Bland-Altman plot of the mean versus the difference in WST and WST-Q 4.1 safety scores. This figure shows a comparison of the WST and the WST-Q 4.1. The mean difference between the WST and the WST-Q was −0.5, with an upper limit of agreement of 5.1 (mean difference + 2 SDs) and a lower limit of −6.1 (mean difference −2 SDs). Five outliers were identified for the safety scores, so 95% of the safety differences fell between 2 SDs.
The correlation between the WST and WST-Q capacity scores was ρ=.89 (P=.000), while the correlation for safety was ρ=.12 (P=.251). Figure 3 illustrates a scatter plot of the WST and WST-Q total percentage capacity scores. Thirty of the 89 participants had the same WST and WST-Q capacity score. For safety scores, the data violated the assumption of linearity. Sixty-three of the 88 participants had the same WST and WST-Q safety score.
Fig 3Total percentage scores for the WST and WST-Q 4.1 capacity scores. Note that all data points (N=89) are not visible because 30 of the WST-Q and WST capacity scores were equal, resulting in many data points presenting as overlapping dots.
The mean differences ± SD for the total WST and WST-Q capacity and safety scores were 3.5%±6.5% (WST-Q higher) (P=.000) and .52%±2.8% (WST-Q lower) (P=.343). The median differences for the WST and WST-Q capacity and safety scores were 3.0% and 0.0%.
The data on individual wheelchair skills are shown in table 3. For the WST-Q, data were missing for 1 subject for all safety items. Otherwise, the reason for the sample size being less than N=89 or N=88 for certain skills was often a result of the WST scoring instructions. For example, for skills that have prerequisites, if the prerequisite for a skill had been failed (eg, 5-cm level change), the skill under consideration (eg, 15-cm level change) was not tested. In such a case, the participant received a “fail” score for capacity and a “not tested” score for safety. Another reason for the sample size being less than N=89 or N=88 involved the tester being unable to test certain skills. For example, at the Vancouver site, there was no alternate access to the top of the stairs. Therefore, if the participant was unable or unwilling to ascend the stairs, the tester was not able to test the descend stairs skill.
Table 3Percentage Agreement Between the WST and the WST-Q and the Percentage Success Rate for Capacity and Safety Scores for Individual Wheelchair Skills
Skill
N
% Agreement Between The WST and the WST-Q
% Success Rate
WST
WST-Q
WST
WST-Q
C
S
C
S
C
S
C
S
C
S
Rolls forward 10m
89
88
89
89
100
100
100
100
100
100
Rolls forward 10m in 30s
89
88
89
89
95.5
100
100
100
95.5
100
Rolls backward 5m
89
88
89
89
98.9
100
100
100
98.9
100
Turns 90 deg while moving forwardL&R
89
88
89
89
100
100
100
100
100
100
Turns 90 deg while moving backwardL&R
89
88
89
89
96.6
100
98.9
100
97.8
100
Turns 180 deg in placeL&R
89
88
89
89
98.9
100
100
100
98.9
100
Maneuvers sidewaysL&R
89
88
89
89
95.5
100
95.5
100
97.8
100
Gets through hinged door in both directions
89
88
89
89
96.6
100
96.6
100
100
100
Reaches 1.5-m high object
89
88
89
89
97.8
100
98.9
100
98.9
100
Picks object from floor
89
88
89
89
100
100
97.8
100
97.8
100
Relieves weight from buttocks
89
88
89
89
96.6
100
96.6
100
95.5
100
Transfers from wheelchair to bench and back
89
88
89
89
94.4
100
91.0
100
96.6
100
Folds and unfolds wheelchair
88
88
89
89
85.2
100
68.2
100
82.0
100
Rolls 100m
89
88
89
89
96.6
100
96.6
100
97.8
100
Avoids moving obstaclesL&R
86
83
89
89
93.0
97.6
93.0
97.6
98.9
100
Ascends 5-deg incline
89
88
89
89
92.1
98.9
94.4
98.9
97.8
100
Descends 5-deg incline
89
88
89
89
98.9
97.7
98.9
100
100
97.8
Ascends 10-deg incline
89
87
89
89
84.3
98.9
80.9
100
92.1
98.9
Descends 10-deg incline
89
87
89
89
91.0
92.0
93.3
97.7
97.8
94.4
Rolls 2m across 5-deg side-slopeL&R
88
88
89
89
93.2
98.9
95.5
100
91.0
98.9
Rolls 2m on soft surface
89
88
89
89
87.6
98.9
78.7
98.9
82.0
100
Gets over 15-cm pothole
89
88
89
89
91.0
94.3
79.8
97.7
84.3
96.6
Gets over 2-cm threshold
89
88
89
89
96.6
100
94.4
100
95.5
100
Ascends 5-cm level change
89
88
89
89
89.9
100
80.9
100
84.3
100
Descends 5-cm level change
89
88
89
89
94.4
100
89.9
100
95.5
100
Ascends 15-cm curb
89
78
89
88
83.1
96.2
20.2
98.7
30.3
97.7
Descends 15-cm curb
89
82
89
88
83.1
91.5
47.2
95.1
59.6
95.5
Performs 30s stationary wheelie
89
88
89
89
93.3
96.6
50.6
98.9
52.8
97.8
Turns 180 deg in place in wheelie position
89
74
89
81
93.3
98.6
43.8
98.6
43.8
100
Gets from ground into wheelchair
89
88
89
89
82.0
97.7
32.6
98.9
48.3
98.9
Ascends stairs
88
87
89
89
93.2
97.7
6.8
100
13.5
97.8
Descends stairs
84
86
89
89
85.7
90.7
13.1
100
30.3
91.0
Abbreviations: C, capacity; deg, degrees; L&R, left and right; S, safety.
The percentage agreement between the WST and WST-Q scores for the individual skills ranged from 82% to 100% for capacity and 90% to 100% for safety. All skills met the threshold of 75% or greater for a clinically significant agreement. Generally, the skills with the lowest percentage success rates were the more difficult community- and advanced-level skills.
Discussion
We met our objectives of determining the relationship between the WST and the WST-Q version 4.1. Our hypothesis that the WST and the WST-Q scores would be highly correlated was confirmed for the capacity scores (ρ=.89) but not for the safety scores (ρ=.12). As hypothesized, WST-Q total scores for capacity were slightly, but significantly, higher than WST scores, but the safety score difference was not significant.
The high correlation between the total WST and WST-Q capacity scores is similar to that found by Mountain et al
(ρ=.95). The nonlinear relationship between the WST and WST-Q safety scores can be explained by the very high concordance between the scores, with 63 of the 88 participants (34.1%) having the same total score for the WST and the WST-Q. This lack of dispersion of values resulted in the nonlinear relationship and low correlation coefficient. Therefore, it is important to note that although the correlation coefficient is low, it is as a result of the very high degree of correspondence between the WST and WST-Q safety scores and is not indicative of a poor relationship between the 2 outcome measures.
The wheelchair users' overestimation of their capacity to perform wheelchair skills (3.5%) was less than that found by Mountain et al
(18%). The results of the 3 studies were similar, however, in that it was generally the more difficult, community- and advanced-level skills that were overestimated, such as negotiating ramps, potholes, and curbs. The overestimation of capacity to perform skills may have been less in this study as the wheelchair users had more wheelchair experience (16.4±13.2y) than did participants in the Mountain et al
study (4.9±10.1y). Participants in this study, therefore, may have been better able to judge their ability to perform the skills. Of significance is the wheelchair users' minimal (0.5%) underestimation of their safety in performing wheelchair skills. This result indicates that, for the most part, the wheelchair users in this study were able to accurately judge their safety in performing the wheelchair skills. This judgment is important because inaccurate judgments in safety may result in serious injury. The significance of this point is emphasized by the lower capacity and safety success rates in the wheelchair skills at the more difficult end of the spectrum.
When using WST-Q results to determine whether or not a wheelchair user requires further skills training, it is important to recognize that some wheelchair users overestimate their ability to perform the more difficult wheelchair skills. Both types of wheelchair skill assessment can yield important information and have potential roles in clinical and research settings. As discussed earlier, the WST and the WST-Q each have merits and limitations that need to be considered when selecting a test for clinical or research purposes. The high correlation between the WST and WST-Q capacity scores, the high concordance between the WST and WST-Q total capacity and safety scores, as well as the low percentage of overestimation of capacity and underestimation of safety to perform the wheelchair skills support the use of either the WST or the WST-Q when administration of one is preferable over the other.
These results are in contrast to findings from some other studies comparing objective and subjective measures. For example, Gandhi et al
found a low correlation between preoperative objective Timed Up-and-Go and the subjective Medical Outcomes Study 36-Item Short-Form Health Survey scores among patients undergoing hip and knee arthroplasty. Wand et al
Self-reported assessment of disability and performance-based assessment of disability are influenced by different patient characteristics in acute low back pain.
found a low correlation between the subjective Roland and Morris Disability Questionnaire and the objective Timed 5 Minute Walk Test and moderate correlations between the Roland and Morris Disability Questionnaire and the objective Timed Sit to Stand, Timed Up-and-Go, and Timed Lying to Stand Test among a sample of adults with acute low back pain. Suchy et al
Instrumental activities of daily living among community-dwelling older adults: discrepancies between self-report and performance are mediated by cognitive reserve.
reported that 38% of the participants in a sample of independent and healthy community-dwelling older adults exhibited a discrepancy between instrumental activities of daily living self-report and instrumental activities of daily living performance.
Increasingly, it has been reported that subjective, self-report measures are influenced by psychological status (eg, confidence, anxiety, and depression) to a greater degree than are objective, performance-based measures.
Self-reported assessment of disability and performance-based assessment of disability are influenced by different patient characteristics in acute low back pain.
Instrumental activities of daily living among community-dwelling older adults: discrepancies between self-report and performance are mediated by cognitive reserve.
However, our study showed little difference between the WST and the WST-Q scores. We know that this sample of manual wheelchair users had high wheelchair confidence and low anxiety and depression.
Therefore, the high level of wheelchair confidence and the low level of anxiety and depression in this group of manual wheelchair users may, in part, explain the high correlation between the WST and WST-Q capacity scores. Although it is tempting to assume that objective assessment is more valid than subjective measures, a wheelchair user could perform poorly in the formal test setting because of anxiety, day-to-day variation, or by narrowly failing to meet an evaluation criterion.
Study Limitations
There were limitations to this study, some of which have already been noted. The sample was an experienced group of manual wheelchair users. The clustering of scores at the higher score end of the WST and the WST-Q demonstrated by the Bland-Altman plot provides some evidence of that. Because our sample did not include new manual wheelchair users, the results should be generalized with caution. Another limitation was that the same tester administered both the WST and the WST-Q, but doing the WST-Q first would have the effect of minimizing any resulting bias. The ceiling effect could be overcome by selectively recruiting a participant population that is more diverse in its abilities and safety. Our sample was also literate and cognitively intact (Mini-Mental State Examination scores of ≥24), so these results should not be generalized to wheelchair users with greater language or cognitive limitations. Also, we confined ourselves to the assessment of capacity rather than performance, the other important dimension about which the WST-Q can provide insight. Inkpen et al
have shown a strong correlation between capacity and performance scores on the WST-Q.
Future work is needed to address the study limitations. As well, this work should be extended to power wheelchair users and caregivers to allow us to determine whether the WST and the WST-Q can be used interchangeably with these populations.
Conclusions
WST and WST-Q version 4.1 scores are highly correlated although the WST-Q scores are slightly higher for capacity. Decisions on which of these assessments to use can safely be based on the circumstances and objectives of the evaluation.
SPSS 16.0 for Windows, SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL 60606.
Acknowledgments
We thank Kristine Vaughan, BSc, Amira Tawashy, MSc, Mark Burley, BScOT, and Haley Augustine, MscOT, for their assistance with data collection and Kathleen Martin-Ginis, PhD, for site support at McMaster University.
References
World Health Organization
Guidelines on the provision of manual wheelchairs in less resourced settings.
Effectiveness and retention of the French-Canadian version of the Wheelchair Skills Training Program for manual wheelchair users: a randomized controlled trial.
Self-reported assessment of disability and performance-based assessment of disability are influenced by different patient characteristics in acute low back pain.
Instrumental activities of daily living among community-dwelling older adults: discrepancies between self-report and performance are mediated by cognitive reserve.
Supported by the Canadian Institutes of Health Research (grant no. 86803 ), the British Columbia Rehab Foundation , and the British Columbia Network for Aging Research . Salary/scholarship funds were provided by the Canadian Institutes of Health Research (grant no. 42703 ) and the Michael Smith Foundation for Health Research .
No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.
Reprints are not available from the author.
In-press corrected proof published online on Aug 9, 2012, at www.archives-pmr.org.
00430-3&id=gr1.jpg){kind=link}
00430-3&id=gr2.jpg){kind=link}
00430-3&id=gr3.jpg){kind=link}