Advertisement

Test-Retest Reliability and Minimal Detectable Change of the SmartWheel Clinical Protocol

      Abstract

      Lui J, MacGillivray MK, Sawatzky BJ. Test-retest reliability and minimal detectable change of the SmartWheel clinical protocol.

      Objectives

      (1) To quantify the intra- and intersession reliability and minimal detectable change (MDC) of the SmartWheel clinical protocol (SCP) and (2) to compare the reliability of the SCP between experienced and naïve wheelchair users.

      Design

      Test-retest study.

      Setting

      Biomechanics laboratory.

      Participants

      Manual wheelchair users (WCUs) (n=10) with 1 to 32 years of wheeling experience and able-bodied (AB) naïve users (n=15).

      Interventions

      Not applicable.

      Main Outcome Measures

      Wheelchair propulsion parameters including average peak force, push frequency, push length, and velocity were measured according to the SCP but with 5 trials per session, for 2 sessions. WCUs and AB users were analyzed separately. Intraclass correlation coefficients ([ICC]2,1 and ICC2,5) were calculated to assess intrasession reliability. ICC2,1 and MDC (with 95% confidence) were calculated for each SCP parameter using a single trial from each session and with the mean of 5 repeated measures to evaluate intersession reliability.

      Results

      Intra- and intersession reliability for WCU parameters ranged from high to very high correlation (ICC range, .70–.99). For AB parameters, intrasession ICC2,1 ranged from moderate to very high (ICC range, .50–.92), while intersession ICC2,1 indicated low to very high correlation (ICC range, .25–.90). Estimates of standard error of measurement and MDC were provided for each parameter. For both WCUs and AB users, using the means from 5 trials increased intra- and intersession ICC and decreased MDC values. All MDC values were lower for WCUs compared with AB users.

      Conclusions

      The SCP is a reliable method for assessing propulsion parameters in WCUs, even if just 1 trial is taken per session. AB users showed lower intra- and intersession reliability compared with WCUs. Therefore, for AB users or individuals with minimal wheeling experience, averaging multiple trials is recommended for the SCP.

      Key Words

      List of Abbreviations:

      AB (able-bodied), ICC (intraclass correlation coefficient), MDC (minimal detectable change), MSE (mean squared error), SCP (SmartWheel clinical protocol), WCU (wheelchair user)
      IT IS WELL DOCUMENTED that long-term use of wheelchairs is associated with repetitive strain injuries to the upper extremities.
      • Silfverskiold J.
      • Waters R.L.
      Shoulder pain and functional disability in spinal cord injury patients.
      • Boninger M.L.
      • Souza L.S.
      • Cooper R.A.
      • Fitzgerald S.G.
      • Koontz A.M.
      • Fay B.T.
      Propulsion patterns and pushrim biomechanics in manual wheelchair propulsion.
      • Mercer J.L.
      • Boninger M.
      • Koontz A.M.
      • Ren D.
      • Dyson-Hudson T.
      • Cooper R.
      Shoulder joint kinetics and pathology in manual wheelchair users.
      Wheelchair configurations and interventions designed to modify propulsion strategies can effectively reduce the forces needed to propel a wheelchair and therefore reduce the load on the shoulders and wrists.
      • Mercer J.L.
      • Boninger M.
      • Koontz A.M.
      • Ren D.
      • Dyson-Hudson T.
      • Cooper R.
      Shoulder joint kinetics and pathology in manual wheelchair users.
      • Shimada S.D.
      • Robertson R.N.
      • Boninger M.L.
      • Cooper R.A.
      Kinematic characterization of wheelchair propulsion.
      • Richter W.M.
      • Rodriguez R.
      • Woods K.R.
      • Axelson P.W.
      Stroke pattern and handrim biomechanics for level and uphill wheelchair propulsion at self-selected speeds.
      • Requejo P.S.
      • Mulroy S.J.
      • Haubert L.L.
      • Newsam C.J.
      • Gronley J.K.
      • Perry J.
      Evidence-based strategies to preserve shoulder function in manual wheelchair users with spinal cord injury.
      To better understand manual wheelchair propulsion, the SmartWheela was developed to allow clinicians and researchers to objectively assess the forces and moments applied to the pushrim.
      • Cooper R.A.
      Smartwheel: from concept to clinical practice.
      The SmartWheel is a specialized wheelchair wheel that measures propulsion parameters, such as the three-dimensional forces applied to the pushrim, push length, push frequency, and velocity during wheeling.
      • Cooper R.A.
      Smartwheel: from concept to clinical practice.
      The SmartWheel is used in conjunction with the SmartWheel clinical software to assist clinicians in assessing these propulsion parameters. The SmartWheel can provide clinicians with objective data needed to prescribe the most appropriate wheelchair configurations or to implement and track the effectiveness of their interventions.
      The SmartWheel users' group, an international network of researchers and clinicians, developed a SmartWheel clinical protocol (SCP), which was designed to accommodate a clinic's availability of space and time.
      • Cowan R.E.
      • Boninger M.L.
      • Sawatzky B.J.
      • Mazoyer B.D.
      • Cooper R.A.
      Preliminary outcomes of the SmartWheel User's Group database: a proposed framework for clinicians to objectively evaluate manual wheelchair propulsion.
      In the SCP, patients are instructed to wheel at a self-selected speed for a maximum of 10 seconds (or 10m) on 3 different surfaces (tile, 5% grade ramp, and carpet) while the SmartWheel software generates data on 4 key measurements: velocity, average peak force, push frequency, and push length.
      • Cowan R.E.
      • Boninger M.L.
      • Sawatzky B.J.
      • Mazoyer B.D.
      • Cooper R.A.
      Preliminary outcomes of the SmartWheel User's Group database: a proposed framework for clinicians to objectively evaluate manual wheelchair propulsion.
      This provides 12 SCP parameters (3 surfaces × 4 measurements), which are considered the most clinically relevant and are used to help guide wheelchair prescription and/or training.
      • Cowan R.E.
      • Boninger M.L.
      • Sawatzky B.J.
      • Mazoyer B.D.
      • Cooper R.A.
      Preliminary outcomes of the SmartWheel User's Group database: a proposed framework for clinicians to objectively evaluate manual wheelchair propulsion.
      Accordingly, velocity, average peak force, push frequency, and push length are important measures that clinicians can modify in order to improve the performance or efficiency of manual wheelchair propulsion.
      Consortium for Spinal Cord Medicine
      Preservation of upper limb function following spinal cord injury: a clinical practice guideline for health-care professionals.
      Therefore, it is important that these measurements are reliably determined by the SCP. It is also important to estimate the minimal detectable change (MDC) of the protocol. This would allow clinicians to distinguish true variability from measurement error and to estimate the minimal difference that can be detected confidently between sessions.
      • Lexell J.E.
      • Downham D.Y.
      How to assess the reliability of measurements in rehabilitation.
      In addition, the concept of reliability depends highly on the population being measured.
      • Dawis R.V.
      Scale construction.
      Therefore, reliability should be evaluated on a sample that most closely resembles the intended target population.
      • Dawis R.V.
      Scale construction.
      In wheelchair users (WCUs), experience has been shown to affect propulsion technique and consistency.
      • Brown D.D.
      • Knowlton J.H.
      • Schneider T.L.
      • Hetzler R.K.
      Physiological and biomechanical differences between wheelchair-dependent and able-bodied subjects during wheelchair ergometry.
      • Robertson R.N.
      • Boninger M.L.
      • Cooper R.A.
      • Shimada S.D.
      Pushrim forces and joint kinetics during wheelchair propulsion.
      • De Groot S.
      • Veeger H.E.
      • Hollander P.A.
      • Van der Woude L.H.
      Wheelchair propulsion technique and mechanical efficiency after 3 wk of practice.
      • De Groot S.
      • Veeger H.E.
      • Hollander A.P.
      • van der Woude L.H.
      Adaptations in physiology and propulsion techniques during the initial phase of learning manual wheelchair propulsion.
      This could mean that the reliability of the SCP may be dependent on general wheelchair experience and therefore should be evaluated accordingly.
      To date, no studies have reported the test-retest reliability or the MDC of the SCP. Therefore, the aims of the study were to: (1) examine the intra- and intersession reliability of the SCP; (2) quantify the MDC for each of the SCP parameters; and (3) compare the intra- and intersession reliability between experienced and naïve WCUs.

      Methods

      Participants

      Based on the methodology of Walter et al
      • Walter S.D.
      • Eliasziw M.
      • Donner A.
      Sample size and optimal designs for reliability studies.
      for calculating sample size for reliability studies, it was determined that a minimum sample size of 9 was needed.
      Ten spinal cord injured manual WCUs with at least 1 year of wheeling experience were recruited through advertisements posted at an outpatient rehabilitation clinic. Only WCUs whose wheelchairs were equipped with 61.0 or 63.5cm (24 or 25in). wheels, quick release axles, and pneumatic high-pressure tires were included in the study. Fifteen able-bodied (AB) individuals were also included and analyzed separately from the WCUs. The AB individuals were comprised of a convenience sample to represent individuals who are naïve to manual wheeling. Therefore, they are to some extent comparable with newly injured individuals with intact upper body function. Inclusion criteria for all participants included being between 18 to 65 years of age, and being able to use a manual wheelchair without pain. Individuals who had muscular or cardiorespiratory illnesses were excluded from this study. The study received approval from the university's clinical research ethics board. All participants provided written informed consent.

      Study Design

      The study used a test-retest design. All participants were evaluated with the same protocol twice, with approximately 7 days between testing sessions.

      Procedure

      WCUs used their own wheelchairs for the study, and AB individuals were provided with appropriately sized (either 40.6 or 43.2cm [16 or 17in]) Elevationb wheelchairs that allowed for seat height and dump adjustments via a built in hydraulic spring system. Individual wheelchair configurations were kept consistent between sessions. The SmartWheel was affixed to the wheelchair on the side of the participant's dominant arm while the wheelchair's standard wheel remained on the nondominant side. Pneumatic high-pressure tires were used on all wheels. All tires were inflated to manufacturer's specifications.
      Once the SmartWheel was attached, all participants were given time to familiarize themselves with their wheelchairs and the different test surfaces prior to testing. The testing orders of the 3 surfaces were randomized. Adequate rest was provided at any time (at participant's request) during testing in order to minimize fatigue.

      Protocol

      Following the SCP, participants were instructed to wheel across 3 different surfaces: tile, ramp, and carpet. Data were collected in accordance with the SCP, but with additional trials for each surface. The current SCP does not specify the number of trials that should be collected. However, a total of 5 trials were collected for each surface in order to (1) evaluate the intrasession reliability and (2) evaluate the effect of averaging multiple trials on the intersession reliability.
      As defined by the SCP for the tile surface, wheeling was done on smooth, level tile. Participants accelerated from a stationary position up to a comfortable, self-selected speed and wheeled continuously for 10 seconds or 10 meters, whichever occurred first. Collection started as soon as the SmartWheel pushrim was contacted and participants were told to keep wheeling at a normal speed until instructed to stop. The SCP was administered for 5 consecutive trials. The ramp and carpet protocols followed the same procedure as the tile protocol, but with a 5% grade ramp and a low-pile carpet, respectively.

      Measures

      Four main measurements were computed with the SmartWheel clinical software for each of the 3 surfaces: average peak force (N/kg), average push frequency (Hz), average push length (°), and average velocity (m/s). These are the 12 SCP parameters currently used by clinicians to assess their patients. Calculations for each parameter were based on the data collected from all pushes except for the first 3. This was done to capture the participants' propulsion when they are assumed to have completed the bulk of their acceleration.
      • Cowan R.E.
      • Boninger M.L.
      • Sawatzky B.J.
      • Mazoyer B.D.
      • Cooper R.A.
      Preliminary outcomes of the SmartWheel User's Group database: a proposed framework for clinicians to objectively evaluate manual wheelchair propulsion.

      Statistical Analysis

      For each parameter, the intrasession reliability was calculated using intraclass correlation coefficients (ICCs): ICC2,1 (single measures) and ICC2,5 (average measures of 5 trials).
      • Weir J.P.
      Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM.
      Both were based on a 2-way (random effects) repeated-measures analysis of variance model with absolute agreement. For each session, intrasession paired t tests were calculated between the first and fifth trials for each parameter in order to detect potential learning effects. The alpha level of .05 was adjusted using a Bonferroni correction for 12 comparisons.
      The intersession reliability was calculated using ICC2,1 with the first trial of each session for each parameter. In addition, to ensure that stable measures were obtained for each wheeling parameter, the mean of 5 trials was also calculated for each parameter of each session. ICC2,1 for intersession reliability was then calculated using the mean parameter values of each session. Although there are many ways to interpret ICC values, we chose Munro's classification of reliability coefficients: .26 to .49 reflects low correlation; .50 to .69 reflects moderate correlation; .70 to .89 reflects high correlation; and .90 to 1.00 indicates very high correlation.
      • Munro B.H.
      Statistical methods for health care research.
      Standard error of measurement was calculated to assess the measurement error across sessions because of variability. Standard error of measurement is an index of absolute reliability; therefore, it estimates the precision of individual scores.
      • Weir J.P.
      Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM.
      The standard error of measurement was estimated using the mean squared error (MSE) from a 2-way analysis of variance, where standard error of measurement = √MSE. This formulation has the advantage of being independent of the ICC values, and therefore is unaffected by the range of measurement values.
      • Weir J.P.
      Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM.
      To estimate the minimal amount of change needed to exceed the measurement error, the MDC for each SCP parameter was estimated based on a 95% confidence interval, where MDC = standard error of measurement × √2 × 1.96.
      • Weir J.P.
      Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM.
      Bland-Altman plots were constructed for each parameter by plotting the mean difference between session measurements (using the mean values of 5 trials) against the mean of the 2 test sessions for each participant.
      • Bland J.M.
      • Altman D.G.
      Statistical methods for assessing agreement between two methods of clinical measurement.
      Ninety-five percent confidence intervals (limits of agreement) were calculated to determine whether there were any systematic biases between sessions. All analyses were done using SPSSc and Microsoft Excel.d

      Results

      WCU Group

      A total of 7 men and 3 women WCUs were recruited with a mean age of 36.2±12.3 years. Eight of the participants were persons with paraplegia and 2 were persons with tetraplegia. Wheeling experience ranged from 1 to 32 years, with a mean of 16.5 years.
      Table 1 shows the means (across all trials) and SDs of all parameters for each session. ICC values for intra- and intersession reliability are shown in table 2. For WCUs, intrasession correlation was high for 7 of 12 SCP parameters and very high for the other 5 parameters. As expected, using ICC2,5 produced higher ratings; all parameters indicated very high correlation. Paired t tests showed no significant differences between the first and fifth trial for all parameters.
      Table 1Means ± SDs for SCP Parameters
      SCP ParameterAB IndividualsWCU
      Session 1Session 2Session 1Session 2
      Tile
       Force (N/kg)8.72±2.428.48±2.897.40±2.258.58±3.23
       Frequency (Hz)0.92±0.230.95±0.181.06±0.171.10±0.20
       Push length (°)64.12±9.1768.00±12.8573.93±14.2875.35±16.24
       Velocity (m/s)1.39±0.311.45±0.281.59±0.411.70±0.50
      Ramp
       Force (N/kg)15.28±2.4814.48±2.7913.60±3.0113.97±3.26
       Frequency (Hz)0.95±0.150.96±0.171.08±0.201.09±0.20
       Push length (°)83.85±10.6786.34±11.6494.90±15.5094.95±15.93
       Velocity (m/s)1.02±0.271.03±0.301.24±0.511.28±0.51
      Carpet
       Force (N/kg)9.21±2.269.14±2.089.60±3.139.47±2.77
       Frequency (Hz)0.94±0.170.98±0.211.06±0.171.10±0.20
       Push length (°)65.06±14.3065.31±15.1278.87±16.9974.6±15.26
       Velocity (m/s)1.12±0.311.21±0.271.45±0.511.49±0.51
      NOTE. Values are mean ± SD.
      Table 2Intrasession and Intersession ICC, Standard Error of Measurement, and MDC Values for the SCP in WCU
      SCP ParameterIntrasessionIntersession (1st trials from each session)Intersession (mean of 5 trials)
      ICC2,1ICC2,5ICC2,1SEMMDCICC2,1SEMMDC
      Tile
       Force (N/kg)0.890.980.701.383.820.801.042.89
       Frequency (Hz)0.890.980.780.080.230.850.070.20
       Push length (°)0.800.950.856.4417.860.934.0411.19
       Velocity (m/s)0.930.990.760.210.590.910.120.33
      Ramp
       Force (N/kg)0.880.970.881.143.170.980.431.19
       Frequency (Hz)0.910.980.970.040.120.970.040.12
       Push length (°)0.930.980.972.647.310.991.845.09
       Velocity (m/s)0.980.990.970.060.180.990.050.15
      Carpet
       Force (N/kg)0.890.980.920.912.510.950.681.89
       Frequency (Hz)0.840.960.870.060.180.940.040.12
       Push length (°)0.800.950.739.2625.680.904.6212.79
       Velocity (m/s)0.980.990.950.100.290.980.050.15
      Abbreviation: SEM, standard error of measurement.
      For intersession reliability, based on the first trials of each session, most parameters had ICC2,1 values that reflected high correlation (see table 2). Seven of 12 SCP parameters had high correlations and 5 had very high correlations (frequency, push length, and velocity on ramp, as well as force and velocity on carpet). ICC2,1 based on the mean of 5 trials resulted in very high correlations for 10 of 12 SCP parameters, while 2 parameters had high correlations (tile force and frequency). Standard error of measurement and MDC values at the 95% confidence level are provided (see table 2). Standard error of measurement and MDC values based on the mean of 5 repeated measures were lower for all parameters compared with using only the first trials. Bland-Altman plots revealed no systematic biases between sessions, for all parameters (fig 1).
      Figure thumbnail gr1
      Fig 1Sample Bland-Altman plots for WCUs: average peak force, push length, push frequency, and velocity. Mean differences (using the mean of 5 trials) between sessions are plotted against the mean of the 2 test sessions for each participant.

      AB Group

      Ten men and 5 women were recruited, with a mean age of 30.3±9.4 years. All 15 AB individuals had no prior wheeling experience. Table 1 shows the test and retest means and SD for all parameters.
      Using ICC2,1, intrasession correlation was moderate for 4 of 12 SCP parameters; 6 of 12 parameters reflected high correlation; and 2 of 12 showed very high correlations (table 3). ICC2,5 values reflected much higher intrasession reliability: 11 of 12 SCP parameters reflected very high intrasession correlation, and 1 of 12 indicated high correlation (see table 3). Paired t tests showed no significant differences between the first and fifth trial for all parameters.
      Table 3Intra- and Intersession ICC, Standard Error of Measurement, and MDC Values for the SCP in AB Individuals
      SCP ParameterIntrasessionIntersession (first trials from each session)Intersession (mean of 5 trials)
      ICC2,1ICC2,5ICC2,1SEMMDCICC2,1SEMMDC
      Tile
       Force (N/kg)0.690.920.381.855.130.392.114.82
       Frequency (Hz)0.910.980.740.090.250.730.110.22
       Push length (°)0.670.910.2511.2931.300.537.5620.96
       Velocity (m/s)0.900.980.700.150.410.720.150.40
      Ramp
       Force (N/kg)0.670.910.272.216.140.392.065.72
       Frequency (Hz)0.760.940.280.120.340.750.080.23
       Push length (°)0.790.950.903.529.770.844.3311.99
       Velocity (m/s)0.830.960.450.220.620.660.170.48
      Carpet
       Force (N/kg)0.730.930.521.815.010.751.103.06
       Frequency (Hz)0.500.83−0.120.210.590.470.140.38
       Push length (°)0.730.930.7011.0130.510.826.3517.60
       Velocity (m/s)0.920.980.540.190.530.690.150.43
      Abbreviation: SEM, standard error of measurement.
      Based on the first trials of each session, intersession ICC2,1 indicated that 6 of 12 SCP parameters had low correlation; 2 of 12 indicated moderate correlation; 3 of 12 had high correlation; and 1 had very high correlation. ICC2,1 values based on the mean of 5 repeated measures increased reliability for most parameters. Six of 12 showed high correlation; 3 of 12 showed moderate correlation; and 3 of 12 showed low correlation.
      Standard error of measurement and MDC values at the 95% confidence level are provided in table 3. Similar to WCUs, standard error of measurement and MDC values based on the average of 5 repeated measures were lower for all SCP parameters. Bland-Altman plots revealed no systematic biases between sessions, for all parameters (see fig 1).

      Discussion

      Previous studies have examined the intra- and intersession reliability of clinical protocols and measurements in gait and manual wheeling.
      • McGinley J.
      • Baker R.
      • Wolfe R.
      • Morris M.E.
      The reliability of three-dimensional kinematic gait measurements: a systematic review.
      • Turcot K.
      • Aissaoui R.
      • Boivin K.
      • Hagemeister N.
      • Pelletier M.
      • de Guise J.A.
      Test-retest reliability and minimal detectable change determination for 3-dimensional tibial and femoral accelerations during treadmill walking in knee osteoarthritis patients.
      • Klejman S.
      • Andrysek J.
      • Dupuis A.
      • Wright V.
      Test-retest reliability of discrete gait parameters in children with cerebral palsy.
      • Cowan R.E.
      • Nash M.S.
      • de Groot S.
      • der Woude L.H.
      Adapted manual wheelchair circuit: test-retest reliability and discriminative validity in persons with spinal cord injury.

      Kirby RL, Dupuis DJ, MacPhee AH, et al. The wheelchair skills test (version 2.4): measurement properties. Arch Phys Med Rehabil 2014;85:794-804.

      • Crane B.A.
      • Holm M.B.
      • Hobson D.
      • Cooper R.A.
      • Reed M.P.
      • Stadelmeier S.
      Test-retest reliability, internal item consistency, and concurrent validity of the wheelchair seating discomfort assessment tool.
      Not all of the protocols in these studies have been found to be reliable.
      • McGinley J.
      • Baker R.
      • Wolfe R.
      • Morris M.E.
      The reliability of three-dimensional kinematic gait measurements: a systematic review.
      • Klejman S.
      • Andrysek J.
      • Dupuis A.
      • Wright V.
      Test-retest reliability of discrete gait parameters in children with cerebral palsy.
      Therefore, characterizing the reliability of protocols is well advised. Using similar methodology, to the best of our knowledge this was the first study to examine the reliability of the SCP.
      The observed ICC2,1 values in this study show excellent intrasession reliability for all SCP parameters in the WCU group. As expected, averaging 5 trials further increased the intrasession reliability for all parameters. This occurs because averaging the results of multiple trials decreases the influence of the error variance compared with the true score.
      • Bravo G.
      • Potvin L.
      Estimating the reliability of continuous measures with Cronbach's alpha or the intraclass correlation coefficient: toward the integration of two traditions.
      The significance of using averages of multiple trials is especially illustrated in the intrasession reliability for the AB group. Using ICC2,1, only 8 parameters had values that indicated high or very high intra-session reliability. When using the average of 5 trials (ICC2,5), all SCP parameters showed very high or high intrasession reliability. This highlights the importance of using multiple trials to obtain stable intrasession measures, especially in individuals with no prior wheeling experience (eg, injured individuals getting their first chair).
      In both AB individuals and WCUs, paired t tests showed no significant differences between the first and fifth trials for all parameters. This means that a learning effect or fatigue was unlikely within each session.
      WCUs showed excellent intersession reliability between the first trials of each session and between the mean of 5 trials, whereas AB individuals showed poor intersession reliability when comparing just the first trials of each session. This was expected because of the large intrasession variability in AB individuals, as indicated by the low intrasession ICC2,1 values when averages of trials were not used. By taking the mean of 5 trials, ICC2,1 values for intersession reliability increased. For AB individuals, however, ICC2,1 based on mean session values for push force and push length on tile, force on ramp, and frequency on carpet still remained low, with unacceptable values that indicated low intersession reliability. As shown by the Bland-Altman plots, no learning effects or systematic differences were observed between sessions for either WCUs or AB individuals when the means of 5 trials were used.
      Differences in wheeling variability between AB individuals and WCUs illustrate the importance of manual wheeling experience. The effect of practice has been shown to improve propulsion technique and coordination.
      • Lenton J.P.
      • Fowler N.E.
      • van der Woude L.H.
      • Goosey-Tolfrey V.L.
      Wheelchair propulsion: effects of experience and push strategy on efficiency and perceived exertion.
      However, for individuals with no prior wheeling experience, de Groot et al
      • De Groot S.
      • Veeger H.E.
      • Hollander A.P.
      • van der Woude L.H.
      Adaptations in physiology and propulsion techniques during the initial phase of learning manual wheelchair propulsion.
      showed that a 12-minute practice session is inadequate for familiarization and developing consistency in timing variables such as push frequency. They observed no improvements in interstroke consistency after a 12-minute practice period or during 3 weeks of practice.
      • De Groot S.
      • Veeger H.E.
      • Hollander P.A.
      • Van der Woude L.H.
      Wheelchair propulsion technique and mechanical efficiency after 3 wk of practice.
      • De Groot S.
      • Veeger H.E.
      • Hollander A.P.
      • van der Woude L.H.
      Adaptations in physiology and propulsion techniques during the initial phase of learning manual wheelchair propulsion.
      This could be indicative of inexperienced users exploring various propulsion strategies because of the novel task.
      • De Groot S.
      • Veeger H.E.
      • Hollander P.A.
      • Van der Woude L.H.
      Wheelchair propulsion technique and mechanical efficiency after 3 wk of practice.
      • De Groot S.
      • Veeger H.E.
      • Hollander A.P.
      • van der Woude L.H.
      Adaptations in physiology and propulsion techniques during the initial phase of learning manual wheelchair propulsion.
      In our study a typical familiarization period ranged from 10 to 15 minutes, similar to the de Groot study.
      • De Groot S.
      • Veeger H.E.
      • Hollander A.P.
      • van der Woude L.H.
      Adaptations in physiology and propulsion techniques during the initial phase of learning manual wheelchair propulsion.
      This shows the importance of sufficient wheeling experience in order to decrease variability in wheeling and may partly explain the large variability observed in AB individuals compared with WCUs in the current study.
      In general, WCUs showed higher intra- and intersession reliability compared with AB individuals. For WCUs, if there is only time to do 1 trial per surface within a given clinical situation, the ICC values for intra- and intersession reliability showed acceptable values for all parameters of the SCP using only the first trial of each session. Thus, it may be clinically acceptable to conduct only 1 trial per surface for experienced WCUs. However, for AB individuals, averaging the 5 trials of each session is needed to increase both the intra- and intersession reliability of the SCP to a more acceptable level. Therefore, for inexperienced or new WCUs, averaging multiple trials is highly recommended. For the parameters that still indicate poor reliability despite using means, MDC values may provide a more helpful guide for clinical judgments.
      ICC is a measure of relative reliability. Therefore, low levels of intersubject variability will lower ICC values even if differences between test sessions are small.
      • Munro B.H.
      Statistical methods for health care research.
      Unlike ICCs, standard error of measurement and MDC are measures of absolute reliability. MDC values could be more useful to clinicians, because they are quantified in the same units as the parameters of interest.
      • Munro B.H.
      Statistical methods for health care research.
      MDC values are especially useful in determining whether changes in values for wheeling parameters are indicative of real change and not measurement error. The MDC values of this study could allow clinicians to estimate with 95% confidence the minimal amount of change in patient status that can be assessed confidently between test sessions. For instance, if a WCU undergoes propulsion training and increases his push length on tile by 12° (as measured by the SCP), based on an estimated MDC of 11.19° (see table 2), we could consider with 95% confidence that the change is because of the intervention and not because of random variation in measurement. This is because the observed change of 12° exceeds the estimated MDC of 11.19°.
      In both WCUs and AB individuals, the responsiveness of the SCP is increased (lower MDC) when the mean of 5 trials is used. The AB group showed higher MDC values for all parameters. Therefore, if our AB group were to reflect newly injured individuals who are prescribed with wheelchairs for the first time, the threshold for real change that can be assessed with 95% confidence would be higher than in individuals with manual wheeling experience. Again, this highlights the benefits (especially in individuals with minimal experience) of averaging multiple trials to increase the intersession reliability of the SCP.

      Study Limitations

      The most significant limitation for this study was the use of AB individuals to represent naïve, newly injured WCUs. This comparison may not be entirely valid because of differences in motor function (ie, trunk control). It would have been difficult to include naïve WCUs with acute injuries, because patients during this stage are in the midst of rehabilitation. However, we and other authors believe that AB individuals naïve to manual wheeling and newly injured WCUs share similar aspects of inexperienced wheeling.
      • De Groot S.
      • Veeger H.E.
      • Hollander A.P.
      • van der Woude L.H.
      Short-term adaptations in co-ordination during the initial phase of learning manual wheelchair propulsion.
      • Van Den Berg R.
      • De Groot S.
      • Swart K.M.
      • van der Woude L.H.
      Physical capacity after 7-weeks of low-intensity wheelchair training.
      For future directions, the SCP may be modified to include the collection of multiple trials. The optimal number of trials to collect should also be determined. The current SmartWheel clinical software does not include a built-in function to allow for the averaging of multiple trials. Future software updates could address this.

      Conclusions

      In this study, the test-retest reliability was quantified for each parameter of the SCP. In general, the SCP showed excellent intra- and intersession reliability in individuals with prior wheelchair experience (at least 1y), even when only the first trial is used from a session. However, for individuals with no wheeling experience, collecting and averaging multiple trials is strongly recommended in order to increase the reliability of the SCP. MDC estimates were presented to provide insight into the sensitivity of the SCP in detecting statistically significant changes in wheelchair propulsion.
      • a
        Three Rivers Holdings, 1826 W Broadway Rd, Ste 43, Mesa, AZ 85202.
      • b
        Instinct Mobility, 818 W 10th Ave, Vancouver, BC, V5Z 1M9 Canada.
      • c
        Version 16.1; SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL 60606.
      • d
        Microsoft, One Microsoft Way, Redmond, WA 98052.

      Acknowledgment

      We thank Rachel E. Cowan, PhD, for providing valuable comments on the manuscript.

      References

        • Silfverskiold J.
        • Waters R.L.
        Shoulder pain and functional disability in spinal cord injury patients.
        Clin Orthop Relat Res. 1991; : 141-145
        • Boninger M.L.
        • Souza L.S.
        • Cooper R.A.
        • Fitzgerald S.G.
        • Koontz A.M.
        • Fay B.T.
        Propulsion patterns and pushrim biomechanics in manual wheelchair propulsion.
        Arch Phys Med Rehabil. 2002; 83: 718-723
        • Mercer J.L.
        • Boninger M.
        • Koontz A.M.
        • Ren D.
        • Dyson-Hudson T.
        • Cooper R.
        Shoulder joint kinetics and pathology in manual wheelchair users.
        Clin Biomech. 2006; 21: 781-789
        • Shimada S.D.
        • Robertson R.N.
        • Boninger M.L.
        • Cooper R.A.
        Kinematic characterization of wheelchair propulsion.
        J Rehabil Res Dev. 1998; 35: 210-218
        • Richter W.M.
        • Rodriguez R.
        • Woods K.R.
        • Axelson P.W.
        Stroke pattern and handrim biomechanics for level and uphill wheelchair propulsion at self-selected speeds.
        Arch Phys Med Rehabil. 2007; 88: 81-87
        • Requejo P.S.
        • Mulroy S.J.
        • Haubert L.L.
        • Newsam C.J.
        • Gronley J.K.
        • Perry J.
        Evidence-based strategies to preserve shoulder function in manual wheelchair users with spinal cord injury.
        Top Spinal Cord Inj. 2008; 13: 86-119
        • Cooper R.A.
        Smartwheel: from concept to clinical practice.
        Prosthet Orthot Int. 2009; 33: 198-209
        • Cowan R.E.
        • Boninger M.L.
        • Sawatzky B.J.
        • Mazoyer B.D.
        • Cooper R.A.
        Preliminary outcomes of the SmartWheel User's Group database: a proposed framework for clinicians to objectively evaluate manual wheelchair propulsion.
        Arch Phys Med Rehabil. 2008; 89: 260-268
        • Consortium for Spinal Cord Medicine
        Preservation of upper limb function following spinal cord injury: a clinical practice guideline for health-care professionals.
        Paralyzed Veterans of America, Washington (DC)2005
        • Lexell J.E.
        • Downham D.Y.
        How to assess the reliability of measurements in rehabilitation.
        Am J Phys Med Rehabil. 2005; 84: 719-723
        • Dawis R.V.
        Scale construction.
        J Couns Psychol. 1987; 34: 481-489
        • Brown D.D.
        • Knowlton J.H.
        • Schneider T.L.
        • Hetzler R.K.
        Physiological and biomechanical differences between wheelchair-dependent and able-bodied subjects during wheelchair ergometry.
        Eur J Appl Physiol. 1990; 60: 179-182
        • Robertson R.N.
        • Boninger M.L.
        • Cooper R.A.
        • Shimada S.D.
        Pushrim forces and joint kinetics during wheelchair propulsion.
        Arch Phys Med Rehabil. 1996; 77: 856-864
        • De Groot S.
        • Veeger H.E.
        • Hollander P.A.
        • Van der Woude L.H.
        Wheelchair propulsion technique and mechanical efficiency after 3 wk of practice.
        Med Sci Sport Exer. 2002; 34: 756-766
        • De Groot S.
        • Veeger H.E.
        • Hollander A.P.
        • van der Woude L.H.
        Adaptations in physiology and propulsion techniques during the initial phase of learning manual wheelchair propulsion.
        Am J Phys Med Rehabil. 2003; 82: 504-510
        • Walter S.D.
        • Eliasziw M.
        • Donner A.
        Sample size and optimal designs for reliability studies.
        Statis Med. 1998; 17: 101-110
        • Weir J.P.
        Quantifying test-retest reliability using the intraclass correlation coefficient and the SEM.
        J Strength Cond Res. 2005; 19: 231-240
        • Munro B.H.
        Statistical methods for health care research.
        Lippincott Williams & Wilkins, Philadelphia1986
        • Bland J.M.
        • Altman D.G.
        Statistical methods for assessing agreement between two methods of clinical measurement.
        Lancet. 1986; 1: 307-310
        • McGinley J.
        • Baker R.
        • Wolfe R.
        • Morris M.E.
        The reliability of three-dimensional kinematic gait measurements: a systematic review.
        Gait Posture. 2009; 29: 360-369
        • Turcot K.
        • Aissaoui R.
        • Boivin K.
        • Hagemeister N.
        • Pelletier M.
        • de Guise J.A.
        Test-retest reliability and minimal detectable change determination for 3-dimensional tibial and femoral accelerations during treadmill walking in knee osteoarthritis patients.
        Arch Phys Med Rehabil. 2008; 89: 732-737
        • Klejman S.
        • Andrysek J.
        • Dupuis A.
        • Wright V.
        Test-retest reliability of discrete gait parameters in children with cerebral palsy.
        Arch Phys Med Rehabil. 2010; 91: 781-787
        • Cowan R.E.
        • Nash M.S.
        • de Groot S.
        • der Woude L.H.
        Adapted manual wheelchair circuit: test-retest reliability and discriminative validity in persons with spinal cord injury.
        Arch Phys Med Rehabil. 2011; 92: 1270-1280
      1. Kirby RL, Dupuis DJ, MacPhee AH, et al. The wheelchair skills test (version 2.4): measurement properties. Arch Phys Med Rehabil 2014;85:794-804.

        • Crane B.A.
        • Holm M.B.
        • Hobson D.
        • Cooper R.A.
        • Reed M.P.
        • Stadelmeier S.
        Test-retest reliability, internal item consistency, and concurrent validity of the wheelchair seating discomfort assessment tool.
        Assist Technol. 2005; 17: 98-107
        • Bravo G.
        • Potvin L.
        Estimating the reliability of continuous measures with Cronbach's alpha or the intraclass correlation coefficient: toward the integration of two traditions.
        J Clin Epidemiol. 1991; 44: 381-390
        • Lenton J.P.
        • Fowler N.E.
        • van der Woude L.H.
        • Goosey-Tolfrey V.L.
        Wheelchair propulsion: effects of experience and push strategy on efficiency and perceived exertion.
        Appl Physiol Nutr Metab. 2008; 33: 870-879
        • De Groot S.
        • Veeger H.E.
        • Hollander A.P.
        • van der Woude L.H.
        Short-term adaptations in co-ordination during the initial phase of learning manual wheelchair propulsion.
        J Electromyogr Kines. 2003; 13: 217-228
        • Van Den Berg R.
        • De Groot S.
        • Swart K.M.
        • van der Woude L.H.
        Physical capacity after 7-weeks of low-intensity wheelchair training.
        Disabil Rehabil. 2010; 32: 1713-1721