Validity and Reliability of the Modified Shuttle Walk Test in Patients With Chronic Obstructive Pulmonary Disease

CHRONIC OBSTRUCTIVE Pulmonary Disease (COPD) is a progressive condition characterized by chronic airflow limitation that is not fully reversible and is associated with an inflammatory process in the lungs.1 The worldwide prevalence of COPD in 1990 was estimated to be 9.34 in 1000 for men and 7.33 in 1000 for women,1 with morbidity increasing with age. COPD is presently the fourth leading cause of death in the world,2 and the burden of the disease is on the rise. In the United States in 2002, direct and indirect costs attributed to COPD were $18.0 and $14.1 billion, respectively.3 More specifically, in the United States, 12.1 million adults aged 25 years and older were diagnosed with COPD in 2001, whereas epidemiologic evidence suggests that approximately 24 million adults actually have COPD, rendering the disease underdiagnosed.3

Deconditioning arises with chronic dyspnea because of decreased exercise tolerance and sedentarism,1 and the reverse is also true. Clinical field tests of walking endurance are therefore of utmost importance in quantifying disability with this clientele. A variety of submaximal exercise walking tests that measure distance exist to predict maximum oxygen consumption (V̇o2max), and hence maximum aerobic power,4 such as the 2- (2MWT), 4- (4MWT), 6- (6MWT), and 12-minute walk tests (12MWT) and the shuttle walk test, to name a few. These tests are valuable to help diagnose exercise intolerance, assess functional limitations, evaluate the outcomes of exercise programs, measure the effects of pharmacologic agents, and assess the recovery strategies on exercise performance.5, 6, 7, 8 To increase the validity and reliability of the data obtained, clinicians are advised to follow the prescribed methods of administration to insure accurate test performance and to use a number of physiologic measures such as heart rate, blood pressure, rate of perceived exertion, pain, and dyspnea to safely monitor patients.4 Clinical submaximal exercise tests have long been favored over maximal exercise tests because some patients are unable to attain V̇o2max without fatiguing first or being limited by musculoskeletal, cardiopulmonary, or neuromuscular impairments.4 Also, clinical submaximal tests have as advantages that they do not require the physician’s presence, are more comfortable to undergo, are easier to administer (because little equipment is required), and more closely resemble exercise training conditions.

One of the most commonly used submaximal exercise tests is to determine a COPD patient’s functional capacity9 is the 12MWT. This test evolved from the 12-minute running test designed by Cooper10 in 1968 and was adapted by McGavin et al11 to a walking test as a means of estimating exercise tolerance in patients with chronic bronchitis. A previous study9 showed that changes in V̇o2 per kilogram were more closely associated with changes measured by the 12MWT compared with changes in the 6MWT, 4MWT, or 2MWT, suggesting that the 12MWT is more accurate in detecting changes in exercise capabilities in patients with COPD. Currently, outside a research protocol, the 6MWT is more commonly used in clinical practice because this measure was used to test many groups, such as people with COPD, heart failure, fibromyalgia, and musculoskeletal and neurologic impairments.12 It was also found to be another highly valid and reliable measure of functional capacity across patient populations.12

Recent literature has now focused on a new submaximal walking test for patients with COPD, the modified shuttle walk test (MSWT). The MSWT was derived from the 20-m shuttle walk test (20SWT). The 20SWT was designed to assess maximum aerobic power in children, adults attending fitness classes, and athletes participating in sports requiring constant starting and stopping.4 This test is unique because it paces the person with the use of sound signals on a prerecorded audiocassette tape and is thus more objective than the traditional informal tests described previously.10 The 20SWT, however, is not appropriate for the elderly and for those with physical disability because it requires patients to run over a 20-m course for a period of time (walking not allowed). This would clearly not be a reasonable request for the elderly or for those with physical disabilities. The 12-level MSWT is an endurance walking test of disability in patients with chronic airway obstruction that was developed by Singh et al.13 This test requires patients to either walk or run (as able) as fast as they can and is conducted over a shorter 10-m course. This 15-level MSWT was developed and tested in adult cystic fibrosis (CF) patients14 and in those with chronic airway obstruction.13 Results indicated that the MSWT is highly reliable, repeatable, and sensitive in those with adult CF.15 Although the 20SWT has been tested in patients with COPD,16, 17, 18 the validity and reliability of the MSWT was not evaluated in patients of varied COPD disease severity. The MSWT is externally paced with the audiocassette and possibly more reproducible,13, 17 whereas the 6MWT and 12MWT are internally paced. Also, the MSWT is a progressive intensity field test that “eliminates the variable effects of motivation and encouragement seen in self-paced tests [such as the 6MWT and 12MWT], and thus more closely approximates the test protocols used in laboratory measures of V̇o2max than do standard walking tests.”12, 13, 17 Because of the MSWT’s more objective approach, it is anticipated to obtain a more accurate estimate of maximal exercise capacity compared with the 6MWT and 12MWT.

The primary objectives of this study were (1) to examine the concurrent criterion validity of the MSWT (number of meters walked) with the number of meters walked on the 6MWT and 12MWT; (2) to examine the concurrent criterion validity of the MSWT (estimated V̇o2max calculated by using a validated regression equation) with the actual V̇o2max score obtained on the Jones stage 1 test; and (3) to determine the test-retest reliability of the MSWT in patients with COPD. The secondary objective of the study was to determine the validity and reliability of the MSWT for subgroups of subjects with COPD (severity of COPD, age, and sex).

Methods 

Participants 

Adult subjects aged 50 to 85 years were recruited prospectively from a rehabilitation hospital outpatient COPD program. The admissibility criteria included clinically stable (<10% variation in forced expiratory volume in 1 second [FEV1] from the best test during the past 6 months, with no overriding comorbidities such as severe coronary artery disease, osteoarthritis, or other neuromuscular impairments) and the ability to come to the hospital for 2 testing sessions. Subjects were monitored for their health status throughout the study by using the Health Questionnaire to ensure continued admissibility.

Procedures 

This was a prospective cross-sectional study that was conducted between January 2003 and June 2004. The study was approved by the institution’s research ethics board. COPD outpatients who embarked on a 10-week rehabilitation program at a rehabilitation hospital were screened by the program nurse for admissibility, and potential subjects received an information letter at the preadmission clinic. Consent was signed in the nurse’s presence, and no incentive was given for participating. To eliminate the effect of order, subjects were randomly assigned to receive either the 6MWT and 12MWT or the MSWT first (both done on the same day) by using prepared envelopes. An inhalation therapist performed the spirometric measures of lung function, and a physiotherapist administered the 6MWT and 12MWT as well as the MSWT at the same time of day during regularly scheduled treatment times. Study subjects performed these tests in the place of a regular treatment session to avoid excessive fatigue. The spirometric tests (FEV1) were performed in the pulmonary rehabilitation department, whereas the walking tests were performed in the corridor of a low-traffic area of the hospital. Both the MSWT and the 6MWT and 12MWT were administered according to recommended guidelines.12, 17, 19

For the 6MWT, subjects were instructed to walk end to end of a 20-m course in a quiet corridor, covering as much ground as possible for 6 minutes. Standardized encouragement12 was given throughout the test. They were told that they may rest if they were too short of breath or too tired to continue but could resume walking when they were able. At the end of the 6-minute duration, subjects were told to stop. The distance walked and the number of rests were noted.12 For the 12MWT, the same protocol described for the 6MWT was used, except that the test was of a 12-minute duration.12 The MSWT was administered by performing a practice test and 1 actual trial. For both the 6MWT and the 12MWT, subjects underwent 1 practice test and 2 trials separated by a 10-minute rest period. The final score for each (6MWT, 12MWT) was the average of the 2 trials. Test-retest reliability data were obtained by repeating the MSWT 1 week after the initial test. Again, 1 practice test and 1 actual test were administered.12, 13 Before testing, subjects were asked to complete the Health Questionnaire to ensure that their clinical status had not changed, and, if stable, the spirometric test was repeated. Many subjects were tested during the same time period such that it was highly unlikely that the evaluators could remember the result of each subject’s performance on the various tests.

Actual V̇o2max values were obtained directly from the Jones stage 1 bicycle exercise test, which was conducted by an inhalation therapist and supervised by a physician. The estimated V̇o2max results obtained from the MSWT were compared with the V̇o2max obtained through the Jones test. The interval between them was no more than 3 weeks to minimize the influence of a true change in clinical status on the comparison of scores. The physiotherapist administering the MSWT was unaware of the subjects’ V̇o2max scores and 6MWT and 12MWT results and the findings from other clinical tests.

FEV1 was measured to classify the patients’ COPD severity (mild, moderate, severe, very severe) in accordance with the Global Initiative for Lung Disease,1 which was performed by the pneumologist at preadmission clinic. In addition, clinical information was collected by using the following measures: the Borg rating of perceived exertion (RPE) scale,20 a measure of perceived exertion; respiratory rate; oxygen saturation (percentage); pulse; and blood pressure. These tests were used to determine the subjects’ level of safety throughout the administration of the testing as well as to describe the effects of the testing procedures on the subjects’ clinical status.

Measures 

Modified shuttle walk test 

The 12-level MSWT requires that patients walk or run depending on their capacity, at increasing speeds back and forth on a 10-m course, demarcated by 2 cones at both ends; this constitutes 1 shuttle. The patient is accompanied by the physiotherapist during the first minute of the test to help him/her pace himself/herself with the audio signal. At the end of each level, the patient is offered a standardized verbal encouragement (“good, keep going, you are doing well”).17 They are reminded to go a little faster at each shuttle/level and that they are permitted to run at any time during the test (only if able to do so). He/she continues until they can no longer or until he/she fails to maintain the set pace,13, 17 meaning that they could not maintain the required speed or failed to complete a shuttle in the time allowed (being 0.5m away from the cone when the beep sounds).

In this study, the number of completed shuttles and the final distance walked were noted by the physiotherapist. Heart rate and oxygen saturation were measured at 15-second intervals by using an oxygen saturation and heart rate monitor. Additionally, pre- and posttest measures of peak heart rate, blood pressure, oxygen saturation, and Borg RPE score were logged. At the end of the test, the physiotherapist recorded the cause of exercise limitation: extreme dyspnea, an inability to maintain the required speed or whether greater than 85% of maximum heart rate was attained.

MSWT estimated V̇o2max 

A validated regression equation was used to calculate the estimated V̇o2max from the distance walked17: V̇o2max=4.19+0.25 (distance), where V̇o2max is in mL/kg and distance is in meters.

Six- and 12-minute walk tests 

These are validated and reliable performance-based tests in which distance walked over 6 and 12 minutes is measured in meters.12 Greater distances indicate better performance, and the tests were performed in accordance with standardized procedure, as described previously.12 The 6MWT and 12MWT have been shown to have excellent intrarater reliability (r=.97)21 and construct validity (r=.897).10 It is important to note that the 6MWT and 12MWT do not include a measure of V̇o2max.

Jones test 

The purpose of the Jones stage 1 test is to identify and define the medical limits to exercise on exertion on a stationary bicycle or treadmill. It is a maximal incremental exercise test that is performed on a bicycle ergometer according to standardized procedure.22, 23, 24 This test is normally performed with COPD patients until clinical exhaustion (symptom limited) or until 85% to 90% of maximum heart rate is obtained, whichever comes first. The test is stopped when 85% to 90% of maximum heart rate is obtained for safety reasons, given age and cardiovascular considerations. The V̇o2max value obtained is clinically considered to be the patient’s maximum exercise capacity.

For this study, this test was performed on a stationary bicycle in a nearby acute care hospital’s laboratory under medical supervision to measure V̇o2max and to determine physical fitness as a standard of comparison with other measures.24, 25 Workload in the form of graded resistance was begun at 10W, and increased by 10W/min until the patient could no longer pedal or until 85% to 90% of maximum heart rate was obtained; a maximal test was deemed to occur at maximal clinical exhaustion or when V̇o2 plateaued despite further increases in workload.4, 23, 24 At the end of the test, the origin of limitation was identified by the physician, and, if necessary, recommendations to exercise training were given (eg, monitoring of blood pressure if hypertension on exertion was detected or considering adding oxygen during training).

Potential confounding variables 

Sex and age in years were recorded.

Health outcome measures 

The following health outcomes were measured: Borg RPE score, FEV1, respiratory rate, oxygen saturation (percentage), pulse, and blood pressure. The Borg RPE scale measures the level of perceived exertion by using a 10-point scale,20 whereas FEV1 was measured using standard spirometry testing.1 The clinical spirometer used was the Microlab 3500US.a The respiratory rate was measured through clinical observation (measured in breaths/min). Oxygen saturation and pulse were recorded by using the miniSpO2T digital pulse oximeter,b which was placed in an adjustable waist belt to allow monitoring at a distance. Blood pressure was measured by using a noninvasive vital signs monitor for blood pressure.c

Analysis 

Data analyses were performed by using SASd software. Descriptive statistics were presented for the group as a whole as well as by COPD severity (mild and moderate vs severe and very severe).1

Pearson product-moment correlations were used to examine the concurrent validity of the MSWT with the 6MWT and 12MWT as well as to examine the relation between the actual V̇o2max measured by the Jones test and the estimated V̇o2max (measured by using the regression equation from the MSWT). Intraclass correlation coefficients (ICCs) were calculated to assess the test-retest reliability of the MSWT.

Results 

Thirty-eight subjects were approached to participate in the study. Four refused to participate. Of the 34 participants, 4 were unable to complete the second evaluation because of COPD exacerbation, as identified by the Health Questionnaire. Thirty subjects completed the study. Their mean age ± standard deviation (SD) was 68.6±7.2 years: 18 were men and 12 were women. Specifically, 9 men and 5 women had mild or moderate COPD, whereas 9 men and 7 women had severe COPD. The subjects’ health status according to severity of COPD is summarized in table 1.

Table 1. Health Status at Time of Testing

	Variable	All (N=30)	Mild and Moderate COPD (n=14)	Severe and Very Severe COPD (n=16)
	FEV1 (L)	1.0±0.5	1.3±0.5	0.8±0.2
	V̇o2max (Jones test) (mL/kg)	12.5±3.9	14.4±4.3	10.9±2.8
	V̇o2max (estimated) (mL/kg)	12.2±3.7	14.5±3.4	10.2±2.7
	Oxygen saturation
	Before testing	94.9±2.4	95.6±2.2	94.4±2.4
	After testing	87.1±5.3	88.0±5.9	86.4±4.9

NOTE. Values are mean ± SD.

The walked distances for our study’s sample in terms of mean ± SD were as follows: for the MSWT, 321.3±148.0m; for the 6MWT, 303.6±99.0m; and for the 12MWT, 566.2±196.3m. There was a moderately high correlation between the MSWT and the 6MWT and 12MWT at initial testing (r=.82 and r=.74) (table 2). The correlation between the estimated V̇o2max (calculated by using the MSWT regression equation) and actual V̇o2max was r equal to .68 (table 3). Results are also presented according to disease severity, sex, and age categories (see Table 2, Table 3).

Table 2. Concurrent Validity: Endurance

	Group	MSWT (initial) vs 6MWT		MSWT (initial) vs 12MWT
		Pearson r	95% CI	Pearson r	95% CI
	Total (N=30)	.82	.65–.91	.74	.52–.87
	COPD staging
	Mild and moderate (n=14)	.75	.36–.92	.64	.17–.87
	Severe and very severe (n=16)	.77	.44–.92	.78	.46–.92
	Sex
	Men (n=18)	.84	.61–.94	.89	.72–.96
	Women (n=12)	.77	.35–.93	.51	−.9 to .84
	Age
	<70 (n=17)	.78	.48–.92	.59	.15–.83
	≥70 (n=13)	.84	.54–.95	.9	.69–.97

Abbreviation: CI, confidence interval.

Table 3. Concurrent Validity: V̇o2max

	Group	Actual V̇o2max vs Estimated V̇o2max
		Pearson r	95% CI
	Total (N=30)	.68	.42–.84
	COPD staging
	Mild and moderate (n=14)	.60	.10–.86
	Severe and very severe (n=16)	.54	.06–.82
	Sex
	Men (n=18)	.69	.33–.88
	Women (n=12)	.74	.29–.92
	Age
	<70 (n=17)	.73	.38–.90
	≥70 (n=13)	.67	.19–.89

Test-retest reliability of the MSWT was high for the group as a whole. For the severity and sex subgroups, reliability coefficients remained high. Results by age indicate that the ICC was higher for subjects aged 70 years or higher compared with those aged younger than 70 years (table 4).

Table 4. Test-Retest Reliability

	Group	ICC	95% CI
	Total (N=30)	.88	.83–.92
	COPD staging
	Mild and moderate (n=14)	.84	.73–.91
	Severe (n=16)	.80	.68–.88
	Sex
	Men (n=18)	.88	.81–.93
	Women (n=12)	.88	.79–.93
	Age
	<70 (n=17)	.93	.89–.96
	≥70 (n=13)	.82	.69–.90

Discussion 

This study examined the validity and reliability of the MSWT in patients with COPD by comparing results with valid normative submaximal tests such as the 6MWT and 12MWT and the maximal Jones stage 1 exercise test. Results indicate that validity is moderately high for endurance and moderate for V̇o2max reliability is high, suggesting that this clinical tool can be used as an alternate method of testing COPD clients to measure endurance and estimate V̇o2max. Because the MSWT is externally paced, it is a more standardized measure and therefore may be more representative of the patients’ maximal capacity and level of physical fitness. This information is important in the research setting and enables clinicians to more clearly define exercise capacity and gain obtained from exercise training.

Overall, for the MSWT and 6MWT or 12MWT, moderate to high correlation coefficients were noted for the group as a whole as well as for the severity, age, and sex subgroups. The associations between the distance walked for the MSWT and the 6MWT were quite consistent across all subgroups, although variations occurred for the 12MWT. Correlations were better for the severe group, the men, and for those 70 years or older. However, the small sample size and the wide 95% confidence interval for the subgroups indicate an uncertainty of the actual correlation coefficient and therefore prevents making any definitive conclusions.

A second outcome measure used to explore concurrent validity issues was the V̇o2max score. The results indicate a moderate correlation between the actual V̇o2max (Jones test) and the regression-based V̇o2max score calculated by the MSWT predictive V̇o2max equation. Although both tests are standardized measures of the same parameter and are externally paced, differences in the mode of administration may explain these findings.

The validated V̇o2max equation was created by testing patients with chronic airflow limitation on a treadmill,17 whereas the Jones stage 1 test was performed on a bicycle. The Jones stage 1 test is a laboratory test that requires sophisticated monitoring equipment that is uncomfortable to patients (eg, mouthpiece, nose clip, blood pressure cuff, bicycle) and measures submaximal to maximal exercise capacity aiming for a safe 85% to 90% maximum predicted heart rate. The MSWT is also a submaximal test that measures V̇o2max in a clinical setting that more closely resembles daily functioning and is easier to tolerate. Because of scheduling difficulties, subjects were tested on these 2 measures within 3 weeks. Although we did not include patients who had a major change in their medical status over that short time, more subtle variations may have occurred and influenced the results.

Results were fairly consistent across subgroups; however, the findings for the severe COPD group were somewhat lower, possibly reflecting their inconsistent performance over time. It is known that there is an intraindividual variation in V̇o2max (Jones test) of 4% to 6% for people with no cardiopulmonary pathology and a day-to-day variation of 6% to 10% for the COPD population.1, 4 Factors such as weather conditions, perceived level of fatigue or dyspnea, pain, and level of motivation may influence performance. Also, it has been noted in previous studies that few people actually attain their true V̇o2max.4, 12, 26 An attempt was made to control for unstable medical conditions through the exclusion criteria and the Health Questionnaire, but given the complex medical background of this clientele, it is likely that some of these factors may have affected the variability in performance over the 2 evaluation times.

Finally, results of the reliability analyses (ICC) between initial and retest MSWT endurance scores were overall consistent and promising. The ICC for the total sample was estimated at .88. Given the natural variability of this population, the high test-retest reliability indicates that the MSWT may be used as a reliable indicator of endurance. Even for those with severe COPD and older subjects, the results remained quite stable over time.

Study Limitations 

There are several limitations to this preliminary study. The sample is small, specifically for the subgroup analyses. Also, scheduling difficulties created a window of 3 weeks between initial testing and the Jones test administration at another hospital site. During this time, subtle changes in medical status, weather, mood, and motivation could have affected performance, possibly reducing the correlation between them. Furthermore, although the Jones test was performed in accordance with medical protocol, it did not yield the absolute V̇o2max but rather was stopped at 85% to 90% of estimated maximal heart rate (calculated by using the following formula: HRmax=210−[.65×age]; secure zone=85%×HRmax)24 because of age and cardiovascular considerations.

Conclusions 

Our study showed that the MSWT has moderate to excellent validity and reliability as an endurance walking test for patients with COPD. This externally paced test of endurance and V̇o2max provides clinicians with an accurate, reproducible, and well-tolerated evaluation for patients with COPD. Future research is required to better understand its usefulness with patients with varying COPD severity and to determine whether evaluation results lead to a more effective rehabilitation evaluation.

Suppliers