Accuracy of Clinical Observations of Push-Off During Gait After Stroke

Methods 

All participants provided informed consent prior to participation in this study. Permission to conduct the study was gained from La Trobe University and Southern Health Ethics Committees.

Participants With Stroke 

We recruited people who had incurred a stroke from the Southern Health Rehabilitation Program. Inclusion criteria required that the participants had sustained a recent stroke resulting in gait dysfunction, and were currently undergoing rehabilitation for gait or balance disorders; and that they were capable of walking trials of 8m with close supervision, were independently ambulant prior to stroke, and were able to follow simple commands and provide informed consent. The group had a mean age ± standard deviation (SD) of 62.7±13.8 years (range, 32–76y), and had incurred strokes a mean of 94±50.5 days (range, 32–183d) prior to the first test occasion (table 1).

Table 1. Characteristics of the Participants With Stroke

	ID No.	Age (y)	Sex	Height (m)	Weight (kg)	Days Since Stroke†	Coexisting Pathology	Type of Stroke	MAS Walk	FIM Walk	FAP
	1⁎	76	M	1.70	69.4	115	No	Hem	3	5	3
	2⁎	49	M	1.67	70.0	39	No	Hem	4	5	3
	3	74	M	1.60	55.6	32	Priorstroke	Infarct	4	5	3
	4	75	M	1.70	51.0	152	MS	Infarct	4	5	3
	5⁎	61	M	1.76	80.0	77	No	Infarct	4	5	3
	6⁎	66	F	1.60	74.8	183	No	Infarct	3	5	4
	7	70	M	1.60	74.8	146	No	Infarct	4	5	4
	8⁎	32	F	1.69	103.6	111	No	Infarct	4	5	3
	9⁎	73	F	1.57	65.4	59	No	Infarct	3	5	3
	10	51	F	1.55	56.2	55	No	Hem	6	7	5
	11	64	M	1.67	66.2	64	No	Infarct	6	7	5
	Mean	62.7		1.64	69.7	94

Abbreviations: F, female; FAP, Functional Ambulation Profile23; Hem, hemorrhage; M, male; MAS, Motor Assessment Scale22; MS, multiple sclerosis.

⁎ These participants were included in the retest subgroup of 6, and were tested on 2 occasions, 3 weeks apart. † Interval between the date of the stroke and the first gait assessment (in days).

Data Analysis 

Accuracy of therapist observations (criterion-related validity) 

Key data for analysis comprised the therapists’ clinical observations of push-off and the corresponding criterion measures of ankle power generation. The raw observational scale data were converted from the two 11-point scales as previously described. Briefly, the observations on the 2 scales were considered as parts of a single 22-point continuous scale that measured push-off; extending from a 0 on the abnormal scale to 10 from the normal scale. Thus values from the 2 scales were converted to this single scale, ranging from 0 to 21. Values from the abnormal scale (0–10), remained unchanged (ie, a “2” remained a “2”); values from the normal scale were converted to values 11 to 21 (ie, a “4” became a “15”). In view of the scale interval construction, the previous exploration of scale properties14 and the high number of categories, parametric statistical methods were employed. Criterion-related validity of the observations was investigated by examining the correlation between the observers’ scores of push-off and the associated criterion measure of peak ankle power generation. A Pearson product-moment correlation was calculated to provide an index of accuracy.

The precision of the observational rating scale was examined by calculation of the standard error of estimate (SEE) of the criterion measure of peak ankle power generation24:

where SD is the standard deviation of the criterion measurements of peak ankle power generation from the 11 participants with stroke and rxy is the Pearson product-moment correlation. This quantifies the judgment error size for the therapists using the observational rating scale. These calculations are in the same units (in W/kg) as the criterion measure, thus providing a direct means to interpret the accuracy of the therapists’ use of the 22-point rating scale.

Discrimination between normal and abnormal push-off 

We investigated the discriminative judgment of normality in 2 ways. First, the observational rating scores were plotted against the criterion measures to provide a clinician judgment model. This model was explored to determine the criterion scores at which the clinicians divided the stroke participants into the abnormal or normal categories. A regression equation fitting the judgment model enabled prediction of this division or cut-off value, which corresponded to the intersection point of the abnormal and normal 0- to 10-point scales. This was determined in the manner consistent with our previous examination of observational gait data.14 A value of 10.5 was substituted into the regression equation as the x axis rating value, as a midpoint to separate the abnormal and normal range. From this, a corresponding y axis criterion value was obtained. This represented the value of the criterion measure at which the judgment model divided the stroke participants’ gait into the categories of normal and abnormal. The value obtained can be described as a “predicted threshold of normality” value.

Second, the categorization of the stroke participants as abnormal or normal and their associated criterion measures of ankle of power generation were examined relative to kinetic data from a reference group of unimpaired participants. We selected an age-, sex-, and height-matched group of 11 control participants. These unimpaired participants had previously undertaken gait analysis in the Kingston Centre Gait Laboratory using an identical gait data acquisition protocol to that used in this study. Paired t tests confirmed no significant differences in age, or height of these participants (age: t10=−.677, P>.05; height: t10=−1.754, P>.05). Peak ankle power generation values were derived from the mean of 3 trials from each of the 11 matched reference control participants. The control group walked with a mean gait speed of 1.30±0.09m/s, and mean peak ankle power generation value of 3.78±0.55W/kg (range, 2.73–4.73W/kg).

Results 

Gait Characteristics of the 11 Stroke Participants 

Gait characteristics of the stroke participants are described in Table 1, Table 2. Although all participants were able to walk independently or with close supervision, 9 of the 11 required supervision of another person to walk a distance of 50m safely, or to negotiate stairs, slopes, or uneven surfaces. Only 2 participants achieved maximal scores of 6 on the MAS walk item,22 indicating difficulty with activities such as walking quickly and turning or picking up objects from the floor. The gait laboratory criterion measures indicated that the group walked slowly (mean, .72m/s; range, 0.20–1.44m/s), with reduced stride length (mean, 0.86m) and cadence (mean, 93.8 steps/min). The speed of the criterion gait laboratory trials was closely matched to the observed clinical gait trials, with near-identical mean speeds (t10=.338, P=.742). Criterion measures of ankle power generation ranged widely from 0.02 to 4.73 (fig 1).

Table 2. Gait Ability of the Participants With Stroke

	Participant	Clinical Observations		Gait Laboratory (Criterion) Measures
		Assistance Level	Gait Speed (m/s)	Gait Speed (m/s)	Stride Length (m)	Cadence (steps/min)	Ankle Power Generation (W/kg)
	1	Supervision	0.43	0.45	0.56	95.2	0.64
	2	Independent	0.81	0.88	1.07	98.4	2.38
	3	Supervision	0.70	0.71	0.98	87.0	1.54
	4	Supervision	0.51	0.58	0.75	92.3	1.81
	5	Supervision	0.60	0.52	0.76	82.2	0.45
	6	Supervision	0.18	0.20	0.46	51.3	0.02
	7	Independent	0.95	0.92	0.87	127.7	1.95
	8	Supervision	0.73	0.71	0.94	90.9	1.22
	9	Supervision	0.26	0.26	0.58	53.6	0.21
	10	Independent	1.44	1.44	1.29	133.3	4.45
	11	Independent	1.25	1.25	1.25	120.0	4.73
	Mean		0.71	0.72	0.86	93.8	1.76
	SD		0.39	0.39	0.27	26.5	1.59

View Large Image Download to PowerPoint

Fig 1. Individual scores of the ankle power generation measures of the 11 participants with stroke, arranged from lowest to highest.

Therapist Accuracy and Use of the Rating Scale 

A strong positive linear relationship existed between the observations and the ankle power generation of the stroke group (fig 2). The Pearson product-moment correlation coefficient was .98 (P<.001). The SEE for the observer group was .32W/kg. The scatterplot showed 2 participants’ performance at the higher end of both axes. It is possible that these 2 distant data points may have unduly influenced the regression line. A Pearson correlation was calculated without these 2 participants, with a similar obtained value (r=.91, P=.001), indicating that these points had not overly inflated the r value.

View Large Image Download to PowerPoint

Fig 2. The relationship between clinical observation of push-off and the criterion measure of ankle power generation.

Distinctions Between Abnormal and Normal Push-Off 

Figure 3 shows how the therapists divided and classified the stroke participants into the categories of abnormal and normal push-off, compared with the push-off values of the reference control group. Observers rated participants with ankle power generation values greater than 2.38W/kg as normal. Participants with values lower than or equal to 2.38W/kg were identified as abnormal. All stroke participants judged as abnormal recorded ankle power generation values that were lower than the control participants; thus the categorization of stroke participant push-off appeared to be accurate. Therapists appear to have used observation in a systematic manner, scoring push-off ability accurately as abnormal or normal, relative to a control reference group.

View Large Image Download to PowerPoint

Fig 3. The classification of push-off observations as normal or abnormal. The dark columns represent participants with stroke rated as abnormal. The light columns represent participants with stroke rated as normal. The striped columns represent the control participants.

The threshold value of normality predicted by the regression analysis was found to be 2.24W/kg (fig 4). This threshold value can be compared with the range of ankle power generation measured in stroke and control reference populations. The value of 2.24W/kg lay below the values achieved by the control reference group of 11 participants and within the range of values recorded by these participants after stroke.

View Large Image Download to PowerPoint

Fig 4. Determination of the threshold prediction of normative values for the 11 participants with stroke.

Discussion 

To be valid, clinical observations need to be accurate with acceptable levels of measurement error. For the current study, the high correlation between real-time clinical observations and the criterion measure values indicates a strong relationship between ankle power generation, as measured by the motion analysis system, and push-off, as observed by the therapists. The criterion-related validity of the observational ratings on the 22-point scale was therefore high. The well-defined linear relationship underpinning the therapist judgment model further suggests that clinicians view abnormal and normal push-off along a movement continuum. These findings are consistent with our previous study14 and similar to those of Bernhardt et al’s25 examination of observation of upper-limb movement after stroke.

Although simultaneous capture of observations and biomechanic gait measures are possible within laboratory settings, we elected to examine clinical observations within the usual therapy environment and context. The high correlation obtained in our study indicates that the nonconcurrent capture of observational and criterion data did not adversely influence the observational ratings. This finding, in conjunction with the closely matched gait velocity in the 2 test environments, strongly suggests that the participants walked in a similar manner in both environments. Had this not occurred, and the walking patterns differed substantially, we would have expected much lower correlations between the observations and criterion measures.

Clinicians are encouraged to consider measurement error when selecting procedures for assessment of therapy outcomes.26, 27 Whether observations of push-off are sufficiently accurate requires consideration of how such observations occur in clinical practice and the size of the associated measurement error. Although the high correlations between the observational ratings and the criterion measure are encouraging, the associated error values provide an indication of whether therapists are able to detect clinically meaningful change. The mean standard error of estimating the criterion scores was found to be .32W/kg, which was less than 10% of the range of ankle power generation values in the stroke participant sample (0.02–4.73W/kg). In using the rating scale, therapists could be 68% confident that an individual rating score would fall ±.32W/kg from the true criterion score. Accordingly, a 95% confidence level would result from around 2 times this range, or ±.64W/kg.

The mean change that occurred in ankle power generation over 3 weeks of gait rehabilitation was .75W/kg, with marked variability evident (range, 0.16–1.62W/kg). This is comparable to the mean change seen in a group of chronic stroke participants after a 10-week exercise program.28 The magnitude of this mean change can be compared with the confidence levels associated with the mean SEE of .32W/kg. This suggests that observations would detect a change of this size with confidence levels of over 95%. The size of the change in ankle power generation is likely to relate to the measured time interval, and individual subject characteristics. Shorter time frames are likely to be associated with smaller changes, longer time periods are likely to be associated with larger changes. People who make rapid recoveries after stroke will make larger changes in ankle power generation than those who recover more slowly or to a lesser degree. This can be seen when comparing the 2 participants who made the greatest and smallest changes over the 3-week period. Participant 2 showed marked improvement in gait function over the 3-week period, with ankle power generation changing from 2.38W/kg (at gait speed .88m/s) to 4.00W/kg (at gait speed 1.24m/s). Participant 6 changed least; from ankle power generation of .02W/kg (at gait speed .20m/s) to .21W/kg (at gait speed .35m/s).

Clinicians should consider measurement error when repeating measures during gait rehabilitation. Knowledge of the error magnitude associated with observations can provide some perspective on how confident therapists can be when they make these observations. For example, it has been shown that the gait speed of a stroke patient would have to improve by more than 9.3m/min before therapists could be 95% confident that change beyond the level of measurement error had occurred.29 These participants were recruited from the Kingston Centre, where assessment occurs at weekly intervals, and at admission and discharge. The error size of .32W/kg compared with the mean change of .75W/kg over the 3-week period suggests that therapists should have very low confidence that observations could accurately detect the small changes that might occur over daily intervals during rehabilitation. These changes are likely to be much lower than .32W/kg and thus lie well within the measurement error range. It also seems unlikely that changes over a 1-week period would be detected with acceptable confidence levels. However, clinicians can be relatively confident in gains observed between admission and discharge, typically around 34 days at the Kingston Centre. Similarly, changes over intervals of 3 or 4 weeks are likely to be detected with higher confidence levels.

This clinical study included therapists observing patients after stroke for whom they provided treatment. This contrasts with many studies of OGA where participants were unknown to observers and limited diagnostic information was available.5, 6, 13, 18 Patla and Clouse6 argued that knowledge of participant pathology can adversely influence accuracy, as therapists may have preconceived notions which incorrectly bias their observations. This form of bias was not apparent in the current study. Indeed, therapists placed 2 participants into the upper ranges of the normal push-off scale, as shown on figure 3. This suggests that physical therapists are aware that gait disorders associated with stroke vary, and that gait may recover to well within normative levels. Therapists in the current study were likely to be aware of poststroke biomechanic impairments potentially influencing push-off, such as reduced strength or length of ankle plantarflexor muscles. This knowledge may have enhanced their accuracy. This is consistent with an investigation of clinical decision-making in OGA that found that some decisions about gait are made prior to the actual observations, based on patient knowledge or previous therapist experience.16

The accuracy of this therapist group compares favorably with the 2 previously published studies13, 14 of accuracy of observations of push-off after stroke. It is also high relative to studies of the accuracy of observations of other gait characteristics in stroke.18, 19, 30, 31 Factors within the clinical environment may also have enhanced the accuracy of observations. The rich life-sized, real-time detail available in the therapists’ clinical observation experience contrasts sharply with the brief, small, 2-dimensional and invariant videotaped images in our prior study.14 Similarly, opportunity to tailor clinical gait observations of known participants to individual preferences contrasts with the group observations of unknown stroke participants in the study by Miyazaki and Kubota.13 The high accuracy indices achieved by therapists in the current study and a previous one using videotaped gait performances14 offer further support for the methods chosen for investigation of observational accuracy, including the rating of a single defined gait variable of a wide range of participants using a simple, yet discriminating, scale. The high accuracy obtained in this study supports the integration of these principles into the development of clinical gait observation measures.

Accurate knowledge of push-off ability during gait is useful to clinicians when planning treatment and monitoring progress after stroke. Previously, these gait characteristics were regarded as inaccessible to clinicians without access to a gait laboratory, with some authors arguing for the specific exclusion of kinetics in OGA.32 Kinetic aspects of gait performance are believed to provide valuable information that relates closely to the causative factors of the observed gait pattern.10, 33 This knowledge can now be gained from observations rather than just from 3-dimensional gait analysis. This is important because reduced push-off during gait is understood to be a potential cause of the slow speed of walking after stroke. Although other readily performed clinical tests, such as muscle strength assessment, are likely to inform the assessment of push-off, the relationship between plantarflexor strength and gait speed after stroke is not entirely straightforward. Several studies have examined this relationship, with highly variable reported correlations of .34,34 .42,35 .64,36 and .84.37 Examination of individual data from the study by Nadeau et al34 reveals some inconsistencies. Some stroke subjects could not walk quickly, even though they had similar levels of plantarflexor strength as the faster walkers. Similarly, some faster walkers had lower plantarflexor strength, suggesting that these individuals produced the same gait velocity using alternative strategies, such as hip flexor activity. Push-off efficacy is likely to relate to the integrated influence of multiple potential factors, including strength, joint range, muscle stiffness, pain, selective motor control and balance. Accurate observations of push-off during gait are therefore necessary.

The positive findings of this study are restricted to the observations of push-off in a small sample of participants with stroke by a small number of therapists from a single rehabilitation hospital. Direct translation of the findings to the wider clinical domain requires caution as the scale was not constructed for clinical utilization and multiple gait variables are typically considered. However, we found that therapists of a wide range of experience levels could make accurate real-time observations of a single kinetic variable in gait of a small group of participants with varied ability. This suggests that accurate judgments are possible in clinical situations where the task is clearly defined and a single parameter is judged. These encouraging findings clearly indicate that continued research is appropriate to define and confirm the role of OGA as part of objective gait assessment procedures. Accumulating biomechanical evidence of gait changes after stroke should guide the selection of other key gait components to be included in observational assessment. Further longitudinal biomechanic studies of gait recovery after stroke are needed to identify which gait components best reflect change in performance and are associated with positive outcomes or predictive measures including gait speed. Clinical OGA tools then need to be developed incorporating a limited number of operationally defined gait variables measured on appropriately constructed observational scales.

Conclusions 

Therapists were able to make accurate judgments of push-off when observing known patients within clinical environments. The error associated with observational judgments was low relative to typical changes in ankle power generation over a 3-week rehabilitation period. These positive findings support the ongoing practice of clinical gait observation of push-off and justify further research to explore the role of observation as a legitimate component of objective gait assessment.