The Hopkins Rehabilitation Engagement Rating Scale: Development and Psychometric Properties

Article Outline

• Abstract
• Methods
  • Instrument Development
  • Sample
  • Procedure
  • Measures
    • Rehabilitation engagement
    • Therapy absences
    • Functional status
    • Emotional functioning
    • Affective state
    • Denial
    • Level of functioning
  • Statistical Analyses
• Results
  • Analysis for Threats to Validity of Results
  • Measure Reliability and Validity
    • Internal consistency
    • Interrater reliability
    • Construct validity
    • Criterion validity
• Discussion
  • Study Limitations
• Conclusions
• Acknowledgment
• Appendix 1: Hopkins Rehabilitation Engagement Rating Scale
• References
• Copyright

Abstract 

Kortte KB, Falk LD, Castillo RC, Johnson-Greene D, Wegener ST. The Hopkins Rehabilitation Engagement Rating Scale: development and psychometric properties.

Objective

To conduct an initial investigation of the psychometric properties of the Hopkins Rehabilitation Engagement Rating Scale (HRERS), a 5-item, clinician-rated measure developed to quantify engagement in acute rehabilitation services.

Design

We used a cross-sectional design to conduct correlational and multivariate analyses to establish the measure’s internal consistency, interrater reliability, construct validity, and criterion validity.

Setting

Acute inpatient rehabilitation in 3 metropolitan hospitals.

Participants

A total of 206 subjects with spinal cord injury, ischemic or hemorrhagic stroke, amputation, or hip or knee replacement.

Interventions

Not applicable.

Main Outcome Measures

The HRERS, Positive and Negative Affect Schedule, Brief Symptom Inventory, Levine’s Denial of Illness Scale, Craig Handicap Assessment and Reporting Technique, and FIM instrument.

Results

The HRERS has good internal consistency (α=.91) and interrater reliability (intraclass correlation coefficient, .73) and represents a unidimensional construct. It correlated negatively with symptoms of depression (r=−.24, P<.01), higher ratings of denial of illness (r=−.30, P<.001), and self-rated negative affect (r=−.23, P<.01), and correlated positively with self-rated positive affect (r=.36, P<.001) and level of functioning 3 months postdischarge (r=.22, P<.01).

Conclusions

The HRERS is a valid and reliable measure of rehabilitation engagement that relates to intermediate-term functional outcomes.

Key Words: Patient participation, Psychometrics, Rehabilitation

COMPREHENSIVE ACUTE rehabilitation is effective in improving outcomes after injuries and illnesses such as stroke, spinal cord injury (SCI), orthopedic injuries, and amputations. Recent investigations of the efficacy of medical rehabilitation techniques have provided evidence of improved functional outcomes¹, ², ³, ⁴ and an increased probability of discharge to home after participation in rehabilitation programs.⁵, ⁶ Further, findings suggest that the level of one’s participation in rehabilitation is related to one’s level of functional improvements.⁷

For this study, we defined participation as the degree or extent to which subjects take part in rehabilitation activities during their acute rehabilitation stay. Although the degree of participation is considered to be crucial in rehabilitation outcomes and has been explored in several studies,⁸, ⁹ the development of an operational definition and of objective tools with which to measure rehabilitation participation has been limited. One reason for the lack of progress may be the complexity and multidimensional nature of rehabilitation participation. Participation appears to encompass more than just a patient attending a therapy session, and measuring attendance alone does not reflect a patient’s participation in the rehabilitation process. Consideration of other factors that are related to participation may provide a basis for understanding this multidimensional construct so that it can be measured.

A review of the literature reveals that the majority of efforts to measure participation levels in medical rehabilitation services have focused on the patient’s motivation to participate in rehabilitation rather than on a direct assessment of participation per se. This line of research is rooted in the assumption that patients with higher levels of motivation will have higher levels of participation.¹⁰ Perceived attitude toward the rehabilitation process is 1 factor that may contribute to motivation and participation. Patients who view rehabilitation as the most important means to recovery,¹¹ or have a perception that the process is useful and valuable to them,¹⁰ are perceived as more motivated to participate. A second element in participation is an understanding by patients of the need for rehabilitation services after an illness or injury.¹¹, ¹², ¹³ It has been suggested that motivation to participate can be increased by effective communication between staff and patients about the purpose of rehabilitation.¹¹

A final component frequently identified by clinicians as important to rehabilitation participation is the need for verbal or physical prompts. Impairments in cognition, auditory or visual sensation, or mood can affect a patient’s ability to fully participate. As a result, the patient may have difficulty following commands, learning new information, initiating behavioral changes, and sustaining attention over longer periods of time, and consequently have difficulty in fully participating in therapy activities. Additionally, patients may have a reduced awareness of their impairments that stems from neuropathologic or psychologic underpinnings, which may result in a decreased motivation to participate.¹⁴ Symptoms such as cognitive impairments and emotional adjustment issues may lead to poorer rehabilitation outcomes, partly because of decreased participation in therapy activities.⁹, ¹⁴ Use of verbal cues and prompts and modifications to the therapeutic environment may be key interventions to facilitate participation of patients with such symptoms.

There have been few published reports about objective and direct measurement of rehabilitation participation. A direct approach for measuring participation is tracking the number of therapy sessions attended. The complexity of the diagnosis, the intensity of the rehabilitation program, and the length of stay (LOS) in the program, however, may confound this measure. Recently, Lenze et al¹⁵ developed a measure of rehabilitation participation that is based on having rehabilitation therapists rate their perceived level of participation by their patients. The Pittsburgh Rehabilitation Participation Scale (PRPS) is a single-item scale designed to capture the therapist’s perception of the “patient’s participation (effort and motivation) in the therapy session.”^15(p383) The therapist is asked to rate the patient’s level of participation at each session on a single-item scale that ranges from “none” to “excellent.” The descriptors for the 6 rating levels for this 1 item cue the therapist to consider such motivation-related concepts as the patient “taking interest in exercises and/or future therapy session” or “requir[ing] much encouragement.”^15(p384) Although the therapists are encouraged to consider these concepts in making their rating, there is no way to determine what concepts their rating was actually based on. The authors acknowledge that the PRPS is a “somewhat blunt instrument”^15(p383) that does not differentiate the different aspects of participation. Although Lenze has demonstrated the utility of the PRPS as a predictor of functional outcomes,³, ⁹, ¹⁵ the components of participation measured by this instrument remain ambiguous.

In summary, findings from studies by Lenze et al,⁹, ¹⁵ Geelen and Soons,¹⁰ Maclean et al,¹¹ Faller,¹² and Jeffrey¹³ suggest that participation may involve multiple elements, including the patient’s attitude toward attending therapy, his/her level of understanding, and/or acknowledgment of the need for treatment, and verbal or physical prompts required for effective communication. A useful measure of rehabilitation participation should include these important elements in a format that allows for consideration of each element separately. This conceptualization extends the construct of participation beyond therapy attendance and motivation and may best be conceptualized as the patient’s engagement in the rehabilitation process.

Engagement in rehabilitation therapy may be defined as an interest in, and an intentional effort to, work toward the rehabilitation goals. To date, however, there is no reported measure of rehabilitation engagement that captures the critical elements of attendance, participation, and effort in working toward those goals.

To provide rehabilitation researchers and clinicians with an instrument that evaluates the multiple elements of engaging in rehabilitation programs, we designed the Hopkins Rehabilitation Engagement Rating Scale (HRERS). The HRERS is a clinician-rated measure that quantifies a patient’s engagement in rehabilitation activities through behavioral observations. Our purpose in this study was to assess the HRERS’s psychometric properties when used within an acute rehabilitation setting.

We hypothesized that the HRERS would (1) have adequate internal consistency, (2) have adequate interrater reliability between occupational and physical therapists, (3) represent a unidimensional construct of engagement across 4 rehabilitation populations, and (4) show a specific pattern of relationships with other important rehabilitation variables. Specifically, we hypothesized that the HRERS would correlate inversely with depression, denial of illness, and negative affective state, and would correlate positively with positive affective state, functional status at time of rehabilitation, and level of functioning at 3 months postdischarge. Additionally, we thought that there were inherent groups of people who would differ on their engagement. We hypothesized that using data-driven cutoff scores on the HRERS, clinically significant group differences could possibly be identified in FIM instrument efficiency, therapy absence rate, and type of absence (refusals vs nonrefusals).

Back to Article Outline

Methods 

Instrument Development 

The HRERS (appendix 1) is a 5-item scale for use in rating behavioral observations of patients during acute inpatient rehabilitation. To evaluate the elements of engagement in rehabilitation activities, the HRERS items rate the following: the level of attendance at therapy sessions, the attitude expressed by the patient toward his/her therapy, the need for verbal or physical prompts to facilitate initiation or maintenance of engagement within the therapy session, the patient’s acknowledgment of the need for therapy, and the patient’s level of active participation in the therapy. The format and structure of this instrument was modeled on a rehabilitation participation measure first proposed by Johnson-Greene (personal communication, 2001), which included 6 items. The research team, whose members have expertise with medical rehabilitation populations, developed a pool of additional items. Redundant items were eliminated and the pool was narrowed to 5 items that were believed to capture the key elements. The 5 items were then reviewed and endorsed by the rehabilitation therapy staff for comprehensiveness in capturing the key elements of engagement with clarity.

The behavioral observations made by a therapist are rated on a 6-point scale, ranging from “never” to “always.” Each patient was rated by 1 physical therapist and 1 occupational therapist; each was instructed to rate the patient’s participation in his/her portion of the rehabilitation program. The ratings were completed at the time of discharge and represent a summary of the therapist’s observations during the patient’s rehabilitation stay.

Sample 

Participants were recruited from the inpatient rehabilitation programs at 3 local hospitals. The institutional review board of each facility approved the study procedures. Inclusion criteria were: (1) first rehabilitation admission and (2) no worse than mild cognitive impairment demonstrated by a Mini-Mental State Examination (MMSE) score higher than 21 (mild cognitive impairment).¹⁶ A more liberal cutoff score of 21 was chosen so to include as many rehabilitation patients with different physical and cognitive impairments as possible. Three subjects who were poststroke scored below the more frequently used cutoff score of 24¹⁶ and were further clinically evaluated by a rehabilitation neuropsychologist to rule out cognitive impairments (including language impairments) that would negatively affect their ability to understand the participation requirements or to complete the study measures. Patients were asked to participate in the study within the first few days of their rehabilitation stay and each signed a written informed consent. Verbal informed consent was obtained and documented from patients who could not sign because of upper-extremity motor impairment.

Of the 406 patients who were invited to participate over a 10-month period, 221 met the inclusion criteria and 208 agreed to participate. Two participants were not rated by both a physical and an occupational therapist and their data were excluded from further analyses, leaving a total of 206 participants in the final study sample. Of that number, 105 had SCI, 36 had ischemic or hemorrhagic strokes (stroke), 25 had amputations (amputation), and 40 had hip or knee replacements (orthopedic). Table 1 shows a summary of the sample’s demographic information.

Table 1. Sample Demographics (N=206)

Characteristics	N	Percent
Sex
Male	113	54.9
Female	93	45.1
Race
White	116	56.3
Hispanic	3	1.5
African American	86	41.8
Asian/Pacific Islander	1	0.5
Marital status
Single	50	24.3
Married	100	48.5
Divorced/separated	24	11.7
Widowed	32	15.5
Employment status
Working/active military	80	38.8
Homemaker	7	3.4
Retired	84	40.8
Student	1	0.5
Unemployed	19	9.2
On disability/medical leave	15	7.3
Diagnosis
Spinal cord	105	51.0
Stroke	37	18.0
Orthopedic	40	19.4
Amputation	24	11.6
	Range	Mean ± SD
Age (y)	18−91	56.7±17.52
Education (y)	4−22	12.7±2.76
LOS (d)	4−63	19.14±13.80
MMSE score (spinal cord, n=105)	25−30	26.77±2.35
MMSE score (stroke, n=36)	21−30	25.05±2.91
MMSE score (orthopedic, n=40)	24−30	28.67±1.82
MMSE score (amputation, n=25)	24−30	28.0±2.14

Abbreviation: SD, standard deviation.

Procedure 

During their inpatient acute rehabilitation, participants were interviewed to obtain demographic information, after which they completed the study measures of emotional functioning, affective state, and denial. The lead author (KBK) and a trained research assistant administered all of the self-report instruments in face-to-face interviews. The instruments were administered in random order and scored as described in the test manuals. Five occupational therapists and 6 physical therapists participated in the HRERS ratings process.

The lead author or a research assistant contacted each participant by telephone 3 months postdischarge to complete the measure of level of functioning. Of the 206 participants who completed the inpatient portion of the study, 183 completed the follow-up interview. The reasons for attrition included death (5 participants), another medical event (2 participants), and lost to follow-up (16 participants). The only significant difference between participants who completed the follow-up interview and those who were lost to follow-up was that the latter had higher ratings on the measure of denial of illness (t_2,204=7.57, P<.01).

Measures 

Each participant’s engagement in the rehabilitation process during his/her inpatient stay was rated on the HRERS by an occupational therapist and a physical therapist. Thus, the rating represents that particular therapist’s summary impression of participants’ engagement in the intervention during their respective therapy sessions. The therapists were unaware of the ratings given by the other therapists. Scoring consisted of adding all of the ratings together, with item 2 being reversed scored. This scoring procedure yields a summary score that can range from between 5 and 30, with a higher score representing greater engagement in the therapy process.

Each therapist documented the number of times a participant was absent from therapy sessions and the reason for the absence. Each rehabilitation program required a minimum of 3 hours of therapy a day, which included at least 1 hour each of physical and occupational therapy. Therapy absences could be because of patient illness or medical contraindication, scheduling conflicts, or participant refusal to participate. Three absences scores were calculated: total number of absences, total number of absences because the subject refused to attend, and total number of absences for reasons other than refusal. The absence rate was calculated by dividing the number of absences by the number of days in rehabilitation, with higher numbers representing a greater number of absences.

The rehabilitation team completed the FIM instrument¹⁷ for each participant on admission and discharge. The FIM is a well-validated clinician rating of a participant’s level of independent functioning in functional skills such as dressing, bathing, walking, and communicating. All rehabilitation team members were trained in the use of this instrument. The FIM’s interrater reliability has consistently been greater than .85.¹⁸, ¹⁹ We calculated a FIM efficiency score by dividing the FIM change score (FIM discharge score minus FIM admission score) by the number of inpatient acute rehabilitation days. FIM efficiency indicates the amount of functional gain per day of acute inpatient rehabilitation and has been found to be a much more sensitive measure than FIM change alone because it takes into consideration both the admission FIM score and the number of days in rehabilitation.²⁰, ²¹

The Brief Symptom Inventory (BSI) is a well-validated rating scale of symptoms of psychopathology that yields 9 subscales.²², ²³ We focused on the depression subscale, given that depression is related to poor rehabilitation outcome.⁹, ²⁴, ²⁵ We used the score on the depression subscale as a continuous variable, with higher scores indicating higher depressive symptomatology. The internal consistency (Cronbach α) of the BSI in our sample was .84. For the depression subscale of the BSI, Cronbach α was .70.

The Positive and Negative Affect Schedule (PANAS) is a list of 20 adjectives that measure positive feelings such as joy and pleasure, and negative feelings such as anxiety or sadness.²⁶ Each participant was asked to rate each adjective on a scale from 1 (very slightly/not at all) to 5 (extremely), according to how much he/she had experienced that feeling during the rehabilitation stay. Responses yielded 2 subscores: positive affect and negative affect, with higher scores indicating higher levels of the particular affective experience. Research involving the PANAS found that it had high internal consistency and appropriate test-retest reliability over a 2-month period.²⁶

The Levine’s Denial of Illness Scale²⁷ is a semi-structured interview in which the interviewer asks questions about participants’ reactions to their medical illness and then rates the responses on a 7-point scale in 24 categories. The item content taps into concepts related to denial of need for care, denial of cognitive symptoms, denial of affective symptoms, unrealistic expectations for care, and denial of impact on future. The ratings are summed to yield a single score, with higher scores interpreted as indicating greater denial of illness. Research with rehabilitation populations has shown that this instrument has good scale reliability (α=.83) and interrater reliability (r=.75).²⁸

The Craig Handicap Assessment and Reporting Technique (CHART) short-form is a 19-item self-report measure of handicap based on the World Health Organization model.²⁹ The CHART uses measurable, behavioral terms to define 6 dimensions: cognitive independence, physical independence, mobility, occupational engagement, social integration, and economic self-sufficiency. We used a total score on the CHART as a measure of overall level of functioning, with higher scores representing a higher level of functioning or less handicap. Three months after discharge, each participant was contacted by phone to complete the CHART. This instrument had good reliability and construct validity when validated in a sample of patients with SCI.²⁹

Statistical Analyses 

Before conducting the main analyses, we tested the data for normality, linearity, and the presence of outliers.³⁰, ³¹ Such analyses provide information about extreme scores on specific measures or participants who are outliers on most measures. Extreme scores and outliers can skew data and result in misleading findings. Additionally, we attempted to identify potential confounding variables by determining whether there were significant Pearson correlations between continuous variables (age, education, LOS) and the HRERS. We conducted Student t tests to determine whether there were sex or racial differences in ratings on the HRERS. Race was separated into 2 groups: white and all other. Analysis of variance (ANOVA) was conducted to determine whether there were differences on the HRERS based on diagnostic group (SCI, stroke, amputation, orthopedic).

To establish the instrument’s reliability and validity, we conducted the following statistical analyses. To determine the internal consistency of the measure when used by physical and occupational therapists, we calculated a Cronbach α for each clinician group.³² The α was accepted as adequate if it was greater than .70.³² Interrater reliability of the HRERS was assessed using intraclass correlation coefficients (ICC) with 2-way random effects design.³³ The ICC can be interpreted as the ratio of within-rater variance to between-rater variance. Thus, the coefficient reflects the true variance between participants on the construct of interest. An ICC above .70 is considered to be adequate.³⁴, ³⁵

Construct validity was demonstrated first by analysis of the factor structure of the HRERS. To demonstrate that the HRERS represents a unidimensional construct labeled “engagement,” a series of principal components factor analyses were computed. For all analyses, an eigenvalue greater-than-1 criterion was used to identify significant factors. Given that instruments can have different factor structures in different populations, we analyzed the factor structure for each diagnostic subgroup. Then, we analyzed the hypothesized relationships between the HRERS and key clinical variables. We used Pearson product correlations to analyze these relationships because the data met criteria for parametric tests rather than nonparametric tests.³⁶ A Bonferroni adjustment was applied to account for multiple comparisons and significance was evaluated against an α level of P less than .01.³⁷

Criterion validity was assessed by testing whether differences in the HRERS correlated with differences in 4 important clinical variables hypothesized to be associated with the level of engagement: FIM efficiency, number of total absences, number of refusals, and number of nonrefusal absences. Three HRERS categories were created, based on histograms that showed a trimodal distribution. The relative cutpoints on the histograms were at scores of less than 20, 20 to 25, and greater than 25.

Back to Article Outline

Results 

Analysis for Threats to Validity of Results 

Histograms of each variable demonstrated relatively well-formed normal distributions. Scatterplots of the residuals for each variable verified pairwise linear relationships without the presence of homoscedasticity. Given that the variables were normally distributed with equal variances, we conducted parametric statistical analyses.³⁴ None of the 206 participants had data missing from the inpatient portion of the study. Five of the 183 subjects who participated in the 3-month follow-up refused to share data about their economic status (a required item from the CHART) with investigators. Consequently, the 5 were excluded from all further analyses of the outcome measure, resulting in a total 3-month follow-up sample of 178 participants. Means and standard deviations for this sample on each of the measures are presented in table 2.

Table 2. Sample Measure Performance

Measure	Mean ± SD	Range
Collected during rehabilitation stay
HRERS: average	26.3±4.2	10−30
HRERS: physical therapist	26.3±4.5	10−30
HRERS: occupational therapist	26.4±4.5	10−30
FIM efficiency	1.19±1.34	0−6
BSI depression	1.5±2.3	0−12
Levine’s Denial of Illness	8.4±6.7	0−35
PANAS positive affect	31.8±7.5	12−49
PANAS negative affect	17.6±7.0	10−39
Absences	2.76±4.56	0−37
Refusals	1.02±3.04	0−29
Collected 3-month postdischarge
CHART	463.7±102.4	182−600

There were no significant relationships between the HRERS and age (r=.11; P, not significant [NS]), education (r=.16, P=NS), or LOS (r=−.13, P=NS). There were no group differences on the HRERS rating concerning sex (t_1,207=−.68, P=NS) or race (t_1,207=.57, P=NS). Finally, the results showed that there were no significant differences in HRERS ratings between diagnostic groups (F_3,205=1.548, P=NS). Thus, subsequent analyses were conducted with the entire sample (N=206).

Measure Reliability and Validity 

The α of the HRERS when ratings were completed by physical therapists was .92; when completed by occupational therapists, it was .91.

The HRERS had an interrater reliability of .733. Given the high interrater reliability and internal consistency of the instrument across disciplines, we used an average score of physical and occupational therapist rating for further analyses. This score, labeled by the measure’s acronym (HRERS), is the average of the physical and occupational therapist ratings for each participant.

Of all the factor analyses, only the first principal component met the eigenvalue greater-than-1 criteria. A single factor with loadings ranging from .69 to .94 was found for physical therapists and from .73 to .93 for occupational therapists. When the factor structure was based on the composite score of physical and occupational therapist ratings, the loadings ranged from .77 to .95 on a single factor. Among the diagnostic groups, factor loadings on the single factor for the orthopedic group ranged from .72 to .95, for the amputation group from .75 to .96, for the stroke group from .62 to .94, and for the SCI group from .83 to .96.

Table 3 shows the correlations between key clinical variables and the HRERS. All hypothesized relationships with the HRERS were supported by the data. Specifically, a higher rating on the HRERS was related to greater FIM efficiency (r=.20, P<.01), greater self-rated positive affect (r=.36, P<.001), and higher level of functioning at 3 months postdischarge (CHART; r=.22, P<.01). Also consistent with the hypotheses, lower ratings on the HRERS were associated with more symptoms of depression (r=−.24, P<.01), higher ratings of denial of illness (r=−.30, P<.001), and greater self-rated negative affect (r=−.23, P<.01).

Table 3. Construct Validity of the HRERS: Correlation Matrix

As shown in table 4, there were statistically significant differences between HRERS categories and the 4 tested clinical outcomes. Equivalent results were obtained by categorizing the measure into quartiles.

Table 4. Criterion Validity of the HRERS

HRERS Score	n	Mean	Standard Error	P⁎
FIM efficacy				.04
<20	22	1.25	.20
20−25	42	1.87	.21
>25	142	2.03	.11

	n	%	Standard Error	P⁎
Total therapy absence rate				<.001
<20	22	28	.05
20−25	42	15	.02
>25	142	9	.01
Therapy refusal absence rate				<.001
<20	22	14	.04
20−25	42	7	.01
>25	142	2	.01
Therapy nonrefusal absence rate				.02
<20	22	14	.03
20−25	42	9	.02
>25	142	7	.01

Back to Article Outline

Discussion 

Our purpose in this study was to establish the psychometric properties of a rehabilitation engagement measure, including its internal consistency, interrater agreement, construct validity, and criterion validity. Rehabilitation engagement is conceptualized as a construct that captures multiple elements, including a patient’s attitude toward the therapy, his/her level of understanding or acknowledgment of a need for treatment, the need for verbal or physical prompts to participate, the level of active participation in therapy activities, and the level of attendance throughout the rehabilitation program. Based on the results, the HRERS demonstrated good internal consistency when completed by either physical or occupational therapists. Interrater reliability between the 2 disciplines indicated that there is good agreement on the patient’s level of engagement in therapy across the disciplines. Although the interrater correlation was lower than that reported in other participation rating systems,¹⁵ it was accepted as sufficient, given that this reliability was based on interdisciplinary ratings. Because of inherent differences between these 2 types of therapy, it was expected that there would be greater variance in the patient’s engagement ratings. This is in contrast to the PRPS study¹⁵ in which the ratings were made between therapists of the same discipline.

The construct validity of the HRERS was determined in 2 ways: through demonstration of its factor structure and through the pattern of relationships with other key clinical rehabilitation variables. In this study, the HRERS measured a unidimensional construct that we labeled “engagement.” The single factor structure was consistently found across disciplines and diagnostic groups. Additionally, we showed that the HRERS was significantly related in the hypothesized direction to other measures that are relevant to rehabilitation participation. Specifically, as hypothesized, greater rehabilitation engagement was related to lower levels of depression, denial of illness, and negative affective states, and higher levels of positive affective states, FIM efficiency scores, and level of functioning 3 months postdischarge.

Finally, the HRERS criterion validity was established by testing whether specific ranges of scores on the HRERS represented distinguishable groups in terms of clinically relevant variables. Using HRERS cutoff scores of 26 and above, 20 to 25, and less than 20, group differences were established based on FIM efficiency, number of total absences, number of total refusals to participate in the therapy session, and the number of nonrefusal absences. This pattern of results provided preliminary evidence of how the HRERS could be used clinically to identify people who need clinical intervention (score of <20) versus those who may be “at risk” for greater absenteeism from therapy, and a consequent poorer FIM efficiency (score range, 20−25). Patients within the “normal” range (score >25) would be considered as fully engaged in their rehabilitation therapies.

One hypothesis regarding participation in rehabilitation is that if patients actively participate in the rehabilitation process, they will have better functional gains and outcomes. There are some data to support this purported connection between rehabilitation participation and functional gains.⁷ Our findings provide preliminary support that greater rehabilitation engagement, as measured by the HRERS, is related to greater functional gain, controlling of the number of days in acute inpatient rehabilitation (ie, greater FIM efficiency), and better outcome 3 months after discharge (CHART score). Future studies should focus on determining the role that rehabilitation engagement plays in rehabilitation outcomes. Should there be substantial evidence that a patient’s rehabilitation engagement is related to the subsequent outcomes, then interventions can be developed and targeted for at-risk patients to modify their level of rehabilitation engagement. Recently, motivational interviewing interventions that are designed to address a subject’s ambivalence about engaging in health behaviors have demonstrated the potential to modify patients’ commitments to participation and to follow through with changes in their health behavior.³⁸ Motivational interviewing was developed in the field of substance abuse treatment but was quickly tested and adapted in several randomized clinical trials designed to promote behavior change in patients with a variety of health problems. Motivational interviewing is a style of counseling designed to assist people to recognize, explore, and resolve ambivalence about change, and consequently, increase their internal motivation to engage in, or change, a behavior. Within a motivational interviewing framework, motivation to engage in a new behavior or change an established one is viewed as an alterable condition that can be increased via the interpersonal, supportive, client-centered interaction.

Rehabilitation specialists have endeavored to demonstrate that rehabilitation services improve outcomes for specific rehabilitation populations.¹, ², ³, ⁴ While considerable progress has been made toward that goal, the field must move to a more sophisticated level of investigation that: (1) identifies person and environmental variables that lead to improved outcomes; (2) identifies prospectively people who are at risk for poor outcomes; and (3) development of interventions that improve rehabilitation outcomes. The construct of rehabilitation engagement may account for variations in rehabilitation outcomes and prove to be a modifiable variable in the complex interplay of individual and environmental factors that determine rehabilitation outcomes.

Study Limitations 

One limitation of the study is its reliance on a summary rating from physical and occupational therapists, which may be influenced by recall bias. Because therapists rated engagement levels only at the end of the rehabilitation stay, the ratings may have been influenced by recency effects in that either more or less engagement in the latter half of the rehabilitation program may have swayed the overall rating. In attempts to prevent this occurrence, therapists were instructed to specifically consider the patient’s engagement throughout the entire program. This does not eliminate the possibility that recall bias occurred, however. If there was a recency effect, then it may have weakened the relationship between the HRERS and self-report instruments that were completed by the participants at the beginning of their rehabilitation stay.

An additional concern regarding the therapists’ ratings is that they were not made totally independently. Therapists were asked to make their ratings privately and independently of one another and to base them on their experience with the patient solely within the context of the particular therapy. It is possible, however, that discussions of patients among therapists during team rounds and during less formal interactions throughout the patients’ stays detracted from the complete autonomy of the HRERS ratings when they were made at the time of discharge. If the therapists shared their thoughts about a patient’s level of engagement, or specifically about the ratings, then the resulting rating would more reflect their shared rating rather than discipline-specific ratings. The ICC provides some indication that the ratings, although related, were not entirely the same. Nevertheless, if future research focuses on differences between engagement levels for different types of therapies, then autonomy of the ratings will be a factor that will need to be controlled for.

To further demonstrate the validity of our findings, the HRERS’s validity and reliability should be assessed in other studies. In this study, we focused on 4 common rehabilitation populations to establish the psychometric properties of this new instrument. To demonstrate the generalizability of these findings and the utility of this instrument in other rehabilitation populations, psychometric properties of the HRERS should be investigated in other populations, such as traumatic brain injury. Given that in this study we focused on physical therapy and occupational therapy as representative of engagement in the rehabilitation process, it remains to be tested whether the HRERS is useful in measuring rehabilitation engagement in other types of rehabilitation interventions, such as speech-language therapy, cognitive therapy, and nursing interventions. Additionally, consideration should be given to the quality of the therapist-patient relationship, including how such factors as cultural or age differences affect that relationship and the extent to which a patient engages in the therapy. Future research should also employ longitudinal designs to further validate the ability of the HRERS to predict short-term and long-term rehabilitation outcomes.

Back to Article Outline

Conclusions 

Overall, our findings indicate that the HRERS is reliable and valid and measures aspects of the rehabilitation process that relate to short- and long-term outcomes. These findings suggest that the HRERS is useful for measuring engagement as part of the medical rehabilitation process. There are now 2 reported instruments for measuring rehabilitation participation through ratings by therapists. It is unclear how the HRERS and PRPS relate to one another. The HRERS is a summary measure designed to measure rehabilitation engagement over the course of an episode of care; it has the potential to capture multiple elements of rehabilitation engagement. The PRPS is completed on a daily basis to provide a summary rating of daily participation. Both measures provide opportunities to determine how participation and engagement relate to short- and long-term outcomes. If further support is found that engagement is an important factor in determining rehabilitation outcomes, then we can look to establishing interventions to modify outcomes.

Back to Article Outline

Acknowledgment 

We thank the therapy staff members for their participation in this study. Without their commitment to investigating ways to improve the care of their patients, this line of research would not have been possible.

Back to Article Outline

Appendix 1: Hopkins Rehabilitation Engagement Rating Scale 

Rater:______________________

Job Title: ___________________

Date: ______________________

Please rate the patient’s participation in your portion of his/her rehabilitation program on the following scales. This rating should be completed at the time of discharge and is a summary of his/her participation during the entire course of your interactions with the patient.

1. The patient regularly attended my therapy/ rehabilitation activity.
□	□	□	□	□	□
Never	Seldom	Someofthe time	Mostofthe time	NearlyAlways	Always
2. The patient required verbal or physical prompts to actively participate in my therapy/ rehabilitation activity.
□	□	□	□	□	□
Never	Seldom	Someof the time	Mostofthe time	NearlyAlways	Always
3. The patient expressed a positive attitude towards my therapy/rehabilitation activity.
□	□	□	□	□	□
Never	Seldom	Someofthe time	Mostofthe time	NearlyAlways	Always
4. The patient acknowledged a need for rehabilitation services and the benefit of therapy exercises or rehabilitation activities.
□	□	□	□	□	□
Never	Seldom	Someofthe time	Mostofthe time	NearlyAlways	Always
5. The patient actively participated in his/her rehabilitation therapy/activity.
□	□	□	□	□	□
Never	Seldom	Someofthe time	Mostofthe time	Nearly Always	Always

Back to Article Outline

References 

1. Cifu D, Stewart D. Factors affecting functional outcome after stroke: a critical review of rehabilitation interventions. Arch Phys Med Rehabil. 1999;80:S35–S39
   • View In Article
   • Abstract
   • Full-Text PDF (669 KB)
   • CrossRef
2. Moffet H, Collet JP, Shapiro SH, Paradis G, Marquis F, Roy L. Effectiveness of intensive rehabilitation on functional ability and quality of life after first total knee arthroplasty: a single-blind randomized controlled trial. Arch Phys Med Rehabil. 2004;85:546–556
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (189 KB)
   • CrossRef
3. Munin MC, Seligman K, Dew MA, et al. Effect of rehabilitation site on functional recovery after hip fracture. Arch Phys Med Rehabil. 2005;86:367–372
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (111 KB)
   • CrossRef
4. Yagura H, Miyai I, Seike Y, Suzuki T, Yanagihara T. Benefit of inpatient multidisciplinary rehabilitation up to 1 year after stroke. Arch Phys Med Rehabil. 2003;84:1687–1691
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (227 KB)
   • CrossRef
5. Freburger JK. An analysis of the relationship between the utilization of physical therapy services and outcomes for patients with acute stroke. Phys Ther. 1999;79:906–918
   • View In Article
   • MEDLINE
6. Freburger JK. Analysis of the relationship between the utilization of physical therapy services and outcomes of care for patients after total hip arthroplasty. Phys Ther. 2000;80:448–458
   • View In Article
   • MEDLINE
7. Fiedler RC, Granger CV, Russell CF. UDSmr^sm: follow-up data on patients discharged in 1994-1996 (Uniform Data System for Medical Rehabilitation). Am J Phys Med Rehabil. 2000;79:184–192
   • View In Article
   • MEDLINE
   • CrossRef
8. Resnick B, Zimmerman S, Magaziner J, Adelman A. Use of the Apathy Evaluation Scale as a measure of motivation in elderly people. Rehabil Nurs. 1998;23:141–147
   • View In Article
   • MEDLINE
9. Lenze EJ, Munin MC, Dew MA, et al. Adverse effects of depression and cognitive impairment on rehabilitation participation and recovery from hip fracture. Int J Geriatr Psychiatry. 2004;19:472–478
   • View In Article
   • MEDLINE
   • CrossRef
10. Geelen R, Soons P. Rehabilitation: an ‘everyday’ motivation model. Patient Educ Couns. 1996;28:69–77
    • View In Article
    • Abstract
    • Full-Text PDF (901 KB)
    • CrossRef
11. Maclean N, Pound P, Wolfe C, Rudd A. Qualitative analysis of stroke patients’ motivation for rehabilitation. BMJ. 2000;321:1051–1054
    • View In Article
    • MEDLINE
    • CrossRef
12. Faller H. [Shared decision making: an approach to strengthening patient participation in rehabilitation] [German] Rehabilitation (Stuttg). 2003;42:129–135
    • View In Article
    • MEDLINE
    • CrossRef
13. Jeffrey D. Cognitive clarity: key to motivation in rehabilitation. J Rehabil. 1981;47:33–35
    • View In Article
    • MEDLINE
14. Kortte KB, Wegener ST. Denial of illness in medical rehabilitation populations: theory, research, and definition. Rehabil Psychol. 2004;49:187–199
    • View In Article
    • CrossRef
15. Lenze EJ, Munin MC, Quear T, et al. The Pittsburgh Rehabilitation Participation Scale: reliability and validity of a clinician-rated measure of participation in acute rehabilitation. Arch Phys Med Rehabil. 2004;85:380–384
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (82 KB)
    • CrossRef
16. Folstein MF, Folstein SE, McHugh PR, Fanjiang G. Mini-Mental State Examination: user’s guide. Odessa: Psychological Assessment Resources; 2001;
    • View In Article
17. Linacre JM, Heinemann AW, Wright BD, Granger CV, Hamilton BB. The structure and stability of the Functional Independence Measure. Arch Phys Med Rehabil. 1994;75:127–132
    • View In Article
    • MEDLINE
18. Hamilton BB, Laughlin JA, Fiedler RC, Granger CV. Interrater reliability of the 7-level functional independence measure (FIM). Scand J Rehabil Med. 1994;26:115–119
    • View In Article
    • MEDLINE
19. Ottenbacher KJ, Hsu Y, Granger CV, Fiedler RC. The reliability of the Functional Independence Measure: a quantitative review. Arch Phys Med Rehabil. 1996;77:1226–1232
    • View In Article
    • Abstract
    • Full-Text PDF (987 KB)
    • CrossRef
20. Bottemiller KL, Bieber PL, Basford JR, Harris M. FIM score, FIM efficiency, and discharge disposition following inpatient stroke rehabilitation. Rehabil Nurs. 2006;31:22–25
    • View In Article
    • MEDLINE
21. Chae J, Zorowitz R, Johnston M. Functional outcome of hemorrhagic and nonhemorrhagic stroke patients after in patient rehabilitation: a matched comparison. Am J Phys Med Rehabil. 1996;75:177–182
    • View In Article
    • MEDLINE
    • CrossRef
22. Derogatis L. The Brief Symptom Inventory (BSI): administration, scoring and procedures manual. 3rd ed. Minneapolis: National Computer Systems; 1993;
    • View In Article
23. Derogatis LR, Melisaratos N. The Brief Symptom Inventory: an introductory report. Psychol Med. 1983;13:595–605
    • View In Article
    • MEDLINE
    • CrossRef
24. MacNeill SE, Lichtenberg PA. Predictors for functional outcome in older rehabilitation patients. Rehabil Psychol. 1998;43:246–257
    • View In Article
    • CrossRef
25. Mast BT, MacNeill SE, Lichtenberg PA. Geropsychological problems in medical rehabilitation: dementia and depression among stroke and lower extremity fracture patients. J Gerontol Biol Sci Med Sci. 1999;54:M607–M612
    • View In Article
26. Watson D, Clark LA, Tellegen A. Development and validation of brief measures of positive and negative affect: the PANAS scales. J Pers Soc Psychol. 1988;54:1063–1070
    • View In Article
    • MEDLINE
    • CrossRef
27. Levine J, Warrenburg S, Kerns R, et al. The role of denial in recovery from coronary heart disease. Psychosom Med. 1987;49:109–117
    • View In Article
    • MEDLINE
28. Jacobsen B, Lowery B. Further analysis of the psychometric properties of the Levine Denial of Illness Scale. Psychosom Med. 1992;54:372–381
    • View In Article
    • MEDLINE
29. Whiteneck GG, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: a new measure of long term rehabilitation outcomes. Arch Phys Med Rehabil. 1992;73:519–525
    • View In Article
    • MEDLINE
30. Bradley JV. The insidious L-shaped distribution. Bull Psychonomic Soc. 1982;20:85–88
    • View In Article
31. Rossner B. Fundamentals of biostatistics. 5th ed. Pacific Grove: Duxbury Pr; 2000;
    • View In Article
32. Nunnally J. Psychometric theory. New York: McGraw-Hill; 1978;
    • View In Article
33. Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;2:420–428
    • View In Article
    • CrossRef
34. Andresen EM, Catlin TK, Wyrwich KW, Jackson-Thompson J. Retest reliability of surveillance questions on health related quality of life. J Epidemiol Community Health. 2003;57:339–343
    • View In Article
35. Raat H, Botterweck A, Landgraf JM, Hoogeveen WC, Essink-Bot ML. Reliability and validity of the short form of the child health questionnaire for parents (CHQ-PF28) in large random school based and general population samples. J Epidemiol Community Health. 2005;59:75–82
    • View In Article
36. Kachigan S. Statistical analysis: an interdisciplinary introduction to univariate and multivariate methods. New York: Radius Pr; 1986;
    • View In Article
37. Miller RG. Simultaneous statistical inference. 2nd ed. New York: Springer Verlag; 1981;
    • View In Article
38. Rubak S, Sandbaek A, Lauritzen T, Christensen B. Motivational interviewing: a systematic review and meta-analysis. Br J Gen Pract. 2005;55:305–312
    • View In Article
    • MEDLINE