Applying the Clinical Practice Improvement Approach to Stroke Rehabilitation: Methods Used and Baseline Results

Article Outline

• Abstract
• Methods
  • Overview
  • PSROP Project Clinical Team
  • PSROP Facilities
  • PSROP Patient Selection Criteria
  • Data Collection
    • Point-of-care data
    • Intervention taxonomy (documentation) development—the black box
    • Intervention documentation training and reliability
    • Intervention documentation form use
    • Intervention documentation validity
    • Postdischarge chart review
    • The CSI: disease-specific severity-of-illness data (signs and symptoms)
    • Validity of the CSI for stroke rehabilitation patients
    • Patient, process, and outcome data
    • Chart review training and reliability
  • Database Management
  • Data Analyses
• Results
  • Patient Characteristics
  • Demographic and Health Plan Status
    • Age and sex
    • Race
    • Payer
  • Health and Functional Status
    • Risk factors
    • Stroke type and location
    • FIM scores
    • CMGs
    • Severity of illness (CSI)
    • Stroke symptoms
  • Prerehabilitation Health Care
    • Time from onset of symptoms to rehabilitation
    • Acute care hospital LOS
  • Process Variables
    • Rehabilitation LOS
    • PT, OT, and SLP
    • Treatments
  • Outcome Variables
    • Comorbidities and complications during rehabilitation
  • Severity of Illness (CSI)
    • Increase in severity during rehabilitation
    • Discharge severity and change in severity from admission to discharge
  • FIM Scores
    • Discharge FIM score and change in FIM scores from admission to discharge
    • Low FIM scores and high severity scores
  • Rehabilitation Discharge Destination
• Discussion
  • Limitations
• Conclusions
• Acknowledgments
• Appendix 1. 
• Appendix 2. 
• Appendix 3. 
• Appendix 4. CSI criteria set for stroke
• Appendix 5. PSROP ADM outline
• References
• Copyright

Abstract 

Gassaway J, Horn SD, DeJong G, Smout RJ, Clark C, James R. Applying the clinical practice improvement approach to stroke rehabilitation: methods used and baseline results.

Objectives

To describe the methods used and baseline data for the Post-Stroke Rehabilitation Outcomes Project (PSROP).

Design

Prospective observational cohort study.

Setting

Seven inpatient rehabilitation facilities (IRFs) in the United States and New Zealand.

Participants

Consecutive convenience sample of 1291 poststroke rehabilitation patients, age older than 18, who were treated between 2001 and 2003 in 7 IRFs (1161 patients in 6 U.S. IRFs).

Interventions

Not applicable.

Main Outcome Measures

Change in FIM score, change in severity of illness, and discharge destination.

Results

For the U.S. sample, the average age was 66 years, 52% were men, 60% were white, and 23% were black. Medicare was the most frequent payer. Seventy-seven percent of strokes were ischemic, with 43% in the left brain, 44% in the right brain, and 11% bilateral. Mean admission total FIM score was 61, with a mean motor FIM score of 40 and mean cognitive FIM score of 21. Lower FIM scores are associated with higher severity-of-illness scores. Mean rehabilitation length of stay was 18.6 days; 78% of patients were discharged home. At discharge, the average increase in total FIM score was 26, in motor FIM score was 22, and in cognitive FIM score was 4.

Conclusions

This article outlines methods used in the PSROP, provides an overview of participating IRFs, describes the database, and summarizes key characteristics to enable readers of subsequent articles to better interpret study findings and determine generalizability.

Key Words:  Outcome assessment (health care) , Rehabilitation , Severity of illness index , Stroke

THE TERM BLACK BOX has been used to describe specific components (interventions) of the stroke rehabilitation process, because specific details about activities used throughout the course of rehabilitation are lacking in rehabilitation literature.¹, ², ³, ⁴, ⁵, ⁶ Stroke rehabilitation practices are customized to meet individual patient needs, which results in variation from one patient to another and from one rehabilitation center to another. Standardized protocols that exist in other areas of medical practice are not common in stroke rehabilitation, which accounts for about 20% of all inpatient rehabilitation admissions. A rationale for the study and detailed literature review substantiating the need to examine rehabilitation processes to improve outcomes for specific types of patients is presented elsewhere.⁷

This article provides an overview of the methods used in a large multisite study of stroke rehabilitation outcomes known as the Post-Stroke Rehabilitation Outcomes Project (PSROP). It was a component of the Rehabilitation Research and Training Center on Medical Rehabilitation Outcomes commissioned by the National Institute on Disability and Rehabilitation Research. The PSROP addressed priority 2: the need for scientific data supporting the effectiveness of rehabilitation treatments for poststroke patients. The article also provides a characterization of the study group, the scope of care received, and an introduction to rehabilitation outcomes realized. It sets the stage for articles that follow, in which the PSROP’s findings are reported.

The PSROP introduces to rehabilitation research a genre of research methodology known as clinical practice improvement (CPI).⁸ CPI’s fit into the pantheon of biomedical and clinical research methodology is described elsewhere.⁹ A CPI study is an observational cohort study that entails the acquisition of prospective and retrospective data while not disrupting the natural milieu of treatment. CPI examines what actually happens in the care process and contains several distinct features, some of which are meant to compensate for the shortcomings commonly attributed to observational studies, particularly the ability to account for patient covariates. Because of CPI’s methodologic complexity, a significant portion of this article is devoted to how CPI concepts were operationalized in the PSROP.

In the context of rehabilitation, the purpose of a CPI study is to discern the relative contributions of specific interventions and therapies to rehabilitation outcomes taking into account patient differences and other contributing factors. On 1 level, CPI studies are straightforward. They resemble other observational studies that take into account demographic-type patient and setting characteristics that may shape outcomes and determine generalizability. CPI then moves to a level beyond traditional observational approaches to create comprehensive, complex databases that include detailed patient-specific descriptions, severity-of-illness measures, and characterizations of rehabilitation treatments for large samples of patients. Moreover, CPI studies entail extensive clinical staff participation in all phases of study design, data collection, and analyses.

Back to Article Outline

Methods 

Overview 

The CPI methodology was central to our approach in the PSROP because it captures in-depth, comprehensive information about patient characteristics (including clinical signs and symptoms), rehabilitation processes of care, and rehabilitation outcomes needed to characterize the process of care and ascertain the contribution of individual rehabilitation processes to outcomes. At the risk of some over simplification, there are 7 components to CPI methodology; the PSROP included the first 5 components, and the sixth and seventh components (validation of findings, incorporation of study findings into care protocols) will be the subject of future work. Each component is described briefly, followed by in-depth descriptions of the first 5 as related to the PSROP:

As noted, the PSROP did not include the validation implementation or protocol incorporation phases (6 or 7), which will be the subject of future work.

The CPI approach offers a naturalistic view of rehabilitation treatment by examining what actually happens in the care process. It does not alter the treatment regimen to evaluate efficacy of a particular intervention. Moreover, CPI’s detailed data on rehabilitation interventions allow researchers to penetrate to the most meaningful level of resolution regarding the types of care rendered—consistent with current knowledge and insights offered by team participants. Thus, the CPI approach can answer study questions and hypotheses initially at a fairly basic level of resolution but also allows researchers to drill down into the data with the help of additional insights offered by clinical team participants.

PSROP Project Clinical Team 

The project clinical team provided expert advice to ensure clinical meaningfulness to PSROP activities and analyses. It began as a core group consisting of the medical director from each of 7 participating inpatient rehabilitation facilities (IRFs). This core clinical team developed and implemented patient selection criteria, provided expert advice for data collection instrument development, obtained institutional review board (IRB) approvals at their respective affiliated university or research organization, oversaw the data collection process, and participated in analyses. Over time and depending on project activities/needs, the PSROP clinical team (hereafter “the team”) expanded to include representatives of each discipline in stroke rehabilitation. Physical, occupational, speech and language, and recreation therapists; social workers; nurses; and psychologists provided expert advice specific to their fields of expertise. No clinicians or patients received monetary reimbursement for participation. Team members participated in weekly conference calls over much of the 5-year project and specialized subgroups teleconferenced as needed. Frequent team meetings contributed to overall collaboration and investment in the study’s processes and findings.

PSROP Facilities 

Table 1 lists the 7 (6 in the United States, 1 in New Zealand) IRFs that participated in the PSROP. IRFs were selected based on their willingness to participate and geographic location only. There were no specific criteria for selection, and thus, they are not a probability sample of IRFs in the United States. All facilities are nonprofit. One facility is free-standing; all others are rehabilitation units within an acute care setting. We did not examine facility-specific patient admission criteria for participating IRFs. Each site contributed detailed data for about 200 consecutive poststroke patients for a total of 1291 patients (1161 patients in the United States). The inclusion of 1 international site (New Zealand) provides somewhat different approaches to rehabilitation care and our data confirm these differences. Thus, we elected to report results from New Zealand as compared with the U.S. sample in a separate article.¹⁰ Apart from this article, the remaining articles in this supplement include only the 1161 U.S. patients, and therefore, this article describes information for U.S. patients only.

Table 1. PSROP Participating IRFs

Rehab, rehabilitation.

Each IRF enrolled consecutively admitted patients with stroke who met inclusion criteria; 5 sites began enrolling patients with stroke in March 2001; 2 sites began in June 2001. Facility size and rate of stroke patient admissions determined the duration of the enrollment period. Some sites enrolled 200 poststroke patients in about 8 months; other sites took up to 2 years. No eligible patients were excluded. Patients with stroke from these 6 U.S. IRFs constitute a convenience sample.

Subsequent articles will use specific subsets of the full PSROP database depending on the topic of each article. When subsets are used they are described fully and reasons for inclusion and exclusion of specific patients are provided.

PSROP Patient Selection Criteria 

Each participating IRF obtained IRB approval for this observational study and enrolled consecutively admitted patients who met the following inclusion criteria:

There were no exclusion criteria that might otherwise limit the generalizability of findings. Because the study did not entail a new or experimental intervention for which patient consent was needed, there were no refusals or study dropouts and, therefore, no need to compare study participants with study dropouts or need to account for patient selection effects that might otherwise occur. The study obtained informed consent from patients at only 2 sites (1 in the United States, 1 in New Zealand) for their participation in the collection of 6-month postdischarge outcomes at these 2 sites (6-mo outcomes were not collected at the other 5 sites and are not included in this or other articles in this supplement).

We also compared the PSROP study group to a national reference group of stroke patients (http://eRehabData.com) to understand better how similar or different PSROP patients are from those who might be found in other IRFs in the nation and to better determine the generalizability of PSROP findings. eRehabData is a subscriber-paid database maintained by the American Medical Rehabilitation Providers Association to monitor national trends and help estimate the programmatic and fiscal impact of federal policy on rehabilitation providers. We used national data only from 2002, mainly because eRehabData was not aggregated across the entire 2001–2003 study, and with only 1 year to choose from, we chose 2002—the midyear of the study period. About 180 rehabilitation providers contributed data to eRehabData for 2002. This is about 15% of all IRFs, but because eRehabData tends to attract larger facilities, its sample of patients is about 20% of the nation’s IRF patients.

Data Collection 

Three types of study data were obtained from multiple sources either at the point of care or from postdischarge chart review in the IRF: (1) patient characteristics (eg, admission severity of illness, functional status measures), (2) process variables (eg, treatments, interventions), and (3) outcome variables (eg, discharge functional status, discharge severity of illness, discharge destination)

An important component of CPI is its attention to the process of care that the patient actually receives; it addresses interventions and patient management strategies. CPI typically relies on information contained in patient medical records, which trained data collectors abstract after patient discharge. The team was confident that many identified acute care hospital and rehabilitation study variables could be obtained from existing documentation at their respective sites. However, they strongly believed that existing patient records did not document adequately specific activities and interventions provided by rehabilitation specialists (eg, physical, occupational, and speech therapists); much patient documentation is oriented to the needs of payment or reimbursement systems. The team agreed that the ability to capture details of what rehabilitation specialists do on a daily basis is essential to opening the black box of rehabilitation care and strongly recommended that we first determine how to get all members of the rehabilitation team to describe accurately what they do. Thus, the concept of point-of-care intervention documentation was incorporated into the study design. This initial taxonomy for stroke rehabilitation is described in detail by DeJong et al.¹²

Discipline-specific specialty teams with representation from each participating IRF met via teleconferences from June 2000 through January 2001 to conceptualize and then create discipline-specific intervention documentation forms to record activities and interventions used with stroke rehabilitation patients. This iterative process took approximately 9 months, because specialty teams learned quickly that what is practiced in 1 site is often different from other sites. Clinicians also realized that definitions of common terms differ from site to site and practitioner to practitioner. This is the black box of stroke rehabilitation—What is it that therapists and other stroke care clinicians provide to stroke patients? How are activities or interventions defined and described to others?

The study’s physicians, nurses, psychologists, social workers, and physical, occupational, recreational, and speech therapists made a first attempt to identify these differences within their practices by creating an intervention documentation form to include a taxonomy of activities used in each clinical area.¹² This work incorporated practices and definitions in existing frameworks—for example, Occupational Therapy Practice Framework and Guide to Physical Therapist Practice and the level of intervention intensity clinicians thought was needed to capture a complete and accurate picture of the contribution made by that discipline to rehabilitation care (beyond what was already contained in traditional medical record documentation). In addition to developing the content of its documentation form, each rehabilitation discipline decided on the frequency with which its form should be completed. The taxonomy provides a format into which clinicians document actual interventions performed with patients. It does not suggest treatment strategies or changes to routine practice.

Intervention documentation forms were standardized for all sites. Because development efforts included representatives from each participating site, the forms contain interventions that may be specific to 1 or more sites but are not used by all. For example, physical therapists in only 1 facility used constrained induced movement therapy, a different site used pet therapy, and several sites used grocery carts as an assistive device. These “unique” interventions are included on each site’s form, even though most places do not use them. Therapists were trained to record only what was done in the actual care process at each site.

What is in the black box of stroke rehabilitation? A partial picture of the black box is presented in Appendix 1, Appendix 2, Appendix 3, which contain 3 therapy intervention documentation forms. Therapy-specific interventions are associated with therapy-specific activities, and time spent within each is recorded. The physical therapy (PT) form, for example, contains time spent on specific functional activities (eg, sitting, transfers, gait) and interventions (eg, balance training, cognitive training, strengthening, education) used with each activity. In addition, the PT form captures time spent on formal patient assessment and on home and worksite evaluations. In the case of group therapy, therapists record and include the number of patients, therapists, and assistants involved in the group. Other therapy intervention documentation forms (occupational therapy [OT] and speech language pathology [SLP]) also contained in Appendix 2, Appendix 3 follow a similar format to capture the amount of time spent on specific activities (eg, dressing, transfers, community integration for OT; verbal expression, problem solving, pragmatics for SLP) and specific interventions within each activity (eg, strengthening, balance training for OT; visual cueing, auditory strategies for SLP). One therapy intervention documentation form was completed for each patient treatment session. Rehabilitation clinicians may provide overlapping services as in the case of physical therapists and occupational therapists who may both, for example, provide balance training. In such instances, therapists from each discipline included discipline-specific applications of the overlapping activities and interventions in their taxonomies. The date and time of therapy was included on each intervention documentation form so that frequency of therapy for specific days of the week could be calculated.

In contrast to the therapy disciplines, other rehabilitation disciplines created intervention documentation forms to meet their needs of capturing information not contained in traditional rehabilitation documentation. The physician form, for example, captured time spent in care management discussions, education of patient/family or medical staff, and supportive activities such as contact with payers and dictation of support letters. The social work form contained information about insurance coordination, crisis intervention, team collaboration, and family communication. Physicians and social workers created multiday forms of patient interaction, with 1 column completed per day, to capture information not documented in traditional documentation. The nursing intervention documentation form was completed for each nursing shift; it contained only information deemed important to the rehabilitation process but not documented elsewhere (eg, frequency of skin checks, frequency and reinforcement of patient/family teaching). Information from these disciplines is not included in this supplement and will be explored in the future to complete the multidisciplinary picture of stroke rehabilitation care. All intervention documentation forms are available on request.

During a 3-month pilot test period after development of each form, practicing clinicians who worked on form development used their draft forms during patient treatment sessions and solicited impact assessments from clinician colleagues. Discipline-specific weekly teleconferences provided the forum for clinicians to discuss pilot findings and agree to add, edit, or delete items from the form. Each discipline’s form was finalized after this 3-month period (January–March 2001).

IRF clinicians were trained to use intervention documentation forms via discipline-specific train-the-trainer teleconferences attended by a lead clinician in each specialty from each IRF. The project team facilitated this training for each clinical specialty using a training manual that included paper and electronic copies of the intervention documentation forms, instructions for completing the forms, and definitions for all terms used on the forms. Written case studies were included; 1 case study was used to demonstrate how to complete each form based on a patient scenario. Additional case studies were used to evaluate trainees’ understanding of instructions by providing examples of how to use the form for different patient scenarios.

After the telephone training session, each clinical leader conducted on-site training sessions for their coworkers. Teleconferences for each group were held throughout the 2 months following training to provide clinicians the opportunity to discuss implementation issues and ask questions of their peers in other participating institutions.

Each site incorporated auditing of intervention documentation form use into routine site practices. Typically, a second therapist (usually the lead therapist) observed a patient session and completed a separate intervention documentation form based on what was observed. The therapist providing the session completed a form as per protocol and the 2 were compared. The lead therapist reviewed and discussed differences in completion with the practicing therapist.

Rehabilitation intervention documentation forms were completed for each therapy session and nursing day for each study patient. After patient discharge, completed documentation forms (141,511 forms in all) were transmitted to the project office for optical character recognition interpretation and incorporation into the project database.

Face validity was built into the therapist intervention documentation forms, because they were developed and used by site therapists as described above. Clinicians came to agree with the content of their respective forms by discussing findings from the pilot test and then agreeing to add, edit, or delete items from the form (content validity).

Predictive validity was assessed as described in other articles¹³, ¹⁴, ¹⁵, ¹⁶ in this supplement by showing significant effects on outcomes of therapist interventions. For example, the amount of variation explained in discharge FIM score, controlling for patient characteristics (including admission FIM score, severity of illness, demographic factors), was 40% for moderate strokes and 45% for severe strokes. When total time per day spent on PT, OT, and SLP was added, there was no increase in variation explained for discharge FIM, consistent with previous findings by Bode et al.¹⁷ However, when time per day spent in specific PT, OT, and SLP activities was added, the amount of variation explained increased to 52% for moderate strokes and 68% for severe strokes, adding 12% to 23% explanation of variation, respectively, in discharge FIM score.

Postdischarge chart review was facilitated by the CSI software system that allows for both the input of severity-of-illness data and the creation of auxiliary data modules (ADMs), which are sets of study-specific data elements that are collected in addition to patients’ illness severity information. The PSROP clinical team identified and defined all patient, process, and outcome variables to include in the PSROP ADM. Using laptop computers, data collectors at each participating IRF entered chart review data into the CSI software system.

The signature component of the CSI software system is the disease-specific severity system. The CSI is an objective method to define illness severity based on individual signs and symptoms of patients’ diseases. Explicit severity criteria were developed by Susan Horn in conjunction with expert clinician panels, originally at the Johns Hopkins Hospital between 1980 and 1992, for each ICD-9-CM diagnosis code or group of similar diagnosis codes. To keep severity criteria up to date with medical practice, the criteria are reviewed and updated via physician panel discussions with each application of the CSI. The CSI defines severity of illness as the physiologic and psychosocial complexity presented to medical personnel due to the extent and interactions of a patient’s disease(s).⁸, ¹⁸, ¹⁹, ²⁰, ²¹, ²², ²³, ²⁴

Inputs to the CSI include over 2200 disease-specific and age-specific severity criteria including physical findings, historical factors, physiologic parameters, and laboratory results at specified levels of abnormality found in a resident’s chart. Treatments provided do not contribute to severity of illness. For example, intubation is not a severity criterion; severity criteria include patient signs and symptoms that led to a clinical decision to intubate (eg, respiratory acidosis, absent breath sounds, cyanosis).

Disease-specific criteria sets are determined by ICD-9-CM codes assigned routinely by trained facility diagnosis-related group (DRG) coding personnel. CSI data collection is performed via retrospective chart review after patient discharge, and thus, all diagnoses assigned by the facility diagnosis coder appear on a front or summary sheet in each patient’s chart. The CSI data collector enters the list of diagnosis codes into the CSI software system, which then displays disease-specific criteria to a trained data collector who abstracts the signs and symptoms that address the criteria from the patient’s medical record for specified time periods. It is important to note that the existence of a diagnosis does not indicate the extent of the disease. The CSI substantiates the diagnosis and allows for stratification based on documented patient signs and symptoms.

The stroke criteria set involves the neurologic, musculoskeletal, cardiovascular, and respiratory systems; vital signs; and laboratory values. The presence of a stroke ICD-9 code (eg, 430 [subarachnoid hemorrhage]) prompts for questions from the stroke criteria set, as listed in appendix 4: highest blood pressure, degree of alertness, ataxia, aphasia, dysarthria, dyspnea, perceptual and sensation impairment, dysphagia, hemiplegia, lesion level, time postinjury, acute confusion, and others. Each criterion is followed by response choices for the data collector to select; possible responses are presented in decreasing order of severity. Responses for the stroke dysphagia question, for example, include unable to swallow liquids, unable to swallow solids, other dysphagia, and no dysphagia. The data collector selects the appropriate response based on information found in the patient’s chart; data collectors are trained to select the most severe response (by order of presentation). A disease-specific criteria set exists for each group of similar ICD-9-CM codes; the CSI contains over 5500 criteria sets for specific diagnoses in 5 health care settings (acute care, rehabilitation, ambulatory, long-term care, hospice) with details similar to the stroke criteria set in appendix 4.

In the PSROP, each CSI criterion was answered separately for 3 time periods: admission to rehabilitation (first 24h), discharge from rehabilitation (discharge day), and maximum. (Maximum CSI score covers the full rehabilitation stay, including admission and discharge periods.) The maximum score reflects the most abnormal signs and symptoms, regardless of when they occur during the stay.

CSI severity scores reflect the interactions of various health conditions and diseases, as derived from variables in the disease-specific criteria sets. The CSI severity calculation engine assigns a severity weight to each criterion response, which then contributes to a severity rating for each diagnosis for each review period. To compute the overall severity score for a patient, the severity scores for all diagnoses are combined using disease-specific weighting rules that reflect the interaction of the diagnoses. The overall patient severity level is scored on a continuous scale with nonnegative integer values that are not subject to any preset maximum limit. The more abnormal the signs and symptoms, the higher the score, indicating that that patient is more severely ill. For example, a patient with stroke and congestive heart failure (CHF) probably would have a higher severity score than a patient with stroke alone. The CHF diagnosis does not indicate higher severity, but the signs and symptoms that determine acuteness of the disease contribute to the patient’s overall severity of illness. If the CHF is controlled and the patient exhibits no abnormal symptoms of the disease, the diagnosis will not contribute to the overall severity score. If, however, the patient exhibits symptoms of CHF such as shortness of breath, abnormal breath sounds, high pulse, low blood pressure, or respiratory acidosis, these symptoms will contribute to the overall CSI score. Thus, to produce the overall CSI score, CSI logic takes into account the interactions of diseases that are present, their severity levels, and the clinical relations of the diseases.

Often a patient is sickest on admission, and thus the admission and maximum CSI scores will be the same. However, when iatrogenic conditions develop, the maximum CSI score becomes larger (more severe) than the admission score; this is referred to as “increase in severity” in the Results section. Discharge CSI scores typically are the lowest because patients have improved and stabilized throughout the rehabilitation stay. Improvements (decreases) in severity scores were measured in 2 ways: (1) gross medical improvement—a decrease from maximum (full stay) CSI score to discharge CSI score—and (2) net medical improvement—a decrease from admission CSI score to discharge CSI score.

Advantages of this approach to measuring severity of illness include disease specificity, based on a concise, carefully chosen set of relevant physiologic characteristics of the particular disease rather than based on a standard set of physiologic factors applied to all diseases; comprehensive scope, with over 5500 disease-specific severity criteria sets representing all diseases for which there is an ICD-9-CM code; independence of treatments; and ability to measure severity during specified time windows in the care process. The CSI has been validated extensively in many inpatient, ambulatory, and long-term care settings since 1982.⁸, ¹⁸, ¹⁹, ²⁰, ²¹, ²², ²³, ²⁴

CSI criteria for stroke were examined and updated by the project clinical team at the beginning of the project to ensure their face validity for stroke rehabilitation patients. Predictive validity of the CSI and its components for stroke rehabilitation patients are shown elsewhere.¹³, ¹⁴, ¹⁵, ¹⁶, ²⁵, ²⁶, ²⁷, ²⁸ Although levels of disability are included in the CSI criteria set for patients with stroke, other components of the CSI remain significant in explaining outcomes after controlling for FIM score and other factors. For example, the amount of variation explained in discharge FIM score by demographic factors alone was 3% for patients with moderate strokes and 4% for patients with severe strokes. When the CSI score and its components were added, the amount of variation explained increased to 15% and 24%, respectively, for patients with moderate and severe strokes.

CPI methodology promotes collection of study-specific patient (in addition to severity-of-illness), process, and outcome data elements, identified and defined by the study team into an instrument referred to as the ADM within the CSI software system. The PSROP ADM contained over 200 variables; most contained date and time fields so that they could be associated with other variables in time sequence, and many have numerous data entries. For example, some data related to vital signs, weight, and pain were collected for each day of the rehabilitation stay, so these single variables have as many entries as the length of stay (LOS). The ADM contained an extensive table of selection choices for each variable; however, data collectors were trained to add to the selection table if a response was not present. For example, the durable medical equipment (DME) selection table contained 173 selection options, but data collectors added another 18 options, including elastic shoelaces and plate guard, during data collection. Appendix 5 presents an outline of the stroke ADM; the outline does not include table selection choices. Rehabilitation activities and interventions contained on each discipline’s point-of-care intervention documentation forms (see Appendix 1, Appendix 2, Appendix 3) were classified as process variables also but are not included in the ADM outline version in appendix 5.

Patient variables included age, sex, race, payer source, stroke risk factors, type of stroke (hemorrhagic, ischemic), side of stroke (left, right, bilateral), location of stroke (brainstem/cerebellum, subcortical, brainstem and subcortical, lobar, unknown), admission FIM score (total, motor, cognitive, and all components), case-mix group (CMG), acute care hospital LOS, and date and time of stroke symptom onset (which is subtracted from rehabilitation admission date and time to determine the number of days from stroke onset to rehabilitation admission).

Process variables included rehabilitation LOS, therapy intensity, and specific activities and interventions from point-of-care documentation forms; oxygen use; medications during rehabilitation care; incontinence interventions (eg, indwelling catheters); and nutritional interventions (eg, diet type, tube feeding).

Outcome variables in the ADM included discharge FIM scores, death, discharge destination (home, community, institution), repeat stroke, deep vein thrombosis (DVT), electrolyte imbalance, anemia, urinary tract infection (UTI), pneumonia, falls, mental disorders including depression, and elevated white blood cell count.

The functional performance for each study patient on admission to and discharge from inpatient rehabilitation was obtained via retrospective chart review using the study site’s recording of FIM scores. We assumed all clinicians providing FIM data within IRFs as part of standard practice were FIM credentialed; no additional documentation of FIM elements was performed for project purposes. The FIM is a widely used measure of performance across 13 motor areas and 5 cognitive areas and has been found to have “acceptably high” interrater reliability.²⁹, ³⁰, ³¹ The FIM describes a patient’s ability to perform various activities of daily living given various levels of assistance. A patient who is independent in completing a task is rated a 7, one who requires only supervision or contact guard is rated a 5, and one who is dependent is rated a 1 for that specific task. In the context of the PSROP project, admission and discharge FIM scores—total, component (motor, cognitive), and subscores (specific domains, eg, dressing upper and lower body, walking, bowel and bladder incontinence, problem solving)—were collected; however, FIM data were incomplete for 6.5% of the sample. FIM ratings also were used to determine “success” or “failure” in a given activity (ie, increasing the component FIM rating to a predetermined level) and to identify a homogeneous group of study patients for comparison of interventions and outcomes (eg, examining differences in interventions among all patients who were rated a 3 in auditory comprehension on admission).

We assigned each study patient to a CMG following stroke CMG definition rules based on motor FIM score, cognitive FIM score, and patient age (table 2).

Table 2. Stroke CMGs and CMG Groupings by Relative Tier Weights

Stroke CMG Groupings	PSROP⁎ Patients, n (%)	CMG	Stroke CMG Definition			Relative Weights
			Motor FIM Score	Cognitive FIM Score	Age (y)	Tier 1	Tier 2	Tier 3	None
Mild (CMG 101—103)	148(11.5)	0101	69—84	23—35	NA	0.478	0.428	0.408	0.386
		0102	59—68	23—35	NA	0.651	0.583	0.555	0.526
		0103	59—84	5—22	NA	0.830	0.743	0.708	0.670
Moderate (CMG 104—107)	511(39.6)	0104	53—58	NA	NA	0.901	0.807	0.769	0.728
		0105	47—52	NA	NA	1.134	1.016	0.968	0.916
		0106	42—46	NA	NA	1.395	1.249	1.191	1.127
		0107	39—41	NA	NA	1.616	1.447	1.379	1.305
Severe (CMG 108—114)	548(42.6)	0108	34—38	NA	≥83	1.748	1.565	1.492	1.412
		0109	34—38	NA	<83	1.890	1.693	1.613	1.527
		0110	12—33	NA	≥89	2.028	1.816	1.730	1.638
		0111	27—33	NA	82—88	2.089	1.871	1.783	1.687
		0112	12—26	NA	82—88	2.478	2.220	2.115	2.002
		0113	27—33	NA	<82	2.238	2.004	1.910	1.807
		0114	12—26	NA	<82	2.730	2.445	2.330	2.205

Source: Centers for Medicare and Medicaid Services.³²

Abbreviation: NA, not applicable (CMG calculation rules for cognitive FIM score apply only to CMG 101—103 and for age apply only to CMG 108—114).

Each IRF medical records abstractor completed a 4-day training session during which efficient and accurate collection of chart review data was explained and practiced. After the training session, each data collector underwent a rigorous manual reliability testing process to ensure complete and accurate data collection that went beyond internal data editing features of the CSI (eg, features that prohibit entry of nonsensible values). Reliability monitoring was conducted at 4 points throughout the PSROP to ensure that data accuracy was maintained. An agreement rate of 95% at the criteria level between each data collector and the project training-team reliability person was required for each reliability test.

Database Management 

The comprehensive PSROP database contains all point of care and chart review patient data. Patients and facilities are identified by study identification number only and cannot be identified directly or through linked identifiers. The entire CSI database was exported to SAS statistical software, release 8.2,^a for analysis.

Data Analyses 

For categoric variables, we used contingency tables to examine differences in frequencies and conducted bivariate analyses using chi-square tests to examine differences across sites. For continuous measures we used descriptive statistics, such as average, median, quartiles, and amount of variation (standard deviation [SD], range), and conducted bivariate analyses using ANOVA to test differences across sites and Pearson correlation to test associations of continuous variables. A 2-sided P value less than .05 was considered statistically significant.

Back to Article Outline

Results 

The study’s database includes 1291 patients from 7 inpatient rehabilitation facilities (1161 U.S. patients, 130 New Zealand patients). Tables 3 through 5 provide data on the key patient, process, and outcome characteristics of the U.S. portion of the PSROP study group and for each site separately.

Table 3. PSROP Patient Characteristics

Characteristics	Site 1 (n=209)	Site 2 (n=198)	Site 3 (n=186)	Site 4 (n=199)	Site 5 (n=206)	Site 6 (n=163)	Site Variation Significance (P)	Full Sample (n=1161)	National Data⁎
Demographic and health plan characteristics
Mean age (y)	68.0	67.5	65.7	64.4	66.1	64.2	.059†	66.0	69.7
Sex (% male)	49.8	56.1	51.1	58.3	47.6	47.9	.175‡	51.8	46.4
Race (%)							<.001‡
White	29.2	68.2	50.0	85.9	82.0	27.6		60.6	ND
Black	61.7	7.1	12.9	7.5	1.0	71.2		23.2	ND
Other, including Hispanic	9.1	24.7	37.1	6.6	17.0	1.2		16.2	ND
Payer (%)							<.001‡
Medicare	56.0	61.1	66.7	50.8	53.4	47.2		56.0	ND
Medicaid	6.7	0.0	15.6	12.1	3.4	23.9		9.7	ND
Commercial	36.8	38.4	9.1	36.2	32.0	28.2		30.5	ND
Self-pay	0.5	0.5	5.9	0.0	7.8	0.6		2.6	ND
Unknown/missing	0.0	0.0	2.7	1.0	3.4	0.0		1.2	ND
Health and functional status characteristics
Stroke risk factors (%)
Previous stroke (excludes TIA)	29.2	27.3	32.8	24.6	28.6	24.5	.485‡	27.9	ND
Hypertension diagnosis	82.3	74.6	75.3	75.4	81.1	82.8	.138‡	78.6	ND
Diabetes diagnosis	37.8	23.2	32.8	24.6	32.5	33.7	.010‡	30.8	ND
Obesity diagnosis	4.8	3.0	4.8	6.0	12.1	1.8	<.001‡	5.6	ND
Smoking history (%)							<.001‡
Never smoked	53.1	52.0	16.1	43.2	60.7	44.8		45.5	ND
Quit >1y before stroke	21.0	25.3	12.4	25.6	17.5	17.2		20.0	ND
Current smoker	19.1	13.6	23.7	23.0	18.9	33.7		21.6	ND
Unknown/missing	5.3	8.6	47.9	7.5	2.9	4.3		12.5	ND
Type of stroke (%)							<.001‡
Hemorrhagic	9.1	28.8	42.5	19.1	9.1	26.2		23.2	ND
Ischemic	90.9	71.2	57.5	90.9	80.9	73.8		76.7	ND
Side of stroke (%)							<.001‡
Right	36.8	46.5	48.9	49.3	39.3	45.4		44.2	42.1§
Left	37.8	43.4	45.7	41.7	44.2	42.3		42.5	42.3§
Bilateral	22.5	7.6	3.8	8.0	12.6	6.8		10.5	3.0§
Unknown	2.9	2.5	1.6	1.0	3.9	5.5		2.8	ND
Location of stroke (%)							<.001‡
Brainstem/cerebellum	19.1	20.2	11.8	18.6	18.0	23.3		18.4	ND
Subcortical	42.6	31.3	37.6	27.6	31.6	39.9		35.0	ND
Brainstem + subcortical	10.1	2.5	4.3	4.0	5.3	1.8		4.8	ND
Lobar	25.4	38.4	40.3	41.7	42.7	33.1		37.0	ND
Unknown	2.9	7.6	5.9	8.0	2.4	1.8		4.8	ND
Mean admission total FIM score	61.1	64.9	47.4	70.9	54.5	67.9	<.001†	61.0	56.7§
Mean admission motor FIM score	38.3	42.2	31.4	46.7	38.3	43.7	<.001†	40.1	35.5§
Mean admission cognitive FIM score	22.9	22.7	16.1	24.3	16.5	24.0	<.001†	21.0	ND
CMG (%)
Mild (101—103)	4.3	10.1	3.2	16.1	13.1	8.6	<.001‡	11.5	ND
Moderate (104—107)	35.4	46.5	25.8	57.8	29.1	59.5	<.001‡	39.6	ND
Severe (108—114)	40.7	39.9	67.7	25.1	50.5	28.2	<.001‡	42.5	ND
Mean admission CSI score	19.5	12.8	27.0	15.3	30.0	19.7	<.001†	20.7	ND
Mean no. of diagnosis codes	11.3	10.3	12.8	7.9	15.4	6.0	<.001†	10.8	ND
Stroke symptoms
Aphasia	19.6	24.2	39.8	18.1	16.5	12.3	<.001‡	21.8	ND
Complete hemiplegia	0.5	0.0	17.7	4.0	15.1	19.0	<.001‡	9.0	ND
Incomplete hemiplegia	91.4	98.0	62.4	86.4	56.8	65.0	<.001‡	77.2	ND
Dysarthria	51.7	41.9	48.5	33.6	56.5	70.6	<.001‡	49.3	ND
Dysphagia	43.1	53.5	71.5	57.8	74.3	52.8	<.001‡	58.8	ND
Ataxia	19.1	15.7	34.4	9.6	61.7	87.7	<.001‡	36.5	ND
Acute confusion	10.5	22.7	62.4	9.1	25.7	30.1	<.001‡	26.1	ND
Bowel/bladder incontinence	64.1	77.8	83.9	57.3	73.3	55.8	<.001‡	68.9	ND
Prerehabilitation health care
Mean no. of days from stroke onset to rehabilitation	15.3	13.0	12.9	21.7	10.0	9.1	<.001†	13.8	ND
Mean acute hospital LOS preceding rehabilitation	10.1	9.0	7.5	8.9	6.8	8.2	.006†	8.6	ND

NOTE. For U.S. patients, n=1161.

Abbreviations: ND, no data; TIA, transient ischemic attack.

Table 4. PSROP Process Variables

Variables	Site 1 (n=209)	Site 2 (n=198)	Site 3 (n=186)	Site 4 (n=199)	Site 5 (n=206)	Site 6 (n=163)	Site Variation Significance (P)	Full Sample (n=1161)	National Data⁎
Mean LOS (d)	21.5	18.2	20.5	16.3	20.2	14.4	<.001†	18.6	17.7
Mean PT (median)
No. of days	15.7 (15)	12.3 (11)	13.5 (12)	13.7 (14)	14.0 (12)	8.0 (7)	<.001†	13.0 (12)	ND
No. of minutes	910 (865)	775 (653)	903 (675)	816 (810)	709 (586)	670 (580)	<.001†	800 (725)	ND
No. of minutes per day	43.1 (43.5)	40.8 (42.0)	42.3 (40.3)	50.0 (50.8)	35.2 (38.6)	45.9 (52.8)	<.001†	42.8 (43.0)	ND
Mean OT (median)
No. of days	10.6 (10)	11.4 (10)	13.0 (11)	12.5 (13)	12.9 (10.5)	5.5 (4)	<.001†	11.1 (10)	ND
No. of minutes	655 (575)	764 (638)	913 (715)	803 (745)	677 (615)	446 (360)	<.001†	715 (625)	ND
No. of minutes per day	30.8 (30.6)	40.7 (41.2)	42.8 (43.2)	49.3 (50.4)	33.4 (36.9)	31.2 (32.3)	<.001†	38.1 (39.1)	ND
Mean SLP (median)
No. of days	8.9 (7)	11.3 (10)	11.9 (10)	8.0 (8)	13.5 (11)	ND	<.001†	10.7 (9)	ND
No. of minutes	425 (355)	625 (500)	749 (560)	360 (315)	735 (613)	ND	<.001†	576 (438)	ND
No. of minutes per day	18.8 (20.0)	32.5 (35.5)	34.6 (35.0)	22.5 (20.8)	36.0 (38.6)	ND	<.001†	28.8 (29.5)	ND
Oxygen use (%)	11.0	0	18.3	3.5	53.4	11.0	<.001‡	16.5	ND
Tube feeding use (%)	13.4	7.1	18.8	7.5	43.7	8.6	<.001‡	16.9	ND
Antidepressant use (%)	32.5	39.9	64.0	43.2	68.5	50.3	<.001‡	49.5	ND
Antipsychotic use (%)	5.7	6.6	4.8	3.0	27.7	6.1	<.001‡	9.2	ND
Opioid pain medication use (%)	8.1	24.8	39.3	16.6	33.5	12.3	<.001‡	22.5	ND
Antiseizure medication use (%)	16.3	14.7	29.0	11.1	21.8	15.3	<.001‡	18.0	ND

NOTE. For U.S. patients, n=1161.

Table 5. PSROP Outcome Variables

Variables	Site 1 (n=209)	Site 2 (n=198)	Site 3 (n=186)	Site 4 (n=199)	Site 5 (n=206)	Site 6 (n=163)	Site Variation Significance (P)	Full Sample (n=1161)	National Data⁎
Complications
Any mental disorder diagnosis (%)	61.2	50.0	56.5	58.8	63.6	19.0	<.001†	52.6	ND
Depression diagnosis (%)	10.5	11.6	12.4	4.5	29.1	4.9	<.001†	12.5	ND
Pneumonia diagnosis (%)	5.3	10.6	9.7	4.5	17.0	2.5	<.001†	8.4	ND
DVT diagnosis (%)	4.8	2.0	7.0	4.0	11.7	3.7	.001†	5.6	ND
UTI diagnosis (%)	29.2	28.3	35.5	14.1	50.5	12.3	<.001†	28.9	ND
Electrolyte imbalance diagnosis (%)	6.7	29.3	38.7	5.5	32.5	12.9	<.001†	20.9	ND
Anemia diagnosis (%)	30.1	11.1	8.1	6.5	34.0	1.2	<.001†	15.9	ND
Falls (%)	21.5	12.1	12.9	9.6	9.2	12.9	.002†	13.1	ND
Elevated white blood cell count (>11.0 × 109/L) (%)	10.1	8.6	26.9	10.6	27.7	8.6	<.001†	15.5	ND
Severity (CSI) during rehabilitation
Mean maximum CSI score	31.4	19.0	39.7	20.8	47.5	29.6	<.001‡	31.4	ND
Mean increase in severity (maximum — admission)	11.9	6.2	12.7	5.6	17.5	9.9	<.001‡	10.7	ND
Mean discharge CSI score	9.8	1.2	16.8	9.1	16.7	9.6	<.001‡	10.5	ND
Mean gross medical improvement (maximum − discharge CSI score)	21.6	17.8	22.8	11.7	30.7	20.0	<.001‡	20.9	ND
Mean net medical improvement (admission − discharge CSI score)	9.7	11.7	10.1	6.1	13.2	10.1	<.001‡	10.2	ND
FIM score
Mean discharge total FIM score	86.9	91.7	73.4	95.0	84.7	91.3	<.001‡	87.2	76.2
Mean increase in total FIM (discharge − admission score)	26.8	27.0	25.9	24.0	30.1	23.2	<.001‡	26.2	19.5
Mean discharge motor FIM score	60.0	64.8	52.7	66.9	62.4	64.4	<.001‡	61.9	51.8
Mean increase in motor FIM score (discharge − admission)	22.4	22.6	21.4	20.2	24.6	20.4	.003‡	21.9	16.3
Mean discharge cognitive FIM score	27.0	26.9	20.7	28.1	21.9	26.9	<.001‡	25.2
Mean increase in cognitive FIM score (discharge − admission)	4.4	4.2	4.6	3.8	5.2	2.8	<.001‡	4.2	ND
Discharge destination (%)
Community vs institution							<.001†
Community discharge includes home	82.3	84.8	73.1	90.0	72.8	82.8		81.0	70.6
Inpatient institutional discharge	17.7	14.1	26.9	10.1	26.2	16.0		18.5	28.5
Died	0.0	1.0	0.0	0.0	1.0	1.2		0.7	0.3
Home only	81.8	80.8	69.4	81.4	72.3	82.8	.003†	78.0	67.5

NOTE. For U.S. patients, n=1161.

Patient Characteristics 

Table 3 provides a profile of the patient characteristics of the U.S. portion of the PSROP (n=1161) and when available, compares study data to national data available in eRehabData.

Demographic and Health Plan Status 

The mean age of the study group was 66.0 years, which is somewhat younger than the mean age in the national eRehabData reference group of 69.7 years. The study group’s age did not vary significantly across the 6 sites, but differences were borderline (P=.059). The study’s sex distribution also did not vary significantly across the sites, but there were proportionately more men in our study group (51.8%) than in the national reference group (46.4%).

The largest difference across the sites was the study group’s racial distribution, where 2 sites were predominately white and 2 sites were predominately black. The 2 remaining sites had a more even racial distribution.

Fifty-six percent of the study group had Medicare as the primary payer, and commercial insurance covered about 30%. A small proportion (2.6%) self paid.

Health and Functional Status 

The most frequent stroke risk factors in the study group were diagnoses of hypertension (78.6%) and diabetes (30.8%). A small portion (5.6%) of the sample had an obesity diagnosis; most had never smoked (45.5%) or had quit smoking more than a year before stroke (20%). Most did not have a history of alcohol abuse (12% current or former abuse).

Most strokes were ischemic in origin (76.7%) and about evenly divided between right (44.2%) or left side (42.5%) of the brain, which is similar to national data (42.1% and 42.3%, respectively). About 10% of the sample had a bilateral stroke (national data, 3.0%). Most strokes were subcortical or lobar, with a smaller percentage of brainstem and cerebellar infarcts.

Mean admission FIM scores (total, 61.0; motor, 40.1) were slightly higher than in the national reference group (56.7 and 35.5, unweighted, respectively). No national reference group data were available for the FIM admission cognitive component (study data, 21.0). Significant differences existed among sites in mean motor, cognitive, and total admission FIM scores (all P<.001).

All stroke CMGs are represented in the study group, with the largest number in the more severe CMGs. We combined CMGs into mild (CMGs 101–103), 11.5% of the sample; moderate (CMGs 104–107), 39.6% of the sample; and severe (CMGs 108–114), 42.5% of the sample. The 6.5% of patients who had incomplete FIM data were not classified into CMG groups.

Severity-of-illness distributions (higher scores indicate more severe) differed significantly among sites for rehabilitation admission (first 24h), ranging from 12.8 to 30.0 (P<.001). The number of diagnosis codes per patient correlated significantly with the patient’s admission severity score (Pearson r=.45, P<.001). Site 5 had the highest severity scores, the most diagnoses, and the second lowest functioning scores. Site 2 had the lowest severity scores; however, it did not have the least number of diagnoses or the highest functioning scores.

As might be expected, hemiplegia was found in the majority of the sample (>86%); bowel and bladder incontinence (as measured by admission FIM bowel and bladder scores of ≤4) was also common. Significant differences were seen among sites; most notably, site 3 had more than twice the number of patients with an aphasia diagnosis than most other sites.

Prerehabilitation Health Care 

The study group was admitted to rehabilitation an average ± SD of 13.8±20.8 days (median, 7d; range, 0–319d) after the first onset of symptoms. Interim stays in acute care facilities and skilled nursing facilities are included here. Significant differences were found among sites ranging from 9.1 to 21.7 days (P<.001).

The average LOS in an acute care hospital before rehabilitation admission was 8.6 days; this differed significantly among sites (site average LOS range, 6.8–10.1; P=.006). The mean number of days from symptom onset to acute care hospital admission was 1.4±4.2 days (median, 0d; range, 0–51d).

Process Variables 

Process variables are detailed by site and overall in table 4.

The mean rehabilitation LOS for our study group was 18.6 days, which is slightly higher than the eRehabData national data (17.7d). Three of our sites had mean LOSs of more than 20 days, and 3 had mean LOSs between 14 and 18 days (P<.001).

Most PSROP patients received at least 1 session of PT (96.7%) or 1 session of OT (94.9%) during their rehabilitation stay. The vast majority of these (94.6%) had at least 1 session of both PT and OT. Only 2.9% of study patients had neither PT nor OT. One site submitted very few SLP intervention documentation forms; therefore, we excluded that site from SLP analyses. After that exclusion, 93.8% of patients received SLP.

Statistically different numbers of days and numbers of minutes of PT, OT, and SLP are seen among study sites. On average, the 3 therapies averaged about 29 to 43 minutes a day when therapy was provided (PT, 42.8min/d; OT, 38.1min/d; SLP, 28.8min/d).

Study sites varied significantly in the use of specific treatments, including use of tube feeding for nutritional support and different types of medications. Forty-nine percent of the sample received an antidepressant medication; 9.2% received an antipsychotic. Almost 23% of study patients received opioid pain medications. Differences in medication use by site²⁵ and differences in the use of tube feeding to provide nutrition²⁶ are discussed elsewhere.

Outcome Variables 

Outcome variables are detailed by site and overall in table 5.

More than half the sample (52.6%) had a documented mental disorder: depression (ICD-9 code 311), 12.5%; organic psychotic condition (ICD-9 code 294), 13.6%; and adjustment reaction (ICD-9 code 309), 8.0%.

The most common medical complications during stroke rehabilitation in our study sample were UTIs (28.9%), anemia (15.9%), and electrolyte imbalances (20.9%); DVTs occurred least frequently (5.6%). We found significant site variation in all measured outcome variables (P≤.003).

Severity of Illness (CSI) 

Some patients (11%) had an increase in CSI score during the rehabilitation stay, indicating that their illness severity increased during the stay from what it was at admission. The mean increase in severity for the full sample was 10.7 (20.7 on admission, 31.4 at maximum); significant site variation was found (P<.001).

Significant differences were found among sites in gross medical improvement (decrease from maximum [full-stay] CSI score to discharge CSI score) and net medical improvement (decrease from admission CSI score to discharge CSI score; ANOVA, P<.001).

FIM Scores 

The mean discharge total and motor FIM scores for the study population were higher than for the national sample (total FIM: 87.2 vs 76.2; motor FIM: 61.9 vs 51.8, respectively); larger increases in total and motor FIM scores also were seen in the study sample (total FIM: 26.2 vs 19.5; motor FIM: 21.9 vs 16.3, respectively). Data for cognitive FIM components are not provided in the national data. Study sites differed significantly in mean motor, cognitive, and total discharge FIM scores (all P<.001).

As seen in table 3, the 2 facilities (sites 3 and 5) that had the lowest functioning patients, as measured by admission FIM scores, also had the highest severity (“sickest”) patients, as measured by the highest admission CSI scores.

Rehabilitation Discharge Destination 

Most study patients (81%) were discharged from the rehabilitation center to a community setting and the vast majority of these were to the resident or family home (78%). This compares with national statistics of 70.6% and 67.5%, respectively. The study sample had about double the percentage of deaths (0.7%) when compared with the national sample (0.34%).

Back to Article Outline

Discussion 

The wide-ranging effects of stroke are a challenge for determining the right match between a stroke survivor’s needs and the appropriate rehabilitation services. Failure to find the right fit can result in too little or too much care for a patient’s individual needs. We cannot clinically and fiscally allocate appropriate rehabilitation services for every patient with stroke without stronger detailed scientific evidence showing the effectiveness of poststroke rehabilitation interventions. By using the CPI approach, the PSROP assembled a comprehensive database providing the opportunity to examine the complex interplay of patient and process factors and their impact on stroke patient outcomes.

Because of the central role played by the project team in all aspects of CPI, this approach can be characterized as a form of participatory action research—a bottom-up approach that values the participation of those actually engaged in the care-providing process and garners their participation in implementing study findings. CPI encourages new findings, even those that challenge conventional wisdom and long-standing practice.

During this study, there were extraordinary contributions of clinical expertise and time to develop new intervention documentation forms by clinical team members at each IRF. The inclusive nature of the CPI approach retained clinician participation for more than 5 years with no financial rewards. Physicians, therapists, and social workers, among others, realized that better understanding of the details of everyday practice (obtained from data, not expert consensus) and the association of these details with patient outcomes can make great contributions to better outcomes for patients with stroke and better training and practice techniques for clinicians. The level of detail about rehabilitation care that became a part of the supplemental intervention documentation forms had never been documented before and provided tremendous potential to discover treatments that are best for specific patient types.

The CSI enabled us to go beyond controlling only for stroke severity: it allowed us to control for many complex comorbidities common to patients with stroke (particularly those with severe stroke), reflecting more accurately the realities of clinical practice. The strength of the CSI’s mechanism for compensating or adjusting for differences among patients allows for a more powerful assessment of the effectiveness of therapeutic interventions. The CSI’s use of very specific, disease-oriented questions produced a highly sensitive measure of severity that could not be produced by using diagnosis and/or procedure codes alone or a limited, fixed set of physiologic criteria no matter what the underlying diagnoses may be. Diagnosis codes indicate existence of disease; they do not indicate extent of disease.

Study sites with higher severity-of-illness scores tended to have lower functioning scores (FIM) on admission. The pattern continued at discharge where again, sites with higher discharge severity scores tended to have lower functional scores. Similarly, study sites with lower severity scores tended to have higher-functioning patients (see Table 3, Table 5). Study sites that had higher severity-of-illness scores also had higher use of more intensive treatments such as oxygen use and nutritional tube feedings (see table 4).

Limitations 

CPI methodology relies on the expertise of participating facility clinicians to guide the development of high-level study hypotheses and identify critical data elements to study. As such, these clinicians are aware of study data elements as they provide care and complete point-of-care intervention documentation forms or perform routine documentation practices. This could be construed as introducing treatment bias; however, the number of clinicians who participated in the development of study instruments was a very small subset of all clinicians who cared for over 1200 stroke rehabilitation patients in 7 facilities in 2 countries. Intervention documentation forms and project hypotheses were designed to capture descriptions of actual practice, not to alter practice patterns. In addition, the novelty of attention to specific study questions would wane over the course of an extended patient enrollment period (8mo to 2y, depending on site size and stroke volume).

As much as supplemental point-of-care intervention documentation forms provide an unprecedented level of detail about rehabilitation interventions, they also have intrinsic limitations. Add-on documentation to traditional IRF practices increases the documentation burden of front-line staff and allotted documentation time may not be sufficient to ensure complete documentation of both. Intervention documentation form training was conducted via a train-the-trainer approach using a lead clinician in each rehabilitation discipline in each IRF. Thus, the training of most clinicians depended on the expertise and time availability of the IRF trainers. Monitoring of documentation accuracy became an obligation of each IRF. The project clinical team received reports of IRF auditing processes and findings but did not intervene directly to determine the level of accuracy of documentation form completion. The project also depended on each IRF to package all intervention documentation forms and send them to the project office for scanning into the project database. Despite these limitations, significant variation in outcomes was found because of differences in time spent per day in various therapy activities.¹⁴, ¹⁵, ¹⁶

The original intent of the PSROP was to collect data from both the acute care hospital and rehabilitation records to cover the full poststroke course for each patient. However, budgetary and time constraints, as well as lack of convenient access to acute care charts, resulted in complete data from acute care hospital records being collected for only a small portion of the study population. In the end, data collection focused primarily on the rehabilitation stay. This led to lack of ability to control for some patient and process variables that could affect functional outcomes—for example, initial stroke severity (CSI score in acute care), blood pressure, temperature, and glucose levels in the early poststroke period; acute care complications such as seizures; and details of therapies and medications received during acute care hospitalization.

In the initial planning stages, efforts were made to identify and use an objective, validated measure of initial stroke severity (ie, the National Institutes of Health Stroke Scale), but no standard measure was in use across all participating IRFs. Hence, we did not include such a measure. We also did not assess admitting criteria for each IRF, which may have had an effect on types of patients admitted and, thus, types of patients included in the study.

A physiologic severity indexing system, such as the CSI, is limited by data availability. Credentialed DRG coding personnel at each facility assign ICD-9-CM codes as part of standard operating IRF procedures; it is these codes that determine reimbursement. We did not evaluate the credentialing procedures, nor did we audit code assignment. However, the difference in average number of ICD-9-CM codes assigned in facilities (range, 6−15) is curious. A smaller number of ICD-9-CM codes may result in lower severity of illness when using a system that is built on ICD-9-CM coding. Indeed, the facility with the highest average number of ICD-9-CM codes (15.4) did have the highest mean severity-of-illness score (30). However, the facility with the lowest average number of ICD-9-CM codes (6.0) did not have the lowest mean severity scores; there were 2 facilities with lower admission and maximum mean severity scores. If laboratory tests are not ordered, findings are not clearly reported, or complications are not documented, the severity or incidence rate for the related conditions will be lower. The incidence and type of test ordering and availability of information was not uniform across sites and could account for a significant portion of the site variability reported in CSI scores. However, the CSI and/or its components were significant predictors of various outcomes.¹³, ¹⁴, ¹⁵, ¹⁶, ²⁵, ²⁶, ²⁷, ²⁸

Lack of a defined time point for measurement of function after stroke was another limitation of the study. Ideally, the FIM score would be measured at some predetermined endpoint (eg, 90d after stroke onset) for all patients. Use of the discharge FIM scores is less satisfactory, because discharge itself is affected by institutional policies, patient preferences, socioeconomic concerns, insurance coverage, rate of recovery, and other variables.

The rehabilitation setting database created from this project is extremely rich in detail. Of course, this leads to an additional limitation—not all data are reported in this supplement. We plan future publications that capture descriptions and contributions (intervention documentation) of other members of rehabilitation care teams, such as nurses and social workers.

Despite these limitations, having micro-level data provided the ability to focus on the individual patient level to explore reasons for our findings. The PSROP was the first application of CPI methodology to a rehabilitation population and process. As with any new application of a process, a certain amount of trial and error occurred. This knowledge will accrue to subsequent CPI studies for other impairment categories and settings, allowing significant incremental improvements in the efficiency, methodology, and reliability of such studies. The knowledge learned also will facilitate completion of CPI process steps 6 and 7 (validation implementation, protocol incorporation) in future work.

Back to Article Outline

Conclusions 

The PSROP created a comprehensive database to assess the importance of such stroke variables as sex, race, severity of illness, baseline level of functioning, and various therapy interventions on patient-centered outcomes. The PSROP allowed us to describe the duration, intensity, and components of treatment regimens. In addition, the PSROP allowed us to discover treatment practices that are associated with better outcomes for patients with various levels of impairment after stroke. These include findings about medications, PT, OT, SLP, timing of rehabilitation, and nutritional support. Subsequent articles in this supplement describe these findings in detail.¹³, ¹⁴, ¹⁵, ¹⁶, ²⁵, ²⁶, ²⁷, ²⁸

Suppliers

Back to Article Outline

Acknowledgments 

We acknowledge contributions of collaborators at each clinical site represented in the Post-Stroke Rehabilitation Outcomes Project: Brendan Conroy, MD (Stroke Recovery Program, National Rehabilitation Hospital, Washington, DC); Richard Zorowitz, MD (Department of Rehabilitation Medicine, University of Pennsylvania Medical Center, Philadelphia, PA); David Ryser, MD (Neuro Specialty Rehabilitation Unit, LDS Hospital, Salt Lake City, UT); Jeffrey Teraoka, MD (Division of Physical Medicine and Rehabilitation, Stanford University, Palo Alto, CA); Frank Wong, MD, and LeeAnn Sims, RN (Rehabilitation Institute of Oregon, Legacy Health Systems, Portland, OR); Murray Brandstater, MD (Loma Linda University Medical Center, Loma Linda, CA); and Harry McNaughton, MD (Wellington and Kenepuru Hospitals, Wellington, NZ). We also acknowledge the role of Alan Jette, PhD (Rehabilitation Research and Training Center on Medical Rehabilitation Outcomes, Boston University, Boston, MA).

Back to Article Outline

Appendix 1. 

Pt intervention documentation form^⁎

©ISIS, 2003. Reprinted with permission.

Abbreviations: FWW, front-wheel walker; KAFO, knee-ankle-foot orthosis; NDT, neurodevelopmental treatment; PNF, proprioceptive neuromuscular fasciculation; PROM, passive range of motion.

Back to Article Outline

Appendix 2. 

OT intervention documentation form^⁎

©ISIS, 2003. Reprinted with permission.

Back to Article Outline

Appendix 3. 

SLP intervention documentation form^⁎

©ISIS, 2003. Reprinted with permission.

Abbreviations: DPNS, direct pharyngeal nerve stimulation; EMG, electromyography; ROM, range of motion.

Back to Article Outline

Appendix 4. CSI criteria set for stroke 

Criteria	Selection Choice Options (from highest to lowest severity)
Three-dimensional impairment array
Degree of impairment	Complete or incomplete high or low quadriplegia, complete or incomplete hemiplegia, upper or lower monoplegia
Ambulatory status	Ambulatory, nonambulatory
Time postinjury	Number of days or number of weeks
Neurologic status	Unresponsive, acute confusion, chronic confusion
Lowest Glasgow Coma Scale score
Degree of alertness	Coma, stupor, lethargic, drowsy, alert
Seizures	Status epilepticus, grand mal seizure, focal or petit mal seizure focal tremors
Pupil reaction	Bilateral pupil dilation, unilateral pupil dilation
Coordination/balance	Severe ataxia, moderate-mild ataxia, dizziness, vertigo, unsteady on feet, clumsiness, other altered coordination
Sensation alteration	Complete loss of sensation, paresthesia, dysthesia, other sensation alteration
Aphasia	Global aphasia, severe-moderate aphasia, mild aphasia
Dysarthria	No speech, incomprehensible sounds, dysphonia, other dysarthria
Dysphagia	Unable to swallow solids, unable to swallow liquids, other dysphagia
Nausea/vomiting	Persistent vomiting, other vomiting, nausea
Headache	Intense headache, moderate to severe headache, other headache
Dyspnea	Dyspnea at rest, dyspnea on exertion, breathing difficulties
Rales	Rales >50% of lung fields, rales ≤50% of lung fields
Breath sounds	Absent breath sounds in >50 of lung fields, decreased breath sounds in >50% of lung fields, decreased breath sounds in ≤50% of lung fields
Apnea	Apnea, no apnea
Perceptual impairment	Acute decline in perceptual function, chronic perceptual impairment requiring internal or external cues, intermittent perceptual limitations
EKG rhythm	Ventricular tachycardia, >6 PVCs/min, SVT, bigeminy, trigeminy, quadrigeminy, atrial fibrillation, PACs, other ectopics
Highest blood pressure, systolic and diastolic
Lowest systolic blood pressure
Highest pulse
Lowest pulse

Abbreviations: EKG, electrocardiogram; PAC, premature atrial contraction; PVC, premature ventricular contraction; SVT, supraventricular tachycardia.

Back to Article Outline

Appendix 5. PSROP ADM outline 

I. Patient demographics/history

1. Race, religion

2. Cardiovascular history

3. Alcohol, smoking, illicit drug use/history

4. Before stroke: ambulation capability, ADLs, DME, medication

5. Education level, career, financial stressors

6. Acute care LOS

II. Rehabilitation information

1. Stroke details

A. Onset of symptoms (date, time)

B. First medical contact (date, time, location)

C. Stroke details (side, type/location of stroke, vascular involvement, cerebral edema)

2. Therapies missed (date, time, reason), oxygen increased requirements during therapy

3. Mental status assessment

4. Vitals from nursing progress notes/flow sheets

A. Daily high temperature, high/low systolic BP, high/low diastolic BP, weight

B. Suction type/frequency

C. Lowest oxygen saturation, maximum oxygen requirements

D. Pain (0–10 scale, location)

E. Diarrhea/constipation

5. Bladder and bowel training programs

A. Indwelling catheter (insertion/removal dates)

B. Intermittent catheter (insertion/removal dates)

C. Postvoid residual measurements (each date)

D. Bladder scan date

E. Prompted bladder program date

F. Prompted bowel program date

6. Respiratory management

A. Ventilator start/stop date/time

B. Tracheostomy start/stop date/time

C. Sleep apnea

D. CPAP/BiPAP

7. Nutrition

A. Dietary consult dates

B. Type of diet

C. By mouth nutritional beverage/supplement

D. Tube feedings

E. Calorie counts

8. Complications

A. Pressure ulcer assessment (date, location, stage, width, length, depth, tissue type)

B. Deep vein thrombosis (date, location)

C. Fall (date, time, type)

D. Physical restraints/electronic monitoring (start/stop date/time)

E. Elevated white blood cell count >11.0×109/L (date, value)

9. Medications (varying levels of dosing information based on drug type)

10. Laboratory test results (albumin, pre-albumin, INR, homocysteine, serum drug levels)

11. FIM scores (all FIM data for admission, discharge, and other times of completion)

12. Family/caregiver assessment

A. Caregivers (who)

B. Physical support available from caregivers after discharge (some, 24 hours)

C. Supervision available from caregivers after discharge (some, 24 hours)

13. Discharge

A. Projected location

B. Actual location

C. Reason different

D. Recommended discharge therapy programs

E. Support programs

F. Acute care DRG (if returned to acute care)

G. Reason for discharge to acute care

H. Discharge DME

Abbreviations: ADLs, activities of daily living; BiPAP, bilevel positive airway pressure; BP, blood pressure; CPAP, continuous positive airway pressure; INR, international normalized ratio.

Back to Article Outline

References 

1. Van Peppen RP , Kwakkel G , Wood-Dauphinee S , Hendricks HJ , Van der Wees PH , Dekker J . The impact of physical therapy on functional outcomes after stroke (what’s the evidence?) . Clin Rehabil . 2004;18:833–862
   • View In Article
   • MEDLINE
   • CrossRef
2. Sulch D , Perez I , Melbourn A , Kalra L . Randomized controlled trial of integrated (managed) care pathway for stroke rehabilitation . Stroke . 2000;31:1929–1934
   • View In Article
   • MEDLINE
   • CrossRef
3. Alexander H , Bugge C , Hagen S . What is the association between the different components of stroke rehabilitation and health outcomes? . Clin Rehabil . 2001;15:207–215
   • View In Article
   • MEDLINE
   • CrossRef
4. Jette DU , Latham NK , Smout RJ , Gassaway J , Slavin MD , Horn SD . Physical therapy interventions for patients with stroke in in-patient rehabilitation facilities . Phys Ther . 2005;85:238–248
   • View In Article
   • MEDLINE
5. Elman RJ , Bernstein-Ellis E . The efficacy of group communication treatment in adults with chronic aphasia . J Speech Lang Hear Res . 1999;42:411–419
   • View In Article
   • MEDLINE
6. Bollinger R , Musson N , Holland AL . A study of group communication intervention with chronically aphasic persons . Aphasiology . 1993;7:301–313
   • View In Article
7. DeJong G , Horn SD , Conroy B , Nichols D , Healton EB . Opening the black box of poststroke rehabilitation (stroke rehabilitation patients, processes, and outcomes) . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S1–S7
   • View In Article
   • MEDLINE
8. In:  Horn SD editors. Clinical practice improvement methodology (implementation and evaluation) . New York: Faulkner & Gray; 1997;
   • View In Article
9. Horn SD , DeJong G , Ryser DK , Veazie PJ , Teraoka J . Another look at observational studies in rehabilitation research (going beyond the holy grail of the randomized controlled trial) . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S8–S15
   • View In Article
   • MEDLINE
10. McNaughton H , DeJong G , Smout RJ , Melvin JL , Brandstater M . A comparison of stroke rehabilitation practice and outcomes between New Zealand and United States facilities . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S115–S120
    • View In Article
    • MEDLINE
11. In:  Hart AC ,  Schmidt KM ,  Aaron WS editor. ICD-9-CM code book . Reston: St. Anthony’s Publishing; 1999;
    • View In Article
12. DeJong G , Horn SD , Gassaway J , Dijkers M , Slavin M . Toward a taxonomy of rehabilitation interventions (using an inductive approach to examine the ‘black box’ of rehabilitation) . Arch Phys Med Rehabil . 2004;55:678–686
    • View In Article
13. Latham NK , Jette DU , Slavin M , et al.   Physical therapy during stroke rehabilitation for people with different walking abilities . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S41–S50
    • View In Article
    • MEDLINE
14. Richards LG , Latham NK , Jette DU , Rosenberg L , Smout RJ , DeJong G . Characterizing occupational therapy practice in stroke rehabilitation . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S51–S60
    • View In Article
    • MEDLINE
15. Hatfield B , Millet D , Coles J , Gassaway J , Conroy B , Smout RJ . Characterizing speech and language pathology outcomes in stroke rehabilitation . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S61–S72
    • View In Article
    • MEDLINE
16. Horn SD , DeJong G , Smout RJ , Gassaway J , James R , Conroy B . Stroke rehabilitation patients, practice, and outcomes (is earlier and more aggressive therapy better?) . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S101–S114
    • View In Article
    • MEDLINE
17. Bode R , Heinemann A , Semik P , Mallinson T . Patterns of therapy activities across length of stay and impairment levels (peering inside the “black box” of inpatient stroke rehabilitation) . Arch Phys Med Rehabil . 2004;85:1901–1908
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (426 KB)
    • CrossRef
18. Horn SD , Sharkey PD , Gassaway J . Managed care outcomes project (study design, baseline patient characteristics, and outcome measures) . Am J Manag Care . 1996;2:237–247
    • View In Article
19. Willson DF , Horn SD , Smout RJ , Gassaway J , Torres A . Severity assessment in children hospitalized with bronchiolitis using the pediatric component of the Comprehensive Severity Index (CSI) . Pediatr Crit Care Med . 2000;1:127–132
    • View In Article
    • MEDLINE
    • CrossRef
20. Horn SD , Sharkey PD , Buckle JM , Backofen JE , Averill RF , Horn RA . The relationship between severity of illness and hospital length of stay and mortality . Med Care . 1991;29:305–317
    • View In Article
    • MEDLINE
    • CrossRef
21. Averill RF , McGuire TE , Manning BE , et al.   A study of the relationship between severity of illness and hospital cost in New Jersey hospitals . Health Serv Res . 1992;27:587–606 discussion 607-12
    • View In Article
    • MEDLINE
22. Thomas TJ , Ashcraft ML . Measuring severity of illness (comparison of severity and severity systems in terms of ability to explain variation in costs) . Inquiry . 1991;28:39–55
    • View In Article
    • MEDLINE
23. Alemi F , Rice J , Hankins R . Predicting in-hospital survival of myocardial infarction, a comparative study of various severity measures . Med Care . 1990;28:762–774
    • View In Article
    • MEDLINE
    • CrossRef
24. Horn SD , Torres A , Willson D , Dean JM , Gassaway J , Smout R . Development of a pediatric age- and disease-specific severity measure . J Pediatr . 2002;141:496–503
    • View In Article
    • Abstract
    • Full Text
    • Full-Text PDF (133 KB)
    • CrossRef
25. Conroy B , Zorowitz R , Horn SD , Ryser DK , Teraoka J , Smout RJ . An exploration of central nervous system medication use and outcomes in stroke rehabilitation . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S73–S81
    • View In Article
    • MEDLINE
26. James R , Gines D , Menlove A , Horn SD , Gassaway J , Smout RJ . Nutrition support (tube feeding) as a rehabilitation intervention . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S82–S92
    • View In Article
    • MEDLINE
27. Maulden SA , Gassaway J , Horn SD , Smout RJ , DeJong G . Timing of initiation of rehabilitation after stroke . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S34–S40
    • View In Article
    • MEDLINE
28. DeJong G , Horn SD , Smout RJ , Ryser DK . The early impact of the inpatient rehabilitation facility prospective payment system on stroke rehabilitation case mix, practice patterns, and outcomes . Arch Phys Med Rehabil . 2005;86(12 Suppl 2):S93–S100
    • View In Article
    • MEDLINE
29. Greenberg E , Treger J , Ring H . Post-stroke follow-up in rehabilitation outpatient clinic . Isr Med Assoc J . 2004;6:603–606
    • View In Article
    • MEDLINE
30. Bohannon RW , Ahlquist M , Lee N , Maljanian R . Functional gains during acute hospitalization for stroke . Neurorehabil Neural Repair . 2003;17:192–195
    • View In Article
    • MEDLINE
31. Hamilton BB , Laughlin JA , Fiedler RC , Granger CV . Interrater reliability of the 7-level functional independence measure (FIM) . Scand J Rehabil Med . 1994;26:115–119
    • View In Article
    • MEDLINE
32. Centers for Medicare and Medicaid Services. Medicare Program; prospective payment system for inpatient rehabilitation facilities, final rule. 66 Federal Register 41315 (2001).
    • View In Article

• a SAS Institute Inc, 100 SAS Campus Dr, Cary, NC 27513.