Impact of Comorbidities on Stroke Rehabilitation Outcomes: Does the Method Matter?

Article Outline

• Abstract
• Methods
  • Study Sample and Database
  • Outcome Measures
  • Measures of Comorbidity
  • Data Analyses
• Results
• Discussion
  • Study Limitations
• Conclusions
• References
• Copyright

Abstract 

Berlowitz DR, Hoenig H, Cowper DC, Duncan PW, Vogel WB. Impact of comorbidities on stroke rehabilitation outcomes: does the method matter?

Objectives

To examine the impact of comorbidities in predicting stroke rehabilitation outcomes and to examine differences among 3 commonly used comorbidity measures—the Charlson Index, adjusted clinical groups (ACGs), and diagnosis cost groups (DCGs)—in how well they predict these outcomes.

Design

Inception cohort of patients followed for 6 months.

Setting

Department of Veterans Affairs (VA) hospitals.

Participants

A total of 2402 patients beginning stroke rehabilitation at a VA facility in 2001 and included in the Integrated Stroke Outcomes Database.

Interventions

Not applicable.

Main Outcome Measures

Three outcomes were evaluated: 6-month mortality, 6-month rehospitalization, and change in FIM score.

Results

During 6 months of follow-up, 27.6% of patients were rehospitalized and 8.6% died. The mean FIM score increased an average of 20 points during rehabilitation. Addition of comorbidities to the age and sex models improved their performance in predicting these outcomes based on changes in c statistics for logistic and R² values for linear regression models. While ACG and DCG models performed similarly, the best models, based on DCGs, had a c statistic of .74 for 6-month mortality and .63 for 6-month rehospitalization, and an R² of .111 for change in FIM score.

Conclusions

Comorbidities are important predictors of stroke rehabilitation outcomes. How they are classified has important implications for models that may be used in assessing quality of care.

Key Words: Cerebrovascular accident, Comorbidity, Rehabilitation, Risk adjustment

List of Abbreviations: ACG, adjusted clinical group, CI, confidence interval, DCG, diagnosis cost group, ICD-9-CM, International Classification of Diseases, Ninth Revision, Clinical Modification, ISOD, Integrated Stroke Outcomes Database, VA, Veterans Affairs

THE IMPORTANCE OF monitoring outcomes of stroke rehabilitation is increasingly recognized.¹ As purchasers and consumers seek to maximize the effectiveness and value of care, outcomes assessments are being used for a variety of purposes, including describing quality of care,² comparing different venues or models of care such as traditional Medicare versus managed care,³, ⁴ and examining temporal trends in care.⁵ When examining such outcomes, baseline characteristics of patients that may affect the outcomes of interest, such as acute illness severity, comorbidities, functional status, and age, must be considered, typically through the process of risk adjustment.⁶, ⁷ This adjustment helps ensure that differences in outcomes reflect the care provided and not the underlying baseline characteristics, or illness severity, of the patients. Developing better approaches to risk adjustment are essential to understanding rehabilitation outcomes and maximizing care.

When performing risk adjustment, comorbidities are often an important consideration. However, studies of stroke rehabilitation outcomes often have not considered such comorbidities⁸ or have not found them to be important predictors of the outcomes.⁹, ¹⁰ When comorbidities have been used, there has been no consensus on the approach to use, resulting in a wide variety of different approaches, including counts of the number of comorbidities,⁵ the Charlson Index,² grouping comorbidities into 3 tiers based on costliness of care,⁴ or using comorbidities in the Elixhauser index¹¹ as individual explanatory variables.³, ¹² In recent years, a variety of different systems for describing the comorbidity burden have been developed and are being used widely in health services research. These systems vary in their basic approach to classifying comorbid diseases and in the weights that they may assign. The extent that these systems differentially impact predictions of stroke rehabilitation outcomes remains uncertain.

We examine the impact of comorbidities on predicting stroke rehabilitation outcomes. We consider 3 different outcomes measures that have been used frequently in past research: 6-month mortality, 6-month rehospitalization, and change in FIM score.¹³ We also consider 3 different methods for describing comorbidities: the Charlson Index,¹⁴ ACGs,¹⁵ and DCGs.¹⁶ We selected these 3 approaches because they are widely used, they are available as pre-existing packages that can be readily used with databases incorporating ICD-9-CM codes, and they are likely to be used by researchers in future studies.

We address the following 3 questions. First, do measures of comorbidity improve predictions of stroke rehabilitation outcomes compared with models that consider only age and sex? Second, are there differences among the 3 comorbidity measures in how well they predict these outcomes? Third, do comorbidity measures appear to be more important in predicting medical outcomes such as mortality and rehospitalization than functional outcomes such as change in FIM score? In addressing these questions, we provide information that is useful in improving outcomes measurement for stroke rehabilitation.

Back to Article Outline

Methods 

Study Sample and Database 

Our study sample consisted of patients beginning stroke rehabilitation in an inpatient setting at a VA medical center in 2001. Patients were identified through their presence in the VA ISOD.¹⁷ The ISOD is a collection of VA clinical and administrative data on patients identified by a clinician as having a stroke, thus avoiding issues with the validity of ICD-9-CM stroke codes, and evaluated with the FIM. Included in the ISOD are FIM data from the Functional Status Outcomes Database, demographics, health care utilization, inpatient and outpatient diagnostic data from the National Patient Care Database, mortality data from the Beneficiary Identification and Records Locator Subsystem, pharmacy data from the Pharmacy Benefits Management database, and health status in the form of the Veterans RAND 36-Item Health Survey collected as part of the Large Survey of Veterans. We supplemented the ISOD with data from its component databases to ensure that information on diagnoses and outcomes included the entire period of interest. A number of studies have supported the reliability and validity of the component databases used in creating the ISOD, particularly the diagnostic data used in describing comorbidity.¹⁸, ¹⁹

Outcome Measures 

We evaluated 3 different outcome measures. First, we considered 6-month mortality. Second, we evaluated 6-month rehospitalization. Rehospitalization was considered present when a patient was readmitted to an acute medical-surgical unit after initiation of rehabilitation. Thus, rehospitalization could occur only after a patient was discharged home or transferred to a rehabilitation or long-term care unit. We only considered events occurring after 3 days of either initiating rehabilitation (for mortality) or of transfer to a rehabilitation or long-term care unit (for rehospitalization). This acknowledges the fact that serious events, such as venous thromboembolism in a patient with hemiplegia, occurring shortly after initiation of rehabilitation or transfer out of an acute medical-surgical unit are unlikely to reflect the quality of rehabilitation care. Longer-term, though, one would expect the rehabilitation team to ensure appropriate care measures such as receipt of anticoagulation when indicated. Our final outcome measure was change in FIM score from the initiation of rehabilitation to its completion. The FIM is an 18-item instrument that captures aspects of motor and cognitive function. Each item is scored on a 7-point ordinal scale, with a 7 indicating independence, so that the maximum possible score is 126. Change in FIM was operationalized as discharge FIM minus admission FIM. One study has suggested that a change in FIM score of 22 represents a clinically important difference in patients with stroke receiving inpatient rehabilitation.²⁰ Because of the different ways that outcomes were determined, the number of patients in each sample differed slightly. For example, patients without a discharge FIM score could not contribute an observation to the change in FIM score but would be considered for mortality and rehospitalization.

Measures of Comorbidity 

We used 3 different measures of comorbidity that vary in how they conceptualize this construct. The Charlson Index consists of a weighted count of 17 serious medical conditions that was originally developed to predict 1-year mortality among hospitalized patients but has since been widely used in health services research.¹⁴ We operationalized the Charlson Index using the Deyo modification that assigns specific ICD-9-CM codes to each diagnosis.²¹

ACGs were originally developed to predict ambulatory care visits among patients of health maintenance organizations but have since been widely used to describe the extent of medical problems and their likely effects on health care resource use.¹⁵ Each ICD-9-CM code for a patient is assigned to 1 of 32 mutually exclusive adjusted diagnosis groups that groups diagnoses based on their similarity on a number of characteristics including expected persistence of the condition, likelihood of requiring hospitalization, or need for specialty referral. Examples of adjusted diagnosis groups are chronic medical unstable conditions and psychosocial recurrent or persistent stable conditions. Based on age, sex, and the number and pattern of adjusted diagnosis groups, each patient is then assigned to 1 of 93 different ACGs. We used ACG, version 6.0,^a in these analyses.

DCGs were originally developed to predict future costs for Medicare beneficiaries.¹⁶ Each ICD-9-CM code is first grouped into diagnostic clusters of clinically related disorders. Clusters are subsequently grouped into hierarchical condition categories that consider the severity and expected costliness of related disorders. Examples of hierarchies are congestive heart failure and diabetes mellitus with chronic complications. Hierarchies may be further clustered into aggregated condition categories. We used DxCG, release 6.0,^b in conducting the analyses.

For each measure of comorbidity, we considered different variations. For the Charlson Index, we evaluated both an unweighted count of conditions and a version incorporating the weights from the original publication. For ACGs and DCGs, we examined weighted versions using coefficients derived from other samples and settings as well as unweighted versions in which coefficients were determined by the regression model. We report results from the models with the best performance based on their c statistic or R². These were a weighted Charlson, an unweighted ACG model that uses individual adjusted diagnosis groups, and an unweighted DCG model that considers each individual aggregated condition category.

Data Analyses 

We modeled each of the 3 outcome measures using a linear regression model for the change in FIM score and a logistic regression model for rehospitalization and mortality. While we followed usual practice in our analyses of change in FIM scores by using linear regression techniques, total FIM scores and differences in FIM scores are not linear equal-interval measures, and hence the assumptions underlying standard regression techniques are not fully met. For each outcome, we compared a model that included just age and sex as independent variables with a model that added information on comorbidity using 1 of the 3 case-mix systems. We evaluated the R² for linear regression models and the c statistic for logistic regression models. These analyses were performed using SAS.^c

We created CIs for each c statistic and R² using a bootstrap method from Stata/SE version 9.2.^d We randomly sampled cases from our data with replacement, creating 1000 samples of the same size as the original data. The models were then run on these samples, producing 1000 values for these samples. The 95% CIs were then calculated by Stata. Statistically significant improvements in model performance when comorbidities were added to the age and sex models were identified based on calculations of the 95% CIs for the differences in the point estimates.

This study was approved by the institutional review boards at both the Bedford VA Hospital and the North Florida/South Georgia Veterans Health System.

Back to Article Outline

Results 

We identified 2402 patients with stroke receiving rehabilitation in a VA facility in 2001. Patients were predominantly elderly with a mean age ± SD of 67.7±11.1 years and were 98.1% men. They had a moderate degree of functional impairment at baseline as represented by their mean FIM score ± SD of 69.6±28.5. They also had a high comorbidity burden as represented by a mean of 2.5±1.4 chronic medical conditions considering those used in the Charlson Index. Among the common diseases, 47% had diabetes, 20% had chronic lung disease, 14% had congestive heart failure, and 13% had peripheral vascular disease. Patients were followed in rehabilitation for a mean of 24.3±24.4 days.

Potentially adverse outcome events were frequent, with 27.6% of patients being rehospitalized and 8.6% dying during the 6 months of follow-up. After completion of rehabilitation, the mean discharge FIM score ± SD was 89.6±29.6, indicating an average 20-point improvement in functional status.

Results from the regression models are presented in table 1. Models that included only age and sex performed poorly, with c statistics of .63 and .54 for 6-month mortality and rehospitalization, respectively. The R² for the change in FIM model was .075. Adding information on comorbidities to the age and sex model improved performance in all cases except the Charlson Index model of change in FIM score. The improvements were greatest for the ACG and DCG models. However, these improvements were not significant for the Charlson and ACG models of change in FIM score or for the Charlson Index with 6-month mortality. Even with information on comorbidities, the c statistic for rehospitalization and the R² for change in FIM were low, with the best values .63 and .111, respectively.

Table 1. Results of Regression Models Examining Outcomes of 6-Month Mortality, 6-Month Rehospitalization, and Change in FIM Score

Independent Variables	6-Month Mortality	6-Month Rehospitalization	Change in FIM (R2 with 95% CI)
Age/sex	.63(.59−.67)	.54(.51−.56)	.075(.054−.096)
Age/sex/Charlson	.68(.64−.72)	.59(.57−.62)⁎	.074(.053−.094)
Age/sex/ACG	.72(.69−.75)⁎	.63(.60−.65)⁎	.100(.075−.124)
Age/sex/DCG	.74(.71−.77)⁎	.63(.60−.65)⁎	.111(.086−.137)⁎

NOTE. Values are c statistic with 95% CI or as otherwise indicated.

Back to Article Outline

Discussion 

The use of outcome measures is an essential element for understanding the quality of rehabilitation care.¹, ⁷ When comparing outcomes, risk adjustment is necessary to ensure that differences in the outcome are not the result of differences in the baseline characteristics of the population being served. Without adequate risk adjustment, poor performers will always be able to argue “our patients are sicker.” For stroke rehabilitation care, the issue is not whether risk-adjustment should be performed, but how it may best be performed to capture differences in important patient characteristics adequately. One potentially important patient characteristic is comorbidity burden. Comorbidities may affect outcomes through a number of mechanisms that compromise the response to or participation in rehabilitation, or through a direct impact on the outcome of interest.²² Our results provide information on the use of comorbidities when evaluating important outcomes for stroke rehabilitation within the VA.

Our results highlight 3 important points. First, information on comorbidities predicts stroke rehabilitation outcomes. In most cases, the addition of comorbidities to the age and sex models resulted in significantly improved model performance as reflected in the increased c statistic and R². This result emphasizes that when comparing outcomes across providers, efforts should be made to collect data on previous diagnoses. While collecting this information is easy in a health care system such as the VA because of its comprehensive clinical and administrative databases, this task may be more challenging in other settings that do not have such a system.

Second, the approach used to describe comorbidities does matter. Based on their 95% CIs, the ACG and DCG models were significantly better than the age and sex models for both 6-month mortality and rehospitalization, while the Charlson model was significantly better only for 6-month rehospitalization. Only DCGs were significant when examining change in FIM. An important difference between the Charlson Index and both ACGs and DCGs is that it considers only a limited number of severe comorbidities and does not capture the full comorbidity burden. ACGs and DCGs use all diagnostic information available. The performance of DCGs and ACGs in predicting the stroke rehabilitation outcomes used in our study was similar, although the c statistic and R² were slightly higher in all cases for DCGs. Interestingly, ACGs and DCGs employ different approaches to classifying diseases. DCG groupings tend to describe the number of distinct clinical conditions better, whereas the ACG system is more oriented toward groupings based on expected resource consumption.¹³, ¹⁴

Finally, the impact of comorbidities in predicting outcomes will vary depending on the outcome. Comorbidities had the largest impact on models predicting 6-month mortality. The observed c statistic of .74 using DCGs, while indicating moderate discriminatory ability, is comparable to that observed with other risk-adjustment models. In the case of 6-month rehospitalization rates, while the c statistic did increase when information on comorbidities was added to age and sex, the performance of the resulting models, with c statistics of .63, would be considered poor. For change in FIM, the R² value was low and increased only slightly when comorbidities were added to the models. This finding suggests that comorbidities explain little of the variation for change in functional status during rehabilitation and emphasize the need for further studies evaluating the impact of comorbidities on other functional outcomes. Several other studies have found that comorbidities generally have a small impact in predicting rehabilitation outcomes.⁹, ¹⁰ For example, R² values increased by only 1% and 2% when comorbidities were added to Functional Independence Measure−Function Related Group measures predicting rehabilitation lengths of stay.⁹ Each additional comorbidity was associated with a 10% increase in the odds of a medical complication during rehabilitation resulting in transfer to the hospital.¹⁰ Further evaluation is clearly required to define better the role of comorbidities in predicting diverse outcomes with stroke rehabilitation.

Study Limitations 

The sample we used was relatively small, including 2402 patients with stroke. This illustrates the difficulty in building risk-adjustment models for rehabilitation where even in a health care system as large as the VA, large numbers of stroke rehabilitation patients are not available. Larger samples may have led to more stable estimates of model performance and narrower 95% CIs. However, it is unlikely our conclusions regarding the role of comorbidities in risk adjustment for stroke outcomes would have changed. In addition, our sample was predominantly men and reflected care provided within the VA, a comprehensive, federally financed health care system. Results could differ in women and in other systems of care.

The models we described are not comprehensive risk-adjustment models, including only age, sex, and comorbidities. Other variables are likely necessary to describe severity fully, including the type of stroke and baseline functional status. These other variables should be included in such comprehensive models and would likely result in further improvements in model performance. It is conceivable that the inclusion of other variables could change the association of comorbidities with rehabilitation outcomes. In addition, there may be interactions that we did not examine. One study found that comorbidities were more important in explaining rehabilitation complications requiring transfer among those patients who were the least functionally dependent.¹⁰ These issues warrant further investigation.

Back to Article Outline

Conclusions 

Understanding reasons for variations in outcomes among stroke rehabilitation patients is critical if we are to enhance the effectiveness of this care. An extensive body of literature, based predominantly on hospital care, has demonstrated that the best risk-adjustment models require comprehensive clinical information to describe the health status of patients. Our work addresses 1 piece of such a comprehensive clinical description, the role of comorbidities. Clearly comorbidities are important for explaining variations in many stroke rehabilitation outcomes. Detailed models for a variety of stroke rehabilitation outcomes will still need to be developed if we are to fully understand care provided to these patients and how best to structure efforts to enhance quality of care.

Suppliers

Back to Article Outline

References 

1. Rao P, Boradia P, Ennis J. Shift happens: using outcomes to survive and thrive under PPS. Top Stroke Rehabil. 2005;12:1–3
   • View In Article
   • MEDLINE
   • CrossRef
2. Hoenig H, Sloane R, Horner RD, Zolkewitz M, Reker D. Differences in rehabilitation services and outcomes among stroke patients cared for in Veterans hospitals. Health Serv Res. 2001;35:1293–1315
   • View In Article
   • MEDLINE
3. Smith MA, Frytak JR, Liou J, Finch MD. Rehospitalization and survival for stroke patients in managed care and traditional Medicare plans. Med Care. 2005;43:902–910
   • View In Article
   • MEDLINE
   • CrossRef
4. Deutsch A, Granger CV, Heinemann AW, et al. Poststroke rehabilitation: outcomes and reimbursement of inpatient rehabilitation facilities and subacute rehabilitation programs. Stroke. 2006;37:1477–1482
   • View In Article
   • CrossRef
5. Ottenbacher KJ, Smith PM, Illig SB, Linn RT, Ostir GV, Granger CV. Trends in length of stay, living setting, functional outcomes, and mortality following medical rehabilitation. JAMA. 2004;292:1687–1695
   • View In Article
   • CrossRef
6. Iezzoni LI. The risks of risk adjustment. JAMA. 1997;278:1600–1607
   • View In Article
   • MEDLINE
7. Kane RL. Improving outcomes in rehabilitation. Med Care. 1997;35:JS21–JS27
   • View In Article
   • MEDLINE
   • CrossRef
8. Ottenbacher KJ, Smith PM, Illig SB, Fiedler RC, Gonzales V, Granger CV. Characteristics of persons rehospitalized after stroke rehabilitation. Arch Phys Med Rehabil. 2001;82:1367–1374
   • View In Article
   • Abstract
   • Full Text
   • Full-Text PDF (110 KB)
   • CrossRef
9. Stineman MG, Tassoni CJ, Escare JJ, et al. Development of Function Related Groups Version 2.0: a classification system for medical rehabilitation. Health Serv Res. 1997;32:529–548
   • View In Article
   • MEDLINE
10. Siegler EL, Stineman MG, Maislin G. Development of complications during rehabilitation. Arch Intern Med. 1994;154:2185–2190
    • View In Article
    • MEDLINE
11. Elixhauser A, Steiner C, Harris DR, Coffey RM. Comorbidity measures for use with administrative data. Med Care. 1998;36:8–27
    • View In Article
    • MEDLINE
    • CrossRef
12. Kind AJH, Smith MA, Frytak JR, Finch MD. Bouncing back: patterns and predictors of complicated transitions 30 days after hospitalization for acute ischemic stroke. J Am Geriatr Soc. 2007;55:365–373
    • View In Article
    • MEDLINE
    • CrossRef
13. Linacre JM, Heinemann AW, Wright BD, Granger CV, Hamilton BB. The structure and stability of the Functional Independence Measure. Arch Phys Med Rehabil. 1994;75:127–132
    • View In Article
    • MEDLINE
14. Charlson ME, Pompei EP, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373–383
    • View In Article
    • MEDLINE
    • CrossRef
15. Weiner JP, Starfield BH, Steinwachs DM, Mumford LM. Development and application of a population-oriented measure of ambulatory care case-mix. Med Care. 1991;29:452–472
    • View In Article
    • MEDLINE
    • CrossRef
16. Ellis RP, Pope GC, Iezzoni L, et al. Diagnosis-based risk adjustment for Medicare capitation payments. Health Care Financ Rev. 1996;17:101–128
    • View In Article
    • MEDLINE
17. Reker DM, Reid K, Duncan PW, et al. Development of an integrated stroke outcomes database within Veterans Health Administration. J Rehabil Res Dev. 2005;42:77–92
    • View In Article
    • MEDLINE
    • CrossRef
18. Kashner TM. Agreement between administrative files and written medical records: a case of the Department of Veterans Affairs. Med Care. 1998;36:1324–1336
    • View In Article
    • MEDLINE
    • CrossRef
19. Borzecki AM, Wong AT, Hickey EC, Ash AS, Berlowitz DR. Identifying hypertension-related comorbidities from administrative data: what's the optimal approach?. Am J Med Qual. 2004;19:201–206
    • View In Article
    • MEDLINE
    • CrossRef
20. Beninato M, Gill-Body KM, Salles S, Stark PC, Black-Schaffer RM, Stein J. Determination of the minimal clinically important difference in the FIM instrument in patients with stroke. Arch Phys Med Rehabil. 2006;87:32–39
    • View In Article
    • Full-Text PDF (33 KB)
    • CrossRef
21. Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–619
    • View In Article
    • MEDLINE
    • CrossRef
22. Iezzoni LI. Risk adjusting rehabilitation outcomes. Am J Phys Med Rehabil. 2004;83:316–326
    • View In Article
    • MEDLINE
    • CrossRef

• a Johns Hopkins School of Public Health, 624 N Broadway, Baltimore, MD 21205.
• b DxCG Inc, 99 Summer St, Ste 520, Boston, MA 02110.
• c Version 9.1; SAS Institute Inc, 100 SAS Campus Dr, Cary, NC 27513.
• d Stata Corp, 4905 Lakeway Dr, College Station, TX 77845.