Advertisement

Predictors of Follow-Up Completeness in Longitudinal Research on Traumatic Brain Injury: Findings From the National Institute on Disability and Rehabilitation Research Traumatic Brain Injury Model Systems Program

Published:November 11, 2013DOI:https://doi.org/10.1016/j.apmr.2013.10.016

      Abstract

      Objective

      To identify baseline participant variables in the domains of demographics, medical/psychosocial history, injury characteristics, and postinjury functional status associated with longitudinal follow-up completeness in persons with traumatic brain injury (TBI) using the TBI Model Systems (TBIMS) National Database (NDB).

      Design

      Exhaustive chi-square automatic interaction detection was used to identify factors that classified participants according to level of follow-up completeness.

      Setting

      Retrospective analysis of a multi-center longitudinal database.

      Participants

      Individuals (N=8249) enrolled in the TBIMS NDB between 1989 and 2009 who were eligible for at least the first (year 1) follow-up up to the fifth (year 15) follow-up.

      Interventions

      None.

      Main Outcome Measures

      Follow-up completeness as defined by 6 different longitudinal response patterns (LRPs): completing all follow-ups, wave nonresponse, dropping out, completing no follow-ups without formally withdrawing, formally withdrawing before completing any follow-ups, and formally withdrawing after completing some follow-ups.

      Results

      Completing all follow-ups was associated with higher levels of education, living with parents or others, and having acute care payer data entered in the NDB. Subgroups more vulnerable to loss to follow-up (LTFU) included those with less education, racial/ethnic minority backgrounds, those with better motor functioning on rehabilitation discharge, and those for whom baseline data on education, employment, and acute care payer were not collected. No subgroups were found to be more likely to have the LRPs of dropping out or formal withdrawal.

      Conclusions

      These data identify subgroups in which retention strategies beyond those most commonly used might reduce LTFU in longitudinal studies of persons with TBI, such as the TBIMS, and suggest future investigations into factors associated with missing baseline data.

      Keywords

      List of abbreviations:

      CHAID (chi-square automatic interaction detection), HS (high school), LRP (longitudinal response pattern), LTFU (loss to follow-up), NDB (National Database), PTA (posttraumatic amnesia), SES (socioeconomic status), TBI (traumatic brain injury), TBIMS (Traumatic Brain Injury Model Systems)
      Longitudinal studies are vital to understanding long-term outcomes after traumatic brain injury (TBI). However, 33% to 50% of participants in these studies have historically been lost to follow-up (LTFU).
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      • Levin H.S.
      • Gary Jr., H.E.
      • Eisenberg H.M.
      • et al.
      Neurobehavioral outcome 1 year after severe head injury. Experience of the Traumatic Coma Data Bank.
      Participants who are lost nearly always differ systematically from those retained,
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      • Levin H.S.
      • Gary Jr., H.E.
      • Eisenberg H.M.
      • et al.
      Neurobehavioral outcome 1 year after severe head injury. Experience of the Traumatic Coma Data Bank.
      which threatens both the internal and external validity of longitudinal outcome studies
      • Touloumi G.
      • Babiker A.G.
      • Pocock S.J.
      • Darbyshire J.H.
      Impact of missing data due to drop-outs on estimators for rates of change in longitudinal studies: a simulation study.
      and promotes biased conclusions about outcomes after TBI. Indeed, factors generally predictive of LTFU (eg, low socioeconomic status [SES]) often adversely affect outcomes,
      • Corrigan J.D.
      • Bogner J.A.
      • Mysiw W.J.
      • Clinchot D.
      • Fugate L.
      Systematic bias in outcome studies of persons with traumatic brain injury.
      which suggests that systematic LTFU might yield outcome data that are positively biased.
      Such bias can be reduced by preventing LTFU (eg, by increasing the frequency of participant contact or using sophisticated tracking systems
      • Bell K.R.
      • Hammond F.
      • Hart T.
      • Bickett A.K.
      • Temkin N.R.
      • Dikmen S.
      Participant recruitment and retention in rehabilitation research.
      • Hunt J.R.
      • White E.
      Retaining and tracking cohort study members.
      ) or by using statistical methods that compensate for loss after it has occurred. However, statistical procedures cannot always reliably estimate or eliminate the impact of LTFU on outcome data.
      • Kristman V.L.
      • Manno M.
      • Cote P.
      Methods to account for attrition in longitudinal data: do they work? A simulation study.
      Prevention techniques are, therefore, more desirable
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      but require what Greenland
      • Greenland S.
      Response and follow-up bias in cohort studies.
      termed subject matter knowledge, such as data regarding which participants are more vulnerable to LTFU.
      Relatively few investigations have focused on factors predictive of LTFU in longitudinal studies of individuals with TBI. A study by Corrigan et al
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      suggested that those with a history of substance abuse, with higher motor function at rehabilitation discharge, and from “socioeconomically disadvantaged groups” (racial and ethnic minorities and those who had little education, were unemployed at time of injury, or were dependent on public funding for acute care) were more frequently lost at 1 and 2 year postinjury follow-ups in 3 large longitudinal studies.
      Of the 3 datasets analyzed by Corrigan, the TBI Model Systems (TBIMS) National Database (NDB) includes the largest number of participants, is the longest running, and has provided the basis for the most analyses, yielding approximately 350 published studies thus far.

      University of Washington. Model Systems Knowledge Translation Center. Available at: http://msktc.washington.edu/research/searchpubs.asp?modelsys=tbi. Accessed September 21, 2011.

      The NDB has experienced LTFU since its inception, as do all longitudinal studies, and instituted targets for follow-up completeness in 2006.
      • Dijkers M.P.
      • Harrison-Felix C.
      • Marwitz J.H.
      The traumatic brain injury model systems: history and contributions to clinical service and research.
      However, between 17% and 23% of participants in the NDB are still lost depending on the specific follow-up point.

      Traumatic Brain Injury Model Systems National Data and Statistical Center. 2012 TBI Model Systems presentation. Available at: https://www.tbindsc.org/StaticFiles/Documents/2012%20TBIMS%20Slide%20Presentation.pdf. Accessed January 10, 2013.

      The goal of the current study was to determine participant-related factors predictive of LTFU in the longitudinal study of persons with TBI using the current TBIMS NDB. Previous work in this area has sought to identify factors associated with either response or nonresponse at 1 or 2 follow-ups, but we aimed to identify factors associated with a number of different longitudinal response patterns (LRPs) (eg, completing all follow-ups, skipping some follow-ups) up to a maximum of 5 follow-ups. This approach allowed us to investigate factors associated with LTFU across the more lengthy sequence of follow-ups needed to best characterize long-term outcomes, thus enhancing the potential usefulness of our findings.
      The NDB was ideal for our purposes because of the large number of available participants, volume of data collected, and number of follow-up data points available, all of which suggest that findings in the NDB would be applicable to many other longitudinal studies of persons with TBI and could inform follow-up practices to maximize participant retention. In addition, there is value in identifying factors associated with LTFU in the NDB for its own sake because of its significant contribution to the outcomes literature, evidence of selective LTFU
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      • Harrison-Felix C.
      • Dijkers M.
      Predictors of loss to follow-up in a longitudinal study of persons with traumatic brain injury.
      in the database, and the association between previously established predictors of LTFU in the NDB and outcomes.
      • Hoofien D.
      • Vakil E.
      • Gilboa A.
      • Donovick P.J.
      • Barak O.
      Comparison of the predictive power of socio-economic variables, severity of injury and age on long-term outcome of traumatic brain injury: sample-specific variables versus factors as predictors.
      • Arango-Lasprilla J.C.
      • Rosenthal M.
      • Deluca J.
      • et al.
      Traumatic brain injury and functional outcomes: does minority status matter?.
      • Gary K.W.
      • Arango-Lasprilla J.C.
      • Stevens L.F.
      Do racial/ethnic differences exist in post-injury outcomes after TBI? A comprehensive review of the literature.
      The overarching goal of the present study was to determine whether various LRPs could be predicted early in study participation and to identify participant characteristics that might signal the need for implementation of specific longitudinal retention strategies. Ideally, these strategies would maximize retention and, therefore, enhance the validity and applicability of longitudinal studies of persons with TBI.

      Methods

       TBI Model Systems

      An extensive description of the TBIMS and its NDB is reported in Dijkers et al.
      • Dijkers M.P.
      • Harrison-Felix C.
      • Marwitz J.H.
      The traumatic brain injury model systems: history and contributions to clinical service and research.
      Further information regarding inclusion criteria and data included in the NDB is available on the TBMIS National Data and Statistics Center website.

      Traumatic Brain Injury Model Systems National Data and Statistical Center. 2012 TBI Model Systems presentation. Available at: https://www.tbindsc.org/StaticFiles/Documents/2012%20TBIMS%20Slide%20Presentation.pdf. Accessed January 10, 2013.

      Briefly, participants must have sustained a moderate-severe TBI as evidenced by a Glasgow Coma Scale score of <13 in the emergency department, loss of consciousness >30 minutes, posttraumatic amnesia (PTA) >24 hours, and/or structural abnormalities indicative of TBI on neuroimaging; must be ≥16 years old; and must have received acute care and rehabilitation from a participating medical center. Informed consent is obtained from the participant or, if unable, a family member or legal guardian.
      Data collected just prior to rehabilitation discharge (baseline) include case mix, acute care, and rehabilitation information obtained through review of emergency and acute care medical records and an interview with the participant or proxy if the participant is unable to respond reliably. Review of rehabilitation records is used to calculate scores on functional status measures (eg, FIM
      • Keith R.A.
      • Granger C.V.
      • Hamilton B.B.
      • Sherwin F.S.
      The functional independence measure: a new tool for rehabilitation.
      and Disability Rating Scale
      • Hall K.
      • Cope D.N.
      • Rappaport M.
      Glasgow Outcome Scale and Disability Rating Scale: comparative usefulness in following recovery in traumatic head injury.
      ). Follow-up data on outcomes are currently collected 1, 2, 5, and 10 years postinjury and every fifth year thereafter. The follow-up survey is completed in-person or via telephone or mail with the participant or a proxy if the participant is unable to respond.

       Analyzed sample

      At the time of our analyses, the NDB included 9310 participants enrolled at 22 centers that are currently (16 centers) or previously (6 centers) participating in the TBIMS. Participants whose LRP, which will be subsequently discussed, could not be determined from the data were excluded from the analysis. These included participants whose first (year 1) follow-up was not yet due (n=589) and those incarcerated during ≥1 follow-ups (n=251) because incarcerated participants are not contacted. Participants who died before completing any follow-ups (n=207) were also excluded. Only 14 participants were eligible for follow-up at year 20 and were also excluded. The sample analyzed included all individuals enrolled in the TBIMS between 1989 and 2009 and who were eligible for at least the first (year 1) follow-up up to the fifth (year 15) follow-up (N=8249).

       Longitudinal response pattern

      LRP was the outcome variable in our analyses. Table 1 contains numbers and examples of each LRP included as a possible outcome. Based on the LRP, participants were classified as 1 of 6 kinds of responders: (1) completers who completed all follow-ups; (2) wave nonresponders who completed follow-up(s) but also skipped follow-up(s); (3) dropouts who completed follow-up(s) but then did not complete subsequent follow-up(s) without formally withdrawing; (4) late withdrawers who formally withdrew after completing ≥1 follow-ups; (5) withdrawers throughout who formally withdrew without completing any follow-ups; and (6) nonresponders who did not complete any follow-ups but did not formally withdraw. Follow-ups scheduled during periods when a TBIMS center was defunded were not considered when determining participants' LRP because follow-up contact is not attempted during such periods. The number of individuals affected by defunding was small (a minimum of 0.5% of those eligible for a year 15 follow-up and a maximum of 8% of those eligible for a year 5 follow-up). Indeed, preliminary analyses did not suggest an association between whether a participant was affected by defunding in the past and their LRP.
      Table 1LRPs in the TBIMS NDB with examples
      Response Patternn%Follow-Up Status (year no.)
      1251015
      Completed all558867.7
      Wave nonresponse
      The pattern of these LRPs can vary among participants. The examples provided reflect one possible pattern.
      102312.4XX
      Dropout
      The pattern of these LRPs can vary among participants. The examples provided reflect one possible pattern.
      6517.9XXX
      Late withdrawal
      The pattern of these LRPs can vary among participants. The examples provided reflect one possible pattern.
      1752.1WWW
      Withdrawal throughout1131.4WWWWW
      Nonresponse6998.5XXXXX
      Abbreviations: W, participant withdrew; X, follow-up not completed; √, follow-up completed.
      The pattern of these LRPs can vary among participants. The examples provided reflect one possible pattern.

       Predictors

      Selected baseline variables were recoded based on the distribution of participants within subgroups or on categories used in previous analyses of the NDB
      • Bushnik T.
      • Englander J.
      • Wright J.
      • Kolakowsky-Hayner S.A.
      Traumatic brain injury with and without late posttraumatic seizures: what are the impacts in the post-acute phase: a NIDRR Traumatic Brain Injury Model Systems study.
      and then used as candidate predictors of LRPs. Predictors fell into 4 broad categories: (1) demographics, including acute care payer (public/private/other), age, marital status (married/unmarried), race (white/black/other), sex, and education at injury (<high school [HS]/HS/>HS); (2) medical/psychosocial history, including history of TBI requiring hospitalization (yes/no), employment status at injury (employed/student/unemployed/retired/other), primary person living with at rehabilitation discharge (alone/spouse or significant other/parents/other family/others), and substance problem use at rehabilitation admission (yes/no/unknown); (3) injury characteristics, including cause of injury (vehicular/violence/sports/falls/other) and duration of PTA (<1/1–7/8–28/>29d); and (4) postinjury functional status, including Disability Rating Scale (none to partial/moderate to moderately severe/severe to extreme vegetative state) and FIM cognitive and motor scores at rehabilitation discharge (complete dependence/moderate dependence/no helper required) averaged across items.
      Many of these candidate predictors have been associated with LTFU in studies of non-TBI populations that experience physical, cognitive, and/or emotional symptoms (eg, marital status in psychiatric and substance abuse populations
      • Bergman P.
      • Ahlberg G.
      • Forsell Y.
      • Lundberg I.
      Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates.
      • Claus R.E.
      • Kindleberger L.R.
      • Dugan M.C.
      Predictors of attrition in a longitudinal study of substance abusers.
      and level of education in persons with spinal cord injury
      • de Groot S.
      • Haisma J.A.
      • Post M.W.
      • van Asbeck F.W.
      • van der Woude L.H.
      Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study.
      ) whereas others are conceptually similar to variables previously associated with LTFU. For example, condition-specific indicators of severity have been associated with LTFU in non-TBI populations (eg, immunologic status in human immunodeficiency virus/acquired immunodeficiency syndrome
      • Brown D.M.
      • Thorne J.E.
      • Foster G.L.
      • et al.
      Factors affecting attrition in a longitudinal study of patients with AIDS.
      and length of hospital stay in burn injury
      • Holavanahalli R.K.
      • Lezotte D.C.
      • Hayes M.P.
      • et al.
      Profile of patients lost to follow-up in the burn injury rehabilitation model systems' longitudinal database.
      ); therefore, we included duration of PTA, an indicator of TBI severity,
      • Malec J.F.
      • Brown A.W.
      • Leibson C.L.
      • et al.
      The mayo classification system for traumatic brain injury severity.
      as a candidate predictor.

       Statistical procedures

      Statistical analyses were conducted using SPSS version 19.0 with SPSS Answer Tree 3.0 software.a Exhaustive chi-square automatic interaction detection (CHAID)
      • Kass G.V.
      An exploratory technique for investigating large quantities of categorical data.
      was used to identify characteristics that classified participants as having a specific LRP. Exhaustive CHAID involves repeatedly merging, splitting, and stopping categories of predictor variables to create a decision tree that can identify group membership based on the relative association between each significant predictor and outcome. Continuous variables are first transformed into categorical variables by dividing continuous distributions into categories with a roughly equal number of observations. Categories of predictors that are least significantly different with respect to the dependent variable are then merged. If certain categories are not statistically different, those categories are merged and the step repeated until statistically different categories of each predictor are determined. This exhaustive repetition is an improvement over the original version of CHAID, which was developed by Biggs et al,
      • Biggs D.
      • deVille B.
      • Suen E.
      A method of choosing multiway partitions for classification and decision trees.
      wherein merging ends when all remaining categories are statistically different, which can result in misidentification of the optimal split for a variable. Chi-square tests are then performed to determine the relation between each predictor and outcome, and a Bonferroni-adjusted P value is calculated for each predictor, which yields a classification tree organized according to the degree of association between predictor and outcome (eg, the most significant predictor of the outcome represents the first split in the data, and the least significant represents the last split). Stopping occurs when statistically different categories of the least significant predictor are determined. At that point, no further splits are performed, which results in a terminal node of the classification tree.
      The CHAID technique was chosen because the analysis is designed to determine variables that identify group membership; allows both predictors and the dependent variable to be nominal, ordinal, or interval; and because predictor variables need not be measured on the same scale (eg, nominal or ordinal). CHAID considers missing data to be a valid category (eg, sex: male/female/missing); therefore, the absence of a value for a particular variable can be considered a value in itself.

      Results

      For categorical criterion variables, such as the LRP, the risk estimate is the proportion of cases in the sample incorrectly classified or the error rate. The overall correct classification of the CHAID decision tree was 68.7% for the entire sample (31.3% error rate), and correct classification of individuals with the most frequent LRP (completed all) was 98%.The most significant predictor of the LRP was education at time of injury (χ2=956.26, df=10, P<.000).
      The CHAID decision tree has been split into 3 figures based on the first split in the data by education at the time of injury. Figure 1 represents the far left portion of the decision tree predicting the LRP for individuals who completed an HS education or less than an HS education at the time of injury (node 1) and its subsequent splits. Figure 2 represents the central portion of the decision tree predicting the LRP for individuals who completed more than an HS education (node 2) and its subsequent splits. Figure 3 represents the far right portion of the decision tree predicting the LRP for those individuals who were missing baseline education data (node 3) and its subsequent splits. The CHAID decision tree consists of 23 groups of participants; each group had different combinations of values on significant predictor variables (and, therefore, each represented by a node) across 3 levels with 15 end groups (terminal nodes).
      Figure thumbnail gr1
      Fig 1CHAID decision tree predicting the LRP for those with an HS education or less and subsequent splits. Abbreviations: Adj., adjusted; FUP01, year 1 follow-up.
      Figure thumbnail gr2
      Fig 2CHAID decision tree predicting the LRP for those with greater than an HS education and subsequent splits. Abbreviations: Adj., adjusted; FUP01, year 1 follow-up; Rehab, rehabilitation.
      Figure thumbnail gr3
      Fig 3CHAID decision tree predicting the LRP for those missing baseline education data and subsequent splits. Abbreviations: Adj., adjusted; FIMMOT, FIM motor score; FUP01, year 1 follow-up; PHI, protected health information.
      Subgroups with a statistically greater likelihood of having a specific LRP are those which are represented by terminal nodes with an index percentage above a cutoff selected a priori. Index percentage is the proportion of participants in a given node with the targeted LRP relative to the number of participants with that LRP in the overall sample. For our analyses, we considered significant terminal nodes to be those with index percentages ≥120%. That is, a subgroup was considered significantly more likely to have a specific LRP if the number of participants in that subgroup with the LRP was ≥120% than the number of participants with the LRP in the overall sample. Using that criterion, several subgroups of participants were more likely to have one of the following LRPs: completed all, wave nonresponse, or nonresponse. No subgroups were more likely to have the LRP of dropout, late withdrawal, or withdrawal throughout.
      A written interpretation of the terminal node representing the subgroup with the highest percentage of completers, wave nonresponders, and nonresponders is subsequently provided, and table 2 contains subgroup characteristics, subgroup total, node number, and index percentage for all terminal nodes representing the subgroups considered more likely to have a specific LRP.
      Table 2Subgroups associated with LRP in CHAID analyses
      Response PatternAssociated SubgroupsnNodeIndex (%)
      Completed all>HS education Acute care payer entered in NDB Living with parents at rehabilitation discharge65414124.6
      >HS education Acute care payer entered in NDB Living with others at rehabilitation discharge37015120.9
      Wave nonresponseEducation missing Black No helper required on FIM motor24921401.6
      Education missing Other race Unmarried or marital status missing17722305.2
      Education missing Black Dependent or missing FIM motor score34320223.3
      Education missing Other race Married9923219.9
      Education missing White <41y of age85618218.5
      ≤HS education Acute care payer missing >41y of age45119126.9
      Nonresponse≤HS education Acute care payer missing695530.2
      >HS education Other or missing acute care payer797268.9
      Education missing Other race Unmarried or marital status missing17722226.7
      Education missing Black No helper required on FIM24921142.2
      Education missing White >41y of age45119141.3
      Education missing Other race Married9923131.4
      Education missing Black Dependent or missing FIM motor score34320120.4
      For the LRP of completed all, 2 subgroups had index percentages >120%, the greater of which was for those with more than an HS education, whose acute care primary payer was entered in the NDB, and who were living with parents after inpatient rehabilitation discharge (node 14). For the LRP of wave nonresponse, 7 subgroups had index percentages >120%, the greatest of which was for black individuals who were missing data on education level at time of injury and were classified as independent based on the FIM motor score (node 21). For the LRP of nonresponse, 7 subgroups had index percentages >120%, the greatest of which was for those with an HS education or less and for whom acute care primary payer data were missing from the NDB.

      Discussion

      Our analyses revealed subgroups more likely to have specific LRPs in the TBIMS NDB. Completers were more likely to have higher levels of education, acute care payer data entered in the NDB, and live with parents or others. Wave nonresponders were more likely to have less education or have no baseline education data in the NDB, have no acute care payer data in the NDB, be physically independent, and be from a racial/ethnic minority background; however, white participants with missing education data and who were <41 years old were also more likely to be wave nonresponders. Nonresponders were also more likely to be physically independent and have less education or have missing baseline education data, but those with more education were also more likely to be nonresponders if they were missing acute care payer data. Nonresponders were also more likely to be from racial/ethnic minority backgrounds; however, white participants were also more likely to be nonresponders if baseline education data were missing and they were >41 years.
      The fact that completers have more education might suggest higher SES among these individuals. Indeed, low SES has been associated with LTFU in the work of Corrigan et al
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      and in analyses of other longitudinal databases.
      • Bergman P.
      • Ahlberg G.
      • Forsell Y.
      • Lundberg I.
      Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates.
      • de Groot S.
      • Haisma J.A.
      • Post M.W.
      • van Asbeck F.W.
      • van der Woude L.H.
      Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study.
      • Holavanahalli R.K.
      • Lezotte D.C.
      • Hayes M.P.
      • et al.
      Profile of patients lost to follow-up in the burn injury rehabilitation model systems' longitudinal database.
      • Brilleman S.L.
      • Pachana N.A.
      • Dobson A.J.
      The impact of attrition on the representativeness of cohort studies of older people.
      In addition, completers' greater likelihood of living with parents or others suggests that supervision and/or social support associated with cohabitation might be protective against LTFU.
      Unlike completers, wave nonresponders and nonresponders tended to belong to historically socioeconomically disadvantaged groups (ie, those with less education, those with ethnic/racial minority backgrounds), which is consistent with previous work.
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      • Bergman P.
      • Ahlberg G.
      • Forsell Y.
      • Lundberg I.
      Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates.
      • de Groot S.
      • Haisma J.A.
      • Post M.W.
      • van Asbeck F.W.
      • van der Woude L.H.
      Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study.
      • Holavanahalli R.K.
      • Lezotte D.C.
      • Hayes M.P.
      • et al.
      Profile of patients lost to follow-up in the burn injury rehabilitation model systems' longitudinal database.
      • Brilleman S.L.
      • Pachana N.A.
      • Dobson A.J.
      The impact of attrition on the representativeness of cohort studies of older people.
      Physical independence was also predictive of both nonresponse and wave nonresponse, again in accord with Corrigan's work
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      showing that severe motor deficits mitigate LTFU risk. Perhaps physically mobile individuals are more difficult to follow because they have less stable living situations, lower likelihood of having caregivers, and/or more responsibilities within their communities.
      Interestingly, having acute care payer data in the NDB was associated with completing all follow-ups whether that payer was private (eg, commercial insurance) or public (eg, state-based financial assistance), suggesting that this finding was not related to SES. Conversely, participants missing baseline education, acute care payer, or baseline employment status data were more likely to be wave nonresponders or nonresponders. Subgroups with missing data often had other characteristics associated with LTFU (eg, less education, minority race).
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      • Bergman P.
      • Ahlberg G.
      • Forsell Y.
      • Lundberg I.
      Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates.
      • de Groot S.
      • Haisma J.A.
      • Post M.W.
      • van Asbeck F.W.
      • van der Woude L.H.
      Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study.
      • Holavanahalli R.K.
      • Lezotte D.C.
      • Hayes M.P.
      • et al.
      Profile of patients lost to follow-up in the burn injury rehabilitation model systems' longitudinal database.
      • Brilleman S.L.
      • Pachana N.A.
      • Dobson A.J.
      The impact of attrition on the representativeness of cohort studies of older people.
      However, missing data was also predictive of LTFU in subgroups with traits typically not associated with LTFU in this study or others (eg, more education, white race),
      • Corrigan J.D.
      • Harrison-Felix C.
      • Bogner J.
      • Dijkers M.
      • Terrill M.S.
      • Whiteneck G.
      Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
      • Bergman P.
      • Ahlberg G.
      • Forsell Y.
      • Lundberg I.
      Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates.
      • de Groot S.
      • Haisma J.A.
      • Post M.W.
      • van Asbeck F.W.
      • van der Woude L.H.
      Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study.
      • Holavanahalli R.K.
      • Lezotte D.C.
      • Hayes M.P.
      • et al.
      Profile of patients lost to follow-up in the burn injury rehabilitation model systems' longitudinal database.
      • Brilleman S.L.
      • Pachana N.A.
      • Dobson A.J.
      The impact of attrition on the representativeness of cohort studies of older people.
      suggesting that missing baseline data—and perhaps subsequent LTFU—in these individuals reflect the quality and/or consistency of data collection methods and not the characteristics of the participants themselves.
      Indeed, collecting acute care payer and baseline education and employment data requires only direct questioning of the participant, a reliable proxy, and/or the hospital billing department. These data might be missing because participants were too impaired to respond and no reliable proxy was available or because the data collection center could not or did not attempt to obtain information not recorded in the initial data collection (eg, by following up with a participant after level of consciousness improved or completing baseline data forms using information obtained at follow-up).
      Standard follow-up practices have been mandated by the TBIMS, but additional strategies to enhance data quality and follow-up completeness have been recommended (eg, periodic check-ins between mandated follow-ups and maintaining extensive records of alternative contact information for participants and proxies).

      Traumatic Brain Injury Model Systems National Data and Statistical Center. Guidelines and strategies for maximizing follow-up. Available at: https://www.tbindsc.org/StaticFiles/SOP/105a%20-%20Guidelines%20and%20Strategies%20for%20Maximizing%20Followup.pdf. Accessed January 11, 2011.

      Such optional strategies have been shown to improve retention even in populations traditionally among the most challenging to retain, such as substance abusers.
      • Scott C.K.
      A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance abusers.
      • Scott C.K.
      • Sonis J.
      • Creamer M.
      • Dennis M.L.
      Maximizing follow-up in longitudinal studies of traumatized populations.
      • Kleschinsky J.H.
      • Bosworth L.B.
      • Nelson S.E.
      • Walsh E.K.
      • Shaffer H.J.
      Persistence pays off: follow-up methods for difficult-to-track longitudinal samples.
      Future analyses should examine the association between use of optional follow-up methods and rates of LTFU in the TBIMS NDB and other longitudinal studies. Further, our data suggest particular subgroups for which these methods might be especially useful; for example, evening or weekend calls might be a preferred strategy for contacting individuals with intact motor functioning who might be less available during traditional working hours whereas participants with low SES might be more easily tracked by regularly searching public records.

       Study limitations

      Several other factors that might predict LTFU could not be included in our analysis because relevant data are not contained in the NDB. These include the specific follow-up practices (eg, retention strategies, participation incentives) used by different sites, the degree of coordination between research and clinical care follow-ups, participants' level of rehabilitation program compliance, and mood symptoms in participants during rehabilitation. The association between these factors and LTFU should be investigated in future studies, and relevant data could be considered for future inclusion in the NDB.
      In regard to limitations of our analysis, including all TBIMS participants eligible for follow-up in our analyses let us examine a broader range of response patterns than previous studies, but our findings might not be as generalizable as those obtained from a more circumscribed analysis. Nesting future analyses of the NDB within a center could yield data on geographic factors (eg, community setting and demographic populations served) that might modify the association between participant characteristics and LTFU. Second, our analysis did not define the LRP based on the number of follow-ups for which participants were eligible; therefore, an individual who completed 1 of 1 follow-up was classified identically to another who completed 5 of 5 follow-ups. The former participant could eventually be lost, yielding a different LRP. Future analyses might classify participants' LRPs while accounting for the number of follow-ups they are eligible to complete. Third, we were able to correctly classify most of those with the LRP of completed all, but correct classification of those with other LRPs was much less accurate, which was likely because of the relatively low frequency of most other LRPs in the NDB and the high degree of association between missing baseline data and LTFU. Therefore, future analyses aimed at developing highly accurate predictive models of LTFU might consider classifying participants according to fewer LRPs than used here (eg, completed all vs nonresponse and/or wave nonresponse) and/or not including missing data as a predictor. Finally, there is no empirically derived cutoff to determine which exhaustive CHAID index percentages are meaningful. Our data were consistent in that subgroups with index percentages around the cutoff of 120% were mostly similar to those with percentages even 2 or 3 times as high. However, the absence of a validated, stringent cutoff means that some findings, especially those supported by index scores around the cutoff, might not be meaningful.

      Conclusions

      Specific demographic, psychosocial, and postinjury functioning factors are predictive of LRPs in the TBIMS NDB. These data provide rehabilitation researchers with subject matter knowledge about risk factors for LTFU among persons with TBI and provide a basis for the addition of specific retention strategies for different subgroups at risk for LTFU. The ultimate aim is to increase retention of participants and reduce biases in data that threaten the validity and applicability of longitudinal outcome studies of persons with TBI.

      Supplier

      • a.
        SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL 60606.

      Acknowledgments

      We thank Wayne Gordon, PhD, for acting as a consultant on the design of this study, Dave Mellick, MA, and Chris Cusack, BA, of the TBIMS National Data and Statistical Center for their assistance with using the TBIMS NDB, and Michael Nguyen, MS, and Jennifer Oswald, BA, for consultation and information about issues relevant to TBIMS NDB data collection.

      References

        • Corrigan J.D.
        • Harrison-Felix C.
        • Bogner J.
        • Dijkers M.
        • Terrill M.S.
        • Whiteneck G.
        Systematic bias in traumatic brain injury outcome studies because of loss to follow-up.
        Arch Phys Med Rehabil. 2003; 84: 153-160
        • Levin H.S.
        • Gary Jr., H.E.
        • Eisenberg H.M.
        • et al.
        Neurobehavioral outcome 1 year after severe head injury. Experience of the Traumatic Coma Data Bank.
        J Neurosurg. 1990; 73: 699-709
        • Touloumi G.
        • Babiker A.G.
        • Pocock S.J.
        • Darbyshire J.H.
        Impact of missing data due to drop-outs on estimators for rates of change in longitudinal studies: a simulation study.
        Stat Med. 2001; 20: 3715-3728
        • Corrigan J.D.
        • Bogner J.A.
        • Mysiw W.J.
        • Clinchot D.
        • Fugate L.
        Systematic bias in outcome studies of persons with traumatic brain injury.
        Arch Phys Med Rehabil. 1997; 78: 132-137
        • Bell K.R.
        • Hammond F.
        • Hart T.
        • Bickett A.K.
        • Temkin N.R.
        • Dikmen S.
        Participant recruitment and retention in rehabilitation research.
        Am J Phys Med Rehabil. 2008; 87: 330-338
        • Hunt J.R.
        • White E.
        Retaining and tracking cohort study members.
        Epidemiol Rev. 1998; 20: 57-70
        • Kristman V.L.
        • Manno M.
        • Cote P.
        Methods to account for attrition in longitudinal data: do they work? A simulation study.
        Eur J Epidemiol. 2005; 20: 657-662
        • Greenland S.
        Response and follow-up bias in cohort studies.
        Am J Epidemiol. 1977; 106: 184-187
      1. University of Washington. Model Systems Knowledge Translation Center. Available at: http://msktc.washington.edu/research/searchpubs.asp?modelsys=tbi. Accessed September 21, 2011.

        • Dijkers M.P.
        • Harrison-Felix C.
        • Marwitz J.H.
        The traumatic brain injury model systems: history and contributions to clinical service and research.
        J Head Trauma Rehabil. 2010; 25: 81-91
      2. Traumatic Brain Injury Model Systems National Data and Statistical Center. 2012 TBI Model Systems presentation. Available at: https://www.tbindsc.org/StaticFiles/Documents/2012%20TBIMS%20Slide%20Presentation.pdf. Accessed January 10, 2013.

        • Harrison-Felix C.
        • Dijkers M.
        Predictors of loss to follow-up in a longitudinal study of persons with traumatic brain injury.
        Rehabilitation Institute of Michigan Del Harder Research Day, Detroit1998
        • Hoofien D.
        • Vakil E.
        • Gilboa A.
        • Donovick P.J.
        • Barak O.
        Comparison of the predictive power of socio-economic variables, severity of injury and age on long-term outcome of traumatic brain injury: sample-specific variables versus factors as predictors.
        Brain Inj. 2002; 16: 9-27
        • Arango-Lasprilla J.C.
        • Rosenthal M.
        • Deluca J.
        • et al.
        Traumatic brain injury and functional outcomes: does minority status matter?.
        Brain Inj. 2007; 21: 701-708
        • Gary K.W.
        • Arango-Lasprilla J.C.
        • Stevens L.F.
        Do racial/ethnic differences exist in post-injury outcomes after TBI? A comprehensive review of the literature.
        Brain Inj. 2009; 23: 775-789
        • Keith R.A.
        • Granger C.V.
        • Hamilton B.B.
        • Sherwin F.S.
        The functional independence measure: a new tool for rehabilitation.
        Adv Clin Rehabil. 1987; 1: 6-18
        • Hall K.
        • Cope D.N.
        • Rappaport M.
        Glasgow Outcome Scale and Disability Rating Scale: comparative usefulness in following recovery in traumatic head injury.
        Arch Phys Med Rehabil. 1985; 66: 35-37
        • Bushnik T.
        • Englander J.
        • Wright J.
        • Kolakowsky-Hayner S.A.
        Traumatic brain injury with and without late posttraumatic seizures: what are the impacts in the post-acute phase: a NIDRR Traumatic Brain Injury Model Systems study.
        J Head Trauma Rehabil. 2012; 27: E36-E44
        • Bergman P.
        • Ahlberg G.
        • Forsell Y.
        • Lundberg I.
        Non-participation in the second wave of the PART study on mental disorder and its effects on risk estimates.
        Int J Soc Psychiatry. 2010; 56: 119-132
        • Claus R.E.
        • Kindleberger L.R.
        • Dugan M.C.
        Predictors of attrition in a longitudinal study of substance abusers.
        J Psychoactive Drugs. 2002; 34: 69-74
        • de Groot S.
        • Haisma J.A.
        • Post M.W.
        • van Asbeck F.W.
        • van der Woude L.H.
        Investigation of bias due to loss of participants in a Dutch multicentre prospective spinal cord injury cohort study.
        J Rehabil Med. 2009; 41: 382-389
        • Brown D.M.
        • Thorne J.E.
        • Foster G.L.
        • et al.
        Factors affecting attrition in a longitudinal study of patients with AIDS.
        AIDS Care. 2006; 18: 821-829
        • Holavanahalli R.K.
        • Lezotte D.C.
        • Hayes M.P.
        • et al.
        Profile of patients lost to follow-up in the burn injury rehabilitation model systems' longitudinal database.
        J Burn Care Res. 2006; 27: 703-712
        • Malec J.F.
        • Brown A.W.
        • Leibson C.L.
        • et al.
        The mayo classification system for traumatic brain injury severity.
        J Neurotrauma. 2007; 24: 1417-1424
        • Kass G.V.
        An exploratory technique for investigating large quantities of categorical data.
        Appl Statis. 1980; 29: 119
        • Biggs D.
        • deVille B.
        • Suen E.
        A method of choosing multiway partitions for classification and decision trees.
        J Appl Statist. 1991; 18: 49
        • Brilleman S.L.
        • Pachana N.A.
        • Dobson A.J.
        The impact of attrition on the representativeness of cohort studies of older people.
        BMC Med Res Methodol. 2010; 10: 71
      3. Traumatic Brain Injury Model Systems National Data and Statistical Center. Guidelines and strategies for maximizing follow-up. Available at: https://www.tbindsc.org/StaticFiles/SOP/105a%20-%20Guidelines%20and%20Strategies%20for%20Maximizing%20Followup.pdf. Accessed January 11, 2011.

        • Scott C.K.
        A replicable model for achieving over 90% follow-up rates in longitudinal studies of substance abusers.
        Drug Alcohol Depend. 2004; 74: 21-36
        • Scott C.K.
        • Sonis J.
        • Creamer M.
        • Dennis M.L.
        Maximizing follow-up in longitudinal studies of traumatized populations.
        J Trauma Stress. 2006; 19: 757-769
        • Kleschinsky J.H.
        • Bosworth L.B.
        • Nelson S.E.
        • Walsh E.K.
        • Shaffer H.J.
        Persistence pays off: follow-up methods for difficult-to-track longitudinal samples.
        J Stud Alcohol Drugs. 2009; 70: 751-761