If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Mental Health and Behavioral Sciences, Defense and Veterans Brain Injury Center at James A. Haley Veterans’ Hospital, Tampa, FloridaDivision of Pulmonary and Sleep Medicine, Morsani College of Medicine, University of South Florida, Tampa, Florida
Division of Pulmonary and Sleep Medicine, Morsani College of Medicine, University of South Florida, Tampa, FloridaMedicine Service, James A. Haley Veterans’ Hospital, Tampa, Florida
Division of Pulmonary and Sleep Medicine, Morsani College of Medicine, University of South Florida, Tampa, FloridaMedicine Service, James A. Haley Veterans’ Hospital, Tampa, Florida
Research Department, Craig Hospital, Englewood, ColoradoDepartment of Physical Medicine and Rehabilitation, University of Colorado School of Medicine, Aurora, Colorado
Division of Pulmonary, Critical Care, and Sleep Medicine and Neuroscience Research Institute, Ohio State University Wexner Medical Center, Columbus, Ohio
Department of Psychiatry and Behavioral Service, Stanford University, Stanford, CaliforniaMental Illness Research Education and Clinical Center, VA Palo Alto Health Care System, Palo Alto, California
To determine the diagnostic sensitivity and specificity and comparative effectiveness of traditional sleep apnea screening tools in traumatic brain injury (TBI) neurorehabilitation admissions.
Design
Prospective diagnostic comparative effectiveness trial of sleep apnea screening tools relative to the criterion standard, attended level 1 polysomnography including encephalography.
Setting
Six TBI Model System Inpatient Rehabilitation Centers.
Participants
Between May 2017 and February 2019, 449 of 896 screened were eligible for the trial with 345 consented (77% consented). Additional screening left 263 eligible for and completing polysomnography with final analyses completed on 248.
Intervention
Not applicable.
Main Outcome Measures
Area under the curve (AUC) of screening tools relative to total apnea hypopnea index≥15 (AHI, moderate to severe apnea) measured at a median of 47 days post-TBI (interquartile range, 29-47).
Results
The Berlin high-risk score (receiving operating curve [ROC] AUC=0.634) was inferior to the Multivariable Apnea Prediction Index (MAPI) (ROC AUC=0.780) (P=.0211; CI, 0.018-0.223) and Snoring, Tired, Observed, Blood Pressure, Body Mass Index, Age, Neck Circumference, and Gender (STOPBANG) score (ROC AUC=0.785) (P=.001; CI, 0.063-0.230), both of which had comparable AUC (P=.7245; CI, −0.047 to 0.068). Findings were similar for AHI≥30 (severe apnea); however, no differences across scales was observed at AHI≥5. The pattern was similar across TBI severity subgroups except for posttraumatic amnesia (PTA) status wherein the MAPI outperformed the Berlin. Youden’s index to determine risk yielded lower sensitivities but higher specificities relative to non-TBI samples.
Conclusion
This study is the first to provide clinicians with data to support a choice for which sleep apnea screening tools are more effective during inpatient rehabilitation for TBI (STOPBANG, MAPI vs Berlin) to help reduce comorbidity and possibly improve neurologic outcome.
Given the significance of sleep after TBI, early detection and facilitation of treatment of sleep disorders should be a central focus of rehabilitation efforts.
While many sleep disorders can occur after TBI, one of the least studied yet most likely to negatively affect recovery is sleep apnea. Sleep apnea is a sleep-related breathing disorder that is characterized by repeated cessation (ie, apnea) or near cessation (ie, hypopnea) of ventilation during sleep. Subtypes of sleep apnea include obstructive (OSA), central, or mixed type. In a meta-analysis of studies across all severities of TBI, the prevalence of OSA was 12 times higher than in community-based samples.
Two additional single-center studies conducted during TBI inpatient rehabilitation found high rates of clinically significant sleep apnea in one-third of participants.
Assessment of sleep apnea and other sleep disorders is critical to the delivery of correctly targeted, evidence-based treatments. Little guidance exists to inform assessment of specific sleep disorders in inpatient neurorehabilitation settings. The Agency for Healthcare Research and Quality has highlighted the need for comparative effectiveness research in risk stratification for sleep apnea diagnostic studies.
Further, multidisciplinary stakeholder panels have prioritized earlier detection of sleep apnea in TBI recovery, and economic modeling has highlighted significant cost savings with successful treatment.
Nonetheless, criterion standard evaluation (level 1 polysomnography [PSG]) is expensive and less accessible during inpatient rehabilitation hospitalization, and confusion and agitation may add to the challenge. Appropriate screening tools are necessary to prioritize referrals for further evaluation by sleep specialists and begin early treatment thus potentially minimizing secondary neurologic injury.
Despite increasing awareness of high rates of OSA in TBI and potential for negative effect, the accuracy, sensitivity, and specificity of the common screening methods have not been evaluated in hospitalized TBI.
These questionnaires assess demographics, physical attributes, comorbidities, daytime sleepiness, and snoring characterization to assess risk of sleep apnea. They were primarily developed and validated with samples of older adults without TBI in outpatient settings.
Their diagnostic utility and precision in populations without TBI varies considerably because of the heterogeneity of content and the demographic (age, sex breakdown) and overall health of study samples (obesity, comorbidities).
Clinical practice guideline for diagnostic testing for adult obstructive sleep apnea: an American Academy of Sleep Medicine clinical practice guideline.
the purpose of this study is to examine the diagnostic accuracy and comparative effectiveness of common sleep apnea screening tools during inpatient rehabilitation for those with moderate to severe TBI relative to the criterion standard, level 1 PSG with electroencephalogram attended by certified sleep technologist (Clinicaltrial.gov no.: NCT03033901).
Methods
Participants
Potential participants were consecutive patients enrolled in the TBI Model Systems (TBIMS) at 6 sites (Tampa, FL, Seattle, WA, Dallas, TX, Columbus, OH, Denver, CO, Philadelphia, PA) over 19 months. Study inclusion/exclusion criteria for the TBIMS and this trial are described in appendix 1. The requirement of TBIMS enrollment at time of consent for the clinical trial was relaxed at study month 11 to allow for earlier enrollment during rehabilitation, but the clinical criteria remained unchanged.
Procedure
All participating sites received institutional review board approval for conduct of the study. Consecutive admissions were screened for eligibility. Participants who passed the first level of screening (or their proxies) were consented and further screened for final eligibility including (1) >2 hours sleep per night based on actigraphy placement or nursing logs and/or reports and (2) medical stability (including no emergent medical issues precluding overnight PSG and minimal to no posttraumatic agitation, as assessed by the Agitated Behavior Scale). Once determined eligible, an overnight PSG study was conducted by a registered polysomnographic technologist (RPSGT) in the participant’s own bed. Within 72 hours of the PSG, questionnaire-based sleep apnea screening measures were completed with the participant and/or best source available using established TBIMS procedures by local staff blinded to PSG results. Sleep-related information during hospitalization was collected from the medical staff (snoring status, daytime sleepiness) or medical record (weight, height). The patient-reported outcome was the primary source of data. When data were missing because of the participant’s inability to respond or with unknown responses, they were imputed using best source data if available; otherwise, they were considered missing data.
Fully attended level 1 PSG was conducted in accordance with the American Academy of Sleep Medicine recommended procedures.
The RPSGT also conducted a physical examination of the participants and rated agitation. Staff were instructed to allow participants to sleep their normal habitual sleep period with a minimum of 2 hours of sleep needed for adequate study.
The lead center (James A. Haley Veterans Hospital, Tampa, FL) served as a centralized scoring center for all sleep studies. All deidentified studies were scored by 1 of 2 certified RPSGTs (CD, LW) and interpreted by a board-certified sleep medicine physician (DS, KC). All staff who scored and interpreted studies were blinded to other sleep assessments.
Measures
Demographic and preinjury medical histories and medical record abstraction were conducted by trained research assistants following the TBIMS protocol. Glasgow Coma Scale (GCS)
score on admission to the emergency department and duration of posttraumatic amnesia (PTA; time elapsed from injury until return of orientation and memory for events surrounding injury used in the TBIMS
protocol) were the primary markers of injury severity. The presence of medications on the day of PSG with sleep effects (opiates, sedatives-hypnotics, antidepressants, neurostimulants, antihypertensives, antihistamines, antiepileptics) were abstracted from the medical record, irrespective of whether they were prescribed for sleep. Sleep duration during hospitalization was recorded using actigraphy (Actiwatch Spectruma) (appendix 2). Level of agitation during polysomnography was rated using the Agitated Behavior Scale
Severity of sleep apnea is measured by the Apnea-Hypopnea Index (AHI), which calculates the number of apneas (≥90% decrease in airflow) and hypopneas (30% reduction in airflow with ≥3% decrease in O2 saturation or an arousal) for a minimum of 10 seconds.
Obstructive AHI is the AHI only due to obstructive events (ie, excluding mixed and central apneas and hypopneas). Severity of sleep apnea was graded by frequency of AHI events per hour with 5-14 denoting mild sleep apnea, 15-29 denoting moderate, and ≥30 indicating severe sleep apnea
(see appendix 2 for an expanded description). PSG was conducted with the Philips Alice 6 LDx Diagnostic Sleep System and scored with Philips Sleepware G3 version 3.8.1.b
Sleep apnea screening tools (comparators)
Appendix 2 summarizes each of the sleep apnea screening comparators in detail. Briefly, the Snoring, Tired, Observed, Blood Pressure, Body Mass Index, Age, Neck Circumference, and Gender (STOPBANG) questionnaire is an 8-item measure that refers to loud snoring, tiredness, observed breathing pauses, high blood pressure, elevated body mass index, older age, large neck circumference, and male sex.
The Berlin Questionnaire is a 10-item measure that evaluates and groups risk factors into 3 categories (snoring severity, excessive daytime sleepiness, and history of high blood pressure or obesity).
The Multivariable Apnea Prediction Index (MAPI) consists of 3 breathing-related questions and information on demographics from which a probability of having sleep apnea (0%-100%) can be calculated.
All statistical analyses were conducted using SAS 9.4.b To determine the predictive utility of the STOPBANG, MAPI probability, and Berlin screening tools for diagnosing mild (AHI≥5), moderate (AHI≥15), and severe (AHI≥30) sleep apnea, receiver operating characteristic (ROC) curve analyses were performed. An ROC curve plots the true positive rate (sensitivity) against the false positive rate (1–specificity) for all possible cutoff scores of the screening tools. The ROC area under curve (AUC) and corresponding 95% CI were estimated to provide a measure of overall discrimination for each screening tool. Standard diagnostic measures including sensitivity (SE), specificity (SP), positive predictive value (PPV), negative predictive value (NPV), and false negative rate (FNR) values were calculated across varying cutoff scores of STOPBANG, MAPI, and Berlin. The Youden index, commonly used to determine the optimal cutoff score where equal weight is given to SE and SP (defined as SE+SP–1, was also calculated).
For MAPI probability, cutoff scores of 0.1-0.9 in increments of 0.1 were summarized along with the optimal cutoff score. For the Berlin, participants were either low risk (0 or 1 category positive) or high risk (≥2 categories positive), so there is only 1 possible cutoff score. The ROC AUCs were compared among the 3 screening tools using a χ2 test. Median with interquartile range are provided.
Results
Sample characteristics
Between May 2017 and January 2019, a total of 896 patients were screened, with 449 initially eligible and 345 consented (77%). Additional screening postconsent resulted in 263 completing PSG and a final analytic sample of 248 (fig 1 and details in appendix 3). Table 1 summarizes participant demographics, injury characteristics, medical and physical status at time of PSG, medications, and sleep apnea diagnostic results. PSG occurred a median of 47 days post-TBI (interquartile range, 29-87 days), with a majority having emerged from posttraumatic amnesia (85%) with low levels of agitation during PSG. Results of PSG revealed elevated (≥5) total and obstructive AHI for a majority of the sample: 68% and 66%, respectively. Given the predominance of obstructive events and high correlation between total and obstructive AHI (r=0.95; P<.0001), total AHI was used in subsequent analyses. Statistical comparison of the subjects excluded because of insufficient sleep (n=12) with those retained (n=248) was not meaningful because of small sample sizes; however, trends indicated that excluded participants were more likely to be male (92% vs 82%), in PTA at time of PSG (33% vs 15%), older at time of PSG (median, 56 vs 40 years), and have higher Agitated Behavior Scale scores (median, 22 vs 14). Groups were similar in terms of GCS and days from injury to PSG.
Fig 1Sample flow diagram. Final sample size obtained for the study is 248 participants.
Table 2 provides summary information for the individual items, subscales, and traditional cutoff scores for the screening measures. Across the 3 scales, the MAPI had the highest number of participant responses (85%-87% across items) compared with the STOPBANG (75%-88%) and Berlin (74%-90%). Significant variability across items was observed across scales. The phrasing of certain items on the Berlin proved to be challenging for some respondents (patient and best source), resulting in high levels of missingness on the Category 2 (n=53) and 1 (n=35) scales resulting in 75 participants without a risk profile. The Berlin item with the highest rate of missingness was related to sleepiness while driving. Across all scales, snoring questions were commonly missing responses; however, the phrasing on the Berlin resulted in greater missingness relative to the STOPBANG and MAPI, which had overall lower rates of missingness on items.
Table 2Description of sleep apnea screening questionnaire comparators completed at time of PSG
No. of cases with data imputed from clinical records. For some cases, a risk profile could be computed because other scale item responses were sufficient to indicate a risk status, thus decreasing the number of cases wherein no risk status could be computed.
STOPBANG
Patient source (%), range across items 1-8
75.4-87.5
Item 1 (S) loud snoring, n (% yes) [missing]
30 (12.4) [6]
Item 2 (T) tired or fatigued, n (% yes) [missing]
170 (69.4) [3]
Item 3 (O) observed stop breathing while sleeping, n (% yes) [missing]
14 (5.7) [1]
Item 4 (P) high blood pressure, n (% yes) [missing]
No. of cases with data imputed from clinical records. For some cases, a risk profile could be computed because other scale item responses were sufficient to indicate a risk status, thus decreasing the number of cases wherein no risk status could be computed.
Item 6 (A) age older than 50 y, n (% yes) [missing]
No. of cases with data imputed from clinical records. For some cases, a risk profile could be computed because other scale item responses were sufficient to indicate a risk status, thus decreasing the number of cases wherein no risk status could be computed.
Item 8 (G) gender male, n (% yes) [missing]
204 (82.3) [0]
STOPBANG score, mean ± SD
2.56±1.30 [9] [0, 2, 2, 3, 7]
STOPBANG score≥3, n (% yes) [missing]
115 (48.1) [9]
STOPBANG score≥5, n (% yes) [missing]
18 (7.5) [9]
Berlin
Patient source (%), range across items 1-13
73.6-89.9
Item 1 (% snore), n (% yes) [missing]
147 (61.8) [10]
Item 2 (% snore louder than talking/very loud), n (% yes) [missing]
21 (9.2) [19]
Item 3 (% snore nearly every day/3-4 times per wk), n (% yes) [missing]
50 (22.3) [24]
Item 4 (% snore bothers others), n (% yes) [missing]
24 (10.4) [18]
Item 5 (% quit breathing), n (% yes) [missing]
22 (9.2) [8]
Category 1 total score (Items 1-5), mean ± SD
0.9±1.5 [35] [0, 0, 0, 1, 6]
Category 1% positive (≥2 points), n (% yes) [missing]
53 (24.9) [35]
Item 6 (% tired after sleep nearly every day), n (% yes) [missing]
141 (58.3) [6]
Item 7 (% tired when awake nearly every day), n (% yes) [missing]
No. of cases with data imputed from clinical records. For some cases, a risk profile could be computed because other scale item responses were sufficient to indicate a risk status, thus decreasing the number of cases wherein no risk status could be computed.
Category 3 (Items 9-10), mean ± SD
0.4±0.6 [6] [0, 0, 0, 1, 2]
Category 3 % positive (≥1 point), n (%yes) [missing]
88 (35.9%) [3]
Berlin risk, n (%)
High risk (≥2 categories positive)
54 (31.2)
Low risk (≤1 category positive)
119 (68.8)
[Missing]
[75]
MAPI
Patient source (%), range across items 1-3
84.8-86.5
MAPI Item 1 (snorting or gasping for air), mean ± SD
0.21±0.77 [9] [0, 0, 0, 0, 4]
Never, n (%)
216 (90.4)
Rarely, less than once/wk, n (%)
11 (4.6)
1-2 times/wk, n (%)
3 (1.3)
3-4 times/wk, n (%)
2 (0.8)
5-7 times/week, n (%)
7 (2.9)
[Missing]
[9]
MAPI Item 2 (loud snoring), mean ± SD
0.38±1.05 [12] [0, 0, 0, 0, 4]
Never, n (%)
204 (86.4)
Rarely, less than once/wk, n (%)
8 (3.4)
1-2 times/wk, n (%)
3 (1.3)
3-4 times/wk, n (%)
9 (3.8)
5-7 times/wk, n (%)
12 (5.1)
[Missing]
[12]
MAPI Item 3 (stop breathing, choking, struggle to breathe), mean ± SD
0.19±0.75 [3] [0, 0, 0, 0, 4]
Never, n (%)
228 (93.1)
Rarely, less than once/wk, n (%)
2 (0.8)
1-2 times/wk, n (%)
4 (1.6)
3-4 times/wk, n (%)
7 (2.9)
5-7 times/wk, n (%)
4 (1.6)
[Missing]
[3]
MAPI (average of items 1-3), mean ± SD
0.34±0.71 [13] [0, 0, 0, 0.33, 4]
MAPI probability of sleep apnea, mean ± SD
0.23±0.20 [13] [0.01, 0.08, 0.15, 0.34, 0.90]
Abbreviation: BMI, body mass index (calculated as weight in kilograms divided by height in meters squared).
∗ No. of cases with data imputed from clinical records. For some cases, a risk profile could be computed because other scale item responses were sufficient to indicate a risk status, thus decreasing the number of cases wherein no risk status could be computed.
Fig 2STOPBANG, Berlin, and MAPI probability ROC curve for AHI≥15. Across all AHI cutoffs for severity of sleep apnea, the STOPBANG generally had the highest ROC AUC, followed by MAP and the Berlin.
The ROC AUC for STOPBANG across all scores was 0.79 (95% CI, 0.72-0.85). SE was high (>0.8) for lower STOPBANG cutoff scores (0-3) but sharply decreased for cutoff scores of ≥4. SP was high (>0.8) for STOPBANG cutoff scores of ≥4 and low for cutoff scores of 0-3. At AHI≥15, the highest Youden’s index was attained at a STOPBANG cutoff score of ≥3 (SE=0.80, SP=0.68, PPV=0.57, NPV=0.87, FNR=0.20). The optimal cutoff score for STOPBANG of ≥3 was consistent for AHI≥5 and AHI≥15.
Berlin
The ROC AUC for Berlin (high-risk score) was 0.63 (95% CI, 0.56-0.71). SE was low (0.50) and SP was moderate (0.77) (PPV=0.48, NPV=0.78, FNR=0.50). Compared with AHI≥15, SE (0.53) and SP (0.73) were similar for AHI≥30; however, SE was lower (0.40) and SP was higher (0.86) for AHI≥5.
MAPI probability
The ROC AUC for MAPI probability across all scores was 0.78 (95% CI, 0.72-0.84). SE was high (>0.8) for MAPI probability cutoff score of ≥0.1 but sharply decreased for cutoff scores of ≥0.2. SP was high (>0.8) for MAPI probability cutoff scores of ≥0.3 and low for cutoff scores of 0.1-0.2. At AHI≥15, the highest Youden’s index was attained at a MAPI probability cutoff score of ≥0.231 (SE=0.69, SP=0.79, PPV=0.61, NPV=0.84, FNR=0.31). The optimal cutoff score for MAPI probability was lower for AHI≥5 (≥0.121) but remained the same for AHI≥30 (≥0.231).
Comparative effectiveness
Pairwise comparisons of the AUC ROCs for STOPBANG, MAPI probability, and Berlin for moderate to severe sleep apnea (AHI≥15) are summarized in table 4. Comparisons for AHI≥5 and AHI≥30 are summarized in supplemental table S3 (available online only at http://www.archives-pmr.org/). Across all AHI cutoffs for severity of sleep apnea, the STOPBANG generally had the highest ROC AUC, followed by MAPI, then the Berlin. For AHI≥15 and AHI≥30, the STOPBANG and MAPI had significantly higher ROC AUC than the Berlin but did not differ significantly from each other. For AHI≥5, there were no significant differences in the ROC AUC among the 3 screening tools. Comparisons of the ROC AUC among the screening tools within the GCS and PTA subgroups are also summarized (see table 4) with largely similar findings except for relatively better performance of the MAPI among those in PTA. Diagnostic summaries for AHI≥15 for STOPBANG, Berlin, and MAPI probability across relevant cutoff scores are summarized in supplemental table S4 and S5 (available online only at http://www.archives-pmr.org/) for GCS (mild/moderate vs severe) and PTA status at time of PSG (in PTA vs out of PTA), respectively.
Table 4Pairwise comparisons of ROC AUC across STOPBANG, MAPI, and Berlin screening tools for AHI≥15 for the total sample and subgroups
Comparison
SB AUC
MAPI AUC
Berlin AUC
n
Difference
95% CI
P Value
Full sample
STOPBANG vs MAPI
0.790
0.780
-
230
0.010
(−0.047 to 0.068)
.725
STOPBANG vs Berlin
0.782
-
0.636
170
0.147
(0.063-0.230)
.001
MAPI vs Berlin
-
0.744
0.624
166
0.121
(0.018-0.223)
.021
Mild/moderate TBI
STOPBANG vs MAPI
0.747
0.739
-
82
0.008
(−0.094 to 0.111)
.874
STOPBANG vs Berlin
0.750
-
0.560
51
0.151
(0.003-0.298)
.045
MAPI vs Berlin
-
0.741
0.600
51
0.141
(0.011-0.271)
.033
Severe TBI
STOPBANG vs MAPI
0.792
0.822
-
119
−0.031
(−0.136 to 0.074)
.568
STOPBANG vs Berlin
0.809
-
0.644
94
0.165
(0.041-0.288)
.009
MAPI vs Berlin
-
0.772
0.601
91
0.171
(−0.002 to 0.344)
.053
Out of PTA at PSG
STOPBANG vs MAPI
0.785
0.761
-
196
0.024
(−0.039 to 0.086)
.462
STOPBANG vs Berlin
0.784
-
0.643
145
0.141
(0.054-0.229)
.002
MAPI vs Berlin
-
0.725
0.630
142
0.095
(−0.016 to 0.206)
.094
In PTA at PSG
STOPBANG vs MAPI
0.822
0.901
-
34
–0.079
(−0.237 to 0.080)
.330
STOPBANG vs Berlin
0.747
-
0.597
25
0.149
(−0.111 to 0.410)
.261
MAPI vs Berlin
-
0.889
0.589
24
0.300
(0.020-0.580)
.036
NOTE. AUC values vary relative to individual scale diagnostics in table 3 because of different sample sizes (data for both scales required for comparative analysis).
This is the first study examining the diagnostic utility of sleep apnea screening tools during TBI inpatient rehabilitation. We compared the AUC value for detecting moderate to severe sleep apnea (ie, AHI≥15 present in 33.4% of participants) across screening measures with significant differences observed between the Berlin relative to both other measures. Overall, the Berlin high-risk score was inferior to the MAPI and STOPBANG, both of which had comparable AUC with nonsignificant differences. Comparative effectiveness at AHI≥30 revealed the same pattern; however, no difference was observed between the 3 scales at AHI≥5. Subgroup comparisons revealed a similar pattern except for the MAPI performing best in the PTA positive subgroup perhaps because of the abbreviated and simpler nature of the scale.
Results highlight the diagnostic sensitivity of sleep apnea screening tools for a hospitalized cohort with TBI across a range of cutoff scores. For example, a score of ≥3 (traditionally considered a high-risk score) on the STOPBANG and ≥0.231 probability on the MAPI provided an equal balance between sensitivity and specificity (Youden’s index) across AHI cutoffs (5, 15, 30). In general, the Youden’s index sensitivity diagnostics perform worse and the specificity values better when comparing this hospitalized population with TBI with populations without TBI who are demographically (younger vs typically middle-age to older male adults) and morphologically different (nonobese vs obese).
Data additionally provide important information about the false negative rates across the tools. For detecting AHI≥15 and AHI≥30, the STOPBANG Youden’s index had the lowest false negative rate (eg, AHI≥15: STOPBANG; n=16/81, 20%; MAPI n=24/77, 31%; Berlin n=26/52, 50%) whereas the MAPI Youden’s index outperformed at AHI≥5 (ie, AHI≥5: STOPBANG; n=64/162, 40%; MAPI n=47/158, 30%; Berlin n=70/116, 60%). Subgroup analyses provided similar findings; however, those in PTA had a Youden’s index of ≥4 at AHI≥15. Variability in the cutoff on MAPI probability scores was observed across TBI severity (≥0.316 to ≥0.188) and PTA status (≥0.231 to ≥0.280) subgroups. Given the small size of the group in PTA, findings warrant caution.
Study limitations
Overall, the screening tools varied sufficiently in content that patterns of missingness were not consistent across scales. Missing item rates highlight the challenge of using existing screening tools in a hospitalized population with TBI. Specifically, content regarding driving or observations of sleep-related behavior (ie, snoring) while participants are commonly without a bed partner were often missing and may have limited the sensitivity of the scales. Further, acutely confused and cognitively impaired persons may not accurately report symptoms that may enhance risk stratification. Alternative approaches to collecting data (best source only) or objective biomarkers may improve risk detection among the most confused and cognitively impaired patients. Additionally, some eligible patients were not able to participate because of discharge prior to the availability of a sleep technologist; we believe that this was a random occurrence and therefore unlikely to create selection bias. However, there were several participants who would have otherwise been eligible but could not participate because they were not sleeping at least 2 hours per night (n=12) or they were not medically stable. These individuals may have been able to complete the screening measures but could not complete the PSG used for comparison; thus, we are not able to generalize results to this portion of the population.
This is the first study to examine the diagnostic sensitivity and comparative effectiveness of the STOPBANG, MAPI, and Berlin in a predominantly young cohort of patients with TBI during early rehabilitation. Additional strengths include the prospective, multicenter design with a large, well-characterized cohort, enhancing generalizability. A key strength is the use of level 1 PSG (criterion standard) administered by trained RPSGT staff with centralized scoring and interpretation by a board-certified sleep medicine physician. Finally, diagnostic sensitivity and relative comparative effectiveness was further explored in key subgroups that may be associated with differential findings (TBI severity, PTA status). However, there are several limitations to the study. The study sample may not represent the full population of inpatient rehabilitation patients with TBI as many were excluded (fig 1), including those with less than 2 hours of sleep on PSG. Shortening lengths of stay (insurance-related) affected study participation, with a significant number of eligible participants discharged prior to RPSGT availability (ie, multiple patients eligible at same time or unanticipated early discharge). Increasing the number of RPSGT staff (ie, contractors) at each site addressed this limitation early in the study. As a result, the study enrolled and completed PSG on 263 TBI admissions over 19 months.
The data reflect diagnostic sensitivity of instruments when prioritizing patient responses and supplementing with best-source data. Future studies may improve diagnostic accuracy of screening tools by examining various combinations of items, sources of information, and other physical and TBI biometrics. Supplementation with objective indices of sleep quality using actigraphy may also improve detection in those with significant cognitive impairments and who are unreliable informants. It is unclear if findings would extend to chronic phases of TBI wherein some risk factors for worsening of OSA increase (weight gain, tobacco use, problematic alcohol use) and others decrease (sedating medications), potentiating a different risk profile.
Obesity and overweight problems among individuals 1 to 25 years following acute rehabilitation for traumatic brain injury: a NIDLRR traumatic brain injury model systems study.
Future analyses planned with these data include subgroup analyses for age and military status.
Conclusions
In conclusion, this study is the first to provide clinicians with data to support a choice for which a sleep apnea screening tool is more effective during inpatient rehabilitation (ie, STOPBANG, MAPI) to help reduce comorbidity and possibly improve neurologic outcome.
Suppliers
a.
Actiwatch Spectrum; Philips Alice 6 LDx Diagnostic Sleep System; Philips Sleepware G3 version 3.8.1; Philips/Respironics.
b.
SAS 9.4; SAS.
Acknowledgments
We thank the following staff for their efforts in recruitment and data collection: Tampa: Danielle O’Connor, MPH, Carlos Diaz-Sein, RPSGT, Lancie Wharton, RPSGT, Emily Noyes, MA; Ohio State: Jacob Goodfleisch, BA, Dominic Sauer; U. Washington: Erica Wasmund; Craig Hospital: Angela Philippus, MS, Jody Newman, MA, CCC-SLP, Emily Almeida, MS, Michael Makley, MD, Alan Weintraub, MD, Eric Spier, MD; Baylor Scott & White Rehabilitation: Amber Lopez-Merfeld, MPH, Lacy Hinkle, Rosemary Dubiel, DO, Terrie Jones, RN, RRT, RCP, David L. Luterman, MD; Moss Rehabilitation Research Institute: Devon Kratchman, Rachel Raucci, Julie Wilson, Kelly McLaughlin, Amber Leon, Brandice Coleman, Grace Loscalzo.
Appendiix
Appendix 1Inclusion and exclusion criteria
Inclusion
PCORI Clinical Trial Exclusion
Damage to brain tissue caused by an external mechanical force
Habitual sleep duration >2 hours/night for 2 consecutive nights not being established prior to PSG
Alteration of consciousness >24 hours, or loss of consciousness >30 minutes, or GCS score in the emergency department of 3-12, or intracranial abnormalities on imaging regardless of GCS
Presence of a physical deformity precluding sensitivity of PSG instrumentation (ie, full-body cast, nasogastric tube that could not be removed prior to PSG)
Infeasibility of tracheostomy placement with decannulation or overnight capping during rehabilitation
Consent to participate by person with brain injury (if able), family member, or legally authorized representative into the TBI Model System lifetime study
Severity of sleep apnea is measured by the AHI, which calculates the number of apnea (≥90% decrease in airflow) and hypopneas (30% reduction in airflow with at ≥3% decrease in O2 saturation or an arousal) for a minimum of 10 seconds. Parameters collected for the purpose of this study: AHI, central AHI, obstructive AHI, and mixed AHI. A diagnosis of sleep apnea is determined by an AHI≥5. Severity of sleep apnea will be graded by AHI events per hour: 5-14 denoting mild, 15-29 denoting moderate, and ≥30 indicating severe sleep apnea. PSG was conducted with the Philips Alice 6 LDx Diagnostic Sleep System and scored with Philips Sleepware G3 version 3.8.1.
OSA screening (comparator)
STOPBANG
The STOPBANG is composed of 8 items that refer to snoring, tiredness, observed breathing pauses during sleep, treatment for high blood pressure, elevated BMI, older age, wide neck circumference, and male sex. An affirmative response to 2 items indicates low risk, 3-4 items intermediate risk, 5-8 items high risk. The validation study found the measure to have good SE, SP, and NPV of obstructive sleep apnea according to the AHI. The sensitivity cutoffs of AHI≥5 was 83.6% (SP=56.4%, NPV=60.8%), AHI≥15 was 92.9% (SP=43.0%, NPV=90.2%), and AHI≥30 was 100.0% (SP=37.0%, NPV=100%) for the presence of mild, moderate, and severe sleep apnea, respectively.
OSA screening (comparator)
Berlin
The Berlin Questionnaire is a 10-item measure that evaluates risk factors for sleep apnea into 3 categories (snoring severity, excessive daytime sleepiness and history of high blood pressure or obesity). Positivity in two or more of these categories is associated with a high likelihood of clinically-relevant sleep apnea. The questions have good internal consistency, with Cronbach’s alpha of 0.86-0.92. Individuals classified as high risk are associated with a Respiratory Distress Index of greater than 5 with 0.86 sensitivity and 0.77 specificity.
OSA screening (comparator)
MAPI
The questionnaire consists of 3 breathing-related questions and information on demographics (sex, weight, height, age), from which a probability of having sleep apnea (0%-100%) can be calculated. The test-retest reliability of the breathing questions is 0.92 and has a good internal consistency with Cronbach α of 0.85-0.93. A MAPI score of 0.50 (ie, calculated 0% likelihood of having clinically significant sleep apnea with a Respiratory Distress Index>10) has a 0.88 sensitivity and 0.55 specificity. Because the MAPI was developed in a general adult population who had been referred to sleep clinics, it will be important to explore whether it has probative value as a screening tool in those with TBI.
Total sleep time screening
Actigraphy
A wrist-worn accelerometer (Actiwatch Spectrum) was used to document sleep metrics during the trial. Activity data and ambient illumination (in lux) were both recorded in 15-second intervals. Data were scored with Actiware 5 software. The software uses validated algorithms to determine whether a 15-second epoch of activity is “sleep” or “wake.” Actigraphy devices in a TBI neurorehabilitation setting have been shown to be feasible and valid for detecting problems with sleep.
Agitation and other problematic behaviors in acute TBI recovery
ABS
The ABS is an observational rating scale describing a range of behaviors rate from 1-4, with 1 being absent and 4 being extreme, based on the extent to which the behavior interferes with functional activities and can be redirected. A total score and 3 subscale scores are derived from individual items (Disinhibition, Aggression, Lability). The ABS has been shown to have strong interrater reliability, internal consistency, and construct validity.
Abbreviations: ABS, Agitated Behavior Scale; BMI, body mass index.
Clinical practice guideline for diagnostic testing for adult obstructive sleep apnea: an American Academy of Sleep Medicine clinical practice guideline.
Obesity and overweight problems among individuals 1 to 25 years following acute rehabilitation for traumatic brain injury: a NIDLRR traumatic brain injury model systems study.
Supported by Patient-Centered Outcomes Research Institute (award no. CER-1511-33005); the VHA Central Office VA TBI Model Systems Program of Research; and a subcontract from General Dynamics Information Technology (W91YTZ-13-C-0015, HT0014-19-C-0004) from the Defense and Veterans Brain Injury Center and National Institute on Disability, Independent Living, and Rehabilitation Research (grant nos. 90DPTB00070, 90DP0084, 90DPTB0013-01-00, 90DPTB0008, 90DPT80004-02). The statements presented in this publication are solely the responsibility of the author(s) and do not necessarily represent the views of the Patient-Centered Outcomes Research Institute, its Board of Governors, or the Methodology Committee.
31364-4&id=gr1.jpg){kind=link}
31364-4&id=gr2.jpg){kind=link}
31364-4&id=figs1.jpg){kind=link}