Advertisement

Reliability and Validity of S3 Pressure Sensation as an Alternative to Deep Anal Pressure in Neurologic Classification of Persons With Spinal Cord Injury

Open AccessPublished:February 24, 2016DOI:https://doi.org/10.1016/j.apmr.2016.02.006

      Abstract

      Objective

      To determine whether pressure sensation at the S3 dermatome (a new test) could be used in place of deep anal pressure (DAP) to determine completeness of injury as part of the International Standards for Neurological Classification of Spinal Cord Injury.

      Design

      Prospective, multicenter observational study.

      Setting

      U.S. Spinal Cord Injury Model Systems.

      Participants

      Persons (N=125) with acute traumatic spinal cord injury (SCI), neurologic levels T12 and above, were serially examined at 1 month (baseline), 3, 6, and 12 months postinjury. There were 80 subjects with tetraplegia and 45 with paraplegia.

      Interventions

      S3 pressure sensation at all time points, with a retest at the 1-month time point.

      Main Outcome Measures

      Test-retest reliability and agreement (κ), sensitivity, specificity, positive and negative predictive values.

      Results

      Test-retest reliability of S3 pressure at 1 month was almost perfect (κ=.98). Agreement of S3 pressure with DAP was substantial both at 1 month (κ=.73) and for all time points combined (κ=.76). The positive predictive value of S3 pressure for DAP was 89.3% at baseline and 90.3% for all time points. No pattern in outcomes was seen in those cases where S3 pressure and DAP differed at 1 month.

      Conclusions

      S3 pressure sensation is reliable and has substantial agreement with DAP in persons with SCI at least 1 month postinjury. We suggest S3 pressure as an alternative test of sensory sacral sparing for supraconus SCI, at least in cases where DAP cannot be tested. Further research is needed to determine whether S3 pressure could replace DAP for classification of SCI.

      Keywords

      List of abbreviations:

      AIS (American Spinal Injury Association Impairment Scale), DAP (deep anal pressure), LT (light touch), PP (pinprick), PPV (positive predictive value), SCI (spinal cord injury), VAC (voluntary anal sphincter contraction)
      The neurologic classification for persons with spinal cord injury (SCI) has been revised several times since it was first reported in 1982.
      American Spinal Injury Association
      Standards for neurological classification of spinal injury patients.
      The classification uses data from a structured neurologic examination to determine the level and severity of injury. The initial measure of severity was the Frankel Scale.
      • Frankel H.L.
      • Hancock D.O.
      • Hyslop G.
      • et al.
      The value of postural reduction in the initial management of closed injuries of the spine with paraplegia and tetraplegia, part I.
      It categorized severity of injury into 5 grades, A through E, with A being “complete” and E being “normal.” A major revision occurred in 1992, when the “sacral sparing” definition of incomplete was introduced.
      American Spinal Injury Association
      International standards for neurological and functional classification of spinal cord injury, revised 1992.
      • Waters R.L.
      • Adkins R.H.
      • Yakura J.S.
      Definition of complete spinal cord injury.
      Before this time, an injury was considered incomplete if sensory or motor function was preserved more than 3 levels below the neurologic level of injury.
      American Spinal Injury Association
      Standards for neurological classification of spinal injury patients.
      The sacral sparing definition required preservation of sensory or motor function in the lowest sacral segments, S4-5. This was defined as either light touch (LT) or pinprick (PP) sensation in the S4-5 dermatome, any sensation of a digital rectal examination, or voluntary anal sphincter contraction (VAC).
      As a consequence of this change, a digital rectal examination is necessary to determine completeness of injury and American Spinal Injury Association Impairment Scale (AIS) grade. Performing a digital rectal examination can be difficult at times. In the acute postinjury setting it is not always possible to have visual access to the anorectal area. Furthermore, patients are not always willing to undergo a digital rectal examination, particularly if they are receiving multiple examinations from different providers. Finally, the reliability and validity of anal sensation have not been well studied. It is thought that a vigorous stimulation of the anorectal wall may result in a vague sensation relayed via the autonomic nervous system rather than the somatosensory system. This was the rationale for changing the term “any anal sensation” to “deep anal pressure” (DAP).
      • Kirshblum S.C.
      • Waring W.
      • Biering-Sorensen F.
      • et al.
      Reference for the 2011 revision of the International Standards for Neurological Classification of Spinal Cord Injury.
      The purpose of this study was to evaluate the suitability of pressure sensation in the S3 dermatome as a substitute for DAP in the neurologic classification of SCI. We selected the S3 pressure sensation for several reasons. First, there are clear anatomic landmarks for the S3 test points, namely the ischial tuberosities. Second, the S3 test point can be reached with the patient in the supine position by slightly flexing the hips. Third, the S3 test point is very close to the S4-5 dermatome and the anal sphincter, so that sensation in these 2 locations should be highly correlated. Fourth, the modality, pressure, is the same as that used for DAP.

      Methods

      This was a prospective, longitudinal study of neurologic recovery after acute traumatic SCI, conducted as a collaborative module among 5 Spinal Cord Injury Model Systems centers. A test-retest reliability study of S3 pressure sensation was incorporated into the 1-month time point. All participants gave informed consent, and the study was approved by the local institutional review board at each participating center.
      Patients with acute traumatic SCI, within 1 month of injury, and neurologic levels T12 and above were recruited. Patients were excluded if they had neurologic deficits unrelated to the SCI, such as traumatic brain injury or peripheral nerve injury, or if they were unable to complete the neurologic examination at 1 month postinjury. This was a consecutive series of consenting patients admitted to the participating centers who met the inclusion/exclusion criteria.
      The International Standards for Neurological Classification of Spinal Cord Injury
      • Kirshblum S.C.
      • Burns S.P.
      • Biering-Sorensen F.
      • et al.
      International Standards for Neurological Classification of Spinal Cord Injury (revised 2011).
      examination was conducted at 1, 3, 6, and 12 months postinjury. Pressure sensation over the ischial tuberosities (S3) was tested at each time point and was repeated within 3 days of the 1-month examination. Testing for S3 pressure sensation was performed as follows: firm pressure was applied over the ischial tuberosity using the index finger. The pressure was firm enough to compress the soft tissue over the ischial tuberosity, but not firm enough to move the patient. Pressure was applied for 1 second and then released. The patient was asked to report whether they felt something when pressure was applied over the ischial tuberosity and to identify the side of the pressure. S3 pressure sensation was recorded as “yes” (present) or “no” (absent) on each side. The complete International Standards for Neurological Classification of Spinal Cord Injury examination including S3 pressure sensation and DAP was performed in 1 session by 1 examiner. Completed neurologic examination forms were uploaded to a password-protected server at the lead center.
      Examiners were selected by the principal investigator at each participating center. All examiners completed standardized International Standards for Neurological Classification of Spinal Cord Injury examination training either through formal, in-person training as part of a multicenter clinical trial or SCI network (eg, North American Clinical Trial Network, NeuroRecovery Network) or through the International Standards Training e-Learning Program. In addition, a videoconference session was held where the protocol was reviewed, and the neurologic examination including S3 pressure sensation was demonstrated for consistency across sites. Bimonthly conference calls were held to review issues with data collection. A total of 46 examiners were involved in testing across the 5 participating sites. The number of examiners per site ranged from 3 to 15.

      Analysis

      For test-retest reliability, percent agreement and the kappa statistic were calculated for S3 pressure on either side, and for S3 pressure on the right and left sides separately. For agreement, percent agreement and kappa were calculated for S3 pressure and DAP, S3 pressure and any sensation at S4-5, and DAP and any sensation at S4-5. Percent agreement and kappa are both reported because the kappa statistic can be misleading with certain distributions of responses.
      • Feinstein A.R.
      • Cicchetti D.V.
      High agreement but low kappa: I. The problems of two paradoxes.
      The 95% confidence interval of each kappa value was determined. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value were determined for S3 pressure predicting DAP at all time points and at the baseline examination. The analysis at baseline was repeated after excluding AIS grade D cases in order to compare results with prior reports.
      • Zariffa J.
      • Kramer J.L.
      • Jones L.A.
      • Lammertse D.P.
      • Curt A.
      • Steeves J.D.
      Sacral sparing in SCI: beyond the S4-S5 and anorectal examination.
      In those cases where 1-month S3 pressure and DAP differed, an “S3P AIS grade” using the S3 pressure result instead of DAP was determined. Follow-up examinations at 6 or 12 months postinjury were examined to see whether there were any apparent differences in final AIS grade based on the use of DAP or S3 pressure.
      We used the criteria of Landis and Koch
      • Landis J.R.
      • Koch G.G.
      The measurement of observer agreement for categorical data.
      to interpret kappa values: 0, poor; .01 to .20, slight; .21 to .40, fair; .41 to .60, moderate; .61 to .80, substantial; and .81 to 1.00, almost perfect.

      Results

      There were 141 participants injured between October 2012 and April 2015 with 1-month neurologic examinations. Sixteen persons were excluded because S3 pressure sensation was missing. The missing assessments were primarily at the time of study initiation. Of the 125 participants with 1-month S3 pressure sensation, 94 had an S3 pressure retest. Repeat S3 pressure testing was missed in the other cases because of lack of participant or examiner availability during the test window. There were 80 participants with tetraplegia and 45 with paraplegia. Injuries were classified as AIS A in 47 individuals, B in 16, C in 22, and D in 40. Demographic information can be found in table 1. Most of the participants were men (82%–83%), and the most frequent etiology of injury was falls followed by vehicular collisions.
      Table 1Demographics and baseline (1mo) characteristics of participants
      CharacteristicsReliability Sample (n=94)Validity Sample (n=125)
      Age (y)41.4±16.941.9±17.8
      Sex
       Male81.983.2
       Female18.116.8
      Etiology
       Falls42.642.4
       Vehicular35.134.4
       Violence14.914.4
       Sports7.48.8
      Level of injury
       Tetraplegia60.664.0
       Paraplegia39.436.0
      AIS grade
       A36.137.6
       B12.812.8
       C21.317.6
       D29.832.0
      NOTE. Values are mean ± SD or percentages.
      Test-retest reliability of S3 pressure on either side at 1 month was almost perfect (κ=.98; 95% confidence interval, .98–1.0). Only 1 subject had a change at the second test. For each side individually, S3 pressure testing gave the same result in 92 of 94 examinations (κ=.96; 95% confidence interval, .90–1.0).
      Agreement for S3 pressure (either side) and DAP for all time points was substantial (agreement, 88.9%; κ=.77), with the disagreements evenly split (table 2). On 20 of 360 examinations, S3 pressure was present when DAP was absent, and on 20 examinations DAP was present when S3 pressure was not. For the 1-month time point (n=125), agreement for S3 pressure and DAP was also substantial (agreement, 88.0%; κ=.75). The 15 disagreements were almost evenly split between DAP present with S3 pressure absent (n=8) and the reverse (n=7).
      Table 2Level of agreement (κ) among S3 pressure, DAP, and S4-5 LT or PP
      ExamsS3 Pressure vs DAPS3 Pressure vs S4-5 LT/PPDAP vs S4-5 LT/PP
      All (n=360).77 (.70–.84).78 (.71–.84).85 (.80–.91)
      1-Month (n=125).75 (.63–.87).79 (.68–.90).84 (.74–.93)
      NOTE. Values are kappa (95% confidence interval).
      Agreement between S3 pressure and any S4-5 LT or PP sensation at 1 month was also substantial (agreement, 89.6%; κ=.79) (see table 2). In 11 of 13 cases of disagreement, S3 pressure was present with absent S4-5 LT and PP. For all time points combined, S3 pressure and DAP were present more often than S4-5 LT/PP sensation (table 3).
      Table 3Frequency of presence of S3 pressure, DAP, and S4-5 LT or PP
      Time PointDAPS3 PressureS4-5 LT or PP
      All time points
       Absent145145171
       Present215215189
      1 Month
       Absent495059
       Present767566
      NOTE. Values are n.
      Sensitivity, specificity, PPVs, and negative predictive values for S3 pressure predicting DAP can be found in table 4. For all time points combined, the PPV and negative predictive value of S3 pressure was .91 and .86, respectively. Similar values were found when limiting cases to baseline. The PPV dropped slightly when AIS D cases were excluded from the baseline, but was still good (.81).
      Table 4Sensitivity, specificity, PPV, and NPV of S3 pressure for DAP
      ExamsNSensSpecPPVNPV
      All cases all times360.907.862.907.862
      All cases baseline125.895.857.907.840
      AIS A–C baseline85.806.857.806.857
      Abbreviations: NPV, negative predictive value; Sens, sensitivity; Spec, specificity.
      There were 40 instances (out of 360 examinations) involving 29 participants where results of DAP and S3 pressure testing differed. Most of these participants had a change in either DAP or S3 pressure to result in disagreement or agreement. Only 3 participants had persistent disagreement between DAP and S3 pressure with no change in sensation in either location. Of the 15 cases with differing DAP and S3 pressure at baseline, 12 had follow-up examinations; 11 of these were at least 6 months postinjury, and the other was 3 months postinjury. No clear pattern emerged in changes in AIS grade or lower extremity motor scores (table 5). None of the participants gained useful lower extremity muscle function. One person gained 2 points and another 4 points. There was 1 case each where DAP or S3 pressure was absent and there was some lower extremity key muscle function.
      Table 5Cases where 1-month DAP and S3 pressure differed
      CaseNLIAny S4-5S3PDAPInitial AISFinal AISInitial S3P AISFinal S3P AISInitial LEMSFinal LEMSNotes
      1T9NYNAABA00S3P Y to N
      2C4NYNABBB00DAP N to Y
      3T9NYNAABC04No change
      4T3NYNAABB00No change
      5C4NYNAABA00S3P Y to N
      6T4NYNAABB00No change
      7C4NNYBBAA00No change
      8C1NNYCCAC68S3P N to Y
      9T9NNYCBCB00VAC Y to N
      10C4NNYBCAC00DAP Y to N, VAC N to Y
      11C4NNYBAAA00DAP Y to N
      12C1NNYCAAA00DAP Y to N
      Abbreviations: LEMS, lower extremity motor score; N, no; NLI, neurologic level of injury; S3P, S3 pressure; Y, yes.

      Discussion

      We have found that S3 pressure sensation has excellent test-retest reliability, as good as or better than other sacral sensory components in the International Standards for Neurological Classification of Spinal Cord Injury. The reliability of S3 pressure sensation (κ=.98) is better than the reliability of LT and PP in the sacral dermatomes reported by Jonsson et al,
      • Jonsson M.
      • Tollback A.
      • Gonzales H.
      • Borg J.
      Inter-rater reliability of the 1992 international standards for neurological and functional classification of incomplete spinal cord injury.
      where the best kappa value was .74 for LT in the S3 dermatome. It is also better than the reliability for anal sensation in adolescents and young adults, aged 16 to 21 years, which was found to be .92.
      • Vogel L.
      • Samdani A.
      • Chafetz R.
      • Gaughan J.
      • Betz R.
      • Mulcahey M.J.
      Intra-rater agreement of the anorectal exam and classification of injury severity in children with spinal cord injury.
      The associations among S4-5 LT or PP, DAP, and S3 pressure sensation were consistent with previous reports. The presence of LT or PP sensation in the S4-5 dermatome was not as sensitive as DAP or S3 pressure. In most cases when there was a discrepancy, S3 pressure or DAP was present when S4-5 sensation was absent. The sensitivity and specificity of S3 pressure in predicting DAP were comparable to those reported by Zariffa et al
      • Zariffa J.
      • Kramer J.L.
      • Jones L.A.
      • Lammertse D.P.
      • Curt A.
      • Steeves J.D.
      Sacral sparing in SCI: beyond the S4-S5 and anorectal examination.
      for DAP predicted by sensation in the S1 dermatome. This analysis of the European Multicenter Study about Spinal Cord Injury database examined persons with baseline AIS grades A through C and found that the PPV of the dermatomes S1 to S3 ranged from .762 to .813, which is similar to our value of .806. There is no similar study in the pediatric population, but Samdani et al
      • Samdani A.
      • Chafetz R.S.
      • Vogel L.C.
      • Betz R.R.
      • Gaughan J.P.
      • Mulcahey M.J.
      The International Standards for Neurological Classification of Spinal Cord Injury: relationship between S4-5 dermatome testing and anorectal testing.
      found poor sensitivity of S4-5 sensation (.60) in predicting DAP in participants with SCI, even for those in the older age group of 16 to 21 years (sensitivity, .63). It would be useful to evaluate the reliability and validity of S3 pressure sensation in the pediatric population to see whether it would be a better substitute than S4-5 LT or PP sensation.
      In 90% of cases in our study, there was agreement between DAP and S3 pressure. We did not find any pattern in the neurologic outcome of individuals with a difference between DAP and S3 pressure at baseline, although the number of cases was low. Both DAP and S3 pressure changed from “yes” to “no” on follow-up examinations, but in different cases. This suggests that the disagreements are random and do not favor one or the other for predicting future neurologic recovery. A larger sample of persons with an initial difference between S3 pressure and DAP is needed to determine whether this is indeed the case.
      Waters et al
      • Waters R.L.
      • Adkins R.H.
      • Yakura J.S.
      Definition of complete spinal cord injury.
      showed that there was less conversion from incomplete to complete SCI using the sacral sparing definition compared with the >3 levels below the injury level definition. In that study, sacral sensory sparing was determined “by the presence of sensation in the perineum at the anal mucocutaneous junction, glans penis or clitoris” and motor sparing “by presence of motor function in the external anal sphincter or the toe flexor muscles.”
      • Waters R.L.
      • Adkins R.H.
      • Yakura J.S.
      Definition of complete spinal cord injury.
      (p575) The International Standards Committee of the American Spinal Injury Association chose to limit sacral sparing criteria to the S4-5 segments but did not provide an explanation for the difference. The reliability and validity of this more restrictive definition of sacral sparing have not been well studied. Marino et al
      • Marino R.J.
      • Ditunno Jr., J.F.
      • Donovan W.H.
      • Maynard Jr., F.
      Neurologic recovery after traumatic spinal cord injury: data from the Model Spinal Cord Injury Systems.
      looked at rates of conversion from motor complete to incomplete based on the 2 criteria for completeness. They found that individuals who were motor complete with extended zones of sensory preservation (>3 levels below the neurologic level) but without sacral sparing were less likely to convert to motor incomplete than those with sacral sparing (13.3% vs 53.6%; P<.001). This supports the use of the sacral sparing of incomplete injury over the prior definition for predicting neurologic recovery.
      The validity of DAP and VAC as an indicator of preserved sensory or motor connectivity across the spinal cord lesion has not been rigorously tested. Few studies have tried to correlate DAP sensation with measures of spinal cord axonal continuity or sensorimotor cortical activity in SCI. It is recognized that being “clinically complete,” that is, without sacral sparing, does not necessarily mean that there are no functioning axons traversing the injury site. Dimitrijevic et al
      • Dimitrijevic M.R.
      • Danner S.M.
      • Mayr W.
      Neurocontrol of movement in humans with spinal cord injury.
      have described motor “discomplete” SCI, where there is supraspinal influence on spinal reflex activity below the level of injury. Recently, magnetic resonance imaging has been used to assess connectivity across the spinal cord lesion. Functional magnetic resonance imaging has shown that somatosensory cortex activity can be seen after below-level sensory stimulation in clinically complete SCI, a “sensory discomplete” injury.
      • Awad A.
      • Levi R.
      • Lindgren L.
      • et al.
      Preserved somatosensory conduction in a patient with complete cervical spinal cord injury.
      Fractional anisotropy values from diffusion tensor imaging have a sensitivity of .70 and a specificity of .53 in predicting DAP.
      • Mulcahey M.J.
      • Samdani A.F.
      • Gaughan J.P.
      • et al.
      Diagnostic accuracy of diffusion tensor imaging for pediatric cervical spinal cord injury.
      Krisa et al
      • Krisa L.
      • Middleton D.
      • Faro S.
      • Calhoun C.L.
      • Mohamed F.B.
      • Mulcahey M.J.
      Cerebral activation during the test of spinal cord injury severity in children: an FMRI methodological study.
      found different patterns of cerebral activation in children with SCI classified as AIS A, B, and C using functional magnetic resonance imaging during rectal wall stimulation.
      Besides the potential false-negative results of DAP testing for complete SCI suggested by “discomplete” injuries, there may also be false-positive results. Sun et al
      • Sun W.M.
      • MacDonagh R.
      • Forster D.
      • Thomas D.G.
      • Smallwood R.
      • Read N.W.
      Anorectal function in patients with complete spinal transection before and after sacral posterior rhizotomy.
      found that 8 of 14 patients with complete supraconal SCI being evaluated for rhizotomy felt a dull pelvic ache during rectal distention, while Wyndaele
      • Wyndaele J.J.
      Investigation of the afferent nerves of the lower urinary tract in patients with 'complete' and 'incomplete' spinal cord injury.
      reported that 15 of 42 patients diagnosed as clinically complete could perceive bladder filling, electrical stimulation, or both. Wietek et al
      • Wietek B.M.
      • Baron C.H.
      • Erb M.
      • et al.
      Cortical processing of residual ano-rectal sensation in patients with spinal cord injury: an fMRI study.
      demonstrated that some patients with complete traumatic SCI display cortical activation in response to nonpainful distention of the distal rectum or anal canal in areas similar to those seen in neurologically intact volunteers. Visceral sensation elicited by DAP testing could be mistaken for tactile sensation by the patient or the examiner. S3 pressure testing would not have this same issue and potentially could be a more valid test of sensory sacral sparing. Studies of functional magnetic resonance imaging cortical activation patterns in response to S3 pressure, DAP, and nonpainful rectal distention may be able to sort out the contribution of somatic versus visceral sensation during DAP testing.
      Even if DAP was not required to determine completeness of injury, the digital rectal examination would still have a role in the evaluation of persons with SCI. The presence or absence of rectal tone and anal reflexes is important acutely to determine whether a person is in spinal shock and later to determine whether there is upper motor neuron or lower motor neuron involvement of the S4-5 segments, which would affect bowel and bladder management.
      • Ko H.Y.
      • Ditunno Jr., J.F.
      • Graziani V.
      • Little J.W.
      The pattern of reflex recovery during spinal shock.
      • Schurch B.
      • Schmid D.M.
      • Kaegi K.
      Value of sensory examination in predicting bladder function in patients with T12-L1 fractures and spinal cord injury.
      At this time, there is no substitute for VAC, which is used in the classification of AIS grades A through C. Zariffa
      • Zariffa J.
      • Kramer J.L.
      • Jones L.A.
      • Lammertse D.P.
      • Curt A.
      • Steeves J.D.
      Sacral sparing in SCI: beyond the S4-S5 and anorectal examination.
      suggested that the ankle plantarflexors are an acceptable substitute, although the PPV of these for VAC was modest at 59.8%. Experienced clinicians have suggested that some of the earliest muscles in the lower extremities to recover are the hip adductors and the toe flexors. We are in the process of evaluating these muscles as an alternative to VAC.
      The high reliability of S3 pressure sensation combined with its high level of agreement with DAP makes it a potential substitute for determining sensory sacral sparing. The ischial tuberosity is easier to access than the anus when a patient is supine and cannot be rolled. In our experience, S3 pressure testing is well tolerated and accepted by patients, unlike the digital rectal examination. In cases where a patient is refusing the rectal examination and completeness of injury cannot be determined, S3 pressure testing would be a reasonable alternative to DAP.

      Study limitations

      We used a baseline examination of 1 month postinjury, although if participating centers admitted patients earlier they were encouraged to perform an examination. The reliability of S3 pressure sensation may not be as good when tested within the first month postinjury. We did not include patients with neurologic injuries below T12, and cannot recommend S3 pressure instead of DAP for conus or cauda equina injuries. Importantly, examinations for this study were conducted by experienced, trained examiners. Results cannot be generalized to untrained clinicians. Although this was a multicenter study, it involved a limited number of centers. Replication of these findings by other investigators would support the generalizability of our results.

      Conclusions

      This new test of S3 pressure sensation shows excellent test-retest reliability and substantial agreement with DAP. Disagreements with DAP appear to be random, with no clear advantage of one over the other in predicting neurologic outcome. Based on these findings, we believe that S3 pressure sensation can be used as a substitute for DAP in classifying persons with traumatic supraconus SCI at least 1 month postinjury, when testing is conducted by trained examiners. Additional studies in the acute postinjury period (<1mo) are needed. We recommend that the International Standards Committee of the American Spinal Injury Association consider this as an alternative test of sensory sacral sparing for supraconus SCI, at least in cases where DAP cannot be tested.

      References

        • American Spinal Injury Association
        Standards for neurological classification of spinal injury patients.
        American Spinal Injury Association, Chicago1982
        • Frankel H.L.
        • Hancock D.O.
        • Hyslop G.
        • et al.
        The value of postural reduction in the initial management of closed injuries of the spine with paraplegia and tetraplegia, part I.
        Paraplegia. 1969; 7: 179-192
        • American Spinal Injury Association
        International standards for neurological and functional classification of spinal cord injury, revised 1992.
        American Spinal Injury Association, Chicago1992
        • Waters R.L.
        • Adkins R.H.
        • Yakura J.S.
        Definition of complete spinal cord injury.
        Paraplegia. 1991; 29: 573-581
        • American Spinal Injury Association
        Standards for neurological classification of spinal injury patients.
        American Spinal Injury Association, Chicago1990
        • Kirshblum S.C.
        • Waring W.
        • Biering-Sorensen F.
        • et al.
        Reference for the 2011 revision of the International Standards for Neurological Classification of Spinal Cord Injury.
        J Spinal Cord Med. 2011; 34: 547-554
        • Kirshblum S.C.
        • Burns S.P.
        • Biering-Sorensen F.
        • et al.
        International Standards for Neurological Classification of Spinal Cord Injury (revised 2011).
        J Spinal Cord Med. 2011; 34: 535-546
        • Feinstein A.R.
        • Cicchetti D.V.
        High agreement but low kappa: I. The problems of two paradoxes.
        J Clin Epidemiol. 1990; 43: 543-549
        • Zariffa J.
        • Kramer J.L.
        • Jones L.A.
        • Lammertse D.P.
        • Curt A.
        • Steeves J.D.
        Sacral sparing in SCI: beyond the S4-S5 and anorectal examination.
        Spine J. 2012; 12: 389-400
        • Landis J.R.
        • Koch G.G.
        The measurement of observer agreement for categorical data.
        Biometrics. 1977; 33: 159-174
        • Jonsson M.
        • Tollback A.
        • Gonzales H.
        • Borg J.
        Inter-rater reliability of the 1992 international standards for neurological and functional classification of incomplete spinal cord injury.
        Spinal Cord. 2000; 38: 675-679
        • Vogel L.
        • Samdani A.
        • Chafetz R.
        • Gaughan J.
        • Betz R.
        • Mulcahey M.J.
        Intra-rater agreement of the anorectal exam and classification of injury severity in children with spinal cord injury.
        Spinal Cord. 2009; 47: 687-691
        • Samdani A.
        • Chafetz R.S.
        • Vogel L.C.
        • Betz R.R.
        • Gaughan J.P.
        • Mulcahey M.J.
        The International Standards for Neurological Classification of Spinal Cord Injury: relationship between S4-5 dermatome testing and anorectal testing.
        Spinal Cord. 2011; 49: 352-356
        • Marino R.J.
        • Ditunno Jr., J.F.
        • Donovan W.H.
        • Maynard Jr., F.
        Neurologic recovery after traumatic spinal cord injury: data from the Model Spinal Cord Injury Systems.
        Arch Phys Med Rehabil. 1999; 80: 1391-1396
        • Dimitrijevic M.R.
        • Danner S.M.
        • Mayr W.
        Neurocontrol of movement in humans with spinal cord injury.
        Artif Organs. 2015; 39: 823-833
        • Awad A.
        • Levi R.
        • Lindgren L.
        • et al.
        Preserved somatosensory conduction in a patient with complete cervical spinal cord injury.
        J Rehabil Med. 2015; 47: 426-431
        • Mulcahey M.J.
        • Samdani A.F.
        • Gaughan J.P.
        • et al.
        Diagnostic accuracy of diffusion tensor imaging for pediatric cervical spinal cord injury.
        Spinal Cord. 2013; 51: 532-537
        • Krisa L.
        • Middleton D.
        • Faro S.
        • Calhoun C.L.
        • Mohamed F.B.
        • Mulcahey M.J.
        Cerebral activation during the test of spinal cord injury severity in children: an FMRI methodological study.
        Top Spinal Cord Inj Rehabil. 2013; 19: 121-128
        • Sun W.M.
        • MacDonagh R.
        • Forster D.
        • Thomas D.G.
        • Smallwood R.
        • Read N.W.
        Anorectal function in patients with complete spinal transection before and after sacral posterior rhizotomy.
        Gastroenterology. 1995; 108: 990-998
        • Wyndaele J.J.
        Investigation of the afferent nerves of the lower urinary tract in patients with 'complete' and 'incomplete' spinal cord injury.
        Paraplegia. 1991; 29: 490-494
        • Wietek B.M.
        • Baron C.H.
        • Erb M.
        • et al.
        Cortical processing of residual ano-rectal sensation in patients with spinal cord injury: an fMRI study.
        Neurogastroenterol Motil. 2008; 20: 488-497
        • Ko H.Y.
        • Ditunno Jr., J.F.
        • Graziani V.
        • Little J.W.
        The pattern of reflex recovery during spinal shock.
        Spinal Cord. 1999; 37: 402-409
        • Schurch B.
        • Schmid D.M.
        • Kaegi K.
        Value of sensory examination in predicting bladder function in patients with T12-L1 fractures and spinal cord injury.
        Arch Phys Med Rehabil. 2003; 84: 83-89