Archives of Physical Medicine and Rehabilitation
Volume 90, Issue 9 , Pages 1469-1477, September 2009

Validation of the Revised Quick Cognitive Screening Test

Presented to the Canadian Colloquium on Dementia, Vancouver, BC, Canada, October 18–20, 2007.

  • C. Charles Mate-Kole, PhD

      Affiliations

    • Department of Psychology, Central Connecticut State University, New Britain, CT
    • Corresponding Author InformationReprint requests to C. Charles Mate-Kole, PhD, Dept of Psychology, Central Connecticut State University, 1615 Stanley St, New Britain, CT 06050
    • ,
  • James Conway, PhD

      Affiliations

    • Department of Psychology, Central Connecticut State University, New Britain, CT
    • ,
  • Katherine Catayong, MA

      Affiliations

    • Department of Psychology, Central Connecticut State University, New Britain, CT
    • Olin Neuropsychiatry Research Center, The Institute of Living, Hartford, CT
    • ,
  • Rachel Bieu, MA

      Affiliations

    • Department of Psychology, Central Connecticut State University, New Britain, CT
    • Olin Neuropsychiatry Research Center, The Institute of Living, Hartford, CT
    • ,
  • Naa Amerley Sackey, BA

      Affiliations

    • Department of Public Health, Drexel University, PA
    • ,
  • Rebecca Wood, PhD

      Affiliations

    • Department of Psychology, Central Connecticut State University, New Britain, CT
    • ,
  • Robert Fellows, MA

      Affiliations

    • Department of Psychology, Central Connecticut State University, New Britain, CT

Article Outline

Abstract 

Mate-Kole CC, Conway J, Catayong K, Bieu R, Sackey NA, Wood R, Fellows R. Validation of the revised Quick Cognitive Screening Test.

Objective

To validate the revised version of the Quick Cognitive Screening Test (QCST).

Design

Cross-sectional.

Setting

Senior homes; hospital; college campus.

Participants

Participants (N=377; 114 men, 263 women) were recruited comprising healthy controls (n=201; 40 men, 161 women), subjects with dementia (n=93; 34 men, 59 women) including Alzheimer disease (n=73) and vascular dementia (n=20); subjects with psychiatric illness (n=35, 15 men, 20 women), specifically schizophrenia or bipolar disorder; and subjects with other neurologic conditions (n=48, 25 men, 23 women) such as traumatic brain injury (n=12) and cerebrovascular disease (n=31). Diagnoses were confirmed by physicians using appropriate criteria. Recruitment was done in the northeastern region.

Interventions

Not applicable.

Main Outcome Measures

In an effort to examine the reliability and validity of the revised QCST, participants were administered the revised QCST with a number of standardized measures (ie, Alzheimer's Disease Assessment Scale-Cognitive, Mini-Mental State Examination, Tests of Oral Fluency, Trail-Making Test, and Functional Activities Questionnaire).

Results

The results revealed that the revised QCST discriminated between healthy controls and the neuropsychiatric participants. Additionally, the revised QCST significantly correlated with other standardized measures, confirming the revised QCST's reliability and validity as a screening instrument for subjects with cognitive deficits.

Conclusions

The revised QCST provides the clinician with a short yet reliable screening instrument in detecting cognitive deficits in subjects with dementia and other neurologic conditions.

Key Words: Psychometrics, Rehabilitation

List of Abbreviations: AD, Alzheimer disease, ADAS, Alzheimer's Disease Assessment Scale, ADAS-cog, Alzheimer's Disease Assessment Scale cognitive subscale, ANOVA, analysis of variance, CCSE, Cognitive Capacity Screening Examination, CNS, central nervous system, COWAT, Controlled Oral Word Association Test, CVD, cerebrovascular disease, FAQ, Functional Activities Questionnaire, MCI, mild cognitive impairment, MDRS, Mattis Dementia Rating Scale, MMSE, Mini-Mental State Examination, QCST, Quick Cognitive Screening Test, ROC, receiver operating characteristic, TMT, Trail-Making Test, WAIS-R, Wechsler Adult Intelligence Scale-Revised

 

AS THE ELDERLY POPULATION steadily increases, complaints of memory loss and decline in cognitive functions are becoming more common. In 1991, 12.5% of the U.S. population was 65 years old and over. By 2030, the elderly population is estimated to number 65 million, 20% of the population.1

The early detection of cognitive impairments among the elderly is important in recognizing the cognitive and behavioral changes associated with normal aging, dementias, or other neurologic disorders. Early detection of impairments can aid clinicians in identifying the most appropriate type of care to slow the progression of a neurologic disease.

Cognitive impairments may not be evident during routine examination. Boustani et al2 noted that more than 50% of subjects with dementia, including many with mild and some with moderate dementia, have never received a diagnosis of dementia from a physician.2, 3

Screening for cognitive changes will help primary care physicians be more aware of the possibility of declining cognition in patients and develop appropriate dementia care and a proactive approach for the care of patients and families.4 Different screening instruments have been developed to address this issue and are categorized into 3 groups: (1) brief instruments, (2) midrange tests, and (3) computerized tests.

Back to Article Outline

Brief Cognitive Screening Instruments 

Brief screening instruments have been developed to ease the overwhelming and rigorous task of a neuropsychologic battery. These include the MMSE, the Mini-Cog, the Clock Drawing Test, and the CCSE.5, 6, 7, 8, 9 They serve as initial assessment tools in detecting cognitive changes.

The most widely used cognitive screening test is the MMSE.10 Kalbe et al9 identified patients with MCI and early-stage dementia. Participants with MCI and AD were tested with the DemTect and the MMSE. Results showed the MMSE to be inferior in detecting mild AD and MCI (80% sensitivity) compared with the DemTect (100% sensitivity). Another study used the MMSE, the Abbreviated Mental Test, and the Informant Questionnaire on Cognitive Decline in the Elderly. Results revealed that the MMSE was not supported for routine population screening.11

The Mini-Cog was developed as a screening for dementia.12 It has shown high sensitivity and specificity in detecting dementia in the elderly population and can be used by relatively untrained raters as a dementia screening tool.6, 12, 13

One of the shorter screening tests is the Clock Drawing Test.7 A review of studies reported a mean sensitivity of 85% and a mean specificity of 85%.7 It is regarded as a poor screening tool for very mild dementia.14, 15

Lorentz et al4 compared brief dementia screening tests for routine use in primary care using 13 instruments grouped according to administration time, ranging from very short (2min) to short (2–5min) to long (5–10min).4 They reported that the shortest tests were inferior to the longer ones. However, the Mini-Cog appeared to be superior to the shorter and longer tests.

Cognitive screening instruments can reduce a physician's burden and provide dementia prevention and treatment.8 However, using 1 specific cognitive measure alone does not effectively detect cognitive changes, especially in the early stages. The MMSE is reported to be insensitive in detecting MCI because it depends on a lot of verbal items. On the other hand, the CCSE uses more nonverbal items but is heavily influenced by education.8

Xu et al8 evaluated the combination of the CCSE and the MMSE in detecting MCI as an early sign of dementia in a retrospective study of 351 elderly volunteers with memory complaints. The MMSE, the CCSE, and a test that combined the MMSE and CCSE, the Combined Mini-Mental-Cognitive Capacity Screening Examinations, were administered to participants. Results showed that the Combined Mini-Mental-Cognitive Capacity Screening Examinations has higher sensitivity (83%) than the MMSE (61%) or the CCSE (74.3%) alone in detecting MCI. In performing a factor analysis, the CCSE has factors that are more sensitive to impairment and can be used as a mental status examination for patients.16

Brief screening tests are more efficient than full neuropsychologic batteries.4 However, shorter administration time and portability can affect specificity and sensitivity in detecting cognitive impairment and discriminating between different diagnoses of mild cognitive impairment, head injury, or dementia.17 Brief screening tests may also be educationally and socioculturally biased. If there is evidence of impairment, a more comprehensive battery can be used to differentiate various deficits to determine the severity of the impairment.18

Back to Article Outline

Midrange Tests 

Midrange tests are slightly more time consuming and often assess multiple cognitive skills. The MDRS and the ADAS are examples of frequently used midrange instruments.19, 20

Administration time varies from 15 to 45 minutes depending on degree of cognitive impairment.19 Van Gorp et al21 examined the sensitivity and specificity of the MMSE, the MDRS, and the Neurobehavioral Cognitive Status Examination in patients with AD and vascular dementia. Overall, the results showed that when the standard cutpoint was used, the tests were less specific and sensitive in identifying patients with dementia. However, with a modified cutpoint, substantial sensitivity and specificity can be achieved in discriminating between mild and moderate dementia.

The ADAS was designed to evaluate the severity of behavior and cognitive characteristics of persons with AD.20 Significant differences in MMSE and ADAS-cog scores were found for patients and controls, showing that the ADAS-cog addressed AD symptoms in more detail and is valuable in the early detection of AD.22 The ADAS-cog has shown adequate sensitivity and specificity in discriminating between patients with AD and healthy controls.23, 24 However, education and age may have an effect on the ADAS-cog outcome, reflecting a need to be sensitive to these variables.25

Back to Article Outline

Midrange Computerized Cognitive Screening Tests 

Computer technology has increased the development of cognitive tests. Advantages in using computerized tests include increased efficiency, reduced scoring errors, short administration time, and reduced differences in administration and responding.26 Computerized neurocognitive tests include the MicroCog and the CNS-vital signs in evaluating cognitive function.27, 28

The MicroCog is designed to detect early signs of cognitive impairment. Its subtests yield 5 domain scores. Reports revealed that the MicroCog is positively correlated with the Wechsler Adult Intelligence Scale–3rd edition.29 However, it is limited because of the lack of motor and divided attention tasks.30, 31

The CNS-vital signs addresses several cognitive domains including verbal and visual memory, finger tapping, digit symbol coding, the Stroop test, shifting attention test, and the continuous performance test.32 In a group of healthy controls and patients diagnosed with MCI and mild dementia, the CNS-vital signs was found to differentiate adequately among age-associated memory impairment, MCI, and mild dementia groups.33

Computerized neurocognitive tests pose a number of potential advantages to paper-and-pencil tests. However, there are some disadvantages. These include a passive stance in interpretation, an altered response to tests when the client is unfamiliar with computers, mixed client acceptance, and inadequate evaluation of the tests' psychometric properties.26, 31, 34 Most computerized tests use considerable visuoperceptual tasks that put patients with decreased visual acuity at a disadvantage.

An ideal cognitive assessment test would include the following characteristics: short administration time; acceptance by patients; ease of scoring; independence from culture, language, and education; good reliability and validity; and high sensitivity and specificity.7 However, certain limitations still need to be addressed for screening instruments. These include false-negative and false-positive results, and test sensitivity and specificity.19 It is also important that the instruments have high sensitivity in correctly identifying patients with cognitive impairment as well as high specificity in correctly identifying patients without cognitive impairment.18

Galvin et al35 took a different approach by testing the ability of patients to rate their own cognitive ability using the Alzheimer disease 8 contrasted with informant and clinician ratings of cognitive status. Galvin35 found that when self-completed, an Alzheimer disease 8 differentiates between nondemented and demented subjects.

Boustani et al36 screened 3340 subjects, subjecting 227 to a formal assessment. They concluded that primary care physicians are ill prepared to screen and diagnose subjects with cognitive deficits.36

Detecting cognitive impairment in the elderly is challenging, especially when the elderly patient is medically ill. Even hospital staff, including physicians, report lower rates of detection of cognitive impairment.37 In a survey of elderly persons in adult day health care programs, the prevalence of cognitive impairment was 60% for persons over age 65 years, with approximately 30% of the sample suggestive of dementia.38

Back to Article Outline

Quick Cognitive Screening Test 

In an attempt to offset problems concerning extensive neuropsychologic testing, Mate-Kole et al39 published the QCST, a brief cognitive screening instruments based on an original work by McFie.40 Participants were classified into 3 groups: neurologic, psychiatric, and control.

Standardized tests like the WAIS-R were administered to examine the QCST's reliability and validity. Results showed that the QCST was able to detect impairment in patients with neurologic disorders, traumatic brain injury, and psychiatric diagnoses. The QCST also significantly correlated with the subtests of the WAIS-R.

In a review of driver assessments measuring cognitive skill and performance, Unsworth et al41 compared the QCST and 19 other assessment instruments, which were divided into those that were developed to assess functionally impaired drivers and those that were not. The QCST was included in the group not developed to screen drivers. Assessment instruments were then ranked, with higher scores indicating more valuable assessments for driving. The QCST was ranked tenth of 20 tests assessing driving capabilities.41

Costs of health care are steadily increasing, and inpatient treatment is continually shortening. There is a need for brevity and economy in clinical examinations. Therefore, a good screening test will be able to examine a variety of abilities, have good normative data, and be reliable and valid.42 However, most of these measures have shown poor diagnostic accuracy and inadequate specificity and sensitivity in differentiating between groups.42

The present study examined the reliability and validity of the revised QCST. The revised QCST is a shorter version of the QCST, taking approximately 10 to 15 minutes, compared with the QCST's administration time of 20 to 30 minutes.39

Back to Article Outline

Methods 

Participants 

The study was approved by the Human Studies Council of Central Connecticut State University. Participants were recruited from a local hospital, senior homes, independent living dwellings, and the community from the northeastern region. Most of the testing was conducted in the psychology laboratory at Central Connecticut State University.

Three hundred seventy-seven subjects participated in the study (114 men, 263 women). They were divided into 4 groups: healthy controls, participants with dementia, subjects with psychiatric illnesses, and those with other neurologic impairments aside from dementia (table 1).

Table 1. Demographics
ChacacteristicsNormal (n=201)Dementia (n=93)Psychiatric (n=35)Neurologic (n=48)
Sex
Male (n)40341525
Female (n)161592023
Age (y)67.20±19.2876.47±9.1353.63±23.755.44±18.71
Education (y)14.58±2.7013.9±2.7711.82±2.8311±4.77
Ethnicity
White European (n)182841522
Black (n)1371925
Latino (n)121
Native American (n)3 1
Other (n)2 1
Handedness (n)
Right291
Left27
Both3
MMSE27.37±5.8317.83±7.21NA26.07±3.83
FAQ3.13±5.8315.46±7.61NA8.67±8.26

NOTE. Values are mean ± SD or as otherwise indicated.

Abbreviation: NA, not applicable.

Control group 

Two hundred one healthy participants comprising 40 men and 161 women were recruited. The age, mean ± SD, of the group was 67.20±19.28 years, and the number of years of education, mean ± SD, was 14.58±2.70. Exclusion criteria included history of neurologic illness and neuropsychiatric disorders. Detailed history was taken to rule out any evidence of cognitive impairment in addition to using the revised QCST and MMSE.

Dementia group 

This group comprised 93 participants. There were 34 men and 59 women. The age, mean ± SD, was 77.47±9.13 years, and the number of years of education, mean ± SD, was 13.9±2.77. The dementia group was composed of participants with AD (n=73) or vascular dementia (n=20) diagnosed by physicians. Diagnosis of dementia was made by attending physicians using the traditional National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association43 criteria and the National Institute of Neurological Disorders and Stroke Association Internationale pour la Recherche et l'Enseignement en Neurosciences.44

Psychiatric group 

The group comprised 35 participants (15 men, 20 women). The age, mean ± SD, was 53.63±23.7 years, and the number of years of education, mean ± SD, was 11.82±2.83. Most of the participants in the group had a diagnosis of schizophrenia or bipolar disorder as diagnosed by psychiatrists.

Nondemented neurologic group 

The group comprised 48 participants. There were 25 men and 23 women. The age, mean ± SD, was 55.44±18.71 years, and the number of years of education, mean ± SD, was 11.8±4.77. Fifteen participants were diagnosed with traumatic brain injury and 33 participants were diagnosed with CVD confirmed by neurologists and computer tomography results. Physicians ruled out evidence of vascular dementia in the CVD group.

Back to Article Outline

Materials 

Revised Quick Cognitive Screening Test 

The revised QCST consists of 48 items sorted into 17 subtests (table 2).

Table 2. Summary of the Revised QCSTs
Verbal TestsNonverbal TestsGlobal Measures
OrientationAttention/concentrationAttention/concentrationTotal verbal
Memory immediate recallSpatial neglectTotal nonverbal
ArithmeticConstructional praxisGlobal score
VocabularyMemory immediate recall
NamingUnusual views
Abstract reasoningSpatial orientation
SimilaritiesMemory delayed recall
Analogies
Memory delayed recall
Memory new learning

Orientation consists of 12 items assessing time, place, person, and age.

Attention/concentration consists of 3 items with 2 parts: verbal and visual. This subtest assesses the individual's ability to maintain attention, concentration, and tracking of a specific problem.

Spatial neglect measures the individual's visual spatial skills and neglect.

Arithmetic consists of 4 items of mathematical calculations such as addition, subtraction, multiplication, and division.

Constructional praxis requires the participant to copy a drawing of 3 interconnected geometric figures to test for the individual's planning, spatial organization, and visual constructional skills.

Memory has 2 parts: nonverbal and verbal. The nonverbal portion requires the participant to draw the 3 geometric figures again from memory immediately after presentation and after a delay period. The verbal portion consists of 5 items verbally presented and requires the participant to repeat the 5 items immediately after presentation and after a 5-minute delay.

New learning asks the participant to repeat the Babcock sentence.39, 40 Ten trials are given for the participant to repeat the sentence successfully without error.

Vocabulary consists of 5 items in which the participant is asked to identify words with the same meanings or definitions.

Naming consists of 5 items requiring the participant to identify pictures of objects such as an umbrella, a butterfly, a teapot, a knife, and a book.

Abstract reasoning comprises 2 parts; similarities and analogies. Similarities require the participant to identify a word or phrase that best describes a pair of words given. Analogies require the participant to identify words that best complete given sentences.

Unusual views requires the participant to identify objects from an unconventional angle. It assesses individual's perception and object recognition.45

Spatial orientation requires the participant to identify matching designs. This subtest measures the individual's visual spatial orientation and relations.

Each of the 17 subtests has its own score. There is a summary score for orientation, for the verbal tests, and for the nonverbal tests. The 3 scores are summed to make up the global score.

Alzheimer's Disease Assessment Scale-cognitive 

The ADAS-cog is one of the most widely used assessment instruments in medical clinical trials for AD and dementia.20, 46

Mini-Mental State Examination 

The MMSE is a widely used brief screening instrument.5

Trail-Making Test parts A and B 

The Trail-Making Test measures scanning, visuomotor tracking, divided attention, and cognitive flexibility.46, 47

Controlled Oral Word Association Test 

COWAT (letters FAS) assesses the person's ability to make verbal associations to given letters.48

Category Naming 

Category naming requires the participant to generate as many words as possible that belongs in a given category such as animals or household items in 60 seconds.46

Functional Activities Questionnaire 

The FAQ is an informant-based questionnaire that measures functional abilities.49

Procedure 

Participants were assessed with the revised QCST, ADAS-cog, MMSE, TMT, COWAT/FAS, and Category Naming tests. The revised QCST was administered first, acting as a screening test, and took approximately 10 to 15 minutes to complete.

Back to Article Outline

Results 

Internal Consistency 

Coefficient alpha values were computed to assess internal consistency reliability of the revised QCST subtests; we supplemented the alphas with item-total correlations. Internal consistency analyses were only done for multi-item scales; a number of the subtests were single items, so reliability could not be computed (table 3).

Table 3. Internal Consistency of Revised QCST Subtests With Multiple Items
Revised QCST SubtestNumber of ItemsCoefficient α
Orientation12.78
Vocabulary5.91
Naming5.80
Similarities4.81
Analogies4.82
Unusual views5.60
Spatial orientation5.87
Total nonverbal23.84
Total verbal15.84
Global score50.92

NOTE. The following revised QCST subtests were single items, so internal consistency analyses could not be conducted: verbal attention/concentration, verbal memory immediate recall, visual attention/concentration, spatial neglect, arithmetic, verbal memory delayed recall, constructional praxis, visual memory immediate recall, visual memory delayed recall, and new learning.

Table 3 shows that most of the subtests had fairly high internal consistency. An exception was the Unusual views, which had a coefficient alpha of .60, and all 5 items had low item-total correlations (<.30). Otherwise, all subtests had coefficient alphas of .78 or higher, which we considered to be adequate. The Orientation subtest, the one with alpha equal to .78, had 2 items with item-total correlations of less than .30; if these items were deleted, the internal consistency would increase.

Effects of Age and Education 

Pearson product moment correlations were used to determine whether age and education were correlated with the revised QCST test variables. All the revised QCST subtests were significantly correlated with age and/or education (P<.05). Before investigating group differences on revised QCST variables, we assessed whether controlling for age and education would impact these differences. An analysis of covariance was performed for each revised QCST subtest to investigate differences between the 4 groups using age and education as covariates. Diagnosis type (ie, normal, dementia, psychiatric, neurologic) was used as the independent variable; the revised QCST subtest was used as the dependent variable. Adding age and/or education as covariates did not change the results, so later we report results of ANOVAs without the covariates.

Percentage of Healthy Control Participants Scoring in Abnormal Range 

To determine the number of healthy participants scoring in the neuropsychiatric range, the data file was split into 2 groups (healthy controls, neuropsychiatric). The group with neuropsychiatric disorders consisted of the dementia, psychiatric, and neurologic groups. The mean revised QCST global score for the healthy control group was 72.84±8.70, and the mean revised QCST global score for the neuropsychiatric group was 44.96±20.18. The range of scores for the healthy controls was 64.14 to 81.54, and the range for the neuropsychiatric group was 24.78 to 65.14. Using the frequency scores of the healthy controls and neuropsychiatric groups, 12% of the healthy controls scored in the neuropsychiatric range, and 20% of neuropsychiatric subjects scored in the normal range.

Group Differences on Revised Quick Cognitive Screening Test Summary Scores 

A 1-way ANOVA was computed to examine any significant differences among the groups. When significant differences were detected, a post hoc test (Tukey Honestly Significant Difference test, P<.05) was computed to establish where the differences existed. The results showed significant differences for all the revised QCST summary scores between the healthy control, dementia, psychiatric, and neurologic groups: total verbal score (F3,366=96.94; P<.001), total nonverbal score (F3,366=88.19; P<.001), and global score (F3,366=119.82; P<.001). Post hoc tests showed that the performance of the healthy control was significantly higher than the dementia, psychiatric, and neurologic groups on all the revised QCST summary scores. The dementia group also obtained significantly lower scores than the neurologic group on the total nonverbal score and the global score (P<.05) (see table 4).

Table 4. Post Hoc Comparisons of Mean and SD of RQCST
Revised QCST SubtestsNormal (n=201)Dementia (n=93)Psychiatric (n=35)Neurologic (n=48)Different From Normal
Orientation11.56±0.926.37†±3.827.65‡±3.409.58†‡±3.12Dementia/psychiatric/neurologic
Verbal attention/concentration1.27±0.900.75±0.930.71±0.901.20±0.92Dementia/psychiatric
Verbal memory Immediate recall4.59±1.033.03±.893.46±1.843.36±1.82Dementia/psychiatric/neurologic
Arithmetic3.75±0.602.40±1.612.58±1.252.71±1.45Dementia/psychiatric/neurologic
Vocabulary4.60±0.782.99±1.882.33±2.012.71±2.03Dementia/psychiatric/neurologic
Naming4.91±0.424.08±1.484.48±0.774.64±0.71Dementia
Similarities3.48±0.682.22±1.602.12±1.522.22±1.52Dementia/psychiatric/neurologic
Analogies3.52±0.762.01±1.462.25±1.622.38±1.45Dementia/psychiatric/neurologic
New learning4.17±3.460.26±1.291.75±2.741.42±2.80Dementia/psychiatric/neurologic
Visual attention/concentration1.65±0.561.12±0.791.67±0.571.42±0.62Dementia
Spatial neglect3.83±3.162.33±1.763.20±1.322.75±1.46Dementia
Visual memory Immediate recall5.03±1.321.77⁎†±1.823.48⁎±2.203.41†±2.31Dementia/psychiatric/neurologic
Unusual views3.09±0.972.13±1.191.58±0.882.30±1.53Dementia/psychiatric/neurologic
Spatial orientation4.85±0.493.45±1.823.46±1.963.87±1.54Dementia/psychiatric/neurologic
Visual memory4.86±1.671.13⁎†±1.993.33⁎±2.412.82†±2.49Dementia/psychiatric/neurologic
Delayed recall Total verbal32.51±5.7217.89±9.5219.94±8.2921.85±9.97Dementia/psychiatric/neurologic
Total nonverbal28.88±4.0315.83†±8.4718.85±9.5020.85†±9.35Dementia/psychiatric/neurologic
Global72.78±8.6340.11†±20.0146.12±18.5352.19†±9.10Dementia/psychiatric/neurologic

⁎Demented significantly different from psychiatric.

†Demented significantly different from neurologic.

‡Neurologic significantly different from psychiatric.

Group Differences on Revised Quick Cognitive Screening Test Summary Subtest Scores 

We used 2 complementary approaches to determine the extent to which revised QCST variables differentiated the healthy control and neuropsychiatric groups. First, we conducted 1-way ANOVAs. Second, we conducted ROC curve analyses.

One-way ANOVAs showed significant differences in mean scores for all the revised QCST subtest scores among the healthy control, dementia, psychiatric, and neurologic groups. Post hoc multiple comparisons revealed group differences among the healthy control group and the groups with neurologic and psychiatric disorders (see table 4). The healthy controls differed significantly from the neurologic group (P<.05) on all the subtests except verbal attention/concentration, visual attention/concentration, and spatial neglect. The healthy controls differed significantly from the dementia group (P<.05) on all the subtests. Finally, the healthy controls differed significantly from the psychiatric group (P<.05) on all the subtests except visual attention/concentration, naming, and spatial neglect.

Within the neuropsychiatric group, the dementia group obtained significantly lower scores than the neurologic group (P<.05) on the orientation and the visual memory immediate and delayed recall. Additionally, the dementia group scored significantly lower than the psychiatric group (P<.05) on the visual memory immediate and delayed recall. The psychiatric group scored significantly higher than the neurologic group on the orientation test (P<.05) (see table 4). Overall, the dementia group performed more poorly than both the neurologic and the psychiatric groups.

Receiver Operating Characteristic Curve Analysis 

To examine how well the revised QCST discriminates normal from impaired individuals, we conducted ROC curve analyses. The ROC analysis compares 2 groups, and we focused on discriminating healthy controls from those with neuropsychiatric conditions. Table 5 shows the area under the curve (with confidence interval), sensitivity and specificity for each subtest, and global measures. Calculating sensitivity and specificity requires a cutoff score. We chose an optimal cutoff based on the ROC curve as described by Akobeng50 as the point that is closest to both perfect sensitivity (true-positive rates for detecting neurologic disorders) and perfect specificity (true-negative rates).

Table 5. ROC Analyses for Revised QCST Subtests
Revised QCST SubtestArea Under Curve95% CI for Area Under CurveSensitivitySpecificityOptimal Cutoff Score
Orientation.71.61–.81.47.9311
Verbal attention/concentration.53.43–.62.47.582
Verbal memory immediate recall.72.62–.81.60.795
Arithmetic.73.64–.83.60.814
Vocabulary.80.71–.88.77.705
Naming.60.50–.70.26.945
Similarities.74.64–.84.58.943
Analogies.74.65–.83.72.654
Verbal memory delayed recall.69.60–.78.67.582
Verbal memory new learning.72.64–.81.74.691
Visual attention/concentration.61.52–.71.53.702
Spatial neglect.69.60–.79.56.814
Constructional praxis.67.56–.78.44.965
Visual memory immediate recall.70.61–.80.58.765
Unusual views.66.55–.77.60.733
Spatial orientation.69.59–.79.47.885
Visual memory delayed recall.73.64–.83.60.745
Total nonverbal.77.68–.85.65.7228
Total verbal.83.76–.90.67.8328
Global score.83.76–.91.77.8168

Abbreviation: CI, confidence interval.

The area under the curve serves as an overall measure of discrimination; perfect discrimination would yield an area of 1, whereas a test that failed to discriminate would have an area of .50. According to Hosmer and Lemeshow,51 an area of at least .70 indicates acceptable discrimination, and at least .80 indicates excellent discrimination. All revised QCST subtests had areas greater than .50, although for verbal attention/concentration and naming, the confidence intervals included .50. Nine subtests had areas between .70 and .80 and 3 had areas of at least .80. The global scores had the largest areas, with .77 for the total nonverbal score, .83 for the total verbal score, and .83 for the global score; vocabulary also had a high area with .80. Sensitivities and specificities are also best for the global scores. The 3 global scores had sensitivities ranging from .65 to .77, while the other subtests were typically lower (vocabulary had the highest sensitivity with .77). Specificities were also high for the global tests with values of .72 to .81. Some subtests did have higher specificities (eg, orientation with .93), but those subtests tended to have low sensitivities. These analyses complement the ANOVAs reported earlier; the ANOVAs are more flexible for comparing multiple groups within a single analysis, and ROC analyses provide more information for a particular comparison, such as an optimal cutoff score. Results were mainly consistent for the 2 sets of analyses—both analyses indicated that most of revised QCST variables distinguished the controls from the neurologic patients, and both showed that the verbal attention/concentration and naming did not distinguish. For visual attention/concentration and spatial neglect, the ANOVAs did not detect a difference, whereas the ROC analyses did.

Correlation of Revised Quick Cognitive Screening Test Subtest With Revised Quick Cognitive Screening Test Summary Scores 

Within-group correlations were performed separately between the healthy control group and the group with a variety of neurologic disorders (psychiatric, dementia, neurologic) to determine whether there were differences between the groups. Results showed that overall, there were no differences between the groups, and therefore the 2 groups were collapsed into 1.

Using the Pearson product moment correlations, significant correlations (P<.01) were found between the revised QCST subtest scores and the revised QCST summary scores (see table 5). The revised QCST total verbal score had significantly higher correlations with the revised QCST verbal subtest scores than the revised QCST nonverbal subtest scores. In addition, the revised QCST total nonverbal score had significantly higher correlations than the revised QCST nonverbal subtest scores. Results also showed significant and high correlations between the global score and the total verbal and nonverbal scores.

Correlation of Revised Quick Cognitive Screening Test Summary Scores With Other Standardized Measures 

Significant correlations were found between the revised QCST summary scores and the other standardized measures, speaking to the revised QCST's validity (table 6). The revised QCST global score correlated significantly with the ADAS-cog global score (r=–.89; P<.01) and the MMSE (r=.88; P<.01). In addition, the revised QCST global score was negatively correlated with TMT part A (r=–.78; P<.01), TMT part B (r=–.79; P<.01), and FAQ (r=–.76; P<.01). The revised QCST global score significantly correlated with MMSE (r=.88; P<.0), category naming (r=.69; P<.01), and oral fluency (r=.36; P<.01) (table 7).

Table 6. Pearson r Correlation Coefficients for Revised QCST Subtest Scores
Revised QCST SubtestNonverbal ScoreVerbal ScoreGlobal Score
Orientation0.75⁎0.73⁎0.83⁎
Verbal attention/concentration0.32⁎0.41⁎0.39⁎
Verbal memory immediate recall0.58⁎0.70⁎0.68⁎
Arithmetic0.77⁎0.80⁎0.82⁎
Vocabulary0.69⁎0.78⁎0.78⁎
Naming0.58⁎0.58⁎0.62⁎
Similarities0.67⁎0.75⁎0.74⁎
Analogies0.70⁎0.78⁎0.79⁎
Verbal memory delayed recall0.50⁎0.61⁎0.58⁎
New learning0.47⁎0.71⁎0.61⁎
Visual attention/concentration0.46⁎0.33⁎0.42⁎
Spatial neglect0.41⁎0.34⁎0.38⁎
Constructional praxis0.82⁎0.62⁎0.74⁎
Visual memory immediate recall0.90⁎0.70⁎0.84⁎
Unusual views0.68⁎0.61⁎0.67⁎
Spatial orientation0.77⁎0.70⁎0.78⁎
Visual memory delayed recall0.86⁎0.67⁎0.80⁎
Total nonverbal1.000.81⁎0.94⁎
Total verbal0.82⁎1.000.95⁎
Global score0.94⁎0.94⁎1.00

⁎P<.01.

Table 7. Pearson r Correlation Coefficients for Revised QCST Summary Scores With Other Measures
MeasureVerbal ScoreNonverbal ScoreGlobal Score
TMT part A−.72⁎−.73⁎−.78⁎
TMT part B−.71⁎−.77−.79⁎
Oral fluency.40⁎.27⁎.362⁎
Category naming.67⁎.64⁎.69⁎
ADAS-cog global−.81⁎−.81⁎−.89⁎
MMSE.83⁎.80⁎.88⁎
FAQ−.71⁎−.70⁎−.76⁎

⁎P<.01.

Back to Article Outline

Discussion 

The results of the present study support the hypothesis that the revised QCST is able to detect cognitive dysfunction in individuals with dementia, psychiatric disorders, and other neurologic disorders. Differences in the revised QCST summary scores and subtest scores were found between the healthy control and the neuropsychiatric groups. Results showed that the healthy control group performed better on the revised QCST than participants with dementia, psychiatric disorders, and neurologic disorders, suggesting that the revised QCST was able to differentiate between healthy individuals and individuals with cognitive impairments, consistent with previous studies.39 However, ROC results suggest that excellent discrimination was achieved for only 3 subtests: the total verbal, global, and vocabulary scores (the total nonverbal score came very close to excellent discrimination).

Results from the post hoc analysis showed differences among groups of individuals with cognitive impairments. Individuals with dementia, psychiatric disorders, and neurologic disorders were different from each other, specifically on their scores on the revised QCST orientation, visual memory immediate and delayed recall, total nonverbal, and global score. Group differences found on the revised QCST summary scores and subtest scores show that the revised QCST is able to discriminate between healthy individuals and individuals with cognitive deficits. Healthy individuals scored higher on all the revised QCST's 17 subtests compared with the groups with dementia, neurologic disorders, and psychiatric disorders. The ROC analyses provide optimal cutoff scores for distinguishing the normal from neurologic groups.

Scores of the dementia, psychiatric, and neurologic groups were reflective of cognitive impairment, suggesting the revised QCST's sensitivity to impairment no matter the type or origin of dysfunction. A number of measures were able to distinguish among the nonhealthy groups. Results showed that the dementia group scored significantly lower than the psychiatric and neurological groups on a number of revised QCST tests: orientation, visual memory immediate and delayed recall, total nonverbal, and the global score. Poor performance by the dementia group compared with the psychiatric and neurologic groups suggests greater cognitive impairment in this population, especially in the areas of orientation, nonverbal skills, and memory. These results confirm the revised QCST's specificity and sensitivity in distinguishing among healthy individuals and individuals with neurologic and psychiatric disorders. However, a number of revised QCST tests were not able to differentiate between the healthy controls and neuropsychiatric groups. Evidence was mixed for the verbal and visual attention/concentration and spatial neglect tests; ANOVAs did not show evidence that they differentiate between healthy individuals and the neurologic group, although ROC analyses did. Additionally, the visual attention/concentration, naming, and spatial neglect tests did not differentiate between the healthy controls and the psychiatric group. However, all of the revised QCST subtests did differentiate between the healthy control group and the dementia group. This supports the findings of previous studies with other standardized tests such as the Clock Drawing Test and the ADAS-cog differentiating between healthy individuals and individuals with cognitive impairment.14, 22, 23, 24

Results of the correlations show the revised QCST's reliability and validity. The revised QCST subtest scores highly correlate with the revised QCST summary scores, indicating good internal consistency reliability (although the results also suggest that dropping certain items can improve internal consistency). Other standardized measures were used to validate the revised QCST. In relation to other standardized measures, the revised QCST global score was highly correlated with the ADAS-cog global score, the MMSE, and the TMTs. The ADAS-cog global and TMTs were all negatively associated with the revised QCST global score. This indicates greater cognitive impairment with higher scores.

The revised QCST is a shorter test than the ADAS-cog.22, 42 The revised QCST is also able to detect cognitive impairment and differentiate among neuropsychiatric groups, in contrast with the MMSE, which does not provide a breakdown of cognitive areas.10

Study Limitations 

The scores of the minority groups would require further investigation in a future study. Cultural factors are crucial in assessing minority patients. Parker et al52 reported that cultural bias is common in traditional brief screening tests, but could be minimized by sensitive administration and interpretation of results.52

Other problems encountered in the present study include the sample sizes of the neurologic and psychiatric groups. The small sample sizes for the dementia, psychiatric, and neurologic groups may have accounted for the few statistically significant differences between the groups. A larger sample size for each of the 3 groups (dementia, psychiatric, neurologic) could differentiate more between the 3, as was shown in the comparison between the normal group and the groups with neurologic and psychiatric disorders. However, the large significant levels still exist, even allowing for a Bonferroni correction (at P<.001).

Another limitation concerns ecological validity. Chaytor and Schmitter-Edgecombe's53 review found moderate ecologic validity for neuropsychologic tests in general, and Higginson et al54 concluded that cognitive measures analogous to everyday tasks should be included in cognitive assessment. We do have evidence that the revised QCST correlated negatively with the FAQ, an informant-based questionnaire measuring functional abilities (see table 7). We suggest additional research on the ecologic validity of the revised QCST.

The present results shed light on the future use of the revised QCST. For example, internal consistency might be improved by dropping some items from the orientation test. We also provide optimal cutoff scores from the ROC analysis for distinguishing those from normal and neurologic populations. Future research should continue to explore the psychometric properties of the revised QCST.

Back to Article Outline

Conclusions 

The revised QCST is useful as a cognitive screening tool because of its quick ease of administration and sensitivity. The breakdown of the cognitive domains may provide the clinician with additional information that may be valuable in a rehabilitation setting.

The use of standardized measures compared with the revised QCST confirmed the revised QCST's reliability and validity in detecting cognitive impairment among different neurologic and psychiatric groups. The revised QCST shows great promise as a cognitive screening tool because of its quick and easy administration, its sensitivity and specificity, and its comprehensive breakdown of cognitive domains.

Back to Article Outline

Acknowledgments 

The revised QCST was based on an original work by the late John McFie, consultant psychologist (formerly of Charing Cross & Westminster Medical School, London, United Kingdom), who introduced C. C. Mate-Kole, University of Ghana, to clinical neuropsychology. We dedicate this article in his memory. We are grateful to colleagues who assisted us in recruiting participants or made comments on the article. They include Samuel Danquah, University of Ghana; John Connolly, McMaster University, Canada; and Jake McDougal, Maria Aresco, Pamela Donis, and Van Dan of Central Connecticut State University.

Back to Article Outline

References 

  1. Elon R, Phillips C, Loome JF, Denman S, Woods A. General issues and comprehensive approach to assessment of elders. In:  Osterweil D,  Brummel-Smith K,  Beck JC editor. Comprehensive geriatric assessment. New York: McGraw-Hill; 2000;p. 1–39
  2. Boustani M, Peterson B, Hanson L, Harris R, Lohr K. Screening for dementia in primary care: a summary of the evidence for the U.S. preventive services task force. Ann Int Med. 2003;138:927–937
  3. Manning L. Neuropsychological assessment of selected clinical disorders. Eur J Psychol Assess. 2001;17:79–86
  4. Lorentz WJ, Scanlan JM, Borson S. Brief screening tests for dementia. Can J Psychiatry. 2002;47:723–733
  5. Folstein MF, Folstein SE, McHugh PR. Mini-Mental State: a practical method for grading the state of patients for the clinician. J Psychiatr Res. 1975;2:189–198
  6. Scanlan J, Borson AS. The Mini-Cog: receiver operating characteristics with expert and naïve raters. Int J Geriatr Psychiatry. 2001;16:216–222
  7. Shulman KI. Clock drawing: is it the ideal cognitive screening test?. Int J Geriatr Psychiatry. 2000;15:548–561
  8. Xu G, Meyer JS, Thornby J, Chowdhury M, Quach M. Screening for mild cognitive impairment (MCI) utilizing combined mini-mental cognitive capacity examinations for identifying dementia prodromes. Int J Geriatr Psychiatry. 2002;17:1027–1033
  9. Kalbe E, Kessler J, Calabrese P, et al. DemTect: a new, sensitive cognitive screening test to support the diagnosis of mild cognitive impairment and early dementia. Int J Geriatr Psychiatry. 2004;19:136–143
  10. Ashla MM. Cognitive function, mood, and behavior assessments. In:  Osterweil D,  Brummel-Smith K,  Beck JC editor. Comprehensive geriatric assessment. New York: McGraw-Hill; 2000;p. 67–119
  11. Flicker L, Logiudice D, Carlin J, Ames D. The predictive value of dementia screening instruments in clinical populations. Int J Geriatr Psychiatr. 1997;12:203–209
  12. Borson S, Scanlan J, Chen P, Ganguli M. The Mini-Cog as a screen for dementia: validation in a population-based sample. J Am Geriatr Soc. 2003;51:1451–1454
  13. Borson S, Scanlan J, Brush M, Vitaliano P, Dokmak A. The Mini-Cog: a cognitive “vital signs” measure for dementia screening in multi-lingual elderly. Int J Geriatr Psychiatry. 2000;15:1021–1027
  14. Powlishta KK, Von Dras DD, Stanford A, et al. The clock drawing test is a poor screening test for very mild dementia. Neurology. 2002;59:898–903
  15. Butler M, Retzlaff P, Vanderploeg R. Brief report: neuropsychological test usage. Prof Psychol Res Pr. 1991;22:510–512
  16. Anderson D, Burton B, Parker J, Godding P. A confirmatory factor analysis of the cognitive capacity screening examination in a clinical sample. Int J Neurosci. 2001;111:221–233
  17. Katzman R, Brown T, Fuld P, Peck A, Schecter R, Schimmel H. Validation of a short Orientation-Memory-Concentration Test of cognitive impairment. J Neurol Neurosurg Psychiatry. 1983;50:734–739
  18. Morgan CD, Baade LE. Neuropsychological testing and assessment scales for dementia of the Alzheimer's type. Geriatr Psychiatry. 1997;20:25–43
  19. Macneill SE, Lichtenberg PA. Screening instruments and brief batteries for assessment of dementia. In:  Lichtenberg PA editors. Handbook of assessment in clinical gerontology. New York: John Wiley and Sons; 1999;p. 417–441
  20. Rosen WG, Mohs RC, Davis KL. A new rating scale for Alzheimer's disease. Am J Psychiatry. 1984;141:1356–1364
  21. Van Gorp WG, Marcotte TD, Sultzer D, Hinkin C, Mahler M, Cummings J. Screening for dementia: comparison of three commonly used instruments. J Clin Exp Neuropsychol. 1999;21:29–38
  22. Hannesdottir K, Snaedal J. A study of the Alzheimer's Disease Assessment Scale-Cognitive (ADAS-Cog) in an Icelandic elderly population. Nord J Psychiatry. 2002;56:201–206
  23. Ihl R, Brinkmeyer J, Janner M, Kerdar MS. A comparison of ADAS and EEG in the discrimination of patients with dementia of the Alzheimer type from healthy controls. Neuropsychobiology. 2000;41:102–107
  24. Schwarb S, Koberle S, Spiegel R. The Alzheimer's Disease Assessment Scale (ADAS): an instrument for early diagnosis of dementia. Int J Geriatr Psychiatry. 1988;3:45–53
  25. Fischer CE. A clinician's guide to interpreting cognitive measures in clinical trials Alzheimer's disease. Can Psychiatr Assoc Bull. 2002;34:16–18
  26. Olson K. Computerized psychological test usage in APA-accredited training programs. Int J Clin Psychology. 2001;57:727–736
  27. Lopez S, Sumerall S, Ryan J. Factor structure of MicroCog in a clinical sample. Appl Neuropsychol. 2002;9:183–186
  28. Gualtieri T. CNS vital signs clinical use: reliability, validity, and standardization. 2006. https://www.cnsvs.com/images/stories/Abstracts/CNS_Vital_Signs_Reliability_and_Validity_v1.0.pdfAccessed August 19, 2006
  29. Johnson JA, Rust JO. Correlational analysis of MicroCog: assessment of cognitive functioning with the Wechsler Adult Intelligence Scale-III for a clinical sample of veterans. Psychol Rep. 2003;93:1261–1266
  30. Elwood R. MicroCog: assessment of cognitive functioning. Neuropsychol Rev. 2001;11:89–100
  31. Larrabee GJ, Crook TH. Computers and memory. In:  Grant I,  Adams KM editor. Neuropsychological assessment of neuropsychiatric disorders. 2nd ed.. New York: Oxford Univ Pr; 1996;p. 102–117
  32. Gualtieri T, Johnson L, Benedict K. Reliability and validity of a new computerized cognitive assessment battery. Baltimore, MD: International Neuropsychological Society Annual Meeting; February 4–7, 2004;
  33. Gualtieri T, Johnson L. Differentiating between mild cognitive impairment and early dementia with a new, computerized neurocognitive assessment battery (Symposium on Dementia Screening, Alzheimer's Association International Conference on Prevention of Dementia, for Washington, DC). 2005;
  34. Skinner H, Pakula A. Challenge of computers in psychological assessment. Prof Psychol Res Pr. 1986;17:44–50
  35. Galvin J, Roe C, Coats M, Morris J. Patient's rating of cognitive ability: using the AD8, a brief informant interview, as self-rating tool to detect dementia. Arch Neurol. 2007;64:725–730
  36. Boustani M, Callahan C, Unverzagt F. Implementing a screening and diagnosis program for dementia in primary care. J Gen Int Med. 2005;20:572–577
  37. Ardern M, Mayou R, Feldman E, Hawton K. Cognitive impairment in the elderly medically ill: how often is it missed. Int J Geriatr Psychiatry. 1993;8:929–937
  38. Teresi JA, Holmes D, Koren MJ, Dichter E, Ramirez M, Fairchild S. Prevalence estimates of cognitive impairment in medical model adult day health care programs. Soc Psychiatry Psychiatr Epidemiol. 1998;33:283–290
  39. Mate-Kole C, Major A, Lenzer I, Connoly J. Validation of the Quick Cognitive Screening Test. Arch Phys Med Rehabil. 1994;75:867–875
  40. McFie J. Assessment of organic intellectual impairment. Oxford: Academic Pr; 1975;
  41. Unsworth CA, Lovell R, Terrington N, Thomas S. Review of tests contributing to the occupational therapy off-road driver assessment. Aust Occup Ther J. 2005;52:57–74
  42. Keys R. Treating traumatic brain injury. http://www.mgh.harvard.edu/forum/TraumaticBrainInjuryF/1998;Accessed November 25, 2003
  43. McKhann G, Drachman D, Folstein M, Katzman R, Price D, Stadlan EM. Clinical diagnosis of Alzheimer's disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer's Disease. Neurology. 1984;34:939–944
  44. Roman GC, Tatemichi TK, Erkinjuntti T, et al. Vascular dementia: diagnostic criteria for research studies (Report of the NINDS-AIREN International Workshop). Neurology. 1993;43:250–260
  45. Warrington E, Taylor A. The contribution of right parietal lobe to object recognition. Cortex. 1973;9:159–164
  46. Lezak MD, Howieson DB, Loring DW. The practice of neuropsychological assessment. In:  Lezak MD,  Howieson DB,  Loring DW editor. Neuropsychological assessment. 4th ed.. New York: Oxford Univ Pr; 2004;p. 3–14
  47. Reitan R, Davison L. Clinical neuropsychology: current status and applications. New York: Wiley & Sons; 1974;
  48. Borkowski J, Benton A, Spreen O. Word fluency and brain damage. Neuropsychologia. 1967;5:135–140
  49. Pfeiffer RI, Kurosaki TT, Harrah CH, Chance JM, Filos S. Measurement of functional activities in older adults in the community. J Gerontol. 1982;37:323–329
  50. Akobeng AK. Understanding diagnostic tests 3: receiver operating characteristic curves. Acta Paediatr. 2007;96:644–647
  51. Hosmer C, Lemeshow S. Applied logistic regression. 2nd ed.. New York: Wiley; 2000;
  52. Parker C, Philip I, Sarai M, Rauf A. Cognitive screening for people from minority ethnic backgrounds. Nurs Older People. 2007;18:31–36
  53. Chaytor N, Schmitter-Edgecombe M. The ecological validity of neuropsychological tests: a review of the literature on everyday cognitive skills. Neuropsychol Rev. 2003;13:181–197
  54. Higginson C, Arnett P, Voss W. The ecological validity of clinical tests of memory and attention in multiple sclerosis. Arch Clin Neuropsychol. 2003;11:185–204

 Supported by the Central Connecticut State University American Association of University Professors.

 No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the authors are associated.

PII: S0003-9993(09)00285-8

doi:10.1016/j.apmr.2009.02.007

Archives of Physical Medicine and Rehabilitation
Volume 90, Issue 9 , Pages 1469-1477, September 2009

Article Tools