Abstract
Objective
To systematically review, summarize, and evaluate published evidence on measurement
properties of real-world versions of the Multiple Errands Test (MET) following Consensus-based
Standards for the Selection of Health Measurement Instruments guidelines.
Data Sources
Four databases were searched in May 2019 using multiple variants of the name of the
MET from 1991 onward following the publication of the original MET.
Study Selection
We included peer-reviewed original research articles in English that provided data
on measurement properties (reliability, validity, and responsiveness to change) on
real-world versions of the MET in any clinical population.
Data Extraction
Data on the MET characteristics, study population, and evidence for each measurement
property were extracted using predefined criteria. The review team critically appraised
the methodological quality and rated the results from each study as sufficient (+),
insufficient (−), or indeterminate (?).
Data Summary
Data on each measurement property were pooled. Pooled results were rated as sufficient
(+), insufficient (−), mixed (±), or indeterminate (?). The overall quality of evidence
per measurement property was graded based on risk of bias, sample size, and consistency
of results. The overall evidence for each measurement property was determined as high,
moderate, low, or very low.
Results
We found 33 studies that provided data on measurement properties of real-world versions
of the MET. Pooled results revealed high-quality evidence for interrater reliability
and moderate-quality evidence for known-group validity. Limited support for other
kinds of reliability and validity was found.
Conclusions
This review suggests the MET should be used cautiously. Reasons for the limited psychometric
support are discussed, the value of generic forms of the MET that do not require site
specific adaptations is noted, and areas for further psychometric work are highlighted.
Keywords
List of abbreviations:
COSMIN (Consensus-based Standards for the Selection of Health Measurement Instruments), GRADE (Grading of Recommendations Assessment Development and Evaluation), ICC (intraclass correlation coefficient), MET (Multiple Errands Test), PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Archives of Physical Medicine and RehabilitationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Neuropsychological asessment. 5th ed.Oxford University Press, New York2012
- Functions of the frontal lobes: relation to executive functions.J Int Neuropsychol Soc. 2011; 17: 759-765
- Executive function, more than global cognition, predicts functional decline.J Gerontol A Biol Sci Med Sci. 2007; 62: 1134-1141
- The cognitive correlates of functional status: a review from the committee on research of the American Neuropsychiatric Association.J Neuropsychiatry Clin Neurosci. 2007; 19: 249-265
- Improving the ecological validity of executive functioning assessment.Arch Clin Neuropsychol. 2006; 21: 217-227
- How do assessments of activities of daily living address executive functions: a scoping review.Neuropsychol Rehabil. 2017; 27: 618-666
- Neuropsychology and the prediction of everyday functioning.in: Marcotte T.D. Grant I. Neuropsychology of everyday functioning. Guilford Press, New York2010: 5-38
- Assessment of executive functions: review of instruments and identification of critical issues.Arch Clin Neuropsychol. 2008; 23: 201-216
- Naturalistic tasks performed in realistic environments: a review with implications for neuropsychological assessment.Clin Neuropsychol. 2017; 31: 16-42
- Ecological validity of a simplified version of the multiple errands shopping test.J Int Neuropsychol Soc. 2003; 9: 31-44
- The case for the development and use of “ecologically valid” measures of executive function in experimental and clinical neuropsychology.J Int Neuropsychol Soc. 2006; 12: 194-209
- Deficits in strategy application following frontal lobe damage in man.Brain. 1991; 114: 727-741
- Ecological validity of the Multiple Errands Test (MET) on discharge from neurorehabilitation hospital.OTJR (Thorofare N J). 2011; 31: S38-S46
- Further development of the Multiple Errands Test: standardized scoring, reliability, and ecological validity for the Baycrest version.Arch Phys Med Rehabil. 2009; 90: S41-S51
- Multiple errands test-revised (MET-R): a performance-based measure of executive function in people with mild cerebrovascular accident.Am J Occup Ther. 2013; 67: 460-468
- Goal management training and mindfulness meditation improve executive functions and transfer to ecological tasks of daily life in polysubstance users enrolled in therapeutic community treatment.Drug Alcohol Depend. 2016; 165: 9-14
- Planning deficits in polysubstance dependent users: differential associations with severity of drug use and intelligence.Drug Alcohol Depend. 2016; 162: 72-78
- Development of a contextualized version of the Multiple Errands Test for people with substance dependence.J Int Neuropsychol Soc. 2018; 24: 347-359
- Predictors of performance in real and virtual scenarios across age.Exp Aging Res. 2019; 45: 180-198
- Using virtual environments in the assessment of executive dysfunction.Presence. 2001; 10: 375-383
- Development, reliability, and validity of the Multiple Errands Test home version (MET-Home) in adults with stroke.Am J Occup Ther. 2019; 73: 1-10
- Ecological validity of the Multiple Errands Test using predictive models of dysexecutive problems in everyday life.J Clin Exp Neuropsychol. 2013; 35: 37-41
- Midlife aging, open-ended planning, and laboratory measures of executive function.Neuropsychology. 2001; 15: 472-482
- Development and validation of a modified Multiple Errands Test for adults with intellectual disabilities.J Appl Res Intellect Disabil. 2017; 30: 255-268
- Development of a simplified version of the multiple errands test for use in hospital settings.Neuropsychol Rehabil. 2002; 12: 231-255
- Validation of the virtual MET as an assessment tool for executive functions.Neuropsychol Rehabil. 2009; 19: 583-602
- Virtual multiple errands test: reliability, usability and possible applications.Annu Rev CyberTherapy Telemed. 2013; 11: 38-42
- Measurement properties of performance-based instruments to assess mental function during activity and participation in traumatic brain injury: a systematic review.Scand J Occup Ther. 2019 Nov 14; ([Epub ahead of print])
- How to select outcome measurement instruments for outcomes included in a “core outcome set”–a practical guideline.Trials. 2016; 17: 449-458
- COSMIN methodology for systematic reviews of patient-reported outcome measures (PROMs): user manual.VU University Medical Center, Amsterdam2018
- COSMIN guideline for systematic reviews of patient-reported outcome measures.Qual Life Res. 2018; 27: 1147-1157
- Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement.Syst Rev. 2015; 4: 1-9
- Understanding measurement properties.in: Law M. Baum C. Dunn W. Measuring occupational performance: supporting best practice in occupational therapy. 3rd ed. SLACK Incorporated, Thorofare2017: 29-41
- Foundations of clinical research: applications to practice. 3rd ed.F. A. Davis Company, Philadelphia2015
- Validity and reliability in quantitative studies.Evid Based Nurs. 2015; 18: 66-67
- Health measurement scales: a practical guide to their development and use. 5th ed.Oxford University Press, Oxford2015
- Executive function measures for children: a scoping review of ecological validity.OTJR (Thorofare N J). 2018; 38: 6-14
- COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures.Qual Life Res. 2018; 27: 1171-1179
- COSMIN methodology for assessing the content validity of PROMs: user manual.VU University Medical Center, Amsterdam2018
- Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach.(Available at:)https://gdt.gradepro.org/app/handbook/handbook.html#ftnt_ref1Date accessed: March 5, 2019
- A neuropsychological battery to detect specific executive and social cognitive impairments in early frontotemporal dementia.Brain. 2009; 132: 1299-1309
- Neuropsychology, social cognition and global functioning among bipolar, schizophrenic patients and healthy controls: preliminary data.Front Hum Neurosci. 2013; 7: 661
- A pilot investigation of the potential for incorporating lifelog technology into executive function rehabilitation for enhanced transfer of self-regulation skills to everyday life.Neuropsychol Rehabil. 2018; 28: 589-601
- Reliability and construct validity of a revised Baycrest Multiple Errands Test.Neuropsychol Rehabil. 2017; 27: 667-684
- Cognitive contributors to Multiple Errands Test (MET) performance.Am J Occup Ther. 2018; 72 (7206205050p1-7)
- The validity, reliability and clinical utility of a performance-based executive function assessment in people with mild to moderate dementia.Aging Ment Health. 2019; : 1-9
- “Real life” executive deficits in patients with focal vascular lesions affecting the cerebellum.J Neurol Sci. 2009; 283: 95-98
- Cognitive deficits in multiple sclerosis correlate with changes in fronto-subcortical tracts.Mult Scler. 2008; 14: 364-369
- Deficits in tasks of executive functioning that mimic real-life scenarios in bipolar disorder.Bipolar Disord. 2012; 14: 118-125
- Rehabilitation of executive functioning with training in attention regulation applied to individually defined goals: a pilot study bridging theory, assessment, and treatment.J Head Trauma Rehabil. 2011; 26: 325-338
- Impaired behavior on real-world tasks following damage to the ventromedial prefrontal cortex.J Clin Exp Neuropsychol. 2007; 29: 319-332
- Training multitasking in a virtual supermarket: a novel intervention after stroke.Am J Occup Ther. 2009; 63: 535-542
- The application of goal management training to aspects of financial management in individuals with traumatic brain injury.Neuropsychol Rehabil. 2012; 22: 852-873
- Use of the Multiple Errands Test-Simplified version in the assessment of suboptimal effort.Neuropsychol Rehabil. 2012; 22: 734-751
- Using video ratings to assess multitasking performance in a naturalistic paradigm.NeuroRehabilitation. 2014; 35: 553-562
- Clinical utility of the Multiple Errands Test in schizophrenia: a preliminary assessment.Psychiatry Res. 2016; 240: 390-397
- Face validity of the youth Multiple Errands Test (yMET) in the community: a focus group and pilot study.Br J Occup Ther. 2019; 82: 248-258
- Effectiveness of executive functions training within a virtual supermarket for adults with traumatic brain injury: a pilot study.IEEE Trans Neural Syst Rehabil Eng. 2013; 21: 182-190
- In-home contextual reality: a qualitative analysis using the Multiple Errands Test Home Version (MET-Home).Neuropsychol Rehabil. 2018; : 1-15
- Developing and validating a big-store Multiple Errands Test.Front Psychol. 2019; 10: 2575
- Measurement in occupational therapy: scientific criteria for evaluation.Can J Occup Ther. 1987; 54: 133-138
Article info
Publication history
Published online: February 27, 2020
Footnotes
Current affiliation for Ruthralingam, Student Wellness Centre, Ontario College of Art and Design University, Toronto, Ontario, Canada; Hnatiw, FunctionAbility Rehabilitation Services, Toronto, Ontario, Canada; Yuzwa, Department of Occupational Science & Occupational Therapy, University of Toronto, Toronto, Ontario, Canada
S.R. undertook this work while a postdoctoral fellow funded by the Canadian Institutes of Health Research.
Disclosures: none.
Identification
Copyright
© 2020 by the American Congress of Rehabilitation Medicine
