Abstract
Objective
To identify, synthesize, and categorize the methodological issues faced by the rehabilitation
field.
Data Sources
A scoping review was conducted using studies identified in MEDLINE, the Cochrane Library,
EMBASE, Web of Science, Scopus, Physiotherapy Evidence Database, and Google Scholar
up to August 2018.
Study Selection
We included all type of publications describing methodological issues in rehabilitation
research where rehabilitation is described as a multimodal process. The methodological
issues have been categorized and classified.
Data Extraction
The synthesis included qualitative and quantitative analysis. To focus the attention
on rehabilitation, we post hoc divided in “specific issues” (highly related to, even
if not exclusive of, rehabilitation research) and “generic issues” (common in biomedical
research).
Data Synthesis
Seventy-one publications were included: 68% were narrative reviews, 15% systematic
reviews, 7% editorials, 4% meta-epidemiologic studies, and 5% others. Specific methodological
issues include the following: problematic application of randomized controlled trials
(32%), absent definition of core outcome sets (28%), poor interventions description
(22%), weak methodological (conducting) and reporting quality (21%), scarce clinical
practice applicability (14%), lack of blinding assessor (10%), inadequate randomization
methods or inadequate allocation concealment (8%), and inadequate participants description
and recruitment (8%). “Generic” issues included the following: data and statistical
description (31%), authors’ methodological training (7%), peer review process (6%,
n=4), funding declaration (6%), ethical statement (3%), protocol registration (3%),
and conflict of interest declaration (1%).
Conclusions
Methodological and reporting issues might influence the quality of the evidence produced
in rehabilitation research. The next steps to move forward in the field of rehabilitation
could be to evaluate the influence of all these issues on the validity of trial results
through meta-epidemiologic studies and to develop specific checklists to provide guidance
to authors to improve the reporting and conduct of trials in this field.
Keywords
List of abbreviations:
PICO (population, intervention, control (comparison), and outcomes), RCT (randomized controlled trial)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Archives of Physical Medicine and RehabilitationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: a systematic analysis for the Global Burden of Disease Study 2015.Lancet. 2016; 388: 1545-1602
- Health, functioning, and disability in older adults–present status and future implications.Lancet. 2015; 385: 563-575
- Global estimates of the need for rehabilitation based on the Global Burden of Disease Study 2019: a systematic analysis for the Global Burden of Disease Study 2019.Lancet. 2021; 396: 2006-2017
- Toward strengthening rehabilitation in health systems: methods used to develop a WHO package of rehabilitation interventions.Arch Phys Med Rehabil. 2019; 100: 2205-2211
- The World Health Organization “Rehabilitation 2030: a call for action.Eur J Phys Rehabil Med. 2017; 53: 155-168
- Prioritising people with disabilities implies furthering rehabilitation.Lancet. 2020; 395: 111
- Rehabilitation: the health strategy of the 21st century.J Rehabil Med. 2018; 50: 309-316
- Rehabilitation is a global health priority.BMC Health Serv Res. 2020; 20: 143
World Health Organization. Rehabilitation 2030: a call for action. Available at:https://www.who.int/disabilities/care/KeyForHealth21stCentury.pdf. Accessed March 18, 2021.
- The struggle for evidence in physical and rehabilitation medicine: publication rate of randomized controlled trials and systematic reviews is growing more than in other therapeutic fields.Am J Phys Med Rehabil. 2019; 98: 258-265
- Cochrane Rehabilitation Methodology Committee: an international survey of priorities for future work.Eur J Phys Rehabil Med. 2017; 53: 814-817
- Usual care: the big but unmanaged problem of rehabilitation evidence.Lancet. 2020; 395: 337
- Real-world evidence - what is it and what can it tell us?.N Engl J Med. 2016; 375: 2293-2297
- Methodological problems in rehabilitation research. Report from a Cochrane Rehabilitation Methodology meeting.Eur J Phys Rehabil Med. 2019; 55: 319-321
- Clinical replicability of rehabilitation interventions in randomized controlled trials reported in main journals is inadequate.J Clin Epidemiol. 2019; 114: 108-117
- Guidance for conducting systematic scoping reviews.Int J Evid Based Healthc. 2015; 13: 141-146
- A scoping review describes methods used to identify, prioritize and display gaps in health research.J Clin Epidemiol. 2019; 109: 99-110
- Scoping studies: towards a methodological framework.Int J Soc Res Methodol. 2005; 8: 19-32
- PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation.Ann Intern Med. 2018; 169: 467-473
- The 3rd Cochrane Rehabilitation Methodology meeting: “rehabilitation definition for scientific research purposes.Eur J Phys Rehabil Med. 2020; 56: 658-660
- A comparison and synthesis of rehabilitation definitions used by consumers (Google), major stakeholders (survey) and researchers (Cochrane Systematic Reviews): a terminological analysis.Eur J Phys Rehabil Med. 2020; 56: 682-689
- Statistical methods for assessing the influence of study characteristics on treatment effects in “meta-epidemiological” research.Stat Med. 2002; 21: 1513-1524
- A typology of reviews: an analysis of 14 review types and associated methodologies.Health Inf Libr J. 2009; 26: 91-108
- Scoping reviews: time for clarity in definition, methods, and reporting.J Clin Epidemiol. 2014; 67: 1291-1294
- Scales to assess the quality of randomized controlled trials: a systematic review.Phys Ther. 2008; 88: 156-175
- Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists.Control Clin Trials. 1995; 16: 62-73
- The structure of research questions in randomized-controlled trials in rehabilitation field: a methodological study.Am J Phys Med Rehabil. 2021; 100: 29-33
- Applying evidence standards to rehabilitation research.Am J Phys Med Rehabil. 2006; 85: 292-309
- Research designs and statistical methods trends in the Annals of Rehabilitation Medicine.Ann Rehabil Med. 2017; 41: 475-482
- Clinical impact research - how to choose experimental or observational intervention study?.Ann Med. 2016; 48: 492-495
- Design and implementation of clinical trials in rehabilitation research.Arch Phys Med Rehabil. 2012; 93: S117-S126
- Practice-based evidence research in rehabilitation: an alternative to randomized controlled trials and traditional observational studies.Arch Phys Med Rehabil. 2012; 93: S127-S137
- Critical evaluation of clinical research.Arch Phys Med Rehabil. 1995; 76: 82-93
- Toward improved evidence standards and methods for rehabilitation: recommendations and challenges.Arch Phys Med Rehabil. 2012; 93: S185-S199
- Sample size calculation in physical medicine and rehabilitation: a systematic review of reporting, characteristics, and results in randomized controlled trials.Arch Phys Med Rehabil. 2011; 92: 306-315
- Systematic reviews of epidemiological studies. A tool for the evidence in physiotherapy [Spanish].Fisioterapia. 2010; 32: 25-32
- Beyond the RCT - understanding the relationship between interventions, individuals and outcome - the example of neurological rehabilitation.Disabil Rehabil. 2010; 32: 1028-1034
- Ensuring inclusion of research reports in systematic reviews.Arch Phys Med Rehabil. 2009; 90: S60-S69
- Conducting multiple-site clinical trials in medical rehabilitation research.Am J Phys Med Rehabil. 2005; 84: 823-831
- Another look at observational studies in rehabilitation research: going beyond the holy grail of the randomized controlled trial.Arch Phys Med Rehabil. 2005; 86: S8-15
- Overview of clinical trials in medical rehabilitation: impetuses, challenges, and needed future directions.Am J Phys Med Rehabil. 2003; 82: S8-15
- Desiderata for clinical trials in medical rehabilitation.Am J Phys Med Rehabil. 2003; 82: S3-S7
- Emerging standards in statistical practice: implications for clinical trials in rehabilitation medicine.Am J Phys Med Rehabil. 2003; 82: S32-S37
- Fundamentals of clinical trials for medical rehabilitation.Am J Phys Med Rehabil. 2003; 82: S22-S25
- Issues of adherence, penetration, and measurement in physical activity effectiveness studies.Med Care. 2001; 39: 409-412
- Considerations for planning and conducting clinic-based research in physical therapy.Phys Ther. 2001; 81: 1446-1454
- A quantitative analysis of research in physical therapy.Phys Ther. 1995; 75 ([discussion: 322-7]): 313-322
- Prevalence of methodologic errors in rehabilitation research.J Rehabil Sci. 1994; 7: 60-62
- Toward a methodology for rehabilitation research.Am J Phys Med Rehabil. 1994; 73: 428-435
- The limitations of randomized controlled trials in rehabilitation research.Clin Rehabil. 1991; 5: 5-8
- How have research questions and methods used in clinical trials published in clinical rehabilitation changed over the last 30 years?.Clin Rehabil. 2016; 30: 847-864
- When is a research question not a research question?.J Rehabil Med. 2013; 45: 513-518
- Mapping of the evidence from systematic reviews of the Cochrane Collaboration for decision-making within physiotherapy.Sao Paulo Med J. 2013; 131: 39-45
- Reported characteristics of participants in physical therapy-related clinical trials.Phys Ther. 2015; 95: 884-890
- Participant recruitment and retention in rehabilitation research.Am J Phys Med Rehabil. 2008; 87: 330-338
- Research in rehabilitation medicine: methodological challenges.J Clin Epidemiol. 2010; 63: 699-704
- Clinical trials in rehabilitation: what are the obstacles?.Am J Phys Med Rehabil. 2003; 82: S16-S21
- Reporting on reliability and validity of outcome measures in medical rehabilitation research.Disabil Rehabil. 2002; 24: 819-827
- Reporting on interventions: issues and guidelines for rehabilitation researchers.Arch Phys Med Rehabil. 2015; 96: 1170-1180
- Evidence-based practice for rehabilitation professionals: concepts and controversies.Arch Phys Med Rehabil. 2012; 93: S164-S176
- Perspectives on rehabilitation research.Arch Phys Med Rehabil. 2010; 91: 169-172
- Control in rehabilitation research.Clin Rehabil. 2009; 23: 675-680
- A systematic review of interventions to increase the use of standardized outcome measures by rehabilitation professionals.Clin Rehabil. 2017; 31: 299-309
- Quality of intervention research reporting in medical rehabilitation journals.Am J Phys Med Rehabil. 2002; 81: 21-33
- Risk adjusting rehabilitation outcomes: an overview of methodologic issues.Am J Phys Med Rehabil. 2004; 83: 316-326
- Rehabilitation outcomes: values, methodologies and applications.Disabil Rehabil. 2010; 32: 961-964
- Physiotherapy trials for the 21st century: time to raise the bar?.J R Soc Med. 2011; 104: 437-441
- Measurement validity in physical therapy research.Phys Ther. 1993; 73 ([discussion: 110-5]): 102-110
- Outcome measures for clinical rehabilitation trials: impairment, function, quality of life, or value?.Am J Phys Med Rehabil. 2003; 82: S26-S31
- Effectiveness versus efficacy: more than a debate over language.J Orthop Sports Phys Ther. 2003; 33: 163-165
- A quantitative analysis of research publications in physical therapy journals.Phys Ther. 2003; 83: 123-131
- Why rehabilitation research does not work (as well as we think it should).Arch Phys Med Rehabil. 1995; 76: 123-129
- Statistical methods in rehabilitation literature: a survey of recent publications.Arch Phys Med Rehabil. 1996; 77: 497-500
- Blinding in physical therapy trials and its association with treatment effects: a meta-epidemiological study.Am J Phys Med Rehabil. 2017; 96: 34-44
- Mapping physical therapy research: the geographical affiliations and methodological quality of 2,959 randomized controlled trials.Physiother Theory Pract. 2018; 34: 723-729
- Rehabilitation research priorities: the next 10 years.J Burn Care Res. 2017; 38: e635-e637
- Practical guidelines for independent assessment in randomized controlled trials (RCTs) of rehabilitation.Clin Rehabil. 1997; 11: 273-279
- What is the influence of randomisation sequence generation and allocation concealment on treatment effects of physical therapy trials? A meta-epidemiological study.BMJ Open. 2015; 5e008562
- Identifying items to assess methodological quality in physical therapy trials: a factor analysis.Phys Ther. 2014; 94: 1272-1284
- PEDro or Cochrane to assess the quality of clinical trials? A meta-epidemiological study.PLoS One. 2015; 10e0132634
- Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: a descriptive analysis.BMC Med Res Methodol. 2013; 13: 116
- Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane Risk of Bias Tool in physical therapy trials.PLoS One. 2014; 9: e96920
- PEDro's bias: summary quality scores should not be used in meta-analysis.J Clin Epidemiol. 2013; 66: 75-77
- Assessment of the risk of bias in rehabilitation reviews.Int J Rehabil Res. 2012; 35: 317-322
- Estimates of quality and reliability with the physiotherapy evidence-based database scale to assess the methodology of randomized controlled trials of pharmacological and nonpharmacological interventions.Phys Ther. 2006; 86: 817-824
- Elevating the quality of disability and rehabilitation research: mandatory use of the reporting guidelines.Int J Rehabil Res. 2014; 37: 103-104
- Small N designs for rehabilitation research.J Rehabil Res Dev. 2012; 49: 175-186
- Improving power and sample size calculation in rehabilitation trial reports: a methodological assessment.Arch Phys Med Rehabil. 2016; 97: 1195-1201
- Small sample research designs for evidence-based rehabilitation: issues and methods.Arch Phys Med Rehabil. 2012; 93: S111-S116
- Measuring clinical significance in rehabilitation research.Rehabil Couns Bull. 2006; 50: 35-45
- Value of confidence intervals in determining clinical significance.Physiother Can. 2006; 58: 205-211
- Measures of effect size in the reporting of rehabilitation research.Am J Phys Med Rehabil. 1989; 68: 52-58
- What are we looking at, and how big is it?.Phys Ther Sport. 2003; 4: 93-97
- Reporting ethical protections in physical therapy research.Phys Ther. 2006; 86: 499-509
- Registration of all rehabilitation clinical trials: an ethical and editorial imperative.Clin Rehabil. 2016; 30: 211-212
- Peer review: issues in physical medicine and rehabilitation.Am J Phys Med Rehabil. 2003; 82: 790-802
- Measuring rehabilitation research capacity: report from the AAPM&R Research Advisory Committee.Am J Phys Med Rehabil. 2005; 84: 955-968
- Blinding in rehabilitation research: empirical evidence on the association between blinding and treatment effect estimates.Am J Phys Med Rehabil. 2020; 99: 198-209
- The Randomized Controlled Trials Rehabilitation Checklist: methodology of development of a reporting guideline specific to rehabilitation.Am J Phys Med Rehabil. 2020; 99: 210-215
- Methodological differences in clinical trials evaluating nonpharmacological and pharmacological treatments of hip and knee osteoarthritis.JAMA. 2003; 290: 1062-1070
- The “architect analogy” of evidence-based practice: reconsidering the role of clinical expertise and clinician experience in evidence-based health care.J Evid Based Med. 2018; 11: 219-226
- Research without good questions is a waste.J Clin Epidemiol. 2019; 108: vi-viii
- What is “quality of evidence” and why is it important to clinicians?.BMJ. 2008; 336: 995-998
- European Physical and Rehabilitation Medicine Bodies Alliance. White book on physical and rehabilitation medicine (PRM) in Europe. Chapter 1. Definitions and concepts of PRM.Eur J Phys Rehabil Med. 2018; 54: 156-165
- Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research.J Eval Clin Pract. 2012; 18: 12-18
- Introduction to the GRADE approach for guideline development: considerations for physical therapist practice.Phys Ther. 2014; 94: 1652-1659
- Novel effect size interpretation guidelines and an evaluation of statistical power in rehabilitation research.Arch Phys Med Rehabil. 2020; 101: 2219-2226
- Current evidence from the Randomised Controlled Trials Rehabilitation Checklist (RCTRACK) reporting guideline project.Am J Phys Med Rehabil. 2021; 100: 2-4
- Toward better reporting standards of patients’ characteristics in rehabilitation trials: applying a new conceptual framework to current standards.Am J Phys Med Rehabil. 2020; 99: 216-223
- Cochrane Rehabilitation Methodology Meeting participants. In search of solutions for evidence generation in rehabilitation: the second Cochrane Rehabilitation Methodology Meeting.Am J Phys Med Rehabil. 2020; 99: 181-182
Article info
Publication history
Published online: May 10, 2021
Accepted:
April 22,
2021
Received in revised form:
March 22,
2021
Received:
June 13,
2020
Footnotes
Disclosures: none
Identification
Copyright
© 2021 The American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.
