Abstract
Objectives
(1) To determine whether new tools and items have been developed to evaluate the risk
of bias (RoB) and reporting of randomized controlled trials (RCTs) in rehabilitation;
(2) to determine which items are included in the existing reporting guidelines, and
to create a matrix of items to report and conduct trials in rehabilitation as the
first step for a starting a rigorous validation process.
Data Sources
Searches were conducted in MEDLINE, EMBASE, and Cumulative Index to Nursing and Allied
Health databases.
Study Selection
Studies should describe a newly developed tool to evaluate the RoB or quality of reporting
for RCTs in the area of rehabilitation.
Data extraction
(1) First, we extracted items from new tools identified by the electronic search strategies
and then (2) we looked at the items provided by the Consolidated Standards of Reporting
Trials statement and its relevant extensions. We determined whether these items were
already included in our matrix of items. Items were classified based on methodological
domains they accomplish, biases they were able to target, and whether they guide reporting
or conduct.
Data Synthesis
Among the 1596 citations found, 23 articles were potentially relevant. From these,
only 3 new scales (National Institute for Complementary Medicine Acupuncture Network,
Quality of reports on spa and balneotherapy [SPAC], Assessment of Study Quality and
Reporting in Exercise) were found. In addition, the newly updated Cochrane RoB tool
(RoB 2.0) was included. Our matrix contained 122 unique items for any rehabilitation
area, 46 items (37.7%) were related to conduct, and 58 (47.5%) were related to the
reporting; 18 (14.8%) were related to both. Overall, 76 new items were added among
all domains.
Conclusions
Many individual and diverse items have been used to guide the reporting and conduct
of rehabilitation trials. This indicates a great variability in number of items and
an apparent lack of consensus on a core set of items to be used in rehabilitation.
Future research should look into developing a core set of items for the rehabilitation
field.
Keywords
List of abbreviations:
CONSORT (Consolidated Standards of Reporting Trials), EXT (pragmatic trials: Extension - Pragmatic Trials), EXT-NPT (Extension - Nonpharmacologic Treatments), EXT-SPI (Extension - Social and Psychological Interventions), EXT- N-of-1-trial (Extension - one trial), NICMAN (National Institute for Complementary Medicine Acupuncture Network), RCT (randomized controlled trial), RCTRACK (Randomized Controlled Trial Rehabilitation Checklist), RoB (risk of bias), SPAC (Quality of Reports on Spa and Balneotherapy), TIDieR (Template for Intervention Description and Replication)To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to Archives of Physical Medicine and RehabilitationAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Inconsistency in the items included in tools used in general health research and physical therapy to evaluate the methodological quality of randomized controlled trials: a descriptive analysis.BMC Med Res Methodol. 2013; 13: 1-19
- Scales to assess the quality of randomized controlled trials: a systematic review.Phys Ther. 2008; 88: 156-175
- The Cochrane Collaboration's tool for assessing risk of bias in randomised trials.BMJ. 2011; 343: d5928
- Construct validity of the Physiotherapy Evidence Database (PEDro) quality scale for randomized trials: item response theory and factor analyses.Res Synth Methods. 2020; 11: 227-236
- Evaluation of the Cochrane Collaboration's tool for assessing the risk of bias in randomized trials: focus groups, online survey, proposed recommendations and their implementation.Syst Rev. 2014; 3: 37
- Risk of bias versus quality assessment of randomised controlled trials: cross sectional study.BMJ. 2009; 339: 1017
- Testing the Risk of Bias tool showed low reliability between individual reviewers and across consensus assessments of reviewer pairs.J Clin Epidemiol. 2012; 66: 973-981
- Poor reliability between Cochrane reviewers and blinded external reviewers when applying the Cochrane Risk of Bias Tool in physical therapy trials.PLoS One. 2014; 9: e96920
- Assessment of study quality for systematic reviews: a comparison of the Cochrane Collaboration Risk of Bias Tool and the Effective Public Health Practice Project Quality Assessment Tool: methodological research.J Eval Clin Pract. 2012; 18: 12-18
- The Randomized Controlled Trials Rehabilitation Checklist: methodology of development of a reporting guideline specific to rehabilitation.Am J Phys Med Rehabil. 2020; 99: 210-215
- Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement.Ann Intern Med. 2009; 151: 264-269
- RoB 2: a revised tool for assessing risk of bias in randomised trials.BMJ. 2019; 366: l4898
- Toward better reporting standards of patients’ characteristics in rehabilitation trials: applying a new conceptual framework to current standards.Am J Phys Med Rehabil. 2020; 99: 216-223
- CONSORT statement for randomized trials of nonpharmacologic treatments: a 2017 update and a CONSORT extension for nonpharmacologic trial abstracts.Ann Intern Med. 2017; 167: 40-47
- CONSORT-SPI 2018 explanation and elaboration: guidance for reporting social and psychological intervention trials.Trials. 2018; 19: 406
- Improving the reporting of pragmatic trials: an extension of the CONSORT statement.BMJ. 2008; 337: a2390
- CONSORT extension for reporting N-of-1 trials (CENT) 2015 statement.J Clin Epidemiol. 2016; 76: 9-17
- Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide.BMJ. 2014; 348: g1687
- The TIDieR checklist will benefit the physiotherapy profession.Physiother Theory Pract. 2017; 33: 267-268
- CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials.BMJ. 2010; 340: 698-702
- Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists.Control Clin Trials. 1995; 16: 62-73
- Bias.J Epidemiol Community Health. 2004; 58: 635-641
- Bias in analytic research.J Chronic Dis. 1979; 32: 51-68
- Development of the RTI item bank on risk of bias and precision of observational studies.Journal of Clinical Epidemiology. 2012; 65: 163-178
- A checklist to assess the quality of reports on spa therapy and balneotherapy trials was developed using the Delphi consensus method: the SPAC checklist.Complement Ther Med. 2013; 21: 324-332
- Validation of a new tool for the assessment of study quality and reporting in exercise training studies: TESTEX.Int J Evid Based Healthc. 2015; 13: 9-18
- Reliability of the NICMAN Scale: an instrument to assess the quality of acupuncture administered in clinical trials.Evid Based Complement Alternat Med. 2017; 20175694083
- Clinical replicability of rehabilitation interventions in randomized controlled trials reported in main journals is inadequate.J Clin Epidemiol. 2019; 114: 108-117
- Standardisation and its discontents.Cogn Technol Work. 2015; 17: 89-94
- Consensus on Exercise Reporting Template (CERT): explanation and elaboration statement.Br J Sports Med. 2016; 50: 1428-1437
- Consensus on Exercise Reporting Template (CERT): modified Delphi study.Phys Ther. 2016; 96: 1514-1524
- A review of CONSORT guidelines about comparison groups with a focused discussion on implications for rehabilitation clinical trials.Am J Phys Med Rehabil. 2020; 99: 191-197
- Sample size calculation in physical medicine and rehabilitation: a systematic review of reporting, characteristics, and results in randomized controlled trials.Arch Phys Med Rehabil. 2011; 92: 306-315
- Improving power and sample size calculation in rehabilitation trial reports: a methodological assessment.Arch Phys Med Rehabil. 2016; 97: 1195-1201
- Sample size calculations are poorly conducted and reported in many randomized trials of hip and knee osteoarthritis: results of a systematic review.J Clin Epidemiol. 2018; 104: 52-61
- Allocation concealment in randomised trials: defending against deciphering.Lancet. 2002; 359: 614-618
- Blinding in physical therapy trials and its association with treatment effects: a meta-epidemiological study.Am J Phys Med Rehabil. 2017; 96: 34-44
- What is the influence of randomization sequence generation and allocation concealment on treatment effects of physical therapy trials? A meta-epidemiological study.BMJ Open. 2015; 5e008562
- Reporting and interpretation of randomized clinical trials.JAMA. 2019; 322: 732-735
- The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study.Qual Life Res. 2010; 19: 539-549
- Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist.Qual Life Res. 2012; 21: 651-657
- Empirical evidence of study design biases in randomized trials: systematic review of meta-epidemiological studies.PLoS One. 2016; 11e0159267
- PEDro or Cochrane to assess the quality of clinical trials? A meta-epidemiological study.PLoS One. 2015; 10e0132634
- Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?.Lancet. 1998; 352: 609-613
- The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study.Arthritis Rheum. 2009; 61: 1633-1641
- The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study.BMJ. 2009; 339: 679-683
- Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study.BMJ. 2010; 341: 241
- Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study.BMJ. 2008; 336: 601-605
- Some Cochrane risk-of-bias items are not important in osteoarthritis trials: a meta-epidemiological study based on Cochrane Reviews.J Clin Epidemiol. 2018; 95: 128-136
Article info
Publication history
Published online: May 11, 2021
Accepted:
April 8,
2021
Received in revised form:
April 7,
2021
Received:
November 20,
2020
Footnotes
Presented to the methodological meeting of the Cochrane Rehabilitation Field, March 3-4, 2020, Orlando, FL.
Disclosures: none
Identification
Copyright
© 2021 The American Congress of Rehabilitation Medicine. Published by Elsevier Inc. All rights reserved.
