Archives of Physical Medicine and Rehabilitation
Volume 90, Issue 11, Supplement , Pages S60-S69 , November 2009

Ensuring Inclusion of Research Reports in Systematic Reviews

Presented to the American Congress of Rehabilitation Medicine, Washington, DC, October 3–7, 2007.

  • Marcel P. Dijkers, PhD

      Affiliations

    • Corresponding Author InformationCorrespondence to: Marcel Dijkers, PhD, Dept of Rehabilitation Medicine, Box 1240, Mount Sinai School of Medicine, One Gustave Levy Place, New York, NY 10029-6574

References 

  1. Bhandari M, Montori VM, Devereaux PJ, et al. Doubling the impact: publication of systematic review articles in orthopaedic journals. J Bone Joint Surg Am. 2004;86-A:1012–1016
  2. Dijkers MP. Task Force on Systematic Reviews and Guidelines (The value of “traditional” reviews in the era of systematic reviewing). Am J Phys Med Rehabil. 2009;88:423–430
  3. Montori VM, Wilczynski NL, Morgan D, Haynes RB Hedges Team. Systematic reviews: a cross-sectional study of location and citation counts. BMC Med. 2003;1:2
  4. American Psychological Association. Publication manual of the American Psychological Association. 5th ed.. Washington (DC): American Psychological Association; 2001;
  5. Iverson C. AMA manual of style: a guide for authors and editors. 10th ed.. New York: Oxford Univ Pr; 2007;
  6. International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals: writing and editing for biomedical publication (updated October 2008). http://www.icmje.org/urm_full.pdfAccessed October 13, 2009
  7. Kalpakjian CZ, Meade M. Writing manuscripts for peer review: your guide to not annoying reviewers and increasing your chances of success. Sex Disabil. 2008;26:229–240
  8. Provenzale JM. Ten principles to improve the likelihood of publication of a scientific manuscript. AJR Am J Roentgenol. 2007;188:1179–1182
  9. Streiner DL. A shortcut to rejection: how not to write the results section of a paper. Can J Psychiatry. 2007;52:385–389
  10. Smart J. Attributes of exemplary research manuscripts employing quantitative analyses. Res Higher Educ. 2005;46:461–477
  11. Bond GR, Mintz J, McHugo GJ. Statistical guidelines for the Archives of PM&R. Arch Phys Med Rehabil. 1995;76:784–787
  12. Wilkinson L. Statistical methods in psychology journals: guidelines and explanations. Am Psychol. 1999;54:594–604
  13. Bailar JC, Mosteller F. Guidelines for statistical reporting in articles for medical journals: amplifications and explanations. Ann Intern Med. 1988;108:266–273
  14. Carraway LN. Improve scientific writing and avoid perishing. Am Midl Nat. 2006;155:383–394
  15. Ludbrook J. Writing intelligible English prose for biomedical journals. Clin Exp Pharmacol Physiol. 2007;34:508–514
  16. Guide for authors (Archives of Physical Medicine and Rehabilitation). http://www.elsevier.com/wps/find/journaldescription.cws_home/623354/authorinstructions#itaAccessed October 9, 2008
  17. Steinert Y, McLeod PJ, Liben S, Snell L. Writing for publication in medical education: the benefits of a faculty development workshop and peer writing group. Med Teach. 2008;30:280–285
  18. Hoffmann T, Bennett S, McKenna K, Green-Hill J, McCluskey A, Tooth L. Interventions for stroke rehabilitation: analysis of the research contained in the OTseeker evidence database. Top Stroke Rehabil. 2008;15:341–350
  19. Bhogal SK, Teasell RW, Foley NC, Speechley MR. Quality of the stroke rehabilitation research. Top Stroke Rehabil. 2003;10:8–28
  20. Fernandez-de-las-Penas C, Alonso-Blanco C, San-Roman J, Miangolarra-Page JC. Methodological quality of randomized controlled trials of spinal manipulation and mobilization in tension-type headache, migraine, and cervicogenic headache. J Orthop Sports Phys Ther. 2006;36:160–169
  21. Moberg-Mogren E, Nelson DL. Evaluating the quality of reporting occupational therapy randomized controlled trials by expanding the CONSORT criteria. Am J Occup Ther. 2006;60:226–235
  22. Burns KE, Adhikari NK, Kho M, et al. Abstract reporting in randomized clinical trials of acute lung injury: an audit and assessment of a quality of reporting score. Crit Care Med. 2005;33:1937–1945
  23. Scales CD, Norris RD, Preminger GM, Vieweg J, Peterson BL, Dahm P. Evaluating the evidence: statistical methods in randomized controlled trials in the urological literature. J Urol. 2008;180:1463–1467
  24. Boutron I, Guittet L, Estellat C, Moher D, Hrobjartsson A, Ravaud P. Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med. 2007;4:e61
  25. Chan KB, Man-Son-Hing M, Molnar FJ, Laupacis A. How well is the clinical importance of study results reported? (an assessment of randomized controlled trials). CMAJ. 2001;165:1197–1202
  26. Dijkers M, Kropp GC, Esper RM, Yavuzer G, Cullen N, Bakdalieh Y. Quality of intervention research reporting in medical rehabilitation journals. Am J Phys Med Rehabil. 2002;81:21–33
  27. Dijkers MP, Kropp GC, Esper RM, Yavuzer G, Cullen N, Bakdalieh Y. Reporting on reliability and validity of outcome measures in medical rehabilitation research. Disabil Rehabil. 2002;24:819–827
  28. Moher D, Schulz KF, Altman D CONSORT Group (Consolidated Standards of Reporting Trials). The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285:1987–1991
  29. Altman DG, Schulz KF, Moher D, et al. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med. 2001;134:663–694
  30. Begg C, Cho M, Eastwood S, et al. Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA. 1996;276:637–639
  31. Stroup DF, Berlin JA, Morton SC, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting (Meta-analysis Of Observational Studies in Epidemiology (MOOSE) Group). JAMA. 2000;283:2008–2012
  32. Moher D, Cook DJ, Eastwood S, Olkin I, Rennie D, Stroup DF. Improving the quality of reports of meta-analyses of randomised controlled trials: the QUOROM statement. Lancet. 1999;354:1896–1900
  33. PLoS Medicine Editors. Many reviews are systematic but some are more transparent and completely reported than others. PLoS Med. 2007;4:e147
  34. Bossuyt PM, Reitsma JB, Bruns DE, et al. The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Clin Chem. 2003;49:7–18
  35. von Elm E, Altman DG, Egger M, et al. Strengthening The Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ. 2007;335:806–808
  36. Vandenbroucke JP, von Elm E, Altman DG, et al. Strengthening the Reporting of Observational Studies in Epidemiology (STROBE): explanation and elaboration. PLoS Med. 2007;4:e297
  37. Des Jarlais DC, Lyles C, Crepaz N TREND Group. Improving the reporting quality of nonrandomized evaluations of behavioral and public health interventions: the TREND statement. Am J Public Health. 2004;94:361–366
  38. Dean ME, Coulter MK, Fisher P, Jobst K, Walach H. Reporting data on homeopathic treatments (RedHot): a supplement to CONSORT. Homeopathy. 2007;96:42–45
  39. MacPherson H, White A, Cummings M, et al. Standards for Reporting Interventions in Controlled Trials of Acupuncture: the STRICTA recommendations. J Altern Complement Med. 2002;8:85–89
  40. National Center for the Dissemination of Disability Research. Reporting of research—checklists. http://www.ncddr.org/ktinfocenter/checklists.htmlAccessed October 7, 2008
  41. EQUATOR Network. http://www.equator-network.org/?o=1001Accessed October 6, 2008
  42. Campbell MK, Elbourne DR, Altman DG CONSORT Group. CONSORT statement: extension to cluster randomised trials. BMJ. 2004;328:702–708
  43. Ioannidis JP, Evans SJ, Gotzsche PC, et al. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141:781–788
  44. Piaggio G, Elbourne DR, Altman DG, Pocock SJ, Evans SJ CONSORT Group. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA. 2006;295:1152–1160
  45. Zwarenstein M, Treweek S, Gagnier JJ, et al. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390
  46. Gagnier JJ, Boon H, Rochon P, et al. Reporting randomized, controlled trials of herbal interventions: an elaborated CONSORT statement. Ann Intern Med. 2006;144:364–367
  47. Gagnier JJ, Boon H, Rochon P, et al. Recommendations for reporting randomized controlled trials of herbal interventions: explanation and elaboration. J Clin Epidemiol. 2006;59:1134–1149
  48. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P CONSORT Group. Methods and processes of the CONSORT group: example of an extension for trials assessing nonpharmacologic treatments. Ann Intern Med. 2008;148:W60–W66
  49. Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P CONSORT Group. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148:295–309
  50. Hopewell S, Clarke M, Moher D, et al. CONSORT for reporting randomised trials in journal and conference abstracts. Lancet. 2008;371:281–283
  51. Hopewell S, Clarke M, Moher D, et al. CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med. 2008;5:e20
  52. CONSORT—transparent reporting of trials. http://www.consort-statement.org/?o=1001Accessed October 7, 2008
  53. Kuhn TS. The structure of scientific revolutions. Chicago: Univ of Chicago Pr; 1962;
  54. Nath SB, Marcus SC, Druss BG. Retractions in the research literature: misconduct or mistakes?. Med J Aust. 2006;185:152–154
  55. Phillips CV. Publication bias in situ. BMC Med Res Methodol. 2004;4:20
  56. Zehetmayer S, Bauer P, Posch M. Optimized multi-stage designs controlling the false discovery or the family-wise error rate. Stat Med. 2008;27:4145–4160
  57. Ebrahimi N. Simultaneous control of false positives and false negatives in multiple hypotheses testing. J Multivar Anal. 2008;99:437–450
  58. Hill CL, LaValley MP, Felson DT. Discrepancy between published report and actual conduct of randomized clinical trials. J Clin Epidemiol. 2002;55:783–786
  59. Devereaux PJ, Choi PT, El-Dika S, et al. An observational study found that authors of randomized controlled trials frequently use concealment of randomization and blinding, despite the failure to report these methods. J Clin Epidemiol. 2004;57:1232–1236
  60. Devereaux PJ, Manns BJ, Ghali WA, et al. Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA. 2001;285:2000–2003
  61. Montori VM, Bhandari M, Devereaux PJ, Manns BJ, Ghali WA, Guyatt GH. In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol. 2002;55:787–790
  62. Gravel J, Opatrny L, Shapiro S. The intention-to-treat approach in randomized controlled trials: are authors saying what they do and doing what they say?. Clin Trials. 2007;4:350–356
  63. Prady SL, Macpherson H. Assessing the utility of the standards for reporting trials of acupuncture (STRICTA): a survey of authors. J Altern Complement Med. 2007;13:939–943
  64. Hopewell S, Altman DG, Moher D, Schulz KF. Endorsement of the CONSORT statement by high impact factor medical journals: a survey of journal editors and journal “instructions to authors.”. Trials. 2008;9:20
  65. Altman DG. Endorsement of the CONSORT statement by high impact medical journals: survey of instructions for authors. BMJ. 2005;330:1056–1057
  66. Mills EJ, Wu P, Gagnier J, Devereaux PJ. The quality of randomized trial reporting in leading medical journals since the revised CONSORT statement. Contemp Clin Trials. 2005;26:480–487
  67. Plint AC, Moher D, Morrison A, et al. Does the CONSORT checklist improve the quality of reports of randomised controlled trials? (a systematic review). Med J Aust. 2006;185:263–267
  68. Smith BA, Lee HJ, Lee JH, et al. Quality of reporting randomized controlled trials (RCTs) in the nursing literature: application of the consolidated standards of reporting trials (CONSORT). Nurs Outlook. 2008;56:31–37
  69. Prady SL, Richmond SJ, Morton VM, Macpherson H. A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials. PLoS ONE. 2008;3:e1577
  70. de Boer MJ, van der Wall EE. Towards better cardiovascular journals. Neth Heart J. 2008;16:151–152
  71. Vlaar AM, Bouwmans AE, van Kroonenburgh MJ, et al. Protocol of a prospective study on the diagnostic value of transcranial duplex scanning of the substantia nigra in patients with parkinsonian symptoms. BMC Neurol. 2007;7:28
  72. Takeda A, Loveman E, Harris P, Hartwell D, Welch K. Time to full publication of studies of anti-cancer medicines for breast cancer and the potential for publication bias: a short systematic review. Health Technol Assess. 2008;12:1–46iii, ix-x
  73. Toma M, McAlister FA, Bialy L, Adams D, Vandermeer B, Armstrong PW. Transition from meeting abstract to full-length journal article for randomized controlled trials. JAMA. 2006;295:1281–1287
  74. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004;291:2457–2465
  75. Chan AW, Hrobjartsson A, Jorgensen KJ, Gotzsche PC, Altman DG. Discrepancies in sample size calculations and data analyses reported in randomised trials: comparison of publications with protocols. BMJ. 2008;337:a2299
  76. Gordon WA, Zafonte R, Cicerone K, et al. Traumatic brain injury rehabilitation: state of the science. Am J Phys Med Rehabil. 2006;85:343–382
  77. Sipski ML, Richards JS. Spinal cord injury rehabilitation: state of the science. Am J Phys Med Rehabil. 2006;85:310–342
  78. Esselman PC, Thombs BD, Magyar-Russell G, Fauerbach JA. Burn rehabilitation: state of the science. Am J Phys Med Rehabil. 2006;85:383–413
  79. Johnston MV, Sherer M, Whyte J. Applying evidence standards to rehabilitation research. Am J Phys Med Rehabil. 2006;85:292–309
  80. Tugwell P, Boers M, Brooks P, Simon L, Strand V, Idzerda L. OMERACT: an international initiative to improve outcome measurement in rheumatology. Trials. 2007;8:38
  81. Merkel PA, Clements PJ, Reveille JD, et al. Current status of outcome measure development for clinical trials in systemic sclerosis: report from OMERACT 6. J Rheumatol. 2003;30:1630–1647
  82. Turk DC, Dworkin RH, Allen RR, et al. Core outcome domains for chronic pain clinical trials: IMMPACT recommendations. Pain. 2003;106:337–345
  83. Dworkin RH, Turk DC, Farrar JT, et al. Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. Pain. 2005;113:9–19
  84. Dworkin RH, Turk DC, Wyrwich KW, et al. Interpreting the clinical importance of treatment outcomes in chronic pain clinical trials: IMMPACT recommendations. J Pain. 2008;9:105–121
  85. Turk DC, Dworkin RH, McDermott MP, et al. Analyzing multiple endpoints in clinical trials of pain treatments: IMMPACT recommendations. Pain. 2008;139:485–493
  86. Johnston MV, Graves DE. Towards guidelines for evaluation of measures: an introduction with application to spinal cord injury. J Spinal Cord Med. 2008;31:13–26
  87. Anderson K, Aito S, Atkins M, et al. Functional recovery measures for spinal cord injury: an evidence-based review for clinical practice and research. J Spinal Cord Med. 2008;31:133–144
  88. Bryce TN, Budh CN, Cardenas DD, et al. Pain after spinal cord injury: an evidence-based review for clinical practice and research: report of the National Institute on Disability and Rehabilitation Research Spinal Cord Injury Measures Meeting. J Spinal Cord Med. 2007;30:421–440
  89. Lammertse D, Dungan D, Dreisbach J, et al. Neuroimaging in traumatic spinal cord injury: an evidence-based review for clinical practice and research. J Spinal Cord Med. 2007;30:205–214
  90. Sherwin E, Whiteneck G, Corrigan J, et al. Domains of a TBI minimal data set: community reintegration phase. Brain Inj. 2006;20:383–389
  91. Biering-Sorensen F, Charlifue S, DeVivo M, et al. International spinal cord injury data sets. Spinal Cord. 2006;44:530–534
  92. DeVivo M, Biering-Sorensen F, Charlifue S, et al. International spinal cord injury core data set. Spinal Cord. 2006;44:535–540
  93. Widerstrom-Noga E, Biering-Sorensen F, Bryce T, et al. The international spinal cord injury pain basic data set. Spinal Cord. 2008;46:818–823
  94. Krogh K, Perkash I, Stiens SA, Biering-Sorensen F. International bowel function extended spinal cord injury data set. Spinal Cord. 2009;47:230–234
  95. Edlund W, Gronseth G, So Y, Franklin G. Clinical practice guidelines process manual—2004 edition. St. Paul: American Academy of Neurology; 2004;
  96. MacCallum CJ. ONE for all: the next step for PLoS. PLoS Biol. 2006;4:e401
  97. West S, King V, Carey TS, et al. Systems to rate the strength of scientific evidence. Rockville, MD: Agency for Healthcare Research and Quality; 2002;AHQR Evidence Report/Technology Assessment Number 47
  98. Higgins JP, Green S, editors. Cochrane handbook for systematic reviews of interventions. Version 5.0.1 (updated September 2008). The Cochrane Collaboration. Available at: www.cochrane-handbook.org. Accessed July 14, 2009.

 Supported by the National Institute on Disability and Rehabilitation Research, Office of Special Education and Rehabilitative Services, Department of Education, to Mount Sinai School of Medicine (grant no. H133B040033).

 No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit on the authors or on any organization with which the author is associated.

 Reprints are not available from the author.

PII: S0003-9993(09)00464-X

doi: 10.1016/j.apmr.2009.04.026

Archives of Physical Medicine and Rehabilitation
Volume 90, Issue 11, Supplement , Pages S60-S69 , November 2009

Article Tools