If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Corresponding author Marcel P. Dijkers, PhD, Department of Physical Medicine and Rehabilitation, Wayne State University, Rehabilitation Institute of Michigan, 261 Mack Ave, Room 209, Detroit, MI 48201.
Kessler Institute for Rehabilitation, West Orange, NJDepartment of Physical Medicine and Rehabilitation, Rutgers, New Jersey Medical School, Newark, NJ
To evaluate the quality of rehabilitation Clinical Practice Guidelines (CPG), specifically with respect to their applicability.
Data Sources
The Medline, Embase, Web of Science, CINAHL, PsycINFO, and Cochrane Library databases were searched for papers published between 2017 and 2019 that applied the Appraisal of Guidelines for Research & Evaluation II (AGREE II) CPG quality assessment tool to rehabilitation CPGs.
Study Selection
Deduplicated abstracts (N=449) were independently screened by 2 authors, resulting in 47 articles. Independent screening of 47 full texts by 2 authors resulted in a final selection of 40 papers appraising 544 CPGs.
Data Extraction
Data were extracted by 1 author using a pretested Excel form and were checked by a second author. Key data on the review papers included: purpose, methods used for finding and appraising CPGs, and observations and conclusions on CPG quality, specifically applicability. Key data on each CPG included the 6 AGREE II domain scores or 23 item scores, as well as 2 global evaluations.
Data Synthesis
The mean AGREE II domain scores for the 544 CPGs (all on a 0-100 scale) were: scope and purpose (72), stakeholder involvement (53), rigor of development (56), clarity of presentation (71), applicability (34), and editorial independence (50). Only 36% of CPGs were recommended without modification. The 40 review authors generally stated that all or most of the CPGs they appraised were poor or mediocre, especially with respect to applicability. They only infrequently pointed out what information, going beyond that specified in AGREE II, would enhance applicability.
Conclusions
CPGs in principle are an ideal means to move knowledge obtained from clinical research into practice. Our review of reviews of rehabilitation CPGs shows that they commonly have deficits, especially in terms of applicability. Much work needs to be done by guideline developers to make it easier for the average rehabilitation organization and clinician to implement CPGs in daily practice.
Since at least the late 1980s, rehabilitation authorities have been stressing that practice needs to be evidence-based—that is, clinicians need to be guided by the results of relevant and reliable research, as well as their own expertise and the patient’s values and preferences.
The problem with that obligation is that the typical rehabilitation practitioner has neither the time nor the expertise to fruitfully study the clinical research producing the evidence: each year PubMed adds 20,000 papers that are relevant to rehabilitation. Several publication types have been developed to summarize or synthesize the information deluge so as to unburden the clinician. However, given an estimate that now there is 1 systematic review (SR) published for every 7 trials in the literature, that route is also problematic.
An attractive method of bridging multiple and wide-ranging research reports to everyday rehabilitation practice is the clinical practice guideline (CPG), defined by the Institute of Medicine in 2011 as “statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an appraisal of the benefits and harms of alternative care options.”
(p29) CPGs commonly make recommendations for screening, diagnosis and assessment, and treatment and management for a particular disorder as a package, rather than providing just one or a few components as the conventional SR does. CPGs are commonly developed by a team of clinicians, patients (or other stakeholders), methodologists, and other researchers, who make those recommendations. They are expected to improve quality of care and decrease variability in practices and costs. However, CPGs have been criticized almost from their start, as they are sometimes poorly developed and inadequately reported.
There is now a large number of instruments and tools to assist clinicians and other stakeholders in vetting available CPGs of interest in terms of the quality of their development process and the usefulness of the recommendations. In 2013, Siering et al
already counted 40 different appraisal tools. Developed by an international consortium and one of the few tools for which psychometric qualities have been studied, the one used most often is the Appraisal of Guidelines for Research & Evaluation version II (AGREE II). AGREE II can be used in developing a CPG, with the CPG authors using its 23 items to guide their writing, and in selecting a quality CPG, with a clinician or a team rating a guideline document on the same 23 items to develop a numerical quality score for 6 domains, making an overall quality rating and a decision on recommendation of the CPG. See appendix 1 for an overview of the instrument.
Many papers have been published in which the authors applied AGREE (version I or II) to available CPGs, with the aim of assisting clinicians in selecting a CPG to adopt. Based on domain scores and overall quality rating, they typically make a recommendation for each CPG available in an area of interest: avoid, use, or use after modification. One issue almost all of these papers have noted is the low score for the AGREE “applicability” domain, which “pertains to the likely barriers and facilitators to [CPG] implementation, strategies to improve uptake, and resource implications of applying the guideline (items 18-21).”
(p2701) Because rehabilitation is always a complex intervention, it would seem that applicability might be a greater problem for rehabilitation CPGs than others. The objective of this study was to review papers that used AGREE II to evaluate rehabilitation CPGs, and to present a synopsis of the quality of rehabilitation CPGs, specifically in terms of applicability.
Methods
This study was registered on PROSPERO with assigned number CRD42019124851.
Literature search
The following bibliographic databases were searched for the period from January 1, 2017 through July 9, 2019: Medline (Ovid), Cochrane Library Databases (Cochrane Library, Wiley), PsycINFO (Ovid), Embase (Embase.com), CINAHL Complete (EBSCO), and Web of Science (Clarivate). Search terms included the acronym and full name of the AGREE II tool; no language limits were applied. The full Medline search strategy is shown in appendix 2. All databases were searched on July 9, 2019. Retrieved records were organized and deduplicated using the bibliographic management software Endnote X9.a
Data selection
This is an overview study, synthesizing information from SRs that applied AGREE II to CPGs, which themselves are based on primary research in a particular area (fig 1). After deduplication, 449 abstracts remained which were reviewed independently by 2 researchers (pairs made up of I.W., T.A., J.F., A.M., and 2 other rehabilitation experts), who selected papers that appeared to use AGREE II to evaluate the quality of rehabilitation CPGs. The definition of rehabilitation used by the 2 reviewers was modified from one developed by Levack et al
: “an intervention provided by or prescribed by rehabilitation professionals to patients to improve their functioning, maximize their independence, prevent or manage secondary complications of a chronic, disabling health condition or to manage functional implications of a chronic health condition.” An operational definition is provided in appendix 3. Because most abstracts only mentioned a diagnostic group (eg, “CPGs for diabetes”) and focused on AGREE II results rather than on the nature of rehabilitation interventions, a conservative judgment resulted in the selection of 47 papers which were obtained in full text. Two researchers independently reviewed the full texts, using the following criteria:
(1)
AGREE II was used to evaluate existing CPGs, not to develop one.
(2)
The report was in English, Spanish, Portuguese, German, French, or Dutch, and was published in a peer-reviewed journal.
(3)
The 6 AGREE II domain scores and/or the 23 item scores were reported, in tables or supplemental digital content, for each CPG, or at least for each CPG that was considered by the review’s author(s) to be of adequate quality.
(4)
All or at least most of the CPGs rated involved rehabilitation as defined here.
(5)
The primary target of the CPGs was a rehabilitation clinician or other health care provider, not the patient or a family caregiver.
Fig 1Relationship of 4 levels of research: primary studies, clinical practice guidelines, systematic reviews of clinical practice guidelines, and overviews of systematic reviews.
Applying the definition of rehabilitation was difficult because the papers’ text frequently mentioned little more than the diagnostic group or health problems dealt with in the CPGs evaluated. The titles of the CPGs appraised were available but were not helpful in most instances, with labels along the line of “XYZ organization guideline for management of diabetes in adults.” The screeners needed to decide that at least 1 member of the rehabilitation team might have a role to play in at least 1 phase or aspect of assessing or treating the problem for which the CPGs made recommendations. In no instance did we consult the individual CPGs (level 2 in fig 1).
Data extraction
A 2-part Excelb form was created and piloted to extract all relevant data. Part 1, with 53 fields, focused on the paper evaluating CPGs (level 3) and included the items at the top of box 1. Part 2, with 46 fields, dealt with the CPGs themselves as reported in the review (level 2) and included the items at the bottom of box 1.
Part 1 focused on the paper evaluating clinical practice guidelines (CPGs) (level 2) and included:
(1)
Purpose of study
(2)
Rehabilitation topic area
(3)
Methods used for identifying rehabilitation CPGs
(4)
Criteria for inclusion of CPGs
(a)
Diagnostic group(s)
(b)
Age group(s)
(c)
Treatment vs diagnosis vs screening vs assessment vs management vs all clinical activities
(d)
Years of publication
(e)
Language(s) of publication
(f)
Process of CPG development
(g)
Nature of guidance document(s) produced
(h)
Other
(5)
Criteria for exclusion of CPGs
(6)
Method of screening of CPG titles/ abstracts for eligibility
(7)
Method of screening of CPG full texts for eligibility
(8)
Number of guidelines included and rated using AGREE II
(9)
Expertise of the AGREE II raters (clinical, research, AGREE II application and/or CPG development/ appraisal)
(10)
Number of AGREE II raters for each CPG
(11)
Efforts to come to a consensus between AGREE II raters, or at least eliminate large score discrepancies
(12)
Comments on interrater reliability of the AGREE II ratings
(13)
Observations/ conclusions/ recommendations regarding the specific set of guidelines evaluated
(14)
Observations/ conclusions/ recommendations regarding the ‘applicability’ of the CPGs evaluated
(15)
Comments on AGREE II itself and its suitability as a measure of CPG quality
Part 2 focused on each of the CPGs evaluated (level 3) and included:
(1)
Full title of the CPG
(2)
Bibliographic reference
(3)
Developing group, organization or authority
(4)
Year of publication
(5)
Geographic or political entity for which the CPG was written (eg, “Brazil”)
(6)
The qualifications of the CPG’s authors
(7)
The nature of the CPG’s intended users
(8)
The 6 AGREE II domain scores, if provided, and their mean, SD, and median across CPGs, if given.
(9)
The 23 AGREE II item scores, if provided, and their mean, SD, and median across CPGs, if given.
(10)
The overall AGREE II quality rating, on the 1 to 7 scale
(11)
The recommendation made with respect to the CPG, using the AGREE II standard terminology: “recommended,” “recommended with modification,” and “not recommended.”
(12)
The intraclass correlation coefficient, W or (weighted) kappa for agreement between raters, whether at the level of the individual CPGs, the individual items and domains, or both.
The extraction of information on papers (part 1) was done by several of the authors (I.W., J.F., T.A., and A.M.), and checked by another (M.D.). In the case of disagreement, discussion took place to resolve the issue. The extraction of CPGs (part 2) was done by one author (D.D.) and was checked by another (M.D.). Disagreements were resolved in a similar way. Many papers reported AGREE II Item scores and overall quality ratings on a percent scale, rather than a 1 to 7 scale. These were rescored using a conversion table that had separate columns for the cases of 2 raters to 7 raters.
Data processing and synthesis
All data were uploaded to SPSSc for processing and analysis. When papers presented AGREE II item scores but not domain scores, the latter were calculated based on the former, using the AGREE II formulas.
We found 40 papers, all in English, that provided AGREE II ratings for a total of 544 CPGs, with a range of 1 to 46 CPGs.
Characteristics of the papers reviewing CPGs
Standard bibliographic databases and guideline depositories were the most frequently used sources to find potential CPGs. Table 1 provides details on additional sources used. In selecting applicable CPGs, age groups were used by 20 reviews (child, n=4; adult, n=14; adult or child, n=2). A year of publication cutoff was used by 73% of the reviews. Most (n=28) appraised English-language CPGs only, but 3 accepted English or another specified language, and an additional 3 reviewed any language. One review accepted non-English languages only, and 5 did not specify the languages accepted. Criteria as to the process of CPG development or the nature of the guidance document, which are closely related items, were specified by 53% and 70%, respectively. Screening of abstracts was done by 2 or more researchers independently in 38% of the reviews (58% did not state anything about abstract screening), and 40% used 2 or more independent reviewers for full text screening (58% did not address the method of full-text screening).
Table 1Characteristics of the 40 studies appraising the quality of CPGs
Characteristics
%
Mean ± SD
Methods used to identify CPGs, searches of:
1. Bibliographic databases (eg, PubMed)
85
2. Grey literature databases (eg, SIGLE)
20
3. Guidelines depositories (eg, G-I-N)
78
4. Professional (rehabilitation) society websites
45
5. Provincial, state, or national government websites
The clinical expertise of the appraisers was stated, however minimally, in 30% of the reviews. We did not analyze degrees and departments given in the papers’ metadata. With respect to training in the AGREE II instrument (the AGREE Trust makes many training materials available), only 38% of papers mentioned it, however minimally. The AGREE II Manual recommends that each CPG is appraised by at least 2 reviewers, and preferably by 4. All reviews satisfied the requirement of 2; only 36% used 4 or more (see table 1). Steps to come to an agreement between appraisers in the case of disagreement on item, domain or overall quality scores (eg, discussion, use of a third person) were mentioned in 45% of the reviews. It was generally unclear whether the ratings provided on individual CPGs were from before or after such consensus building.
The average number of CPGs appraised ± SD was 13.6±11.5, ranging from 1 to 46, with a median of 8.5. Most reviews (78%) reported AGREE II domain scores only, 8% reported item scores only, and 15% reported both.
The Discussion sections of the 40 papers tended to call the quality of the CPGs appraised poor to mediocre, although there were exceptions.
How should clinicians rehabilitate patients after ACL reconstruction? A systematic review of clinical practice guidelines (CPGs) with a focus on quality appraisal (AGREE II).
It is unclear whether an overall CPG quality designation was associated with topic area or with rater severity. Many reviews noted variability in quality among the guidelines, and if an explanation was attempted, this often referred to years of publication (with more recent CPGs receiving better scores) or to the nature of the CPG development group, with specialized agencies (especially the National Institute for Health and Care Excellence [NICE]) being singled out as creating high-quality CPGs. Many reviews noted that patient input to guideline development was absent, or at least not mentioned.
Most reviews in their Discussion paid limited attention to the issue of applicability, beyond noting that the mean domain score for this area was the lowest of the 6 domains, and remarking that the CPGs did not offer the information listed in the four applicability domain items (items 18-21 in appendix 1). Only a few authors went beyond that, by making such suggestions as:
(1)
Providing a brief CPG summary, algorithm, and flow chart.
Exploration of the methodological quality and clinical usefulness of a cross-sectional sample of published guidance about exercise training and physical activity for the secondary prevention of coronary heart disease.
A systematic review of clinical practice guidelines on the use of low molecular weight heparin and fondaparinux for the treatment and prevention of venous thromboembolism: Implications for research and policy decision-making.
Professional organizations, whether discipline-focused or disorder-focused, created more CPGs than other groups distinguished here (table 2). The origin of the CPGs often was North America (37%) or Europe (26%), whether individual countries or combinations of countries. Almost 70% were published in 2010 or later, but 23% were from before that year, presumably because there was no recent update available.
Table 2Characteristics of the 544 CPGs appraised
Characteristics
%
Mean ± SD
Agency, group, or authority publishing the CPG
Discipline- or disorder-focused professional organization
45
University or hospital
3
CPG creating specialist entity (eg, NICE, SIGN)
11
Provincial, state, or national health department
5
Not stated, combination, other, unclear
36
Continent of origin (single country or combination of counties)
North America
37
Europe
26
Australia and New Zealand
5
Asia
3
South America
2
Africa
<1
Combination
3
Not reported
24
Year of publication
1994-2009
23
2010-2012
25
2013-2014
23
2015-2019
21
Not reported
8
ICC for appraiser agreement on AGREE II Domains (8 studies with 135 CPGs; study mean ± SD across 8 studies shown)
Eight papers reported the intraclass correlation coefficient (ICC) for agreement between appraisers on each of the domain scores across 135 CPGs. ICCs ranged from 0.73 to 0.90 (see table 2). When the medians across the 8 papers were calculated, the values were very similar. Five papers reported ICC values across the 6 domains, for each of 41 CPGs. ICCs ranged from 0.47 to 0.98, with a mean ± SD of 0.82±0.12 (median, 0.85).
The mean scores across 544 CPGs for the 6 AGREE II domains ranged from 34 (applicability) to 72 (scope and purpose), with quite some variation in each domain. The data in table 3 indicate that applicability received the lowest mean score in all reviews but 2. Typical mean scores range between 40 and 80, with the only stark exception the paper by Guo et al.
Exploration of the methodological quality and clinical usefulness of a cross-sectional sample of published guidance about exercise training and physical activity for the secondary prevention of coronary heart disease.
How should clinicians rehabilitate patients after ACL reconstruction? A systematic review of clinical practice guidelines (CPGs) with a focus on quality appraisal (AGREE II).
Quality assessment of recent evidence-based clinical practice guidelines for management of type 2 diabetes mellitus in adults using the AGREE II instrument.
Is the development of clinical practice guidelines for non-neurogenic overactive bladder trustworthy? A critical appraisal using the Appraisal of Guidelines, Research and Evaluation (AGREE) II instrument.
A critical appraisal of the quality of low back pain practice guidelines using the AGREE II tool and comparison with previous evaluations: a EuroAIM initiative.
An umbrella review of clinical practice guidelines for the management of patients with hip fractures and a synthesis of recommendations for the pre-operative period.
A systematic review of clinical practice guidelines on the use of low molecular weight heparin and fondaparinux for the treatment and prevention of venous thromboembolism: Implications for research and policy decision-making.
A systematic critical appraisal of evidence-based clinical practice guidelines for the rehabilitation of children with moderate or severe acquired brain injury.
Clinical practice guidelines for treating headache with Traditional Chinese Medicine: quality assessment with the appraisal of guidelines for research and evaluation Ⅱ instrument.
The Global Spine Care Initiative: a systematic review for the assessment of spine-related complaints in populations with limited resources and in low- and middle-income communities.
Methodological quality of clinical practice guidelines with physical activity recommendations for people diagnosed with cancer: a systematic critical appraisal using the AGREE II tool.
Assessing the quality of the OPQ's Guidelines for the Evaluation of Dyslexia in Children using the Appraisal of Guidelines for Research and Evaluation (AGREE) II: a brief report.
Clinical practice guidelines for the noninvasive management of low back pain: a systematic review by the Ontario Protocol for Traffic Injury Management (OPTIMa) Collaboration.
The AGREE II overall CPG quality rating was only available for 306 CPGs (56% of the total) (see table 2); it had an approximately normal distribution, with a mean ± SD of 4.6±1.4 on the 1 to 7 scale. A recommendation for a CPG was only available for 139 CPGs (25% of the total). Only 36% were recommended to be used as published; for a further 44%, the recommendation was conditional on modification. Lastly, the average ratings for the 4 applicability items, available for 67 CPGs (12%), were poor all around (see table 2).
Figure 2 suggests that there is a trend toward improvement in guidelines, especially in the domains of stakeholder involvement, clarity of presentation, and editorial independence. The data presented in figure 3 indicate that whatever the domain, the best guidelines are produced by agencies such as NICE that specialize in CPG development. These CPGs are rated as better than those produced by professional organizations, hospitals, or government agencies, especially in the domains of stakeholder involvement, rigor of development, applicability, and editorial independence.
Fig 2Mean domain scores by year of clinical practice guideline publication. D1, scope and purpose; D2, stakeholder involvement; D3, rigor of development; D4, clarity of presentation; D5, applicability; and D6, editorial independence.
Fig 3Mean domain scores by nature of the organization or group that produced the clinical practice guideline. D1, scope and purpose; D2, stakeholder involvement; D3, rigor of development; D4, clarity of presentation; D5, applicability; and D6, editorial independence.
The AGREE II Manual defines guideline quality as “the confidence that the potential biases of guideline development have been addressed adequately and that the recommendations are both internally and externally valid, and are feasible for practice.”
(p0) Our findings suggest that rehabilitation CPGs as a whole have not yet achieved high quality, especially in the area of applicability. Lack of consideration of the facilitators of and barriers to application, advice and tools for putting recommendations into practice, specification of resource implications, and monitoring and auditing tools and criteria are some of the key issues.
This conclusion is based on the AGREE II domain and other ratings (overall quality appraisal and recommendation) that were extracted from the level 3 reviews, and on the observations and conclusions these papers offered, each based on the 1 or multiple CPGs they assessed.
Although some authors, especially those who considered a small number of CPGs, judged the guidelines they evaluated of adequate quality, the majority, especially authors who appraised 10 or more CPGs, found multiple problems. The average domain scores across 544 CPGs in table 2 suggest the same: even the highest-rated domains (scope and purpose and clarity of presentation), with a score just above 70, are far from “very satisfactory.” At 34, the mean score for applicability indicates that the typical rehabilitation CPG has major problems in this area. Unfortunately, many reviews did not report all the standard AGREE II items, or did not report them in the standard format. Therefore, we have overall quality ratings for only 56% of the CPGs, and recommendations for only 26%. However, on the 1 to 7 “overall quality” scale, only 30% were scored a 6 or 7 (with 7 representing the “highest possible quality”), and only 36% of CPGs were recommended without modification.
Our findings correspond with the results from earlier similar overview studies (not focused on rehabilitation) concluding that CPGs often are of low quality, especially in terms of applicability, and cannot be recommended, even with modifications, in 18% to 38% of cases. Table 4 summarizes the findings of these level 4 studies. Between them, they addressed more than 1000 CPGs. There is presumably some overlap, so the numbers in the CPG column cannot be added. The average domain scores are less than 100 throughout. With 1 exception (Knai et al
), applicability received the lowest mean score. It is not clear that there has been progress in the almost 40 years that CPGs have been published. Alonso et al
who focused on applicability. Our data suggest improvement, but not in all domains, and a very limited improvement in applicability, which is the area that is presumably more difficult in rehabilitation than in other health care fields.
Table 4Summary of the findings of 6 previous reviews of CPG appraisal articles
Guidelines are protracted and expensive to develop, and even excellent ones may fail to be utilized for a number of reasons, both intrinsic (ie, a consequence of the CPG itself) and extrinsic (ie, a consequence of the structure of health care organizations and the knowledge and attitudes of providers). “Leakage” in their use may be related to: (1) clinicians’ awareness of what is recommended, (2) their agreement with the recommendations, (3) actual adoption in practice, and (4) long-term follow-through with all patients. Leakage in the utilization of CPGs has been studied insufficiently,
but it is clear that it contributes to the waste that characterizes the research in health care.
There are several resources available to rehabilitation experts who want to issue a new CPG or update an old CPG (see box 2 for an incomplete listing). Another positive development is the recent explosion of implementation science, “the scientific study of methods to promote the uptake of research findings.”
(p1) Although clinical research focuses on what is to be implemented, implementation science concentrates on shortening the pipeline from research to implementation, which is often stated to be 17 years.
For the results of individual primary studies and the SR findings underlying CPGs alike, it develops and evaluates interventions to prevent leakage. Lynch et al
The Guideline Implementability Decision Excellence Model (GUIDE-M): a mixed methods approach to create an international resource to advance the practice guideline field.
There are 21 questions to be applied to each individual recommendation in the CPG, rather than to all recommendations combined, as in AGREE II and other CPG appraisal tools.
(6)
Mazza offers a taxonomy of implementation strategies and refers to 2 others.
We used a functional definition of rehabilitation, which was applied to level 3 papers that often provided minimal information on the level 2 documents (the CPGs) and the degree that rehabilitation was involved in their recommendations. In these level 3 studies, there was a fairly high frequency of missing values for AGREE II items, agency, country, overall AGREE II rating, and recommendation for a CPG. The level 3 papers tended to focus on the AGREE II scores per se rather than on deficits in applicability and ways of improving it. Lastly, these studies often did not use the standardized AGREE II procedures or scales for item, overall quality rating, and CPG recommendation, forcing us to recalculate, often making assumptions that were uncertain.
Conclusions
In principle, CPGs are an ideal means to move knowledge obtained from clinical research into rehabilitation practice. However, our review of reviews shows that rehabilitation CPGs commonly have deficits, especially in terms of applicability. Much work needs to be done by guideline developers to make it easier for the average rehabilitation organization and clinician to implement CPGs in daily practice.
Acknowledgments
We thank Jennifer Moore, PT, DHS, NCS, and Patricia Heyn, PhD, FACRM, FGSA, who contributed to early phases of this research.
Appendix 1 The AGREE II Instrument
The following are the 23 AGREE II items, and the domain to which they contribute. The items are scored on a 1 to 7 scale, where 1 is the poorest possible quality and 7 is the best possible quality. There are no descriptors for values 2 to 6.
Domain 1. Scope and purpose
(1)
The overall objective(s) of the guideline is (are) specifically described.
(2)
The health question(s) covered by the guideline is (are) specifically described.
(3)
The population (patients, public, etc) to whom the guideline is meant to apply is specifically described.
Domain 2. Stakeholder involvement
(4)
The guideline development group includes individuals from all relevant professional groups.
(5)
The views and preferences of the target population (patients, public, etc.) have been sought.
(6)
The target users of the guideline are clearly defined.
Domain 3. Rigor of development
(7)
Systematic methods were used to search for evidence.
(8)
The criteria for selecting the evidence are clearly described.
(9)
The strengths and limitations of the body of evidence are clearly described.
(10)
The methods for formulating the recommendations are clearly described.
(11)
The health benefits, side effects, and risks have been considered in formulating the recommendations.
(12)
There is an explicit link between the recommendations and the supporting evidence.
(13)
The guideline has been externally reviewed by experts prior to its publication.
(14)
A procedure for updating the guideline is provided.
Domain 4. Clarity of presentation
(15)
The recommendations are specific and unambiguous.
(16)
The different options for management of the condition or health issue are clearly presented.
(17)
Key recommendations are easily identifiable.
Domain 5. Applicability
(18)
The guideline describes facilitators and barriers to its application.
(19)
The guideline provides advice and/or tools on how the recommendations can be put into practice.
(20)
The potential resource implications of applying the recommendations have been considered.
(21)
The guideline presents monitoring and/or auditing criteria.
Domain 6. Editorial independence
(22)
The views of the funding body have not influenced the content of the guideline.
(23)
Competing interests of guideline development group members have been recorded and addressed.
Domain scores are calculated by averaging the 2 to 8 item ratings within a domain across the 2 or more appraisers, and expressing the result on a percentage scale, where 0% means all appraisers assigned a score of 1 on all relevant items, and 100% means all appraisers gave a score of 7 on all relevant items.
In addition, AGREE II offers the following 2 “overall quality” judgements:
(1)
Rating of the overall quality of a guideline on a 1 to 7 scale, with the extremes meaning 1 equals the lowest possible quality and 7 equals the highest possible quality.
(2)
Recommendation of a guideline for use: yes, yes with modifications, or no.
Appendix 2 Ovid Medline Search Strategy (Medline Version: "Epub Ahead of Print, In-Process & Other Non-Indexed Citations, Daily and Versions")
(1)
"AGREE II".mp.
(2)
(AGREE adj II).mp.
(3)
"AGREE 2".mp.
(4)
(AGREE adj "2").mp.
(5)
"Appraisal of Guidelines Research and Evaluation II".mp.
(6)
"Appraisal of Guidelines Research and Evaluation 2".mp.
(7)
Or/1-6
(8)
limit 7 to yr="2017-2019"
Appendix 3 Operational Definition of “Rehabilitation” Used in Abstract and Full Text Screening
Inclusion criteria: clinical services provided by or prescribed by rehabilitation professionals, including, but not limited to:
(1)
Physical therapist/physiotherapist
(2)
Occupational therapist
(3)
Kinesiotherapist
(4)
Speech (-language) therapist
(5)
Recreation therapist
(6)
Rehabilitation psychologist
(7)
Neuropsychologist
(8)
Vocational counselor
(9)
Rehabilitation counselor
(10)
Rehabilitation nurse
(11)
Orthotist, prosthetist
(12)
Rehabilitation engineer
(13)
Physiatrist, PM&R physician
to
(1)
Enable people with disabilities to attain or maintain their maximum independence; to improve their physical, mental, social, and vocational ability; or to improve their inclusion and participation in one or more aspects of their life.
(2)
Prevent secondary health conditions or complications arising from a primary, disabling health condition (eg, interventions to prevent poststroke depression).
(3)
Manage a chronic health condition, when a goal of the intervention is explicitly to address functional abilities (eg, pulmonary rehabilitation for chronic lung disease with the goal of improving physical function was included).
Exclusion criteria: CPGs of interventions:
(1)
Prevention of first time, disabling health conditions (eg, prevention of first-time stroke)
(2)
First aid, paramedical, and emergency care.
(3)
Acute medical management of health conditions unless a goal of the intervention was explicitly to address functional abilities (eg, early mobilization after stroke).
(4)
Surgical management of impairments of body structure, except where a rehabilitation professional is involved or where surgical care is compared with non-surgical (rehabilitation) management.
(5)
Not currently within the scope of the practice of rehabilitation professionals in the state(s)/country(ies) covered by the CPG (eg, homeopathy, invasive procedures for deep brain stimulation, hyperbaric oxygen therapy, specialized medicine prescribed by a specialist other than a rehabilitation physician, such as a rheumatologist or neurologist).
(6)
Services for people with mental health disorders as the disabling condition, where the primary outcome is reduction in psychiatric impairment (eg, mood, psychosis).
(7)
Arrangements not specific to rehabilitation, but potentially relevant to all health services (eg, interventions to reduce non-attendance rates).
Suppliers
a.
Endnote X8; Clarivate Analytics.
b.
Excel Microsoft Office 365 ProPlus; Microsoft Corp.
c.
SPSS, version 25.0; IBM Corp.
References
Dijkers M.P.
Murphy S.L.
Krellman J.
Evidence-based practice for rehabilitation professionals: concepts and controversies.
How should clinicians rehabilitate patients after ACL reconstruction? A systematic review of clinical practice guidelines (CPGs) with a focus on quality appraisal (AGREE II).
Exploration of the methodological quality and clinical usefulness of a cross-sectional sample of published guidance about exercise training and physical activity for the secondary prevention of coronary heart disease.
A systematic review of clinical practice guidelines on the use of low molecular weight heparin and fondaparinux for the treatment and prevention of venous thromboembolism: Implications for research and policy decision-making.
Quality assessment of recent evidence-based clinical practice guidelines for management of type 2 diabetes mellitus in adults using the AGREE II instrument.
Is the development of clinical practice guidelines for non-neurogenic overactive bladder trustworthy? A critical appraisal using the Appraisal of Guidelines, Research and Evaluation (AGREE) II instrument.
A critical appraisal of the quality of low back pain practice guidelines using the AGREE II tool and comparison with previous evaluations: a EuroAIM initiative.
An umbrella review of clinical practice guidelines for the management of patients with hip fractures and a synthesis of recommendations for the pre-operative period.
A systematic critical appraisal of evidence-based clinical practice guidelines for the rehabilitation of children with moderate or severe acquired brain injury.
Clinical practice guidelines for treating headache with Traditional Chinese Medicine: quality assessment with the appraisal of guidelines for research and evaluation Ⅱ instrument.
The Global Spine Care Initiative: a systematic review for the assessment of spine-related complaints in populations with limited resources and in low- and middle-income communities.
Methodological quality of clinical practice guidelines with physical activity recommendations for people diagnosed with cancer: a systematic critical appraisal using the AGREE II tool.
Assessing the quality of the OPQ's Guidelines for the Evaluation of Dyslexia in Children using the Appraisal of Guidelines for Research and Evaluation (AGREE) II: a brief report.
Clinical practice guidelines for the noninvasive management of low back pain: a systematic review by the Ontario Protocol for Traffic Injury Management (OPTIMa) Collaboration.
The Guideline Implementability Decision Excellence Model (GUIDE-M): a mixed methods approach to create an international resource to advance the practice guideline field.
Supported by the National Institute on Disability, Independent Living, and Rehabilitation Research (NIDILRR; grant no. 90DPKT0001). NIDILRR is a Center within the Administration for Community Living (ACL), Department of Health and Human Services (HHS). The contents of this article do not necessarily represent the policy of NIDILRR, ACL, HHS, and you should not assume endorsement by the Federal Government. M.D. and D.D. were supported by the American Institutes for Research's Center on Knowledge Translation for Disability and Rehabilitation Research, which is supported by NIDILRR. This research was suggested by participants in the Center’s Community of Practice on Evidence for Disability and Rehabilitation Research.
30266-5&id=gr1.jpg){kind=link}
30266-5&id=gr2.jpg){kind=link}
30266-5&id=gr3.jpg){kind=link}