If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
The overall objectives of this project were to implement and sustain use of a gait assessment battery (GAB) that included the Berg Balance Scale, 10-meter walk test, and 6-minute walk test during inpatient stroke rehabilitation. The study objective was to assess the effect of the study intervention on clinician adherence to the recommendations and its effect on clinician perceptions and the organization.
Design
Pre- and post-training intervention study.
Setting
Subacute inpatient rehabilitation facility.
Participants
Physical therapists (N=6) and physical therapist assistants (N=2).
Intervention
The intervention comprised a bundle of activities, including codeveloping and executing the plan with clinicians and leaders. The multicomponent implementation plan was based on the Knowledge-to-Action Framework and included implementation facilitation, implementation leadership, and a bundle of knowledge translation interventions that targeted barriers. Implementation was an iterative process in which results from one implementation phase informed planning of the next phase.
Main Outcome Measures
Clinician administration adherence, surveys of perceptions, and organizational outcomes.
Results
Initial adherence to the GAB was 46% and increased to more than 85% after 6 months. These adherence levels remained consistent 48 months after implementation. Clinician perceptions of measure use were initially high (>63%), with significant improvements in knowledge and use of one measure after implementation.
Conclusions
We successfully implemented the assessment battery with high levels of adherence to recommendations, likely because of using the bundle of knowledge translation activities, facilitation, and use of a framework to codevelop the plan. These changes in practice were sustainable, as determined by a 4-year follow-up.
Measurement is integral to physical rehabilitation, as it can determine a patient’s outcomes, assess the effect of care, increase patient engagement, and determine the value of rehabilitation.
Smith M. Saunders R. Stuckhardt L. McGinnis J.M. Best care at a lower cost: the path to continuously learning health care in America. The National Acadamies Press,
Washington DC2013
In this system, measures assess patients’ perspectives and improve care, increase the transparency of outcomes, link clinicians’ performance to patient outcomes and benchmarks, strengthen public health, and generate knowledge.
Smith M. Saunders R. Stuckhardt L. McGinnis J.M. Best care at a lower cost: the path to continuously learning health care in America. The National Acadamies Press,
Washington DC2013
The American Physical Therapy Association initiated task forces in 2009 to develop measurement recommendations and, more recently, published a clinical practice guideline on a core set of measures, including gait assessments, to be used at admission, discharge, and, whenever possible, in between these periods.
Outcome measures for persons with moderate to severe traumatic brain injury: recommendations from the American Physical Therapy Association Academy of Neurologic Physical Therapy TBI EDGE Task Force.
Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association.
Individual clinician-level barriers and facilitators include the perceived value of measurement, as well as the knowledge, skill, and educational level of the practitioner.
Clinicians’ perceptions of patient-related barriers include concerns of sharing measurement information with patients, questioning the relevance of measures, and the potential for a negative effect if measures fail to show progress.
The field of knowledge translation (KT) aims to identify efficient and effective methods to implement evidence into clinical practice. KT is the dynamic and iterative process that includes synthesis, dissemination, exchange, and ethically-sound application of knowledge to improve health, provide more effective health services and products, and strengthen the health care system.
A recent systematic review assessed the types and effectiveness of KT interventions (ie, strategies to change behavior and promote adoption) to increase knowledge of, improve attitudes toward, and enhance the use of standardized outcome measures in the field of rehabilitation.
Of 11 studies, 9 used educational strategies (eg, educational workshops or seminars) and 2 used indirect strategies (eg, providing resources and written guidelines) to disseminate measurement information. Five studies combined 3 or more strategies, 3 studies applied audit and feedback, and 1 study used knowledge brokers (ie, individuals to guide and support KT processes).
Nine of the 10 studies that assessed rates of measurement use reported improvements in perceived (n=5) or actual (n=4) use. KT theory informed only 5 studies in this review, and all studies targeted individual clinicians (vs organizations). Two studies reported on sustainability.
reported that effects were sustained for 3 of 4 implemented measures after 12 and 18 months based on clinician report of “high” amounts by approximately 60% to 85% of survey respondents. Schreiber et al
The use of a knowledge translation program to increase use of standardized outcome measures in an outpatient pediatric physical therapy clinic: administrative case report.
documented an increase in the frequency of 5 of 6 implemented measures and sustained over 8 months. However, the authors did not provide the percentage of patients assessed with these measures.
The use of a knowledge translation program to increase use of standardized outcome measures in an outpatient pediatric physical therapy clinic: administrative case report.
The Knowledge-to-Action Cycle (KTA), a planned-action framework, is the product of a metasynthesis of more than 30 planned action models. This framework is composed of 7 phases to implement evidence into clinical practice and includes the knowledge creation funnel at the center, which represents the publication, synthesis of research, and knowledge products and tools such as practice guidelines.
The overarching goal of this KT project was to implement an evidence-based gait assessment battery (GAB) with high levels of adherence in inpatient stroke rehabilitation and determine the sustainability of this new practice. The intervention included codeveloping and executing an implementation plan with clinicians and leaders. The multicomponent implementation plan was based on the KTA Framework and included implementation facilitation, implementation leadership, and a bundle of KT interventions that targeted barriers. Implementation was an iterative process in which results from one implementation phase informed planning of the next phase. The study objective was to assess the effect of the study intervention on clinician adherence to the recommendations and its effect on clinician perceptions and the organization.
Methods
We implemented the GAB at Mary Free Bed Rehabilitation Hospital (MFB) in Grand Rapids, Michigan. This nonprofit hospital has 119 acute rehabilitation and 48 skilled nursing rehabilitation beds. In 2015, there were 342 stroke admissions, 6 physical therapists (4.0 full-time positions), and 2 physical therapist assistants (1.0 full-time position) on the stroke unit. Clinicians reported using measures at their discretion before implementation, but measures varied among clinicians.
A primary goal of this project was to implement a GAB into routine clinical practice on a stroke rehabilitation unit. However, patients provided informed consent for research use of the GAB results. Inclusion criteria were adult patients (aged <90y) with subacute stroke (<2mo poststroke) and goals to improve walking. If patients were unable to provide informed consent, a legally authorized representative could provide it. Exclusion criteria included an inability to ambulate more than 50 meters before the stroke. Clinicians provided informed consent before taking online surveys. Inclusion criteria for clinicians included full- or part-time physical therapist or physical therapist assistant on the MFB stroke unit. The Northwestern University, Indiana University (IU), and MFB Institutional Review Boards approved the project.
The KTA Framework guided the development of the KT plan, and the methods are described according to the KTA action cycle phases (see subheadings). Integrated KT was a core principle of the KT plan, which is “a model of collaborative research, where researchers work with knowledge users who identify a problem and have the authority to implement the research recommendations.”
(p299) The research, leadership, and clinical teams worked collaboratively on the development and implementation of the KT plan. The team also considered issues that might affect sustainability of the new practice at the beginning of and throughout the project. When possible, the team integrated the KT interventions into standard organizational processes (ie, team conference reporting, performance goals). The team used an iterative approach to implement the plan and repeated phases until the achievement of desired results occurred. In addition, results from early phases informed the activities of later phases. Therefore, the KTA phase activities and associated results are presented here in the methods (table 1). The results section presents the implementation outcomes.
Table 1Implementation plan and results with activities described according to each phase of the KTA cycle
KTA Phase
Methods for Each Phase
Results
Identify problem; determine the know-do gap; identify, review, and select knowledge
on perceptions, barriers, and facilitators related to standardized measures and informal discussions
2.
Selected knowledge (ie, gait and balance assessments) for implementation
1.
Assessment of know-do gap: positive perception of measures, several measures used at the therapists’ discretion, but none were required or standardized
2.
Knowledge selected: BBS, 10MWT, 6MWT
Adapt knowledge to local context
1.
Review of current evidence for GAB in subacute patient populations
2.
Adaptation of the standardized administration procedures to fit into local context; recommendations for adaptations made by clinicians, administrators, and researchers
Recommendations for:
1.
Use of GAB in subacute stroke to support interpretation of results
2.
Administration procedures: a. Administration protocol adapted to the MFB context
b.
Administration timing (within 72 h of admission, weekly, and at discharge)
c.
Documentation recommendations
Assess barriers and facilitators to knowledge use
1.
Survey to MFB clinicians that included adapted survey
Informal discussions about barriers and facilitators
3.
An iterative process of barrier and facilitator assessment, implementation of KT interventions, and monitoring occurred for 6 months until adherence consistently achieved >85%
1.
Barriers included knowledge, skills, social influences, and environmental context and resources (see table 3)
Select, tailor, implement interventions
1.
Barriers were categorized according to the Theoretical Domains Framework and KT interventions were selected
Design of KT interventions codeveloped with the MFB clinicians and research team
1.
A multi-component KT intervention was delivered that included educational interventions, leadership support, process changes, audit and feedback, purchasing of equipment, and environmental modifications (see table 3)
Monitor knowledge use
1.
Monitored conceptual use with previously administered surveys
2.
Monitored instrumental use with audits and during team conference reporting
1.
Survey results demonstrated statistically significant increase in use of the 10MWT
2.
>85% adherence achieved and sustained after 6 mo
Evaluate outcomes
1.
Clinician outcomes assessed using previously administered surveys
2.
Organizational outcomes assessed anecdotally
1.
Survey results demonstrated statistically significant difference identified in clinicians’ knowledge and skill related to administration of measures
Process changes: ongoing monitoring of adherence, modification of electronic health record, weekly reporting in team conference
2.
Staff: adopted standardized testing day, standardization processes integrated into new hire orientation, practice incorporated into performance appraisals and job description
3.
Organization: incorporated into the goals/vision, implemented throughout system of care
1.
All KT interventions phased out, with the exception of scanning of assessment form into medical record.
2.
>85% adherence continued until last audit at 24 mo
3.
100% of clinicians agreed or strongly agreed on the survey that the project increase value of GAB and the culture is data-driven
NOTE. The activities describe the methods used for each phase, and the results column provides an overview of the findings from each KTA phase.
Identify the problem, assess know-do gap, select knowledge to implement
The research and clinical team discussed appropriate measurement areas and the average impairment level of the patients at admission to identify an appropriate GAB. An important consideration in measurement selection was the team’s future plans to implement a high-intensity gait training program. Therefore, the measures should target areas of function that improve with this intervention. The clinical team reported that patients often required moderate to total assistance (ie, 50%-100% assistance) for mobility and balance when admitted to inpatient rehabilitation. Therefore, we aimed to select measures that assess static and dynamic sitting and standing balance, independence with walking, gait speed, and walking distance. Measurement selection was based on the measures’ potential to demonstrate change in patients with the typical level of function at admission. In addition, the measure needed excellent psychometrics and clinical utility. The research team evaluated recommendations made by the American Physical Therapy Association StrokeEDGE group
Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association.
Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association.
Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association.
Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association.
Two years later, the Core Set of Outcome Measures for Neurologic Clinical Practice also recommended these measures for clinical practice, which further supports the measures selected for implementation.
Once selected, the research team reviewed the GAB with the clinical and leadership team and did not recommend changes. To formally assess perceptions, barriers, and current use of outcome measures in clinical practice (ie, the know-do gap), we used a previously validated survey
with additional questions about the selected measures (table 2). We administered the online surveysa before implementation (2015), and 1 and 3 years after implementation.
Table 2Investigator developed survey questions
Questions Added to the 2015, 2016, and 2018 Survey, With the 2015 Results Described
Please read the following questions on the effect of the FIRST project on your measurement related practice. Indicate whether you completely disagree (1), disagree (2), are neutral (3, neither agree nor disagree), agree (4), or completely agree (5).
1. I am familiar with the Berg Balance Scale
4.5 (4.0-5.0)
2. I use the Berg Balance Scale in routine practice
5.0 (4.0-5.0)
3. I am familiar with the 10-meter walk test
4.0 (3.25-4.0)
4. I use the 10-meter walk test as part of routine practice
3.0 (2.0-4.0)
5. I am familiar with the 6-minute walk test
4.0 (4.0-5.0)
6. I use the 6-minute walk test as part of routine practice
4.0 (4.0-4.75)
Questions Added to the 2018 Survey, With 2018 Results Described
Please read the following questions on the effect of the project on your measurement related practice. Indicate whether you completely disagree (1), disagree (2), are neutral (3, neither agree nor disagree), agree (4), or completely agree (5).
1. As a result of this project, I have increased the use of outcome measures in my clinical practice.
5.0 (4.25-5.0)
2. As a result of this project, I use the outcome measure results to guide my clinical decision-making.
5.0 (4.0-5.0)
3. As a result of this project, I have more discussions with my patients about their outcome measurement results.
5.0 (4.0-5.0)
4. As a result of this project, I have more discussions with colleagues about outcome measurement results.
4.5 (4.0-5.0)
5. As a result of this project, the culture in our department has shifted to discuss patient-related data (ie, outcome measurement results) instead of patient observations (ie, patient walks slowly, has poor balance, etc).
4.0 (4.0-5.0)
6. As a result of this project, I better understand the value outcome measures add to clinical practice.
5.0 (4.0-5.0)
7. After this project is over, I plan to use outcome measures at the same or higher frequency that I currently use them.
Adapt the knowledge about the GAB to the local context
The research and clinical team reviewed and adapted standardization procedures for application in the MFB context. Adaptations included recommendations for GAB administration within 72 hours of admission, weekly, and at discharge. Measurement-specific adaptations included standing on the weaker limb during the BBS single-limb support and tandem items,
The Organizational Readiness to Implement Change (ORIC) questionnaire assessed the clinicians’ perception of the organization’s readiness for this change.
Areas assessed included the staff’s confidence, motivation, and determination to implement the GAB. The ORIC identified relevant barriers and organizational readiness for change, and guided the selection of organizational level KT interventions.
Informal barrier assessments during meetings and daily interactions with clinicians occurred until adherence levels reached more than 85%. The research team categorized the barriers and facilitators using the theoretical domains framework, which is a comprehensive, theory-informed approach to identify determinants of behavior.
Barriers identified by clinicians included a lack of knowledge and skills to administer the tests and incorporate results into clinical decision-making. Clinicians also reported conflicts between the recommended practice and current beliefs. Some examples are the belief that prioritizing assessments would decrease the time available for treatment and that assessment should not occur on patients who might perform poorly (eg, BBS ≤5). Other barriers included the environment and resources, such as the lack of equipment and space. One clinician stated, “No [barriers]….just need the motivation.” The identified categories of barriers were knowledge, skills, social influences, and environmental context and resources (table 3).
Table 3Barrier categories and knowledge translation interventions.
Barrier
KT Intervention
Knowledge and skills
Education sessions: training and standardization of GAB Consultations, structured meeting, and informal discussion Education session on interpretation of results Cheat sheet developed for SEMs, MDCs, goal writing Annual standardization
Social influences
Leadership support (articulation of clear expectations and reinforcement) Required reporting at team conference Specific day for all PTs to perform outcome measures Audit and feedback Rewards
Environmental context and resources
Purchase equipment (measuring wheels, 10-meter rope, stopwatch, etc) Research assistant support for data collection, timing, putting equipment out for easy access, and reminders Development of data collection forms, scanned into EMR
Facilitators included organizational, social, individual, and financial factors. MFB’s vision included the integration of outcome measures and the creation of a learning health system, which aligned with the goals of this project and could serve as an organizational KT strategy. Key stakeholders, including the leaders, clinicians, and MFB researchers, were also engaged and actively involved in the project. Lastly, funding supported personnel, training, and equipment.
Select and tailor KT interventions to implement the GAB
The research team and clinicians codeveloped the multicomponent KT plan that targeted the identified barriers. Using the Theoretical Domains Framework
to map KT interventions to barriers, we selected interventions that were deemed feasible and suitable in the MFB context. To target barriers of knowledge and skills, we held an educational session on measurement concepts, psychometric properties, and clinical utility of the GAB, and standardization of its administration (see appendix 1 for details). Clinicians completed a post-test by rating patients on a video demonstration of the assessments. Clinician scores averaged 96% initially and 89% at 1 year later. Items that had lower levels of correct responses were clarified. The research-clinician team developed a resource sheet to assist in test interpretation and inclusion of the GAB in goal writing. Clinicians also received training on discussing GAB results with patients.
The KT interventions that targeted social influences included leadership support, team conference reporting, audit and feedback,
and the determination of an administration day. The Research Director, Stroke Manager, and research team collaboratively set an adherence goal of administration of greater than 85% of expected assessments, and clinicians reported that the goal was feasible. The leadership team required reporting of GAB results at the weekly team conference. They also reinforced these expectations often and provided rewards (eg, catered lunches, baked goods) for the achievement of desired adherence levels. The research team audited charts and provided feedback on adherence every 4 to 6 weeks. This process consisted of data extraction from each patient’s chart (ie, assessment collected: yes or no), calculation of percentage administered (number of yes and total number of expected assessments), and reporting back to the team by e-mail or team meeting. E-mails were sent to the clinicians by the MFB research team with a copy to the unit manager. Adherence rates of less than 85% triggered additional barrier assessments and KT interventions. A full description of the audit and feedback intervention
is available in appendix 2. Lastly, adaptations to the administration timing recommendations occurred to implement a standard weekly interim testing day (nicknamed “Testing Tuesday”).
The research-clinician team secured several environmental resources. Equipment, including measuring wheels and sticks, were placed in the gym. MFB staff painted lines to mark distances for the 10MWT on baseboards. The research assistant reminded clinicians about testing and assisted in data collection, timing, and setting up equipment. The research-clinical team codeveloped data collection forms that were scanned into the medical record to minimize documentation burden.
Motivation was identified as a barrier and targeted by KT interventions that also targeted other barriers. Specifically, we stated a goal of greater than 85% adherence and provided information about the outcome of the behavior change interventions (ie, audit and feedback). We also provided rewards and used social processes, such as team conferences, to increase motivation for behavior change.
During the first 6 months, we used an iterative process that included barrier assessment, codevelopment of KT interventions, and monitoring until adherence was consistently greater than 85%. We stopped providing feedback after 14 months but audited for 24 months. When adherence consistently achieved greater than 85%, the research team phased out the KT interventions. Appendix 1 describes the KT interventions according to Workgroup for Intervention Development and Evaluation Research criteria.
Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations.
Monitor knowledge use of the GAB and assess outcomes (Effect)
The research team monitored the GAB throughout the 24-month project. We assessed conceptual knowledge use by readministering the initial surveys on perceptions, barriers, and use of outcome measures before and at 1 and 3 years after implementation. The research team monitored instrumental or actual use (adherence to GAB) by conducting chart audits and sharing the adherence rates with the leadership and clinical teams. The ORIC questionnaire assessed the effect of the project on the organization’s confidence, motivation, and ability to implement the change. The leadership team also performed a nonsystematic observation of organization impact.
The research-clinician team integrated the GAB into routine clinical and organizational processes using a number of sustainability interventions. Clinicians routinely report GAB results and interpretation during team conferences, and training on the GAB is a core component of new hire orientation. The electronic medical record incorporated GAB data fields. The leadership team integrated the GAB into organizational goals and vision. Managers wrote adherence performance appraisal goals and added the GAB to job descriptions. A follow-up chart audit assessed sustainability at 48 months after implementation.
Statistical analysis
Descriptive statistics were used to described clinician demographics. Adherence data were collected, and a monthly average was generated for 24 months and at a 48-month follow-up. Friedman’s test determined differences in baseline (2015), early-implementation (2016), and late implementation (2018) survey data. For statistically significant results, pairwise comparisons were performed using SPSS Statistics, version 23.b
Results
Clinician demographics are described in table 4. During 24 months, the research team approached 197 eligible patients and 157 patients enrolled (80% enrollment rate). Data were collected over 24 months, and we conducted a 48-month follow-up chart audit on 45 patients.
Table 4Clinician demographics (n=8)
Demographics
n (%)
Age, y
20-29
3 (38)
30-39
2 (25)
40-49
1 (13)
50-59
2 (25)
Sex
Men
2 (25)
Women
6 (75)
Clinical role
Physical therapist
6 (75)
Physical therapist assistant
2 (25)
Years of practice
<5
4 (50)
5-10
0 (0)
11-15
1 (12.5)
>15
3 (37.5)
Entry-level professional degree
Associates
2 (25)
Baccalaureate or equivalent
1 (12.5)
Master’s
2 (25)
PhD
0 (0)
Doctorate
2 (25)
Other advanced degree
1 (12.5)
Clinical specialist (ie, NCS)
Yes
0 (0)
No
8 (100)
Hours worked per week
≤20
1 (12.5)
21-30
2 (25)
31-39
1 (12.5)
≥40
4 (50)
Number of patients seen daily
3-4
0 (0)
5-6
5 (71.43)
7-8
0 (0)
>8
2 (28.7)
Abbreviation: NCS, Board Certified in Neurologic Physical Therapy.
At the first chart audit after starting to use KT interventions, adherence to recommendations was 46%. Adherence increased to greater than 85% by 6 months after the initiation of implementation interventions (fig 1). The 48-month follow-up chart audit indicated that adherence levels were at 95.2%.
Each clinician on the stroke unit (n=8) participated in the survey in 2015, 2016, and 2018. In 2015, 63% of participants agreed or strongly agreed that they used measures for diagnosis, 88% used measures for prognosis, and 100% monitored change over time. However, 38% of clinicians reported measuring 40% of patients, 13% reported measuring 60% of patients, 38% reported measuring 80% of patients, and only 13% reported measuring 100% of patients. Over the study data collection periods (2015, 2016, and 2018), clinicians' knowledge of how to administer the measures significantly improved (2015: median, 4.0; range, 4.0-5.0; 2016: median, 4.0; range, 4.0-5.0; 2018: median, 5.0; range, 5.0-5.0; χ2(2)=8.82; P=.01). Post hoc testing indicated differences between 2015 and 2018 (P=.02). Use of the 10MWT also significantly improved between 2015, 2016, and 2018 (2015: median, 3.0; range, 2.0-4.0; 2016: median, 5.0; range, 4.0-5.0; 2018: median, 5.0; range, 4.25-5.0; χ2(2)=10.33; P=.006). Post hoc testing indicated differences between 2015 and 2018 (P=.03).
Organizational outcomes
The ORIC score (median, range) increased from 2015 to 2018 (2015: 52 (39-53.5); 2016: 53.5 (45.5-59.0); 2018: 59.5 (47.5-60.0), but they were not significantly different from each other (P>.05) at the 3 data collection points. The MFB leadership team observed the effect of the project on the organization, including a positive effect on patients and families, demonstration of the value of outcome measurement, and facilitation of achievement of an organizational goal of routine clinical outcome measurement.
Discussion
In this project, we codeveloped a multicomponent implementation plan that was based on the KTA Framework and included implementation facilitation, implementation leadership, and a bundle of KT interventions that targeted barriers. Six months after implementation of the GAB, greater than 85% adherence was achieved and sustained for 48 months. Clinician perceptions of measurement improved during the study, and perceptions of barriers decreased. The project positively affected the organization, as illustrated by actions taken by MFB to implement the GAB throughout the organization and through observations described by Butzer et al.
The KT interventions were designed using theory, targeted barriers, and engaged end-users, all of which are considered essential components of designing effective behavior change interventions.
Although our KT interventions included education-based strategies, they constituted a small percentage of the approach. Audit and feedback was frequently used, which alone results in modest improvements in clinical practice.
Although the critical elements of audit and feedback are unknown, the intervention included components that may increase its effectiveness, such as feedback by a supervisor or respected colleague, frequent feedback, specific goals, and action-plans (≥85% adherence and codevelopment of KT interventions to target reported barriers).
How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field.
approach taken, which included ongoing codevelopment of KT interventions to target barriers and selection and tailoring of implementation and sustainability strategies, may have positively affected outcomes.
Systematic reviews of KT interventions report frequent use of interventions that target individuals.
The KT interventions used in this project also targeted organizational leadership and other environmental and resource barriers. The leadership team emphasized GAB use and articulated its importance to the organization’s vision, which may have facilitated this project’s success.
Clinicians perceived that the organization was ready for this change as indicated by a relatively high aggregate ORIC score. Implementation research funding provided the financial support for MFB staff time, external guidance, and equipment, as well as legitimation to the project goal. Staff had time to prepare testing packets, complete medical record modifications, and prepare the environment for testing (eg, develop BBS test kits, create rope with 10-meter marks on it, etc). It is important to note that this external funding may initiate projects and generate implementation knowledge. However, relying on implementation research funding is not sustainable for long-term or widespread practice changes. Without this implementation research funding at MFB, it is unlikely that the implementation of the GAB would have achieved a high level of adherence. Organizations may benefit from budgeting for KT activities to ensure that adequate resources are available to support successful implementation efforts. Future research should also assess the funding needs for successful implementation.
Facilitation is a social process that focuses on evidence-informed practice change and includes project management, leadership, relationship building, and communication.
guided the implementation process. Open communication between the MFB and the IU and Shirley Ryan AbilityLab (SRALab) team positively affected the success of the project. The MFB research team tailored information to the local context, collaborated to remove barriers, monitored progress, and maintained effective communication.
They also shared information about organizational events, morale, and other conflicting organizational priorities with the IU and SRALab research team, which ensured the timing of the KT interventions met and did not conflict with local needs. Involving clinicians in key roles may have also increased the relevance, applicability, and ease of translation of the GAB into clinical practice.
How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field.
Importantly, this KTA project resulted in sustainable changes in practice. Recent systematic reviews on the sustainability of new practices postimplementation indicate that fewer than 50% of providers sustained practices at high levels of fidelity.
Factors that may have contributed to the sustainability of the use of the GAB in practice may include the integration of the staff and consideration of the organizational context and processes into the KT plan. These are core components in the National Health Service Sustainability model, which is a diagnostic tool for identifying strengths and weaknesses in sustainability aspects of implementation plans.
The KT plan included many critical sustainability factors, including processes such as identifying and communicating the benefits of the GAB beyond helping patients and the use of a system for monitoring progress. At a staff level, we used an integrated KT approach that involved clinicians throughout the project. The clinicians were actively involved in the codevelopment of implementation and sustainability interventions. From a context perspective, the organization was very supportive of this change. For example, the project aligned with the organization’s vision to become a learning health system. The integrated KT approach using the KTA framework allowed the development of a customized KT plan for the MFB context. Senior leaders were also highly involved, visible in the project, and undertook implementation leadership behaviors.
External facilitation also supported the organization in making these changes successful. In addition, we integrated expectations for the use of the GAB into organizational processes such as performance appraisals, job descriptions, and new hire orientation.
Nineteen KT interventions were included in the bundle of implementation strategies used in this project. Although research demonstrates that implementation strategies are most effective when they include multiple components and target site-specific barriers, the most efficient or effective method of translating evidence into clinical practice have not been determined.
A critical discourse analysis of the attitudes of occupational therapists and physiotherapists towards the systematic use of standardised outcome measurement.
Organizational leaders and clinicians should evaluate different models to identify those appropriate for their context. In this study, key principles included integrated knowledge translation with engagement at all levels of the organization, use of a KT framework, and selection of a bundle of KT strategies that targeted barriers. Clinicians and leaders considering implementation projects, may consider working collaboratively with organizational stakeholders to identify barriers and implementation strategies that may be successful in their context. Providing internal funding for staff to have time to participate in the implementation project may also be beneficial.
Study limitations
Limitations of this study demonstrate the need for research in this area. This study used a pre-post observational study design without a control group, and we were unable to make definitive conclusions about the intervention tested. This project was implemented at a single facility and with a relatively small number of professionals. Therefore, research should be conducted to determine the effectiveness of this approach when these methods are replicated in other settings and with more practitioners. We implemented the GAB using an approach that included a bundle of KT activities that were codeveloped with clinicians and leaders and tailored to a specific facility. Research should determine adaptations required and the effectiveness of this approach when tailored to other settings. Further work should examine the contributions of using a framework, such as the KTA, as compared with specific activities in the KT bundle. In addition, research should examine the different components of this approach (ie, codevelopment of the KT plan, facilitation, implementation leadership, and tailoring of interventions) to determine the active ingredients. During the implementation of the GAB, we did not experience any staff turnover. Future research should assess sustainability after staff turnover occurs. We implemented the GAB as a standard of care but included data from patients who provided consent in the analysis. Patients who were unable to provide consent and did not have a written power of attorney were excluded, many with substantial cognitive or communication impairments. Approximately 30% of individuals with acute stroke have a cognitive impairment.
Therefore, the generalization of these findings to the measurement of patients with substantial cognitive impairment could be limited. The measures selected for the GAB are well known among physical therapists. We are unsure of the effect of the clinicians’ familiarity with the measures on adherence rates. Implementation of newer or unfamiliar measures may have required different strategies, more time to implement, or may not have been implemented successfully.
Conclusions
We developed a comprehensive plan based on the KTA cycle to implement evidence-based gait-related measurement recommendations. An iterative approach of barrier assessment, selection of new KT interventions, and monitoring facilitated the achievement of the desired adherence levels. Successful implementation of the GAB occurred within 6 months, and the clinicians sustained these practice changes for 4 years after implementation. Active ingredients that may have contributed to the success of this project include an integrated KT approach, use of the KTA framework to guide the development of the KT plan, implementation facilitation, and leadership.
Suppliers
a.
SurveyMonkey; SurveyMonkey.
b.
SPSS Statistics, version 23; IBM Corp.
Acknowledgments
We thank the clinicians and administrators at Mary Free Bed Rehabilitation Hospital for their contributions to data collection and knowledge translation plan development.
Appendix
Appendix 1KT interventions described using workgroup for intervention development and evaluation research criteria
KT Intervention
Characteristics of Those Delivering Intervention
Setting
Mode of Delivery
Contact Time
Duration (No. of Times)
Adherence/Fidelity to Delivery Protocols
Detailed Description of Intervention Content
Interventions Targeting Knowledge and Skill Barriers
Consultation ∼1 mo after education session
IU/SRALab research team and research PTs
NA
Phone conversation with MFB PT liaison and research coordinator
1 h
1
NA
MFB developed a list of questions after using assessments; IU/SRALab discussed answers
Education session
PT from IU/SRALab research team
MFB conference room
In-person to MFB stroke team (PTs/PTAs)
2 h and 50 min
1
Assessed % correct rating on video demonstration and standardization questions
Presentation and quiz on standardization
In-Person discussion ∼8 wk after education session
IU/SRALab PI/Co-PI
MFB conference room
In-person lunch meeting with stroke team PTs/PTAs and manager
50 min
1
NA
IU/SRALab PI/CoPI provided lunch and discussed barriers, facilitators to using the GAB; answered questions about GAB use
Education session on test interpretation
MFB PT liaison
MFB stroke unit
In-person to MFB stroke team (PTs/PTAs), e-mail
50 min
1
Reviewed PPT with IU/SRALab CoPI
Developed handout on interpretation of test results and goal writing and discussed with staff
Informal discussions
IU/SRALab Co-PI
NA
Phone conversation with MFB research coordinator
Varied from 5-60 min
1 time a mo, for 15 mo
NA
Problem solving sessions, discussions about adherence, barriers, and KT interventions
Training of noncore staff (coverage PTs/PTAs, weekend, etc)
MFB PT liaison
MFB conference room or gym
In-person
90 min
5 (1 per team)
Used educational materials, video consistency ratings from GAB project
Standardization training for 5 additional MFB teams (Brain Injury, Medically Complex Team, Sub-Acute Rehab Team, Core Weekend PTs/PTAs, coverage services PT/PTAs), using same training methods and tested for consistency using videos PT and PTA students also trained.
Restandardization 1 year after implementation
MFB PT liaison
MFB conference room
In-person lunch meeting with stroke team PTs/PTAs and manager
1 h
1
Assessed % correct rating on video demonstration and standardization questions
Presentation of adherence data for group and individual PT staff; discussion of barriers and facilitators for adherence; review of standardization items with which clinicians had the most difficulty
Development of “cheat sheets”
MFB PT liaison
MFB stroke unit
Distributed via email, discussed in meetings
1-h discussion
2 (initial, updated∼1 y later)
NA
Information to support test interpretation in subacute stroke (provided online)
Interventions Targeting Social Influences
Obtain leadership support
IU/SRALab PI/CoPI and MFB research team
MFB
In-person and phone meetings
∼8 h
2
NA
Discussion of the project, expectations, and suggestions for adherence and to demonstrate leadership support; discussion with stroke team manager about processes and equipment to support program, and with Medical Director to require GAB reporting weekly
Articulate leadership support
MFB stroke unit manager, physicians, and executives
MFB
In-person during team meetings and conferences, and e-mails
1 h
3
NA
Re-enforced project importance, adherence expectations, and reporting of results in team conference
Required team conference reporting
MFB research and leadership team
MFB, during team conference
E-mailed expectations, in-person during team conference when reporting did not occur
Reporting is weekly, monitored in-person by leadership and physicians
Intermit-tent for 12 mo
Reporting requirements monitored by physicians and managers
E-mails and in-person encouragement for MFB clinicians to report patient test results and changes demonstrated in outcome measurement
Testing “Tuesday”
MFB research team and leadership
MFB stroke unit
In-person, by e-mail
Weekly during therapy session
Ongoing, integrated into processes
NA
Tuesdays were designated the regular testing days; clinicians were notified by e-mails and team meetings; initially reminded by research assistant; clinicians and patients continue to remind each other/support use
Audit and feedback
MFB research team e-mailed, cc’d manager
NA
E-mail to the entire group of PTs/PTAs, in-person meetings
3 in-person 1-h meetings, other times by e-mail
Feedback provided every 6 wk for 15 mo
NA
Detailed in appendix
Rewards
MFB Leadership Team (Director of Research)
MFB team meeting
In-person, provided food and awards
1 h
2
NA
Director of Research provided 2 lunches and rewarded staff for high adherence; Most improved award was also provided
Interventions Targeting Environmental Context and Resources
Equipment purchase
MFB research team
MFB stroke unit
Purchased and stored in rehabilitation gym
NA
1
NA
Yardsticks, measuring wheels, file folders; Plant Services department painted baseboards in halls for 10MWT
Research assistant support for measurement
MFB research assistant
MFB Stroke Unit: gym
In-person
Available 8 am-4 pm on “Testing Tuesday” to support test administration
6 mo
NA
Research assistant was available in the gym to assist with testing (ie, set up equipment, obtain testing forms, and assist with testing as needed)
Data collection forms
Data collection forms developed by IU/SRALab research team, revised by MFB team; clinicians provided feedback, which was integrated into each version
Developed at each site, used during testing at MFB
Word documents
Forms reviewed during initial education session; ongoing verbal and e-mail communication with PTs to optimize form
1, then stored in gym; revisions occurred 10 times and were communicated by e-mail and verbally
Assessment forms were reviewed by MFB research team
Data collection forms can be reviewed online; data collection forms were stored alphabetically in file folder in main gym; updated weekly; uploaded into electronic health record after patient discharge
Scanning of data collection forms
MFB research assistant
MFB
Electronic: data collection forms scanned into medical record
Forms scanned weekly, ∼1 h
Ongoing
NA
RA scans paper data collection forms into the medical chart after discharge
Abbreviations: CoPI, co-principal investigator; NA, not applicable; PI, principal investigator; PT, physical therapist; PTA, physical therapist assistant; RA, research assistant.
1. Was the feedback given to an individual, a group, or both?
Provided to the group, reporting individual data
2. Was it given to the person in whom the practice change was desired (eg, healthcare provider vs hospital administrator)?
Provided to the clinicians directly, cc’d manager
What?
3. Was there feedback about the processes of care?
Yes, feedback about the rate of compliance with administration recommendations (actual administration/expected administration)
4. Was there feedback about patient outcomes?
No
5. Was there feedback about something other than processes of care or patient outcomes?
No
6. Was the feedback about individual provider performance?
Yes, individual level data reported to the whole group
7. Was the feedback about the performance of the provider group?
Yes, performance of the group was also provided
8. Was the feedback about individual patient cases?
No
9. Was the feedback about an aggregate of patient cases?
Yes, all patients discharged in the last month
10. Did the feedback identify a specific behavior(s) to be changed?
Yes, assessment administration rate
11. What was the comparison provided in the feedback?
Yes, historical administration rates and target rate
12. Were graphical elements included in the feedback?
Graphical feedback was provided at team meetings, 3 times in year 1
When?
13. What was the lag between the time of the audit and the delivery of the feedback?
First feedback delivered 4 mo after staff training; feedback delivered approximately every 2 mo thereafter
Why?
14. What is the rationale for using audit and feedback?
To monitor and improve compliance with assessment recommendations using clear communication with the team and manager
Who?
15. Was the feedback given face to face?
Feedback was given face to face 3 times: 2 times when compliance was low and once at the 1-y restandardization meeting; the remaining times were provided by e-mail or verbal communication
16. Were providers explicitly asked to consider the implications the audit and feedback had for their practice?
No
How much?
17. What was the total number of times the feedback was given?
8 times
NOTE. The intervention is described according to the modifiable design elements as reported by Colquhoun et al.
Smith M. Saunders R. Stuckhardt L. McGinnis J.M. Best care at a lower cost: the path to continuously learning health care in America. The National Acadamies Press,
Washington DC2013
Outcome measures for persons with moderate to severe traumatic brain injury: recommendations from the American Physical Therapy Association Academy of Neurologic Physical Therapy TBI EDGE Task Force.
Guidelines for adult stroke rehabilitation and recovery: a guideline for healthcare professionals from the American Heart Association/American Stroke Association.
The use of a knowledge translation program to increase use of standardized outcome measures in an outpatient pediatric physical therapy clinic: administrative case report.
Development of a checklist to assess the quality of reporting of knowledge translation interventions using the Workgroup for Intervention Development and Evaluation Research (WIDER) recommendations.
How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field.
A critical discourse analysis of the attitudes of occupational therapists and physiotherapists towards the systematic use of standardised outcome measurement.
Supported by the Department of Health and Human Services, National Institute on Disability, Independent Living, and Rehabilitation Research (grant no. 90RT5027). Ian D. Graham is a Canadian Institutes of Health Research Foundation Grant recipient (FDN no. 143237).
Disclosures: Ian D. Graham is an originator of the Knowledge-to-Action framework. The other authors have nothing to disclose.
This paper is part of a supplement from the American Congress of Rehabilitation Medicine.
31255-7&id=gr1.jpg){kind=link}