Advertisement

Integrated Knowledge Translation Guiding Principles for Conducting and Disseminating Spinal Cord Injury Research in Partnership

Open AccessPublished:October 28, 2020DOI:https://doi.org/10.1016/j.apmr.2020.09.393

      Abstract

      Objective

      To address a gap between spinal cord injury (SCI) research and practice by rigorously and systematically co-developing integrated knowledge translation (IKT) guiding principles for conducting and disseminating SCI research in partnership with research users.

      Design

      The process was guided by the internationally accepted The Appraisal of Guidelines for REsearch & Evaluation (AGREE) II Instrument for evaluating the development of clinical practice guidelines.

      Setting

      North American SCI research system (ie, SCI researchers, research users, funders).

      Participants

      The multidisciplinary expert panel (n=17) and end users (n=35) included individuals from a North American partnership of SCI researchers, research users, and funders who have expertise in research partnerships.

      Interventions

      Not applicable.

      Main Outcome Measures

      Clarity, usefulness, and appropriateness of the principles.

      Results

      Data regarding 125 principles of partnered research were systematically collected from 4 sources (review of reviews, scoping review, interviews, Delphi consensus exercise). A multidisciplinary expert panel held a 2-day meeting to establish consensus, select guiding principles, and draft the guidance. The panel reached 100% consensus on the principles and guidance document. The final document includes a preamble, 8 guiding principles, and a glossary. Survey data showed that the principles and guidance document were perceived by potential end users as clear, useful, and appropriate.

      Conclusions

      The IKT Guiding Principles represent the first rigorously co-developed, consensus-based guidance to support meaningful SCI research partnerships. The principles are a foundational tool with the potential to improve the relevance and impact of SCI research, mitigate tokenism, and advance the science of IKT.

      Keywords

      List of abbreviations:

      AGREE II (Appraisal of Guidelines for Research and Evaluation II), CBPR (community-based participatory research), IKT (integrated knowledge translation), KT (knowledge translation), OSF (Open Science Framework), PAR (participatory action research), SCI (spinal cord injury)
      The gap between discovery and the application of research is of particular concern for people living with spinal cord injuries (SCIs) whose needs and voices are often not reflected in the research process and whose lives could be enhanced by research discoveries.
      • Noonan V.
      • Wolfe D.
      • Thorogood N.
      • et al.
      Knowledge translation and implementation in spinal cord injury: a systematic review.
      • Barrable B.
      • Thorogood N.
      • Noonan V.
      • et al.
      Model for bridging the translational "valleys of death" in spinal cord injury research.
      • Conner M.
      • Rhodes R.E.
      • Morris B.
      • McEachan R.
      • Lawton R.
      Changing exercise through targeting affective or cognitive attitudes.
      • Graham I.D.
      • Tetroe J.
      Some theoretical underpinnings of knowledge translation.
      • Ward V.
      • House A.
      • Hamer S.
      Developing a framework for transferring knowledge into action: a thematic analysis of the literature.

      Rick Hansen Institute. Praxis 2016 bridging the gaps from SCI research to improved outcomes: conference report. November 2016, Vancouver, BC. Available at: https://praxisinstitute.org/praxis-2016-conference-report. Accessed November 17, 2020.

      • Hammell K.R.W.
      Spinal cord injury rehabilitation research: patient priorities, current deficiencies and potential directions.
      • Anderson K.D.
      Targeting recovery: priorities of the spinal cord-injured population.
      • Simpson L.A.
      • Eng J.J.
      • Hsieh J.T.
      • Wolfe D.L.
      the Spinal Cord Injury Rehabilitation Evidence Scire Research Team. The health and life priorities of individuals with spinal cord injury: a systematic review.
      • Woodill G.
      • Willi C.
      Centre for Independent Living in Toronto. Independent living and participation in research: a critical analysis.
      Funders, researchers, and research users have emphasized the importance of knowledge translation (KT) to address this gap between discovery and the application of knowledge.
      • Tetroe J.M.
      • Graham I.D.
      • Foy R.
      • et al.
      Health research funding agencies' support and promotion of knowledge translation: an international study.
      KT broadly refers to a range of activities that aim to ensure research leads to real-world effects and outcomes for individuals or groups who can benefit from or use the research (i.e., research users).
      CIHR
      Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches.
      , Establishing research partnerships between researchers and research users is one approach that is gaining attention for its potential to close the gap between research and practice. Increasingly, research funders, SCI organizations, and people with lived experience of SCI are encouraging or requiring researchers to partner with research users on grants.
      • Tetroe J.M.
      • Graham I.D.
      • Foy R.
      • et al.
      Health research funding agencies' support and promotion of knowledge translation: an international study.
      ,
      • Gainforth H.L.
      • Baxter K.
      • Baron J.
      • Michalovic E.
      • Caron J.G.
      • Sweet S.N.
      RE-AIMing conferences: evaluating the adoption, implementation and maintenance of the Rick Hansen Institute’s Praxis 2016.
      ,
      • Sweet S.N.
      • Michalovic E.
      • Baxter K.
      • Baron J.
      • Gainforth H.L.
      RE-AIMing conferences: a reach, effectiveness, and maintenance evaluation of the Rick Hansen Institute’s Praxis 2016.
      Research partnerships have been defined as “individuals, groups, or organizations engaged in collaborative research activities involving at least one researcher and any stakeholder actively engaged in any part of the research process (p. 4).”
      • Hoekstra F.
      • Mrklas K.J.
      • Sibley K.M.
      • et al.
      A review protocol on research partnerships: a Coordinated Multicenter Team approach.
      A variety of the research partnership approaches have emerged within and outside the field of disability research, such as community-based participatory research (CBPR),
      • Israel B.A.
      • Parker E.A.
      • Rowe Z.
      • et al.
      Community-based participatory research: lessons learned from the Centers for Children's Environmental Health and Disease Prevention Research.
      participatory action research (PAR),
      • Ehde D.M.
      • Wegener S.T.
      • Williams R.M.
      • et al.
      Developing, testing, and sustaining rehabilitation interventions via participatory action research.
      emancipatory research,
      • Stone E.
      • Priestley M.
      Parasites, pawns and partners: disability research and the role of non-disabled researchers.
      and integrated knowledge translation (IKT).
      • Hoekstra F.
      • Mrklas K.J.
      • Khan M.
      • et al.
      A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: a first step in synthesising the research partnership literature.
      • Nguyen T.
      • Graham I.D.
      • Mrklas K.J.
      • et al.
      How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field.
      • Jull J.
      • Giles A.
      • Graham I.D.
      Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge.
      While these approaches share similarities in terms of a commitment to including research users as partners in the research process and the co-creation of knowledge, there are also some differences in terms of their historical roots and motivations.
      • Nguyen T.
      • Graham I.D.
      • Mrklas K.J.
      • et al.
      How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field.
      ,
      • Jull J.
      • Giles A.
      • Graham I.D.
      Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge.
      IKT is unique because it was introduced by a funding agency as a research partnership approach that aims to ensure research is translated into practice by engaging research users as partners throughout the entire research process.
      CIHR
      Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches.
      ,
      • Jull J.
      • Giles A.
      • Graham I.D.
      Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge.
      An IKT approach aligns with disability communities’ calls for there to be “nothing about us without us” and with efforts to enhance participant empowerment,
      World Health Organization
      Ninth futures forum on health systems governance and public participation.
      elevate disenfranchised voices,
      • O'Mara-Eves A.
      • Brunton G.
      • McDaid D.
      • et al.
      Community engagement to reduce inequalities in health: a systematic review, meta-analysis and economic analysis.
      and engage in real-world problem solving to improve research relevance and effect.
      • Jull J.
      • Giles A.
      • Graham I.D.
      Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge.
      ,
      • Mockford C.
      • Staniszewska S.
      • Griffiths F.
      • Herron-Marx S.
      The impact of patient and public involvement on UK NHS health care: a systematic review.
      • Bowen S.
      • Graham I.D.
      Backwards design or looking sideways? Knowledge translation in the real world: comment on "A call for a backward design to knowledge translation.
      • Brett J.
      • Staniszewska S.
      • Mockford C.
      • et al.
      Mapping the impact of patient and public involvement on health and social care research: a systematic review.
      • Rycroft-Malone J.
      • Burton C.R.
      • Bucknall T.
      • Graham I.D.
      • Hutchinson A.M.
      • Stacey D.
      Collaboration and co-production of knowledge in healthcare: opportunities and challenges.
      Despite IKT’s growing importance, it remains a relatively understudied and sometimes poorly implemented aspect of KT.
      • Gagliardi A.R.
      • Kothari A.
      • Graham I.D.
      Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know.
      Instead of meaningfully partnering with research users, SCI researchers have often been accused of using a “tokenistic” approach.
      • Woodill G.
      • Willi C.
      Centre for Independent Living in Toronto. Independent living and participation in research: a critical analysis.
      This approach is the antithesis of IKT and refers to SCI research users only being asked to endorse research to legitimize research programs over which they have little real control.
      • Woodill G.
      • Willi C.
      Centre for Independent Living in Toronto. Independent living and participation in research: a critical analysis.
      To foster meaningful research partnerships, evidence-informed partnership principles (i.e., fundamental norms, rules, beliefs) need to be rigorously identified and developed in partnership with SCI researchers, research users, and funders.
      • Hoekstra F.
      • Mrklas K.J.
      • Sibley K.M.
      • et al.
      A review protocol on research partnerships: a Coordinated Multicenter Team approach.
      Although recommendations and advice for establishing research partnerships including principles for various research partnership approaches (eg, CBPR, emancipatory research) are available,
      CIHR
      Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches.
      ,
      • Israel B.A.
      • Parker E.A.
      • Rowe Z.
      • et al.
      Community-based participatory research: lessons learned from the Centers for Children's Environmental Health and Disease Prevention Research.
      ,
      • Stone E.
      • Priestley M.
      Parasites, pawns and partners: disability research and the role of non-disabled researchers.
      rigorously co-developed guidance for SCI-specific partnerships is lacking. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) Instrument is a rigorous, internationally accepted, consensus approach to transparently evaluating the development of clinical practice guidelines.
      • Brouwers M.C.
      • Kho M.E.
      • Browman G.P.
      • et al.
      AGREE II: advancing guideline development, reporting and evaluation in health care.
      Although this instrument has not been used to evaluate the establishment of partnership guidance, the framework has been used to create evidence-informed guidelines and tools for SCI research users.
      • Martin Ginis K.A.
      • Hicks A.L.
      • Latimer A.E.
      • et al.
      The development of evidence-informed physical activity guidelines for adults with spinal cord injury.
      • Arbour-Nicitopoulos K.P.
      • Ginis K.M.
      • Latimer-Cheung A.E.
      • et al.
      Development of an evidence-informed leisure time physical activity resource for adults with spinal cord injury: the SCI Get Fit Toolkit.
      • Ginis K.A.M.
      • van der Scheer J.W.
      • Latimer-Cheung A.E.
      • et al.
      Evidence-based scientific exercise guidelines for adults with spinal cord injury: an update and a new guideline.
      Accordingly, the purpose of this article is to transparently present the consensus-based “IKT Guiding Principles for Conducting and Disseminating SCI Research in Partnership” and describe the co-development process using the AGREE II reporting critieria.

      Methods

      Design

      Adapted versions of AGREE II instrument and reporting guidance
      • Brouwers M.C.
      • Kho M.E.
      • Browman G.P.
      • et al.
      AGREE II: advancing guideline development, reporting and evaluation in health care.
      ,
      • Brouwers M.C.
      • Kerkvliet K.
      • Spithoff K.
      • Consortium A.N.S.
      The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines.
      (supplemental appendices S1 and S2, available online only at http://www.archives-pmr.org/) were used to design the procedures and methodological approach for co-developing the IKT Guiding Principles. The instrument includes 6 domains that outline reporting criteria: (1) scope and purpose, (2) stakeholder involvement, (3) rigor of the process used to synthesize evidence and develop guidance, (4) clarity and presentation, (5) applicability and implementation of the guidance, and (6) editorial independence and acknowledgment of biases. Aligned with the AGREE II reporting criteria (see supplemental appendices S1 and S2), information pertinent to each domain are presented in the results and discussion sections of this article. For transparency, additional files associated with the project are available in the Open Science Framework (OSF) repository (https://osf.io/9mq24/; DOI: 10.17605/OSF.IO/9MQ24).

      Procedures and methodological approach

      In 2017, an initial core expert panel (n=9) was established to co-develop the methods for creating the first IKT guiding principles for the North American SCI research system (ie, SCI researchers, research users, funders involved in the planning, conduct, and/or dissemination of research). During a 1-day meeting in September 2017, this core panel consisting of SCI researchers, SCI research users, and funders of SCI research discussed the first 3 AGREE II domains (ie, scope and purpose, methods for stakeholder involvement, rigorous processes for synthesizing evidence and developing guidance). The panel decided to co-conduct research that would allow for the guiding principles to be informed by multiple data sources. Since 2017, the core panel met several times to identify new partners, collect and synthesize data sources, and establish processes to co-develop, co-implement, and co-evaluate the IKT Guiding Principles. In November 2019, the expanded multidisciplinary panel (n=17) (supplemental appendix S3, available online only at http://www.archives-pmr.org/), which included SCI researchers, clinicians, people with SCI, representatives from SCI community organizations, and funding agencies of SCI research, held a 2-day meeting to establish consensus on the IKT Guiding Principles. After the meeting, expert feedback was sought and the principles were refined. Figure 1 provides an overview of the project procedures and timeline.

      Results

      The final IKT Guiding Principles for conducting and disseminating SCI research in partnership are presented in fig 2. The document includes a preamble, 8 guiding principles, and a glossary of terms. Aligned with the AGREE II domains, the outcomes of our co-development process are presented below.
      Figure thumbnail gr2
      Fig 2IKT Guiding Principles document. NOTE. Data sources included (1) review of reviews on principles used in research partnerships, (2) a scoping review on research partnership principles used in SCI research, (3) an interview study examining how SCI researchers and research users have partnered to conduct research, and (4) a Delphi consensus exercise.

      Scope and purpose (AGREE II Domain 1)

      The objectives, practical questions, target population, and potential end users of the principles were determined by the core expert panel and refined by the multidisciplinary expert panel. Through discussion, the panel members agreed on the following statements:
      • Objectives: To co-develop IKT guiding principles to support the meaningful engagement of the right users at the right time throughout the SCI research process.
      • Practical questions: What principles should be used to support the meaningful engagement of the right research users at the right time throughout the SCI research process?
      • Target population: Researchers and research users who adopt an IKT approach in an SCI-related research project in a North American setting. Research users included but were not limited to persons with lived experience of SCI, policymakers, service providers, other researchers, professional organizations, funders, and industry partners.
      • Potential end users: SCI researchers, SCI research users, and funders of SCI research.

      Stakeholder involvement (AGREE II Domain 2)

      Supplemental appendix S3 describes panel members’ expertise and roles within the core panel and the multidisciplinary panel. Panel members reviewed findings from the data sources, attended the 2-day AGREE II meeting, reviewed the IKT Guiding Principles, provided feedback on the first draft of IKT Guiding Principles via an online survey, and reviewed the manuscript. The IKT Guiding Principles were also reviewed by potential end users with experience in research partnerships. Moreover, the research associated with the data sources was conducted using an IKT approach.

      Rigor of development (AGREE II Domain 3)

      A 4-step process was used to synthesize evidence and co-develop the IKT Guiding Principles: (1) creating a long list of research partnership principles, (2) conducting a Delphi consensus study, (3) establishing consensus on the IKT Guiding Principles, and (4) collecting expert feedback about the resulting IKT Guiding Principles (see fig 1). These steps are described in detail below.

      Step 1: Creating a long list of research partnership principles

      A rigorous and systematic approach was used to create a long list of research partnership principles using 3 key data sources: (1) a review of reviews (n=86) on research partnership principles, (2) a scoping review of (n=13) SCI research partnership principles, and (3) interviews with SCI researchers and SCI research users (n=10) with experience partnering to conduct research (see OSF: https://doi.org/10.17605/OSF.IO/9MQ24). Of note, data regarding the effectiveness of principles was not found. From these data sources, research partnership principles were extracted using the following definition: “Fundamental norms, rules, or beliefs that represent what is desirable and positive for a person, group, organization, or community, and help it in determining the rightfulness or wrongfulness of its actions. Principles are more basic than policy and objectives and are meant to govern both.”
      • Hoekstra F.
      • Mrklas K.J.
      • Sibley K.M.
      • et al.
      A review protocol on research partnerships: a Coordinated Multicenter Team approach.
      To aid interpretation, the extracted principles were then synthesized into overarching principles and corresponding subprinciples using direct content analysis.
      • Hsieh H.F.
      • Shannon S.E.
      Three approaches to qualitative content analysis.
      After removing principles with similar meanings, a list of 125 principles was created (18 overarching principles linked to 107 corresponding principles). Supplemental appendix S4 (available online only at http://www.archives-pmr.org/) provides an overview of the final principles and how these principles link to the data sources, and OSF provides further details on decision processes and corresponding subprinciples.

      Step 2: Delphi consensus

      To establish consensus on which principles should be used to guide SCI research partnerships, a Delphi consensus study was conducted among SCI researchers and SCI research users who had completed and/or were currently involved in 1 or more SCI research projects that involve a research partnership. The Delphi consisted of 3 rounds including at least 20 participants per round. In the first 2 rounds, participants indicated the extent to which they agreed that a principle should be used to guide SCI research partnerships. In the last round, participants categorized principles as “essential,” “desirable,” “irrelevant,” or “unsure.” In the last round, 37 principles were categorized as either essential (n=29) or desirable (n=8). Final results are available elsewhere (see OSF: https://doi.org/10.17605/OSF.IO/9MQ24).

      Step 3: AGREE II consensus meeting

      Prior to the AGREE II meeting, panel members (n=13) reviewed the summary of the Social Sciences and Humanities Research Council (SSHRC) grant proposal as well as abstracts and results from the 4 data sources and completed a conflict of interest declaration (ie, AGREE II Domain 6). First, the project lead outlined and confirmed the scope and purpose of the project, the relevant terms and definitions, and the consensus-based framework on research partnership domains.
      • Hoekstra F.
      • Mrklas K.J.
      • Sibley K.M.
      • et al.
      A review protocol on research partnerships: a Coordinated Multicenter Team approach.
      Furthermore, the project lead outlined the different histories, motivations, and social locations of an IKT and CBPR approach
      • Jull J.
      • Giles A.
      • Graham I.D.
      Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge.
      to ensure that these differences were taken into account in the discussions and decisions.
      In small working groups (n=3-4 members), panel members discussed and reviewed the overarching principles using a structured discussion guide that presented the Delphi results for each guiding principle and its corresponding principles (see OSF). Each group discussed 13 of the 19 overarching principles. Panel members discussed if (1) the principle is applicable to all research groups (researchers, research users, funders); (2) if the meaning is specific and unambiguous; and (3) if the principles should be adopted, adapted, or removed and provided suggestions for adaptations if needed. Findings from these discussions were synthesized and presented to the panel. The panel discussed the findings and established consensus on the final principles to be included. The panel also discussed the contents of the preamble of the principles and a glossary. Rationale for decisions to remove and include principles, as well as the content of the preamble and glossary, are available via OSF.
      On day 2, the panel was presented with a draft version of the preamble, guiding principles, and glossary. The panel edited and refined the document through discussion. The panel reviewed the resulting document, and a consensus vote was held regarding the meaning of the preamble and principles. All panel members voted in favor of both the preamble, glossary, and the principles (100% consensus). Finally, clarity of presentation of the principles and barriers and facilitators to the application of the principles were discussed (see OSF for meeting agenda).

      Step 4: Expert feedback

      The IKT Guiding Principles were reviewed by panel members and potential end users with experience working in research partnerships using an online survey (Tables 1 and 2). Survey items were used to rate the principles for their clarity, usefulness, and appropriateness. End users were recruited through our panel’s extensive network and had the opportunity to provide open-ended feedback. In accordance with the regulations of the Tri-Council Policy Statement governing research ethics in Canada, ethics approval was not required for these online surveys because they are classified as quality improvement and not research (ie, to receive feedback on the principles not to answer a research question). All surveys are provided on OSF.
      Table 1Panel member questionnaire item responses
      Questionnaire ItemMean ± SDMedianMinMax
      Preamble
       The preamble of the IKT Guiding Principles is clearly expressed.6.2±0.8657
       The preamble of the IKT Guiding Principles is specific and unambiguous.5.5±1.2537
      Principles
       The principles are clearly expressed.6.1±0.7657
       The principles are specific and unambiguous.5.5±1.1537
      Glossary
       The glossary is clearly expressed.6.2±0.9657
       The glossary provides definitions of key terms that are specific and unambiguous.5.5±1.5637
      IKT Guiding Principles document
       The IKT Guiding Principles document is relevant for SCI researchers who want to adopt an IKT approach on an SCI-related research project.6.7±0.5767
       The IKT Guiding Principles document is relevant for SCI research users who want to adopt an IKT approach on an SCI-related research project.6.6±0.5767
       The IKT Guiding Principles document is relevant for research funders who are funding SCI-related research projects.6.3±0.8657
       The IKT Guiding Principles document is useful to support the meaningful engagement of the right users at the right time throughout the SCI research process.6.2±0657
       The IKT Guiding Principles document is useful to support the conduct of quality and ethical research that is relevant, useful, and/or useable for SCI research users.5.7±1657
       The IKT Guiding Principles can be applied throughout the entire research process.6.7±0.6757
       I intend to use the IKT Guiding Principles document.6.5±0.8757
       The IKT Guiding Principles document uses language that is appropriate for SCI researchers.6.6±0.5767
       The IKT Guiding Principles document uses language that is appropriate for SCI research users.6.1±0.8657
       The IKT Guiding Principles document uses language that is appropriate for research funders who are funding SCI-related research projects.6.1±0.8657
      NOTE. All items were rated on a 7-point Likert scale: 1= strongly disagree, 2=disagree, 3=somewhat disagree, 4=neither disagree nor agree, 5=somewhat agree, 6=agree, 7=strongly agree.
      Table 2Potential end-user expert questionnaire item responses
      Questionnaire ItemMean ± SDMedianMinMax
      Preamble
       The introduction of the IKT Guiding Principles is clearly expressed.5.6±1617
       The introduction of the IKT Guiding Principles is specific and unambiguous.5.7±1.2617
      Principles
       The principles are clearly expressed.6±1627
       The principles are specific and unambiguous.5.6±1.2627
      Glossary
       The glossary is clearly expressed.6±0.8647
       The glossary provides definitions of key terms that are specific and unambiguous.5.8±1.2627
      IKT Guiding Principles document
       The IKT Guiding Principles document is relevant for SCI researchers who want to adopt an IKT approach on an SCI-related research project.6±1627
       The IKT Guiding Principles document is relevant for SCI research users who want to adopt an IKT approach on an SCI-related research project.5.9±1.1627
       The IKT Guiding Principles document is relevant for research funders who are funding SCI-related research projects.5.9±1.1627
       The IKT Guiding Principles document is useful to support the meaningful engagement of the right users at the right time throughout the SCI research process.5.4±1.3527
       The IKT Guiding Principles document is useful to support the conduct of quality and ethical research that is relevant, useful, and/or useable for SCI research users.5.5±1.3627
       The IKT Guiding Principles can be applied throughout the entire research process.6±1.2617
       I intend to use the IKT Guiding Principles document.5.5±1.4617
       The IKT Guiding Principles document uses language that is appropriate for SCI researchers.6.1±1627
       The IKT Guiding Principles document uses language that is appropriate for SCI research users.5.5±1.3617
       The IKT Guiding Principles document uses language that is appropriate for research funders who are funding SCI-related research projects.6±1627
      NOTE. All items were rated on a 7-point Likert scale: 1=strongly disagree, 2=disagree, 3=somewhat disagree, 4=neither disagree nor agree, 5=somewhat agree, 6=agree, 7=strongly agree.

      IKT Guiding Principles and clarity of presentation (AGREE II Domain 4)

      Expert feedback: panel members

      Mean ratings indicated that panel members’ responses (n=6) reflected that they agreed or strongly agreed that the preamble, principles, and glossary were clear, useful, and appropriate (see table 1).

      Expert feedback: potential end users

      Thirty-five potential end users with experience working in research partnerships responded to the survey. Of these end users, 18 were researchers, 8 were research users, 9 were both researchers and research users, and/or 10 had an SCI. Most end users conducted or used SCI research primarily (n=24). Mean ratings indicated that potential end users agreed or strongly agreed that the preamble, principles, and glossary were clear, useful, and appropriate (see table 2). Based on end-user feedback, minor changes were made to the document to improve clarity (eg, changes to wording, additions to definitions). Changes that would alter the meaning of the document were not implemented (supplemental appendix S5, available online only at http://www.archives-pmr.org/).

      Discussion

      Using a systematic and rigorous consensus process, we reached consensus on 8 guiding principles to guide meaningful engagement of researchers and research users in conducting and disseminating research that is relevant, useful, and/or useable. The IKT Guiding Principles document also includes a preamble and glossary to support end users’ application of the principles and understanding of key concepts and terms. Given that data regarding the effectiveness of the principles were not found, the document is presented as guiding principles and not guidelines. Both the panel and potential end users perceived the IKT Guiding Principles to be clear, useful, and appropriate for supporting meaningful research user engagement in SCI research. Ultimately, these principles have the potential to reduce the likelihood of tokenism, contribute to more co-production of knowledge, and foster meaningful SCI research partnerships. Aligned with AGREE II reporting, implications for applicability and implementation of the IKT Guiding Principles are discussed below (ie, AGREE II Domain 5).

      Applicability

      The panel outlined potential implications for disseminating and implementing the IKT Guiding Principles within the North American SCI Research System, which aligned with barriers and facilitators to adopting IKT identified in the broader literature.
      • Gagliardi A.R.
      • Berta W.
      • Kothari A.
      • Boyko J.
      • Urquhart R.
      Integrated knowledge translation (IKT) in health care: a scoping review.
      Barriers specific to members of a research partnership primarily related to a lack of capability to implement the principles. Adopting the principles will require appropriate budgeting, an understanding of research processes, and strong interpersonal and communication skills. Training in conducting research in partnership is limited,
      • de Moissac D.
      • Bowen S.
      • Botting I.
      • et al.
      Evidence of commitment to research partnerships? Results of two web reviews.
      and without capacity building and incentives for both researchers and research users, these principles may be perceived as intimidating, time consuming, and/or risky to adopt. The structure of research funding could also present barriers. Timelines and eligible funding costs may be perceived as too stringent to allow for the flexibility and resources needed to build interpersonal relationships prior to submitting a grant application and throughout the research process. Likewise, research budgets need to account for the environmental and financial barriers that may prevent people with disabilities from engaging in research partnerships (eg, accessible transport and travel, personal support workers, adapted equipment, limited income).
      The panel also highlighted facilitators that could support adoption and implementation. Research funders’ increased emphasis on the importance of partnered research and partnered funding opportunities may influence researchers and research users to consider adopting the principles. The SCI research system has a strong network of IKT champions both within and outside academia that can help promote and champion the principles. Researchers already using an IKT approach can be leaders and act as ambassadors for the IKT Guiding Principles. There is a strong network of research user organizations and funders who are committed to ensuring that there is “nothing about us, without us.”
      • Woodill G.
      • Willi C.
      Centre for Independent Living in Toronto. Independent living and participation in research: a critical analysis.
      ,
      • Gainforth H.L.
      • Baxter K.
      • Baron J.
      • Michalovic E.
      • Caron J.G.
      • Sweet S.N.
      RE-AIMing conferences: evaluating the adoption, implementation and maintenance of the Rick Hansen Institute’s Praxis 2016.
      ,
      • Sweet S.N.
      • Michalovic E.
      • Baxter K.
      • Baron J.
      • Gainforth H.L.
      RE-AIMing conferences: a reach, effectiveness, and maintenance evaluation of the Rick Hansen Institute’s Praxis 2016.
      Together these groups have strong potential to build capacity among research users who want to be involved in research partnerships (eg, the North American SCI Consortium, Praxis Spinal Cord Institute, SCI Canada).

      Dissemination and implementation

      The IKT Guiding Principles will only be as effective as our dissemination and implementation efforts. Our panel has co-developed a team that will support implementation. In addition to disseminating the principles via scientific articles and conferences, they will create bilingual tools, resources, and interventions that address barriers to implementation and support each of our potential users to disseminate and use the IKT Guiding Principles (see www.iktprinciples.com). This process will build on our initial understanding of barriers and facilitators and will involve interviewing researchers, research users, and funders to understand how to support implementation. Example resources and interventions may include educational videos, guidance for reviewing grant applications, and trainings.

      Future directions for monitoring and co-evaluation

      To monitor the dissemination and implementation of the IKT Guiding Principles, the panel has co-developed an evaluation plan, which includes monitoring implementation via quarterly online surveys sent to our partners’ membership and via Google Analytics. The surveys will be guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance
      • Glasgow R.
      • Vogt T.
      • Boles S.
      Evaluating the public health impact of health promotion intervention: the RE-AIM framework.
      aligned with previous work examining community-university partnerships.
      • Sweet S.N.
      • Ginis K.A.
      • Estabrooks P.A.
      • Latimer-Cheung A.E.
      Operationalizing the RE-AIM framework to evaluate the impact of multi-sector partnerships.
      To gain an in-depth understanding of how the principles are implemented in a SCI research system, an in-depth mixed-methods evaluation will also be conducted within the International Collaboration on Repair Discoveries, an SCI research center focused on the development and translation of effective strategies to promote prevention, functional recovery, and improved quality of life after SCI. The panel will consider a process for updating the principles once these evaluations are complete.

      Future directions for advancing the science of integrated knowledge translation

      While the potential users of the guiding principles are members of the SCI research system, potential users of our established methodology are broader. Our systematic and transparent processes for co-developing guiding principles can be adopted to develop other domain-specific principles for other groups (eg, research related to people with multiple sclerosis, diabetes, arthritis, and/or stroke). The IKT-specific principles will contribute to better understanding the differences and similarities between IKT and other research partnership approaches (eg, CBPR, PAR). Future research should focus on identifying partnership strategies that could support the use of these principles in different phases of a research project as well as in different settings and areas.
      • Hoekstra F.
      • Mrklas K.J.
      • Khan M.
      • et al.
      A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: a first step in synthesising the research partnership literature.
      The principles and related strategies provide a common language to clearly describe and improve the reporting of research partnerships,
      • Hoekstra F.
      • Mrklas K.J.
      • Sibley K.M.
      • et al.
      A review protocol on research partnerships: a Coordinated Multicenter Team approach.
      which will be invaluable for improving our understanding of effective IKT processes.
      • Gagliardi A.R.
      • Kothari A.
      • Graham I.D.
      Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know.
      ,
      • Graham I.D.
      • Kothari A.
      • McCutcheon C.
      Integrated Knowledge Translation Research Network Project Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation.

      Study limitations

      The development of the principles relied on the quality of the data reported in the literature. Data regarding the effectiveness of the IKT Guiding Principles are not yet available. Therefore, these are guiding principles and not guidelines. The principles were co-developed by a multidisciplinary North American team. Whether the principles are relevant and applicable to research systems in other countries and/or research domains remains to be seen.

      Conclusions

      The IKT Guiding Principles for conducting and disseminating SCI research in partnership are an important, needed, and foundational innovation that can be used to improve SCI research. By applying the AGREE II instrument, we have presented a systematic, rigorous, and engaged co-development process. These guiding principles have strong potential to support the development of meaningful and effective research partnerships that champion the expertise of research users and mitigate tokenism.

      Acknowledgments

      We thank Kristy Baxter (BHK), Dr Michael Kennefick (PhD), and Kelsey Wuerstl (MSc) for administrative support for the Consensus Meeting. We thank Dr Hugh Anton for his contribution to the first panel meeting. We also thank Dr Ian Graham for his guidance and feedback. Finally, we thank the staff at SCI British Columbia for hosting the Consensus Meeting.

      Supplementary Data

      References

        • Noonan V.
        • Wolfe D.
        • Thorogood N.
        • et al.
        Knowledge translation and implementation in spinal cord injury: a systematic review.
        Spinal Cord. 2014; 52: 578
        • Barrable B.
        • Thorogood N.
        • Noonan V.
        • et al.
        Model for bridging the translational "valleys of death" in spinal cord injury research.
        J Healthc Leadersh. 2014; 6: 15
        • Conner M.
        • Rhodes R.E.
        • Morris B.
        • McEachan R.
        • Lawton R.
        Changing exercise through targeting affective or cognitive attitudes.
        Psychol Health. 2011; 26: 133-149
        • Graham I.D.
        • Tetroe J.
        Some theoretical underpinnings of knowledge translation.
        Acad Emerg Med. 2007; 14: 936-941
        • Ward V.
        • House A.
        • Hamer S.
        Developing a framework for transferring knowledge into action: a thematic analysis of the literature.
        J Health Serv Res Policy. 2009; 14: 156-164
      1. Rick Hansen Institute. Praxis 2016 bridging the gaps from SCI research to improved outcomes: conference report. November 2016, Vancouver, BC. Available at: https://praxisinstitute.org/praxis-2016-conference-report. Accessed November 17, 2020.

        • Hammell K.R.W.
        Spinal cord injury rehabilitation research: patient priorities, current deficiencies and potential directions.
        Disabil Rehabil. 2010; 32: 1209-1218
        • Anderson K.D.
        Targeting recovery: priorities of the spinal cord-injured population.
        J Neurotrauma. 2004; 21: 1371-1383
        • Simpson L.A.
        • Eng J.J.
        • Hsieh J.T.
        • Wolfe D.L.
        the Spinal Cord Injury Rehabilitation Evidence Scire Research Team. The health and life priorities of individuals with spinal cord injury: a systematic review.
        J Neurotrauma. 2012; 29: 1548-1555
        • Woodill G.
        • Willi C.
        Centre for Independent Living in Toronto. Independent living and participation in research: a critical analysis.
        CILT, Toronto2006
        • Tetroe J.M.
        • Graham I.D.
        • Foy R.
        • et al.
        Health research funding agencies' support and promotion of knowledge translation: an international study.
        Milbank Q. 2008; 86: 125-155
        • CIHR
        Guide to knowledge translation planning at CIHR: integrated and end-of-grant approaches.
        Canadian Institutes of Health Research, Ottawa2012
        • SSHRC
        Guidelines for effective knowledge mobilization.
        (Available at:)
        • Gainforth H.L.
        • Baxter K.
        • Baron J.
        • Michalovic E.
        • Caron J.G.
        • Sweet S.N.
        RE-AIMing conferences: evaluating the adoption, implementation and maintenance of the Rick Hansen Institute’s Praxis 2016.
        Health Res Policy Syst. 2019; 17: 39
        • Sweet S.N.
        • Michalovic E.
        • Baxter K.
        • Baron J.
        • Gainforth H.L.
        RE-AIMing conferences: a reach, effectiveness, and maintenance evaluation of the Rick Hansen Institute’s Praxis 2016.
        Transl Behav Med. 2020; 10: 123-135
        • Hoekstra F.
        • Mrklas K.J.
        • Sibley K.M.
        • et al.
        A review protocol on research partnerships: a Coordinated Multicenter Team approach.
        Syst Rev. 2018; 7: 217
        • Israel B.A.
        • Parker E.A.
        • Rowe Z.
        • et al.
        Community-based participatory research: lessons learned from the Centers for Children's Environmental Health and Disease Prevention Research.
        Environ Health Perspect. 2005; 113: 1463-1471
        • Ehde D.M.
        • Wegener S.T.
        • Williams R.M.
        • et al.
        Developing, testing, and sustaining rehabilitation interventions via participatory action research.
        Arch Phys Med Rehabil. 2013; 94: S30-S42
        • Stone E.
        • Priestley M.
        Parasites, pawns and partners: disability research and the role of non-disabled researchers.
        Br J Sociol. 1996; 47: 699-716
        • Hoekstra F.
        • Mrklas K.J.
        • Khan M.
        • et al.
        A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: a first step in synthesising the research partnership literature.
        Health Res Policy Syst. 2020; 18: 51
        • Nguyen T.
        • Graham I.D.
        • Mrklas K.J.
        • et al.
        How does integrated knowledge translation (IKT) compare to other collaborative research approaches to generating and translating knowledge? Learning from experts in the field.
        Health Res Policy Syst. 2020; 18: 35
        • Jull J.
        • Giles A.
        • Graham I.D.
        Community-based participatory research and integrated knowledge translation: advancing the co-creation of knowledge.
        Implement Sci. 2017; 12: 150
        • World Health Organization
        Ninth futures forum on health systems governance and public participation.
        World Health Organization, Copenhagen2006
        • O'Mara-Eves A.
        • Brunton G.
        • McDaid D.
        • et al.
        Community engagement to reduce inequalities in health: a systematic review, meta-analysis and economic analysis.
        NIHR Journals Library, Southampton (UK)2013
        • Mockford C.
        • Staniszewska S.
        • Griffiths F.
        • Herron-Marx S.
        The impact of patient and public involvement on UK NHS health care: a systematic review.
        Int J Qual Health Care. 2012; 24: 28-38
        • Bowen S.
        • Graham I.D.
        Backwards design or looking sideways? Knowledge translation in the real world: comment on "A call for a backward design to knowledge translation.
        Int J Health Policy Manag. 2015; 4: 545-547
        • Brett J.
        • Staniszewska S.
        • Mockford C.
        • et al.
        Mapping the impact of patient and public involvement on health and social care research: a systematic review.
        Health Expect. 2012; 17: 637-650
        • Rycroft-Malone J.
        • Burton C.R.
        • Bucknall T.
        • Graham I.D.
        • Hutchinson A.M.
        • Stacey D.
        Collaboration and co-production of knowledge in healthcare: opportunities and challenges.
        Int J Health Policy Manag. 2016; 5: 221-223
        • Gagliardi A.R.
        • Kothari A.
        • Graham I.D.
        Research agenda for integrated knowledge translation (IKT) in healthcare: what we know and do not yet know.
        J Epidemiol Community Health. 2017; 71: 105-106
        • Brouwers M.C.
        • Kho M.E.
        • Browman G.P.
        • et al.
        AGREE II: advancing guideline development, reporting and evaluation in health care.
        CMAJ. 2010; 182: E839-E842
        • Martin Ginis K.A.
        • Hicks A.L.
        • Latimer A.E.
        • et al.
        The development of evidence-informed physical activity guidelines for adults with spinal cord injury.
        Spinal Cord. 2011; 49: 1088-1096
        • Arbour-Nicitopoulos K.P.
        • Ginis K.M.
        • Latimer-Cheung A.E.
        • et al.
        Development of an evidence-informed leisure time physical activity resource for adults with spinal cord injury: the SCI Get Fit Toolkit.
        Spinal Cord. 2013; 51: 491
        • Ginis K.A.M.
        • van der Scheer J.W.
        • Latimer-Cheung A.E.
        • et al.
        Evidence-based scientific exercise guidelines for adults with spinal cord injury: an update and a new guideline.
        Spinal Cord. 2018; 56: 308
        • Brouwers M.C.
        • Kerkvliet K.
        • Spithoff K.
        • Consortium A.N.S.
        The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines.
        BMJ. 2016; 352: i1152
        • Hsieh H.F.
        • Shannon S.E.
        Three approaches to qualitative content analysis.
        Qual Health Res. 2005; 15: 1277-1288
        • Gagliardi A.R.
        • Berta W.
        • Kothari A.
        • Boyko J.
        • Urquhart R.
        Integrated knowledge translation (IKT) in health care: a scoping review.
        Implement Sci. 2016; 11: 38
        • de Moissac D.
        • Bowen S.
        • Botting I.
        • et al.
        Evidence of commitment to research partnerships? Results of two web reviews.
        Health Res Policy Syst. 2019; 17: 73
        • Glasgow R.
        • Vogt T.
        • Boles S.
        Evaluating the public health impact of health promotion intervention: the RE-AIM framework.
        Am J Public Health. 1999; 89: 1322
        • Sweet S.N.
        • Ginis K.A.
        • Estabrooks P.A.
        • Latimer-Cheung A.E.
        Operationalizing the RE-AIM framework to evaluate the impact of multi-sector partnerships.
        Implement Sci. 2014; 9: 74
        • Graham I.D.
        • Kothari A.
        • McCutcheon C.
        Integrated Knowledge Translation Research Network Project Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation.
        Implement Sci. 2018; 13: 22