If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Department of Kinesiology and Physical Education, McGill University, Montreal, Quebec, CanadaCenter for Interdisciplinary Research in Rehabilitation of Greater Montreal (CRIR), Montreal, Quebec, Canada
School of Health and Exercise Sciences, University of British Columbia Okanagan, Kelowna, British Columbia, CanadaInternational Collaboration on Repair Discoveries, Vancouver, British Columbia, CanadaDepartment of Medicine, Division of Physical Medicine and Rehabilitation, University of British Columbia, Vancouver, British Columbia, Canada
School of Health and Exercise Sciences, University of British Columbia Okanagan, Kelowna, British Columbia, CanadaInternational Collaboration on Repair Discoveries, Vancouver, British Columbia, CanadaFaculty of Medicine, Department of Occupational Science and Occupational Therapy, University of British Columbia, Vancouver, British Columbia, Canada
International Collaboration on Repair Discoveries, Vancouver, British Columbia, CanadaFaculty of Medicine, Department of Occupational Science and Occupational Therapy, University of British Columbia, Vancouver, British Columbia, Canada
International Collaboration on Repair Discoveries, Vancouver, British Columbia, CanadaDepartment of Medicine, Division of Physical Medicine and Rehabilitation, University of British Columbia, Vancouver, British Columbia, Canada
To address a gap between spinal cord injury (SCI) research and practice by rigorously and systematically co-developing integrated knowledge translation (IKT) guiding principles for conducting and disseminating SCI research in partnership with research users.
The process was guided by the internationally accepted The Appraisal of Guidelines for REsearch & Evaluation (AGREE) II Instrument for evaluating the development of clinical practice guidelines.
North American SCI research system (ie, SCI researchers, research users, funders).
The multidisciplinary expert panel (n=17) and end users (n=35) included individuals from a North American partnership of SCI researchers, research users, and funders who have expertise in research partnerships.
Main Outcome Measures
Clarity, usefulness, and appropriateness of the principles.
Data regarding 125 principles of partnered research were systematically collected from 4 sources (review of reviews, scoping review, interviews, Delphi consensus exercise). A multidisciplinary expert panel held a 2-day meeting to establish consensus, select guiding principles, and draft the guidance. The panel reached 100% consensus on the principles and guidance document. The final document includes a preamble, 8 guiding principles, and a glossary. Survey data showed that the principles and guidance document were perceived by potential end users as clear, useful, and appropriate.
The IKT Guiding Principles represent the first rigorously co-developed, consensus-based guidance to support meaningful SCI research partnerships. The principles are a foundational tool with the potential to improve the relevance and impact of SCI research, mitigate tokenism, and advance the science of IKT.
The gap between discovery and the application of research is of particular concern for people living with spinal cord injuries (SCIs) whose needs and voices are often not reflected in the research process and whose lives could be enhanced by research discoveries.
KT broadly refers to a range of activities that aim to ensure research leads to real-world effects and outcomes for individuals or groups who can benefit from or use the research (i.e., research users).
Establishing research partnerships between researchers and research users is one approach that is gaining attention for its potential to close the gap between research and practice. Increasingly, research funders, SCI organizations, and people with lived experience of SCI are encouraging or requiring researchers to partner with research users on grants.
Research partnerships have been defined as “individuals, groups, or organizations engaged in collaborative research activities involving at least one researcher and any stakeholder actively engaged in any part of the research process (p. 4).”
While these approaches share similarities in terms of a commitment to including research users as partners in the research process and the co-creation of knowledge, there are also some differences in terms of their historical roots and motivations.
IKT is unique because it was introduced by a funding agency as a research partnership approach that aims to ensure research is translated into practice by engaging research users as partners throughout the entire research process.
To foster meaningful research partnerships, evidence-informed partnership principles (i.e., fundamental norms, rules, beliefs) need to be rigorously identified and developed in partnership with SCI researchers, research users, and funders.
rigorously co-developed guidance for SCI-specific partnerships is lacking. The Appraisal of Guidelines for Research and Evaluation II (AGREE II) Instrument is a rigorous, internationally accepted, consensus approach to transparently evaluating the development of clinical practice guidelines.
Accordingly, the purpose of this article is to transparently present the consensus-based “IKT Guiding Principles for Conducting and Disseminating SCI Research in Partnership” and describe the co-development process using the AGREE II reporting critieria.
Adapted versions of AGREE II instrument and reporting guidance
(supplemental appendices S1 and S2, available online only at http://www.archives-pmr.org/) were used to design the procedures and methodological approach for co-developing the IKT Guiding Principles. The instrument includes 6 domains that outline reporting criteria: (1) scope and purpose, (2) stakeholder involvement, (3) rigor of the process used to synthesize evidence and develop guidance, (4) clarity and presentation, (5) applicability and implementation of the guidance, and (6) editorial independence and acknowledgment of biases. Aligned with the AGREE II reporting criteria (see supplemental appendices S1 and S2), information pertinent to each domain are presented in the results and discussion sections of this article. For transparency, additional files associated with the project are available in the Open Science Framework (OSF) repository (https://osf.io/9mq24/; DOI: 10.17605/OSF.IO/9MQ24).
Procedures and methodological approach
In 2017, an initial core expert panel (n=9) was established to co-develop the methods for creating the first IKT guiding principles for the North American SCI research system (ie, SCI researchers, research users, funders involved in the planning, conduct, and/or dissemination of research). During a 1-day meeting in September 2017, this core panel consisting of SCI researchers, SCI research users, and funders of SCI research discussed the first 3 AGREE II domains (ie, scope and purpose, methods for stakeholder involvement, rigorous processes for synthesizing evidence and developing guidance). The panel decided to co-conduct research that would allow for the guiding principles to be informed by multiple data sources. Since 2017, the core panel met several times to identify new partners, collect and synthesize data sources, and establish processes to co-develop, co-implement, and co-evaluate the IKT Guiding Principles. In November 2019, the expanded multidisciplinary panel (n=17) (supplemental appendix S3, available online only at http://www.archives-pmr.org/), which included SCI researchers, clinicians, people with SCI, representatives from SCI community organizations, and funding agencies of SCI research, held a 2-day meeting to establish consensus on the IKT Guiding Principles. After the meeting, expert feedback was sought and the principles were refined. Figure 1 provides an overview of the project procedures and timeline.
The final IKT Guiding Principles for conducting and disseminating SCI research in partnership are presented in fig 2. The document includes a preamble, 8 guiding principles, and a glossary of terms. Aligned with the AGREE II domains, the outcomes of our co-development process are presented below.
Scope and purpose (AGREE II Domain 1)
The objectives, practical questions, target population, and potential end users of the principles were determined by the core expert panel and refined by the multidisciplinary expert panel. Through discussion, the panel members agreed on the following statements:
Objectives: To co-develop IKT guiding principles to support the meaningful engagement of the right users at the right time throughout the SCI research process.
Practical questions: What principles should be used to support the meaningful engagement of the right research users at the right time throughout the SCI research process?
Target population: Researchers and research users who adopt an IKT approach in an SCI-related research project in a North American setting. Research users included but were not limited to persons with lived experience of SCI, policymakers, service providers, other researchers, professional organizations, funders, and industry partners.
Potential end users: SCI researchers, SCI research users, and funders of SCI research.
Stakeholder involvement (AGREE II Domain 2)
Supplemental appendix S3 describes panel members’ expertise and roles within the core panel and the multidisciplinary panel. Panel members reviewed findings from the data sources, attended the 2-day AGREE II meeting, reviewed the IKT Guiding Principles, provided feedback on the first draft of IKT Guiding Principles via an online survey, and reviewed the manuscript. The IKT Guiding Principles were also reviewed by potential end users with experience in research partnerships. Moreover, the research associated with the data sources was conducted using an IKT approach.
Rigor of development (AGREE II Domain 3)
A 4-step process was used to synthesize evidence and co-develop the IKT Guiding Principles: (1) creating a long list of research partnership principles, (2) conducting a Delphi consensus study, (3) establishing consensus on the IKT Guiding Principles, and (4) collecting expert feedback about the resulting IKT Guiding Principles (see fig 1). These steps are described in detail below.
Step 1: Creating a long list of research partnership principles
A rigorous and systematic approach was used to create a long list of research partnership principles using 3 key data sources: (1) a review of reviews (n=86) on research partnership principles, (2) a scoping review of (n=13) SCI research partnership principles, and (3) interviews with SCI researchers and SCI research users (n=10) with experience partnering to conduct research (see OSF: https://doi.org/10.17605/OSF.IO/9MQ24). Of note, data regarding the effectiveness of principles was not found. From these data sources, research partnership principles were extracted using the following definition: “Fundamental norms, rules, or beliefs that represent what is desirable and positive for a person, group, organization, or community, and help it in determining the rightfulness or wrongfulness of its actions. Principles are more basic than policy and objectives and are meant to govern both.”
After removing principles with similar meanings, a list of 125 principles was created (18 overarching principles linked to 107 corresponding principles). Supplemental appendix S4 (available online only at http://www.archives-pmr.org/) provides an overview of the final principles and how these principles link to the data sources, and OSF provides further details on decision processes and corresponding subprinciples.
Step 2: Delphi consensus
To establish consensus on which principles should be used to guide SCI research partnerships, a Delphi consensus study was conducted among SCI researchers and SCI research users who had completed and/or were currently involved in 1 or more SCI research projects that involve a research partnership. The Delphi consisted of 3 rounds including at least 20 participants per round. In the first 2 rounds, participants indicated the extent to which they agreed that a principle should be used to guide SCI research partnerships. In the last round, participants categorized principles as “essential,” “desirable,” “irrelevant,” or “unsure.” In the last round, 37 principles were categorized as either essential (n=29) or desirable (n=8). Final results are available elsewhere (see OSF: https://doi.org/10.17605/OSF.IO/9MQ24).
Step 3: AGREE II consensus meeting
Prior to the AGREE II meeting, panel members (n=13) reviewed the summary of the Social Sciences and Humanities Research Council (SSHRC) grant proposal as well as abstracts and results from the 4 data sources and completed a conflict of interest declaration (ie, AGREE II Domain 6). First, the project lead outlined and confirmed the scope and purpose of the project, the relevant terms and definitions, and the consensus-based framework on research partnership domains.
to ensure that these differences were taken into account in the discussions and decisions.
In small working groups (n=3-4 members), panel members discussed and reviewed the overarching principles using a structured discussion guide that presented the Delphi results for each guiding principle and its corresponding principles (see OSF). Each group discussed 13 of the 19 overarching principles. Panel members discussed if (1) the principle is applicable to all research groups (researchers, research users, funders); (2) if the meaning is specific and unambiguous; and (3) if the principles should be adopted, adapted, or removed and provided suggestions for adaptations if needed. Findings from these discussions were synthesized and presented to the panel. The panel discussed the findings and established consensus on the final principles to be included. The panel also discussed the contents of the preamble of the principles and a glossary. Rationale for decisions to remove and include principles, as well as the content of the preamble and glossary, are available via OSF.
On day 2, the panel was presented with a draft version of the preamble, guiding principles, and glossary. The panel edited and refined the document through discussion. The panel reviewed the resulting document, and a consensus vote was held regarding the meaning of the preamble and principles. All panel members voted in favor of both the preamble, glossary, and the principles (100% consensus). Finally, clarity of presentation of the principles and barriers and facilitators to the application of the principles were discussed (see OSF for meeting agenda).
Step 4: Expert feedback
The IKT Guiding Principles were reviewed by panel members and potential end users with experience working in research partnerships using an online survey (Tables 1 and 2). Survey items were used to rate the principles for their clarity, usefulness, and appropriateness. End users were recruited through our panel’s extensive network and had the opportunity to provide open-ended feedback. In accordance with the regulations of the Tri-Council Policy Statement governing research ethics in Canada, ethics approval was not required for these online surveys because they are classified as quality improvement and not research (ie, to receive feedback on the principles not to answer a research question). All surveys are provided on OSF.
Table 1Panel member questionnaire item responses
Mean ± SD
The preamble of the IKT Guiding Principles is clearly expressed.
The preamble of the IKT Guiding Principles is specific and unambiguous.
The principles are clearly expressed.
The principles are specific and unambiguous.
The glossary is clearly expressed.
The glossary provides definitions of key terms that are specific and unambiguous.
IKT Guiding Principles document
The IKT Guiding Principles document is relevant for SCI researchers who want to adopt an IKT approach on an SCI-related research project.
The IKT Guiding Principles document is relevant for SCI research users who want to adopt an IKT approach on an SCI-related research project.
The IKT Guiding Principles document is relevant for research funders who are funding SCI-related research projects.
The IKT Guiding Principles document is useful to support the meaningful engagement of the right users at the right time throughout the SCI research process.
The IKT Guiding Principles document is useful to support the conduct of quality and ethical research that is relevant, useful, and/or useable for SCI research users.
The IKT Guiding Principles can be applied throughout the entire research process.
I intend to use the IKT Guiding Principles document.
The IKT Guiding Principles document uses language that is appropriate for SCI researchers.
The IKT Guiding Principles document uses language that is appropriate for SCI research users.
The IKT Guiding Principles document uses language that is appropriate for research funders who are funding SCI-related research projects.
NOTE. All items were rated on a 7-point Likert scale: 1= strongly disagree, 2=disagree, 3=somewhat disagree, 4=neither disagree nor agree, 5=somewhat agree, 6=agree, 7=strongly agree.
IKT Guiding Principles and clarity of presentation (AGREE II Domain 4)
Expert feedback: panel members
Mean ratings indicated that panel members’ responses (n=6) reflected that they agreed or strongly agreed that the preamble, principles, and glossary were clear, useful, and appropriate (see table 1).
Expert feedback: potential end users
Thirty-five potential end users with experience working in research partnerships responded to the survey. Of these end users, 18 were researchers, 8 were research users, 9 were both researchers and research users, and/or 10 had an SCI. Most end users conducted or used SCI research primarily (n=24). Mean ratings indicated that potential end users agreed or strongly agreed that the preamble, principles, and glossary were clear, useful, and appropriate (see table 2). Based on end-user feedback, minor changes were made to the document to improve clarity (eg, changes to wording, additions to definitions). Changes that would alter the meaning of the document were not implemented (supplemental appendix S5, available online only at http://www.archives-pmr.org/).
Using a systematic and rigorous consensus process, we reached consensus on 8 guiding principles to guide meaningful engagement of researchers and research users in conducting and disseminating research that is relevant, useful, and/or useable. The IKT Guiding Principles document also includes a preamble and glossary to support end users’ application of the principles and understanding of key concepts and terms. Given that data regarding the effectiveness of the principles were not found, the document is presented as guiding principles and not guidelines. Both the panel and potential end users perceived the IKT Guiding Principles to be clear, useful, and appropriate for supporting meaningful research user engagement in SCI research. Ultimately, these principles have the potential to reduce the likelihood of tokenism, contribute to more co-production of knowledge, and foster meaningful SCI research partnerships. Aligned with AGREE II reporting, implications for applicability and implementation of the IKT Guiding Principles are discussed below (ie, AGREE II Domain 5).
The panel outlined potential implications for disseminating and implementing the IKT Guiding Principles within the North American SCI Research System, which aligned with barriers and facilitators to adopting IKT identified in the broader literature.
Barriers specific to members of a research partnership primarily related to a lack of capability to implement the principles. Adopting the principles will require appropriate budgeting, an understanding of research processes, and strong interpersonal and communication skills. Training in conducting research in partnership is limited,
and without capacity building and incentives for both researchers and research users, these principles may be perceived as intimidating, time consuming, and/or risky to adopt. The structure of research funding could also present barriers. Timelines and eligible funding costs may be perceived as too stringent to allow for the flexibility and resources needed to build interpersonal relationships prior to submitting a grant application and throughout the research process. Likewise, research budgets need to account for the environmental and financial barriers that may prevent people with disabilities from engaging in research partnerships (eg, accessible transport and travel, personal support workers, adapted equipment, limited income).
The panel also highlighted facilitators that could support adoption and implementation. Research funders’ increased emphasis on the importance of partnered research and partnered funding opportunities may influence researchers and research users to consider adopting the principles. The SCI research system has a strong network of IKT champions both within and outside academia that can help promote and champion the principles. Researchers already using an IKT approach can be leaders and act as ambassadors for the IKT Guiding Principles. There is a strong network of research user organizations and funders who are committed to ensuring that there is “nothing about us, without us.”
Together these groups have strong potential to build capacity among research users who want to be involved in research partnerships (eg, the North American SCI Consortium, Praxis Spinal Cord Institute, SCI Canada).
Dissemination and implementation
The IKT Guiding Principles will only be as effective as our dissemination and implementation efforts. Our panel has co-developed a team that will support implementation. In addition to disseminating the principles via scientific articles and conferences, they will create bilingual tools, resources, and interventions that address barriers to implementation and support each of our potential users to disseminate and use the IKT Guiding Principles (see www.iktprinciples.com). This process will build on our initial understanding of barriers and facilitators and will involve interviewing researchers, research users, and funders to understand how to support implementation. Example resources and interventions may include educational videos, guidance for reviewing grant applications, and trainings.
Future directions for monitoring and co-evaluation
To monitor the dissemination and implementation of the IKT Guiding Principles, the panel has co-developed an evaluation plan, which includes monitoring implementation via quarterly online surveys sent to our partners’ membership and via Google Analytics. The surveys will be guided by the Reach, Effectiveness, Adoption, Implementation, and Maintenance
To gain an in-depth understanding of how the principles are implemented in a SCI research system, an in-depth mixed-methods evaluation will also be conducted within the International Collaboration on Repair Discoveries, an SCI research center focused on the development and translation of effective strategies to promote prevention, functional recovery, and improved quality of life after SCI. The panel will consider a process for updating the principles once these evaluations are complete.
Future directions for advancing the science of integrated knowledge translation
While the potential users of the guiding principles are members of the SCI research system, potential users of our established methodology are broader. Our systematic and transparent processes for co-developing guiding principles can be adopted to develop other domain-specific principles for other groups (eg, research related to people with multiple sclerosis, diabetes, arthritis, and/or stroke). The IKT-specific principles will contribute to better understanding the differences and similarities between IKT and other research partnership approaches (eg, CBPR, PAR). Future research should focus on identifying partnership strategies that could support the use of these principles in different phases of a research project as well as in different settings and areas.
Integrated Knowledge Translation Research Network Project Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation.
The development of the principles relied on the quality of the data reported in the literature. Data regarding the effectiveness of the IKT Guiding Principles are not yet available. Therefore, these are guiding principles and not guidelines. The principles were co-developed by a multidisciplinary North American team. Whether the principles are relevant and applicable to research systems in other countries and/or research domains remains to be seen.
The IKT Guiding Principles for conducting and disseminating SCI research in partnership are an important, needed, and foundational innovation that can be used to improve SCI research. By applying the AGREE II instrument, we have presented a systematic, rigorous, and engaged co-development process. These guiding principles have strong potential to support the development of meaningful and effective research partnerships that champion the expertise of research users and mitigate tokenism.
We thank Kristy Baxter (BHK), Dr Michael Kennefick (PhD), and Kelsey Wuerstl (MSc) for administrative support for the Consensus Meeting. We thank Dr Hugh Anton for his contribution to the first panel meeting. We also thank Dr Ian Graham for his guidance and feedback. Finally, we thank the staff at SCI British Columbia for hosting the Consensus Meeting.
Integrated Knowledge Translation Research Network Project Leads. Moving knowledge into action for more effective practice, programmes and policy: protocol for a research programme on integrated knowledge translation.
Supported by an Social Sciences and Humanities Research Council Partnership Development Grant (SSHRC, Canada, no. 890-2018-0044). Dr Gainforth is supported by a Michael Smith Foundation for Health Research Scholar Award (MSFHR, Canada, no. 16910), and Dr Mortenson holds a Canadian Institutes of Health Research (CIHR) New Investigator award. Of note, SSHRC did not have influence over the content of the guiding principles.
Disclosures: All authors completed conflict of interest declarations prior to the consensus meeting. The authors do not have competing financial interests in relation to the work. Of note, several authors play a leadership role within the SCI Research System (C.M., K.A., J.C., T.C., L.M., P.A.). Additionally, all authors are funded and involved in conducting research using and/or investigating an integrated knowledge translation (IKT) approach.