(Updated) Targeting Stakeholders and Tailoring Knowledge as Communication Strategies in Assistive Technology: Three Randomized Controlled Case Studies

University at Buffalo (SUNY)
Center on Knowledge Translation for Technology Transfer (KT4TT)
Submitted by Vathsala Stone

Focus

This case study focuses on implementation of an NIDILRR-supported research project that developed and evaluated knowledge translation (KT) strategies designed to communicate new knowledge to rehabilitation technology stakeholders. Two strategies were addressed: tailor-and-target and target-only. Their communication effectiveness was evaluated in a series of three randomized controlled studies, focusing respectively on stakeholders of augmentative and alternative communication (AAC), recreational access (RecAcc) and wheeled mobility (WhMob) technologies. Each study used the two strategies as interventions that communicated new knowledge. The traditional diffusion method was used as the control. Participants represented diverse stakeholders of the respective technology.

In March 2018, a panel presentation focused on the KT Casebook occurred at the 40th Annual NARRTC (formerly known as the National Association of Rehabilitation Research and Training Centers) Conference. View the presentation materials about this Casebook entry below.

• PowerPoint Presentation
• Text Description of the PowerPoint
• Edited transcript of presentation by Vathsala Stone

Context

The context of research and development (R&D) projects, such as that exemplified by NIDILRR’s technology grantees, has unique implications for KT. Moving new knowledge to end users who could benefit from it involves a crucial step of technology transfer (TT). R&D projects generate technology-based knowledge. For ready use by consumers, this knowledge must be transformed from conceptualized form to prototype to product (Lane & Flagg, 2010). The effort involves engaging multiple stakeholders in a KT-for-TT process. In addition to researchers, clinician practitioners, policymakers, and consumers, it includes manufacturers as well as consumer advocates and knowledge brokers. For impact to happen, the knowledge should be applied first by each of the stakeholder types in their own context. Communicating new knowledge to these diverse and geographically dispersed stakeholders is a unique challenge.

KT Strategies

In response to the challenge of communicating new knowledge to a diverse, geographically scattered stakeholder population, two communication approaches have been proposed as showing promise for implementing KT for TT: deliberately targeting potential users for dissemination (Graham, Logan, Harrison et al., 2006) and tailoring the knowledge to specific stakeholder contexts (Graham, Logan, Harrison et al., 2006). These approaches provided the basis for the target-only and the tailor-and–target strategies that were designed and tested.

In each of the three studies, the KT4TT Center selected a peer-reviewed publication by an NIDILRR technology grantee as the new knowledge to be communicated to stakeholders. For the tailor-and-target strategy, the KT4TT Center tailored the research findings (new knowledge) to five types of stakeholders. The KT4TT Center developed a contextualized knowledge package (CKP) written in five different versions, one for each stakeholder type. For increased accessibility, the KT4TT Center adapted both language and format to stakeholder background, emphasizing readability and preparation of both printed and digital versions. The KT4TT Center presented the potential value of the findings to each stakeholder type through concrete examples of how to apply them in their specific living and working contexts, so they might be persuaded to derive benefit from applying the knowledge. The KT4TT Center pilot tested each CKP by a representative stakeholder prior to its use as intervention in the three studies. The strategy combined the CKP with a tailored webcast, which demonstrated application of the findings in the context of the specific stakeholder type. Optional technical assistance was offered on request.

To maintain authenticity of content, the KT4TT Center involved the authors of the publications in the development of both the CKPs and the webcasts. The webcasts featured presentation of findings by the respective author and demonstration by a user. The CKPs and the webcasts are available for view on the KT4TT website.

The tailor-and-target strategy used a targeted approach for delivering the CKPs and the webcast to stakeholders. The KT4TT Center identified potential stakeholders by interviewing organizations of their affiliation, and mapping the values and preferences of these stakeholders (Lane & Rogers, 2006). The organizations also helped us recruit participants for the studies.

For the target-only strategy, the KT4TT Center preidentified and recruited participants as described earlier. The KT4TT Center then mailed the published research article containing the new knowledge directly to each individual participant. No tailoring was done to the content of the publication (the findings).

Impact

All three studies used the same randomized controlled design to evaluate the two strategies. To measure participants’ level of knowledge use as evidence of strategy effectiveness, the KT4TT Center developed the Level Of Knowledge Use Survey (LOKUS) instrument. The instrument demonstrated strong validity, reliability, and responsiveness to change (Stone, Nobrega, & Lane, 2014). A copy of the instrument can be viewed on the KT4TT Center website.

Participants in the three studies (n1=207; n2=288; and n3=210) represented five stakeholder types: brokers/advocates, practitioners/therapists, industry, researchers, and consumers. Each study assigned participants randomly to tailor-and-target group, target-only group, and control group. All three groups answered the LOKUS instrument three times: as baseline (pretest), at 4 months (posttest 1) and at 8 months (posttest 2). During the interval between tests, the tailor-and-target and the target-only groups received the corresponding tailored or nontailored materials as intervention. The tailor-and-target group received the CKP in the first 4-month interval, and the webcast-plus-technical-assistance offer in the second 4-month interval. The target-only group received the research publication in the first 4-month interval but nothing afterward. The control group received no intervention materials at any time.

Responses were analyzed for differences in knowledge use levels among the three groups, using nonparametric statistics. To reach a conclusion about the effectiveness of the two strategies, the KT4TT Center required that changes observed between pretests and posttests in these two groups should surpass any change seen in the control group, which might reflect an effect due to repeated testing.

Effectiveness of strategies: The tailor-and-target strategy was found to be effective in all three studies. Pretest-to-posttest changes were significant in the first 4 months (p<.001), as well as over the 8-month study period (p<.001). Thus, the CKP was an effective format, and the strategy as a whole was also effective.

In the case of the target-only strategy, there were significant pretest-to-posttest changes (p=.001; p<.001; and p<.001) across all three studies, over the first 4-month period when active intervention had occurred through targeted delivery of the publication. Thus the target-only strategy was also shown to be effective.

Although both strategies were effective compared to control, they were not significantly different from each other in any study (p=.086; p=.323; and p<.615). Neither strategy was better than the other for the collective sample of five stakeholder types.

Strategies and Stakeholders: The two strategies were variously effective with different stakeholder types, with different benefit implications. Details are available in the full article (Stone, Lane, Tomita, et al., 2015).

Raising stakeholder awareness: Both strategies effectively moved nonaware participants to awareness, interest, and use levels in all three studies. The effect was observed at 4 months and was retained over the total 8-month period.

Persuading non-users to use the knowledge: In terms of statistical significance, results were not consistent across the three studies. In the first two studies, both strategies moved non-users (i.e., individuals who were nonaware, aware, or interested) to the Use level in the initial 4-month period. But the effect was not retained beyond that point. Same results were observed also in the third study, but only for the target-only strategy. The tailor-and-target strategy did not move non-users to the Use level initially, but did so over the total 8-month period.

Nevertheless, from the viewpoint of practical significance, data from all three studies consistently showed that only a small number of individuals either decided to use the knowledge or maintained such a decision. While this low number may be explained by the length of the study or contextual constraints to use, it also raises the issue of how much value the user attributes to the new knowledge. It underscores the importance of ensuring relevance in the knowledge being generated.

Lessons Learned

The logistical and operational processes of the randomized controlled studies were seamless, and they effectively supported the rigor of the design. One limitation was lack of provision for a follow-up interview of the participants in the design. The LOKUS instrument called for self-reported responses and required participants to recall information for repeated responses over the study period. So, learning in depth about the actual use of the findings and the reasons for not using them would have enlightened the results further. Future designs could also include shorter or longer study periods, to aid in understanding how long after dissemination it is reasonable to expect awareness, interest, or use.

Contact Information

Center on Knowledge Translation for Technology Transfer (KT4TT)
Center for Assistive Technology
100 Sylvan Parkway, Suite 400
Amherst, NY 14228
716-204-8606
Fax: 716-204-8610
http://publichealth.buffalo.edu/cat/kt4tt.html
vstone@buffalo.edu

---