Translating Evidence to Support Transitions (TEST)

Transitions to Adulthood Center for Research, Systems & Psychosocial Advances Research Center, Department of Psychiatry, University of Massachusetts Medical School
Submitted by Marsha Ellison PhD, Sloan Huckabee PhD, Laura Golden, and Emma Pici-D’Ottavio

Focus

The Translating Evidence to Support Transitions (TEST) grant aims to increase the use and adoption of research-based best practices by transition planners who support high school students with emotional behavioral disturbance (EBD) who receive special education services. The National Implementation Research Network (NIRN) Staged-based Implementation Framework directs the process of developing and implementing practice guides for special education transition planners at the high school level.

Context

Research clearly points out challenges and poor outcomes faced by many students with EBD. Students with EBD dropout of high school more than any other group of students with disabilities.^1,2 A shortened education often equates to lower wages, lower rates of employment, and poorer health later in life.^3-5 Students with EBD also enroll in postsecondary education less frequently and have lower rates of postschool employment than many other categories of students with disabilities.^6-9

Special education transition planners at the high school level help students with EBD plan for life after high school. The goals and services for such a transition are outlined in each student’s Individualized Education Program (IEP). However, without a set of best practices to support their work, transition planners may not always have the tools and guidance to best prepare students for life after high school.

To help address these poor outcomes, the TEST team at the Transitions to Adulthood Center for Research, (Transitions ACR) has partnered with knowledge translation experts—the National Technical Assistance Center on Transition (NTACT), the American Institutes for Research (AIR), and SRI International. Together, their goal is to increase the use and adoption of research-based best practices by transition planners who support high school students with EBD. Research has identified three best practices for transition planning for such students:

1. Hold transition planning meetings that are directed by the students themselves.
2. Ensure that students take four or more Career and Technical Education courses during high school,
3. Include adult services agencies and organizations in the transition planning process, and

Practice Guides have been created to address each of these best practices:

• Supporting Student-Led Transition Planning for Students With Emotional Behavioral Disturbance
• Incorporating Career and Technical Education in Transition Planning for Students With Emotional Behavioral Disturbance
• Partnering With Community Agencies in Transition Planning for Students With Emotional Behavioral Disturbance

KT Activity

For its activities, the TEST team uses the NIRN Stage-based Implementation Framework to promote the integration of research-based evidence into transition planning practices. Built on implementation science, the framework serves as an implementation planning tool for early care and education programs and systems and is relevant to stakeholders invested in issues specific to children and education.¹⁰ The NIRN Framework includes four stages: exploration, installation, initial implementation, and full implementation. Each stage incorporates three core components: implementation teams, data feedback loops, and implementation infrastructure. The TEST team relies on these components to continually improve its materials.

The design of objectives and tasks reflects the four stages and three core elements:

1. Develop research-informed materials and procedures for use by transition planning teams that are tailored to students with EBD (Exploration). The TEST team engaged an implementation team of transition planning stakeholders and members of the Transition Alliance of South Carolina (TASC). TASC provides resources to increase the capacities of interagency teams to collaboratively and effectively serve young adults with disabilities who are transitioning from high school to adult life.¹¹ During in-person meetings, members of TASC provided feedback and guidance on procedures and draft versions of the practice guides from TEST. Members of TEST and TASC worked together to better understand the needs, goals, capacities, and challenges of transition practitioners at the local level in South Carolina.

2. Pilot test procedures and materials, in partnership with implementation teams, in a South Carolina school district to inform the development of final TEST procedures and materials (Installation and Initial Implementation). The TEST team:

   • Identified and recruited three high schools in Darlington County, South Carolina;
   • Trained three transition teams on TEST procedures and use of materials;
   • Submitted TEST procedures and materials for implementation by three transition teams;
   • Collected ongoing feedback on the use of TEST practice guides and procedures;
   • Reviewed and revised TEST practice guides and procedures; and
   • Evaluated the pilot process.

3. Provide implementation support and technical assistance to transition teams in two states that will result in a TEST implementation guide (Full Implementation). The TEST team

   • Selected two states (Maine and Connecticut, who collaborate with NTACT) for intensive technical assistance on implementing TEST procedures and three TEST practice guides;
   • Involved stakeholder implementation teams at each school;
   • Provided technical assistance on the use and adoption of TEST practice guides and procedures through a 2-day, in-person training and monthly virtual coaching calls;
   • Collected iterative feedback and evaluation data about the use, quality, and relevance of TEST practice guides and procedures and on technical assistance via coaching calls, pre- and post-training evaluations, student surveys, implementation checklists, de-identified IEPS, etc.;
   • Gathered information for a TEST Implementation Guidance and Technical Assistance Toolkit that will provide instruction on anticipated challenges and useful strategies for implementing TEST materials and procedures in school and transition planning settings.

4. Present TEST best practices, practice guides, and implementation guide at a national capacity building institute for special education transition planning teams at the high school level. This forum puts the piloting and implementation of TEST into a national arena for adoption and use by local stakeholders across the nation. In Year 5, the TEST team will lead presentations at NTACT’s biannual Capacity Building Institute in Charlotte, North Carolina. Here, representatives from state and local education agencies from across the United States will engage in content sessions, networking, and other activities to increase their knowledge of evidence-based and promising practices—with the purpose of implementing new knowledge in their home states. In Year 5, the TEST team will also lead presentations during Cadre meetings. NTACT holds Cadre meetings every 6 months. Here, stakeholders gather to discuss the “how” for implementing the “what.” The goal is to make changes to practices in the education system that facilitate improved educational outcomes.¹²

5. Disseminate TEST materials nationally. The TEST team will:

   • Create tip sheets, webinars, social media posts, etc. (for distribution by Transitions ACR) about TEST materials, procedures, and implementation guidance.
   • Evaluate the reach and impact of materials from TEST. Existing materials will be reviewed and synthesized, using plain language summaries, to highlight key messages.

Impact

The TEST grant aims to improve the outcomes of students with EBD who receive special education services and are transitioning to life after high school. Achieving this goal starts with improving the practices of special education transition planners who support such students as they plan for the future. The TEST team strives to meet several direct, intermediate, and long-term outcomes:

Direct outcomes:

• Each of three transition teams in South Carolina deliver a best practice in transition planning.
• School districts in two states receive technical assistance on implementing best practices across schools.
• High school transition teams across the country are knowledgeable about best practices in transition planning for students with EBD who receive special education services.

Intermediate outcomes:

• Adoption and use of best practices in transition planning for students with EBD is widespread.
• Students with EBD have improved transition plans.

Long-term outcomes:

• Students with EBD have improved employment and education outcomes after high school.

Throughout implementation, the TEST team measures its impact and progress toward the overall goal. Mixed-methods evaluations produce qualitative and quantitative data.

The TEST team also collects and relies on iterative feedback from end users and stakeholders. Feedback is used to (a) gauge the usefulness, quality, and relevance of TEST practice guides, procedures, and technical assistance; (b) assess whether activities are occurring as planned and intended outcomes are being achieved, and (c) determine whether progress is being made toward improving transition outcomes for students with EBD. The TEST team collects feedback in many ways, for example:

• Coaching calls;
• Pre- and post-training evaluations;
• Surveys of students, parents, and community partners;
• Teacher interviews;
• Implementation checklists;
• De-identified IEPs; and
• Evaluations of utility and satisfaction with content sessions.

Lessons Learned

• Implementation can be derailed by contextual factors in a state’s education system or in a school district. Despite careful planning, the TEST team could not control all factors or prepare for all contingencies. Some contextual factors presented challenges during Installation and Initial Implementation.
• Getting evaluation data and feedback from some end users, especially parents and community partners, can be difficult. Parents are not always engaged with their child’s education and/or appear to be distrustful toward or frustrated with their child’s school. Community partners may not respond to outreach attempts from educators or participate fully in transition planning meetings.
• Districts and states were excited to participate in the full implementation of TEST practice guides and procedures. The TEST team selected sites from a wide range of applicants. School districts and state educational systems appear hungry for TEST resources regarding transition planning for students with EBD who receive special education services.
• Despite great interest and having received a stipend for participation, many schools did not follow through with full implementation.

Contact Information

Transitions to Adulthood Center for Research, Department of Psychiatry, University of Massachusetts Medical School
https://www.umassmed.edu/TransitionsACR/research/projects-by-grant/translating-evidence-to-support-transitions/
222 Maple Avenue, Chang Building,
Shrewsbury, MA 01545
Key contact: Laura Golden, 508-856-8483, Laura.Golden@umassmed.edu

References

1. Duchnowski, A., & Kutash, K. (2011). School reform and mental health services for students with emotional disturbances educated in urban schools. Education and Treatment of Children, 34(3), 323–346.
2. National Center on Secondary Education and Transition. (2002). NLTS2 data brief: Introducing the NLTS2. Retrieved from http://www.ncset.org/publications/nlts2/NCSETNLTS2Brief_1.1.pdf
3. Pleis, J. R., Ward, B. W., & Lucas, J. W. (2010). Vital and health statistics: Summary health statistics for U.S. adults: National Health Interview Survey, 2009 (Vol. 10). Hyattsville, MD: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics.
4. Rouse, C. E. (2007). Quantifying the costs of inadequate education: Consequences of the labor market. Washington, DC: Brookings Institution Press.
5. U.S. Department of Labor, Bureau of Labor Statistics. (2010). Tabulations.
6. Anderson, J. A., Kutash, K., & Duchnowski, A. J. (2001). A comparison of the academic progress of students with EBD and students with LD. Journal of Emotional and Behavioral Disorders, 9(2), 106–115. doi: 10.1177/106342660100900205
7. Lane, K. L., Carter, E. W., Pierson, M. R., & Glaeser, B. C. (2006). Academic, social, and behavioral characteristics of high school students with emotional disturbances or learning disabilities. Journal of Emotional and Behavioral Disorders, 14(2), 108–117.
8. Newman, L., Wagner, M., Knokey, A. M., Marder, C., Nagle, K., Shaver, D., & Wei, X. (2011). The post-high school outcomes of young adults with disabilities up to 8 years after high school: A report from the National Longitudinal Transition Study-2 (NLTS2). Menlo Park, CA: SRI International.
9. Wagner, M., & Davis, M. (2006). How are we preparing students with emotional disturbances for the transition to young adulthood? Findings from the National Longitudinal Transition Study—2. Journal of Emotional and Behavioral Disorders, 14(2), 86–98.
10. Metz, A., Naoom, S. F., Halle, T., & Bartley, L. (2015). An integrated stage-based framework for implementation of early childhood programs and systems (OPRE Research Brief OPRE 2015 48). Washington, DC: Office of Planning, Research and Evaluation, Administration for Children and Families, U.S. Department of Health and Human Services.
11. Transition Alliance of South Carolina. (n.d.). About us. Our history. Retrieved from https://transitionalliancesc.org/about-us/#history
12. Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M., & Wallace, F. (2005). Implementation research: A synthesis of the literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FIMHI Publication #231).

---