4 Ways Implementation Support Centers Assist in the Delivery of Evidence-Based Programs

Overview

Evidence-based policymaking allows government leaders to achieve the best possible outcomes by investing in programs that have been rigorously studied and proved to work. How these programs are implemented, however, plays a critical role in determining whether they are successful. If delivered in a manner that is inconsistent with the original model—for instance, in an inappropriate setting, to a different population, or with unequipped staff— evidence-based programs can fail to achieve the outcomes they promise and that policymakers expect.

States face several challenges to ensuring that evidence-based programs are effectively delivered. First, state governments often rely on community-based providers to deliver services but have limited capacity to carefully monitor these organizations. Second, communities vary widely, and what works in one may not always work in another. Understanding what adaptations can be made to an evidence-based program—and how such changes may affect outcomes—can make the difference between a successful program and one that is ineffective or even harmful. Yet, few state agencies have the expertise to recognize which program adaptations should be made, and they are rarely equipped to provide this level of support to service providers.

To address these challenges, some states have formed what are called implementation support centers. These entities work alongside and often at the direction of state or local government to support efforts to effectively implement evidence-based programs. As states consider ways to monitor program implementation, these centers provide an alternative to having to build such capacity inside of a government agency.

This fact sheet highlights four ways these centers can support more effective implementation of evidence-based programs, thereby increasing the likelihood that these investments will achieve better outcomes and generate positive net benefits for taxpayers.

1. Provide training and technical assistance on how to deliver evidence-based programs.

Evidence-based policymaking relies on the assumption that staff who provide services to the public are trained to deliver those services according to a prescribed approach, also known as fidelity. Implementation support centers¹ can train staff on the delivery of evidence-based programs and practices or coordinate this training with program developers (organizations with expertise in delivering a particular intervention). Such training offers greater assurance to policymakers that funded evidence-based programs will achieve consistently positive results.

For example, since 1999, the Child Health and Development Institute (CHDI) of Connecticut has supported the implementation of eight evidence-based mental health programs across the state.² The institute, an independent nonprofit organization initially established with funding from the Children’s Fund of Connecticut, helps disseminate evidence-based practices, provides ongoing coaching and supervision, and works with providers to use data to monitor and continuously improve their services. CHDI staff has worked with program developers to train more than 800 clinicians and staff in 35 community mental health centers to deliver Trauma-Focused Cognitive Behavioral Therapy, an evidence-based treatment for child victims of trauma.

“A lot of our work is as a convener, bringing together academic institutions, state agencies, community-based providers, family advocates, and others to bridge the research-to-practice gap,” said Jason Lang, director of dissemination and implementation at the CHDI. “We help to develop practical solutions for implementing evidence-based programs in the real world.”³

Implementation centers also bring challenges faced by service providers to the attention of government administrators and leadership and help to identify solutions. In Pennsylvania, the EPISCenter, which has supported more than 300 replications of evidence-based programs across the state over the past 10 years, regularly surveys community-based service providers to identify key implementation barriers and communicates these findings back to the government funding agency, the Pennsylvania Commission on Crime Delinquency (PCCD). This has resulted in policy and funding changes that have helped providers to improve service delivery and make better use of taxpayer dollars.

2. Create tools and processes to strengthen implementation oversight by state agencies.

Governments operate many programs, including those that are evidence-based, that lack clear instructions that enable providers to implement them consistently. Similarly, many programs lack common outcome measures that would allow state agency leaders to monitor and compare their progress. Implementation support centers can help address these challenges by creating tools that make it easier to monitor implementation, measure common outcomes, and make sure that programs are on the right track to achieve results.

The EPISCenter has developed common indicators for measuring implementation fidelity and outcomes along with spreadsheet-based tools for reporting these data for 16 evidence-based programs.⁴ The tools support providers in monitoring their own programs and allow the center and PCCD staff to monitor implementation remotely and address problems earlier. Staff from the EPISCenter use the tools to analyze the performance of individual providers, as well as to aggregate and report these data to key stakeholders in state government, including PCCD staff and leadership.

“We work to connect the needs of these different stakeholders,” said Stephanie Bradley, director of EPISCenter. “This includes policymakers who want to invest in evidence-based programs but don’t know the intricacies of implementing them, researchers and program developers who understand the science but are not involved in ongoing implementation and the challenges that arise, and staff in community-based organizations who are doing their best to deliver effective services but don’t have the time to digest all of the research and have many administrative demands to meet.”

3. Help jurisdictions to identify needs and select appropriate evidence-based interventions.

Prior to implementing any evidence-based intervention, policymakers and providers must understand the needs of a particular jurisdiction and identify the programs that can best address these needs. Implementation support centers can help bolster these efforts by leading communities through a structured process that gathers data about target populations, the prevalence of key problems, and the risk factors that could be addressed through interventions. Doing this work upfront can help ensure that communities have the right mix of programs to meet their local needs and gives state and local officials confidence that they aren’t funding ineffective or unnecessary programs.

For example, in 2014, the EPISCenter conducted a statewide gap analysis to determine whether additional evidence-based programs were needed to help prevent youth delinquency. These analyses led to the recommendation of nine new programs to address unmet, statewide needs, from which communities can choose to deliver services using their existing state funding. The center now supports implementation of 18 evidence-based interventions across the state, which has generated an estimated $55.8 million return on investment, based on avoided long-term costs.⁵

4. Build broad support across government for using evidence-based programs.

Effective implementation of programs is fundamental to evidence-based policymaking because it gives government leaders the confidence that their investments are likely to yield better results. Yet for evidence-based policymaking to gain traction and become sustainable requires extensive outreach and education to line staff, government agency leadership, policymakers, and community advocates. Implementation centers can play an important role

in educating these stakeholders on the importance of using programs that research has shown to be effective in achieving positive outcomes, and the critical role that implementation plays in ensuring services are delivered correctly.

In Washington state, the Evidence-Based Practice Institute, a university-based organization created by the state Legislature in 2008, was instrumental in leading a stakeholder engagement process to support the implementation of landmark legislation calling for significantly increasing the percentage of funding to evidence-based programs across three major youth-focused service sectors.⁶ As part of a stakeholder engagement process, staff from the institute helped facilitate community outreach meetings with citizens, provider organizations, and agency leadership; lead an informal needs assessment to identify service gaps, particularly in communities of color that were underrepresented in the public comment process; and develop a communications and education strategy throughout the state, including a website.

Conclusion

While state and local governments are increasingly investing in evidence-based programs, some struggle to implement them effectively. Some service providers and government agencies are still in the early stages of building their capacity to effectively deliver these programs and monitor their quality, which often requires additional staff training or certification requirements, and development of data systems. Implementation support centers have shown promise in eliminating barriers to effective program implementation and dissemination of evidence-based practices, helping to ensure that the benefits of investments in these programs are realized.

Endnotes

1. To date, only two reports have defined what an implementation center is: Mettrick, J., Harburger, D.S., Kanary, P.J., Lieman, R.B., and Zabel, M., “Building Cross-System Implementation Centers,” The Institute for Innovation and Implementation (2015): 3, https:// theinstitute.umaryland.edu/newsletter/articles/bcsic.pdf; and Robert P. Franks and Christopher T. Bory, “Who Supports the Successful Implementation and Sustainability of Evidence-Based Practices? Defining and Understanding the Role of Intermediary and Purveyor Organizations,” New Direction for Child and Adolescent Development 149 (2015): 41–56, http://dx.doi.org/10.1002/cad.20112. Both of these reports informed the contents of this brief and how we define implementation centers.
2. For more information, see the Child Health and Development Institute of Connecticut’s website: http://www.chdi.org/about/history/.
3. Jason Lang (director, Child Health and Development Institute of Connecticut), interview by the Pew-MacArthur Results First Initiative, April 19, 2016.
4. Stephanie Bradley (director, EPISCenter), interview by the Pew-MacArthur Results First Initiative, May 13, 2016.
5. EPISCenter, “2014 Annual Report” (2014), http://www.episcenter.psu.edu/sites/default/files/outreach/EPISCenter-Annual-Report-2014. pdf.
6. Gabrielle D’Angelo et al., “Community Engagement Strategies for Implementation of a Policy Supporting. Evidence-Based Practices: A Case Study of Washington State,” Administration and Policy in Mental Health and Mental Health Services Research (2015): 1–10, https://link. springer.com/article/10.1007/s10488-015-0664-7.