Abstract

As “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice,” implementation science (IS) offers the potential to translate effective innovations in policing across agencies with fidelity and sustainability in support of a commitment to evidence-based policing (EBP). Despite this potential, and its widespread use in adjacent fields facing similar challenges, implementation science remains almost completely unstudied and unutilized in police settings. To fill these gaps in research and practice, this paper provides an orientation to IS for police researchers and practitioners. It recounts EBP’s historical roots in an evidence-based approach to health care, demonstrates the commonalities that make IS as natural to policing as medicine, and surveys the paucity of existing literature on the employment of IS in policing. It adapts a conceptual model of IS to policing, presents two well-developed frameworks, and introduces three types of hybrid implementation/effectiveness trials suitable for use in dynamic police settings. It then provides illustrative cases in policing where the use of IS would be apt, and highlights the importance of the de-implementation of substandard or problematic practices as a key aspect of IS. It concludes by discussing how police practices that fully embrace evidence will nonetheless be guided by contestable values and norms, and how IS provides a way to address this concern. The paper provides research and practice agendas for integrating IS into EBP as police strive to adopt evidence-informed practices that deliver public safety, respect rights, and increase community satisfaction and trust.

Introduction

Evidence-based policing (EBP),1 the movement to identify and adopt police practices supported by scientific evidence, is frequently discussed in policing circles but has been slow to catch on in the United States (Todak & Huey, 2021). The earliest well-known call to systematically embrace scientific evidence2 in the conduct of police work dates to Lawrence Sherman’s (1998) monograph “Evidence Based Policing,” published by what is now the National Policing Institute. In it, he surveyed the paucity of rigor in police practice and made a case for using evidence to improve it. He called for transforming police agencies into organizations that internalize a scientific, evidence-based approach to both their operations and evaluation of their performance. However, a gap remains in better understanding how police can effectively identify, adopt, implement, and sustain practices supported by evidence, which has proven to be one of the most considerable obstacles to the embrace of EBP. Implementation science (IS) offers tools to improve implementation outcomes and methods to assess such efforts.

As “the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice” (Eccles & Mittman, 2006), implementation science has guided practice in major fields including healthcare and social services (Estabrooks, et al., 2018; Rudd et al., 2020), various forms of correctional practice (Knight, et al., 2016; Zielinski, et al., 2020), criminal legal institutions (Ducharme, et al., 2013), and public and global health (Madon, et al., 2007; McIssac et al., 2018). Put more directly, IS concerns the “how to” of translating proven interventions from academic, scientific settings to the field in all its varying conditions, sustaining them with fidelity over time, and adapting them as necessary. Grounded in examining contextual factors that impact the uptake of EBP, implementation science utilizes strategies that address those contextual factors to enhance the receptivity, acceptance, and sustainment of a practice. Implementation science is an emerging field that is largely untapped and acutely underutilized in policing. It should not be confused with other evaluation methods, such as process or formative evaluations, which are largely focused on structural aspects of a program. Instead, IS focuses on contextual factors that might influence these structural factors such as leadership and/or staff level support, external stakeholders’ influence, and so on.

As an experimental science, IS therefore focuses on two things that are necessarily entwined: 1) the careful and deliberate use of strategies for the successful implementation of EBP by police practitioners, and 2) the experimental evaluation of the effectiveness of the implementation strategies themselves. It is beneficial to be mindful of careful implementation, but that does not, in and of itself, constitute a scientific practice. The science of implementation will emerge in policing when these mindful researchers and practitioners use experimental designs to test the comparative effectiveness of various implementation techniques and strategies.

In this article, we lay the groundwork for integrating IS into EBP for use by researchers and practitioners, proceeding as follows. We will: 1) Recount EBP’s historical roots in an evidence-based approach to health care, and demonstrate the commonalities that make IS as natural to policing as medicine; 2) Survey the existing literature on the employment of IS in policing; 3) Present and summarize well-developed IS frameworks available to researchers and practitioners of EBP; 4) Introduce three types of hybrid implementation/effectiveness trials used in health research suitable for use in dynamic police settings; 5) Provide illustrative cases where the use of IS would be apt; 6) Highlight the importance of the effective de-implementation of substandard or problematic practices as a key aspect of IS; 7) Discuss how police practice that fully embraces evidence will be guided by contestable values and norms, and IS provides a way to reconcile this concern, and 8) Conclude by suggesting a research and practice agenda for integrating IS into EBP as police contend with calls to adopt evidence-informed practices, and address the counterinfluences in policing that hamper its effectiveness.

THE THING: A brief orientation to implementation science

Before proceeding, a primer on the essence of IS may help orient unacquainted readers. IS examines the context of an organizational change—its setting, the relevant populations, and the sociopolitical environment—to understand how these factors impact the implementation and sustainment of that change. The approach may be best conveyed by a series of questions that relate to different parts of the undertaking, presented here by adapting the concept of “THE THING,” an IS teaching tool developed by Curran (2020):

• What is the THE THING that an agency seeks to implement? For EBP, THE THING may range from a small internal change such as an update to a data intake form, to a systematic overhaul of suspect interview procedures, to a new crime control strategy or technological innovation.

• Which factors will impact successful implementation of THE THING? This can be broadly understood as barriers and facilitators to implementing a particular change. Such factors exist in both the inner context (i.e., within an organization and among its staff) and the outer context, where external actors and forces exert influence on a program’s feasibility and prospects of success (Aarons et al., 2011; Damschroder et al., 2009).

• How will an agency actually implement THE THING? Implementation strategies are empirically supported procedures (e.g., education, coaching, collaboration techniques, new or revised policies and procedures) that address contextual factors (i.e., barriers and facilitators), and coalesce into a protocol to implement the intervention, i.e., the EBP.

• What happens when THE THING is implemented? Evaluating outcomes is essential, and should involve three types:

  • Implementation outcomes such as feasibility and acceptance of THE THING within the organization, how well it was adapted to local conditions, and how sustainable the new practice is;

  • Service-related outcomes (in this case, policing-related outcomes), which are the effects of implementing THE THING: for example, the identification of suspects, increased closure rates, the effective use of arrest and its alternatives;

  • Distal outcomes: if THE THING is effective and successfully implemented, it should yield desirable ultimate outcomes such as the reduction of crime, the reduced incidence of property loss, injury and death, and increased community trust and satisfaction.

The goal of the process illustrated above is twofold: to ensure successful organizational change in the setting under study, and to create generalizable knowledge that can yield the same type of successful change in other settings. With this basic outline of IS in mind, we will proceed to demonstrate the role IS can play in police innovation and reform, and present ways to make it a valuable component of police research.

Evidence-based policing’s roots in health services

Policing, public safety, and healthcare have commonalities that emphasize the need for scientific evidence in the improvement of operations and practices. Although education and licensing are largely standardized across medicine—in contrast to policing—the size and scope of healthcare practice rivals the police profession in its complexity and fragmentation. There are over a million physicians with over 230,000 practices, including over 6,000 hospitals (American Hospital Association, 2022). Similarly, in the United States, there are approximately 750,000 police officers employed by 18,000 police departments and other agencies. This makes the implementation of new practices in both settings challenging. In police departments, this challenge has sometimes been met when a charismatic leader with the right support brings about transformative change. But given that the average police chief of a large, complex agency is in command for about five years (Police Executive Research Forum, 2022), the likelihood of lasting change is slim given any organization’s tendency to privilege the status quo. Sustained innovation in policing therefore requires methods that do not depend on bold, visionary leaders, but instead on a much more systematic approach.

Two things may surprise practitioners and researchers who study policing’s relationship with evidence. First, EBP saw its birth in an explicit attempt to take the lessons learned from improving the practice of medicine and applying them to policing (Sherman, 1998). In other words, policing’s growing emphasis on scientific evidence was based on a similar movement in health care that sought to embrace evidence-based practices to improve its outcomes (Institute of Medicine, 2001). If IS comes to policing, it would follow the same path as the origins of the EBP movement itself. Second, despite being popularly regarded as setting the standard of evidence-based practice, medicine actually has a history of resistance to using scientific findings in practices (Greenhalgh et al., 2014; Pope, 2003). At times, some have argued, such resistance has matched or exceeded that of police (Sherman, 1998). Drawing attention to this resistance in medicine, implementation science research papers have opened with the observation that it takes 17 years, on average, for health services to adopt about 14% of the innovations well-supported by clinical evidence (Bauer et al., 2015; Morris et al., 2011), and many practices with a solid base in evidence are never introduced into clinical practice at all (Kirchner et al., 2020).

Police would see reflections of their own profession in them in the sources of this delay. Physicians have resisted evidence-based medicine by contending the scientific knowledge produced by controlled studies is too narrow and rigid, and fails to respect the “art” of medicine, namely, the practical knowledge of front line practitioners dealing with patients in time- and resource-constrained settings (Pope, 2003). Such resistance isn’t only an artifact of medicine’s culture, but derives from observations that many rigorous trials, including randomized controlled trials, may lack consideration of real-life conditions, have diminished reliability given common selection biases, fail to conduct long enough follow-up periods, and are not replicated when re-examined or implemented in different settings (Berk, 2005; Deaton & Cartwright, 2018; Tucker & Roth, 2006). Researchers who have worked with police have reported analogous sentiments, with a preference for practical knowledge, for example asserting that “we must acknowledge the critical importance of practitioner knowledge and experiences… too often this knowledge and experience is belittled or ignored, with serious consequences for the advance of police reform” (Bradley & Nixon, 2009), and there is an acknowledged replication problem with policing’s most rigorous research as well. Hospital administrators would likely have much to share with their peers in policing about resistance to change among the rank and file, while physicians and police officers might be having parallel conversations in the break room about the misunderstood artistry of their work. IS can therefore help convince resistant practitioners to accept evidence, and ensure the studied practices account for conditions in ways that preserve their validity.

The use of IS may help overcome another important obstacle to innovation: risk averseness. A study of evidence-based mental health services across different agencies in Ohio found that an agency’s decision to adopt innovative practices was negatively related to the perceived risk of doing so, and positively related to the agency’s capacity to manage risk, as well as past comfort in taking risks (Panzano & Roth, 2006). If police leaders are averse to taking risks, IS offers the potential to manage and reduce them. The careful use of IS can improve the odds that effective practices can be introduced with fidelity to the features that made them successful in another setting, while gaining institutional and public acceptance by considering local conditions and needs. Such a process inherently reduces the risks of innovation, thereby lowering perceived risks of change, and increasing comfort in adopting new practices.

This illustrates one of the key gaps that IS helps to articulate and bridge: the difference between the (internal) efficacy of an evidence-based practice and its (external) effectiveness. It is the difference between a new medicine working in patients under tightly controlled, almost abstract circumstances, and changing the way doctors diagnose and treat a diverse range of patients to make that new medicine an integral part of the profession’s standard of care. Implementation science provides a method to take an internally valid, efficacious EBP and maximize its real-world validity by determining what the key features of the EBP are, how the EBP can be implemented with fidelity in resistant or risk-averse organizations, and what the range of benefits are in implementing a particular EBP. We will untangle this in the remainder of the article.

The state of implementation science in EBP: A review of the literature

The use of IS in policing remains almost completely unstudied, especially in a way that could provide initial guidance to researchers. A search in 2023 by the authors and a research librarian yielded six publications: two peer-reviewed empirical articles, two pieces of gray literature, and a book chapter (Santos & Santos, 2022) that reprised the peer-reviewed presentation of an implementation framework developed for police settings (Santos & Santos, 2019) (see Table 1). Of the peer reviewed empirical publications, one used IS constructs to identify barriers and facilitators to implementing a community-based participatory approach to addressing the determinants of crime and health (Stalker et al., 2020). The other looked at the barriers and facilitators surrounding the carriage and administration of naloxone, the opioid overdose reversal drug, by police (Berardi et al., 2021). Both were retrospective studies.

The gray literature consists of advocacy for the value of IS in policing. One publication is a component of “The Better Policing Toolkit,” a pilot project developed by the RAND Corporation (Hollywood et al., 2018). It presents evidence-based policing strategies, suggests IS can help bring them to an agency, and provides resources to learn more about the field. The other emphasizes the importance of evidence-based practices in criminal justice, suggests IS can foster effective organizational change, and provides and overview of the considerations (Gleicher, 2017). As overviews outside of the peer-reviewed literature, neither lays out the approach to IS research in police settings in sufficient detail to provide direction to researchers as they partner with practitioners in developing experimental projects.

It is also critical to note that the literature discussed here does not contain policing studies that prospectively employed experimental IS to ensure the success and sustainability of adopting an EBP, which is the most rigorous way researchers can improve policing’s knowledge about organizational change. For example, to provide researchers and practitioners with the means to design such experiments, Powell et al. (2015) assembled a panel of experts to generate 73 implementation strategies in health care settings. In contrast, the framework offered by Santos & Santos (2019) for use in policing has yet to form the basis of an implementation experiment, while other frameworks have been utilized experimentally in adjacent criminal justice fields.

The present gap between evidence-based practices and their successful implementation is well-illustrated by the Evidence-Based Policing matrix, a tool to assist police agencies in translating research to practice (Lum et al., 2022). A catalog of over 110 peer reviewed empirical studies about police interventions that was exhaustive when it was compiled, the matrix situates them in a three-dimensional array based on an intervention’s unit of analysis (e.g., people, places, cities, states, etc.), its level of proactivity, whether the intervention was found to be effective, and the study’s level of rigor. The matrix covers gang violence, street crime, domestic violence, narcotics enforcement, and several other police concerns. It is one of the most accessible and comprehensive collections of research about police practice, and the only one we are aware of that points practitioners toward relevant, high-quality studies. However, its emphasis is entirely on efficaciousness. The matrix does not identify the core practical features of each intervention (therefore reducing their transportability to other police settings), offer guidance about what made them effective (i.e., internally valid) in a given setting, discuss what variables are important to consider when generalizing practices across police agencies to best ensure external validity, or relate to how the practice affects a range of different possible outcomes.

For example, studies in the matrix about reducing gang violence are not distilled into a set of discrete practices that can be implemented across police departments. This prevents interested agencies from assessing their appropriateness, the acceptability of the practice to local communities, the staff’s receptiveness to them, or what resources are necessary in a given agency to ensure successful implementation and sustainment. When it comes to practices deemed ineffective, the matrix doesn’t offer insight into whether an intervention was ineffective because it wasn’t implemented with fidelity, wasn’t well-integrated into police practice and culture, failed to gain support among critical non-police actors, or because it simply lacked internal validity because its premises were causally inert. All of these are outcomes that can assist in understanding what was implemented and how it fared in the police agency. In other words, the matrix provides a valuable foundation for EBP, but does not address the elephant in the room: whether a range of police organizations can actually implement promising interventions with fidelity, and why many have failed. This is where IS can further the impact of EBP, and increase the confidence of police organizations that the research literature is relevant to their own agency.

An example of how implementation research can assess effectiveness and promote generalizability can be found in a study in the juvenile justice setting: JJ-TRIALS, Juvenile Justice—Translational Research on Interventions for Adolescents in the Legal System (Knight et al., 2016), a 36-site effort to improve the delivery of substance use disorder (SUD) treatment services in juvenile probation agencies through a novel behavioral health services cascade (Belenko et al., 2017). The study utilized EPIS, a framework discussed below, to ensure the intended cascade of care was acceptable to practitioners, had goals set by a collaborative JJ-SUD process, and used a data-driven approach (Leukefeld et al., 2017) to assess if different approaches to implementation were met with different results (Becan et al., 2018). The results showed that careful strategic planning and preparation for institutional change produced improvements in the number of youth receiving substance use treatment in the juvenile justice setting (Belenko et al., 2022; Knight et al., 2022; Robertson et al., 2023), yielding promising techniques for implementation going forward (Becan et al., 2020).

To fill the gaps between studies of efficacy and the realization of EBP, police researchers must examine a range of variables they have never systematically operationalized before. In their foundational paper on the criteria for measuring implementation outcomes, Proctor et al. (2011) sort these variables into eight categories: acceptability of the innovation, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability. In doing so, they provide bases for rigorously studying how to assess and generalize effective practices across police agencies. Research has long sought to answer questions about police culture, attitudes toward certain practices, what type of policing communities desire, and what the outcomes of policing should be beyond raw crime statistics (Goulka et al., 2021). IS calls for research about all of these things, but coordinates and focuses it toward one goal: the improvement and sustainment of police reforms, one innovation or reform at a time. To that end, Proctor et al. (2009) presented a conceptual model for integrating IS into mental health service delivery settings. We have adapted it here as a provisional model for use by police researchers and practitioners as the field continues to evolve and mature (see Figure 1).

An overview of implementation science frameworks

Given the complexity of understanding the broad range of contextual factors that affect implementation, a slew of frameworks for understanding and executing IS has emerged in the past three decades. Frameworks generally guide the user in identifying which factors should be considered, and how they will impact the implementation.3 Nilsen (2015) described five categories of frameworks, including process based (e.g., how knowledge is translated), determinants-focused (which factors impact implementation), classic theories (what accounts for people’s behavioral change), implementation theories (specific implementation features such as adaptation), and evaluation (assessing effectiveness and sustainability). The selection of the most suitable framework can be a daunting task, but careful selection will help ensure the most relevant concerns are identified and managed.

Here, we illustrate how two of the more popular implementation frameworks can be applied to EBP. One has been selected because it is intentionally broad, presenting users with a wide range of implementation domains and constructs that can be utilized across fields and projects to create a thorough plan for both researchers and practitioners, a breadth that has led to its extensive utilization in implementation research. The other has already been adapted for use in settings adjacent to policing such as juvenile justice, and acknowledges the cyclical approach to implementation inherent to fields like policing.

The Consolidated Framework for Implementation Research (CFIR) is one of the broadest and most widely used determinant frameworks for understanding the factors that influence implementation. It consists of more than 50 constructs that can be considered when assessing the implementation of an evidence-based treatment or program. They are grouped into five domains: Innovation, the Outer Setting, the Inner Setting, Individuals, the Implementation Process, and Individual Characteristics (see Table 2).

The innovation domain, for example, identifies the aspects of a practice that will impact implementation. One factor is trialability – can the innovation be tested or piloted on a small scale and easily undone? A police department can implement body-worn cameras in one of its precincts for a year, for example, then switch to another brand, or decide to stop the pilot. However, the practicalities associated with adding mental health professionals to a police department are much more complicated. It would require hiring, training, and deploying a new type of employee or issuing a contract for services, then possibly removing these professionals after a trial period if the implementation proved unsuccessful. Other inner setting determinants include agency culture, IT infrastructure, performance and incentive systems, and compatibility with other practices. Recognizing these internal constraints will allow police departments to consider the availability of existing resources and distill the EBP into operational parts.

Other constructs across the five domains include High and Mid-Level Leaders, Financing, and the level of Motivation in an agency. Each construct enables police leaders to understand barriers and facilitators to implementation and devise tailored implementation strategies with the goal of improving success and sustainability that can be tested for their effectiveness and future use. Such an approach is bound to be much more advantageous than plodding forward with a new practice relying only on changes to policy and procedure, or a generally ad-hoc approach. These CFIR constructs are not set in stone, however: they are periodically expanded or redefined based on user feedback (Damschroder et al., 2022b), as users provide practical feedback about what determinants are most relevant for measuring success in implementation in particular settings (Damschroder et al., 2022a).

Another critical domain is the outer setting. It includes external determinants (i.e., outside of the police department) that impact implementation by focusing on stakeholders and their support for EPBs. Some examples include local attitudes, community relations, partnerships, and external pressure related to the EBP. Using the example of body-worn cameras, how does the local community perceive this practice? What state and federal laws exist about recording civilian and police officer interactions? Have there been local or national events that create an external pressure to adopt body-worn cameras? What do national police unions advocate around their use? Future research using the CFIR in police settings should endeavor to identify which of its 50 constructs across its five domains are likely to be germane to implementation in police settings, and develop the means to operationalize them using what we know about policing.

A second framework, Exploration, Preparation, Implementation, and Sustainment (EPIS, 2023), is structured around the natural phases of implementation and the factors that impact each phase (see Figure 2). The EPIS model began as a linear process, but has evolved to emphasize that successful implementation can be a dynamic, adaptive, recursive, or cyclical process rather than simply linear (Becan et al., 2018). The exploration phase of EPIS gives a police organization the chance to consider the evidence for a new practice, discuss its relevance to their setting, and assess the actual need for it; the preparation phase calls for careful strategizing and building implementation processes tailored to the culture, structure and needs of the agency, and the barriers and facilitators they create; the implementation phase allows for structured implementation and evaluation, employing specific strategies created during preparation; finally, the sustainment phase allows an agency to determine what it will take for the new practice to remain an effective, ingrained practice within the agency without constant attention and supervision. As conditions change and priorities shift, the cyclical nature of EPIS can adapt an EBP to new evidence and changing circumstances. In doing so, EPIS allows police departments to remain agile and innovative.

The EPIS model categorizes implementation factors into four domains: inner context, outer context, bridging factors, and innovation factors (EPIS, 2023). Bridging factors are features that serve to connect inner and outer contexts such as inter-agency collaborations, linkages, and relationships. This feature may be especially useful when implementing programs across agencies, such as between the police and healthcare providers. The innovation factors speak to the adaptability of the intervention (or the program) itself to different contexts, suggesting that certain programs that have a high number of core components with little room for change may not be appropriate for translation to different settings or agencies. Each of the domains is important to consider, and some may be especially relevant during a particular phase. For example, a department that is considering an arrest diversion program may need to be especially attentive to the inner context (e.g., department resources, organizational staffing practices, and leadership support) during exploration and preparation, and subsequently shift to focusing largely on bridging factors, i.e., connections with the mental health and substance use programs necessary to effectively implement and sustain the police department’s initiative.

The design of IS trials for policing

Controlled and quasi-experimental trials

Experiments in IS can assist police practitioners by evaluating not only the outcomes of a particular strategy or innovation, but the effectiveness of different strategies to implementing it. Consider, for example, a large police agency that will incorporate aerial drones into its emergency response capacity. It might experiment with different approaches to implementation and compare outcomes in one part of the department to others that do not receive the implementation technique. It might start by conducting a needs assessment (Gupta, 2011), where planners confer with officers in the field to better understand where drones could best fill gaps or improve responses. The chief could create a change team (Belenko et al., 2013; McCarty et al., 2007), led by an influential executive and consisting of police of different ranks and assignments. The chief would give them the task of talking to police and the public, identifying the barriers and facilitators to a successful drone program, developing an implementation plan, and designating the standards of success. As the program starts, the department could create implementation toolkits (Yamada et al., 2015) that go beyond policy and procedure to walk officers through drone deployment, troubleshoot problems, answer frequent questions, and overcome common obstacles. As particular officers gain expertise in drone use, they could be designated as coaches (Fleddermann et al., 2023), making themselves available to officers still in the process of learning how to utilize them. Designated supervisors could conduct performance audits of situations where drones were—-or should have been—used, and provide constructive feedback for the future, an intervention shown to generate improvements in professional practice (Ivers et al., 2012). Finally, the agency could engage in continuous quality improvement (Shortell et al., 1995), where it tweaks and improves its procedures and approach to ensure drones deliver the greatest value in ways officers embrace. All these are proven implementation strategies that could be translated with an eye toward the police officers whose work will be impacted by drone use, as well as the public and governmental stakeholders who have an interest in creating effective and fair public safety.

As this work proceeds, it can shift from implementation toward sustainment. In the case here it would involve assessing to what extent drone use has penetrated the agency and its culture. Have bodies of policy and procedure evolved to signal drones are accepted and useful tools for the agency and to set the expectation that they are used? Is proper drone use an aspect of performance evaluations and a means of assessment for promotion or specialized assignments? Are particularly successful uses applauded by command staff and advertised to the public? Are new officers instructed in drone deployment in the academy classroom, at roll call, and in practice by more senior users? Is there a recurring budget line for drone acquisition, maintenance, and upgrading drone technology? Collectively, these strategies and others can ensure a successful innovation has staying power.

It is critical to note IS about two things: methodical implementation, and testing the relationship between given implementation strategies and implementation outcomes (Proctor et al., 2023). Experimental design in IS therefore relies on the same principles as clinical trials and intervention evaluations, including the comparative rigorousness of various methods. The most rigorous ones would randomize implementation variables in different parts of an agency, allowing researchers to see which approach yields the greatest impact of an EBP, or use techniques such as cluster randomization or stepped wedge implementation to see if a given approach generates improved uptake and fidelity (Handley et al., 2011). Among simpler approaches, survey research can prospectively or retrospectively measure feasibility and acceptability of an intervention and identify the type and intensity of barriers and facilitators across a range of variables. To that end, researchers can use validated instruments that assess organizational readiness for implementing change such as the one developed by Shea, et al. (2014). Likewise, qualitative methods offer considerable value across study designs, since considerable insight can be gained by talking to practitioners and stakeholders and translating their responses into the language of IS frameworks and constructs. As the use of IS in policing grows, the ultimate goal should be the design of a priori implementation science hypotheses to test the variables associated with implementation using an experimental or quasi experimental study design.

Hybrid implementation and effectiveness trials

At the core of IS are hybrid trials designed to test both the effectiveness of the proposed intervention, and the effectiveness of its implementation in a practice setting (Landes et al., 2019). They are designs well suited to police settings because they acknowledge that experimentation in public administration is a complex and imperfect endeavor, and it is often difficult or impossible for agencies to rigorously test an EBP, then, afterward, rigorously evaluate its implementation in real-life settings. They have been codified as three distinct types:

Type I hybrid trials emphasize testing the efficaciousness of an intervention while assessing the concerns about implementation necessary to design a more extensive implementation study when time and resources permit.

Type II hybrid trials reverse the emphasis: they utilize an implementation trial to collect data about the efficaciousness of a practice in a particular implementation setting. This type can be useful when the effectiveness of a practice is already well-established by evidence, and the main task is to implement it in a new and uncertain way, or in a challenging setting.

Type III hybrids seek to robustly test both impact and implementation. For example, a violence reduction initiative that showed promising results in Chicago might need to be tested in other cities before researchers can be certain it has a meaningful impact (Prudente, 2020). At the same time, in taking the model on the road for this purpose, it might face implementation challenges in other cities due to localized concerns about its feasibility and acceptability (Charles, 2020). Experimentation would therefore equally emphasize both effectiveness and implementation.

Sample use cases for implementation science in policing

Two brief use cases for IS in policing may help orient the reader toward ways it can assist in organizational change, and stimulate thinking about its potential value. One takes up CompStat, a well-established practice, and the other takes up de-escalation, an important developing one.

CompStat

CompStat, the data-driven police management and crime reduction process, saw its origins in the New York City Police Department in the early 1990’s, but quickly spread across the nation to many hundreds of agencies (Weisburd et al., 2004). Scholars observed that its diffusion was marked by the embrace of particular aspects in lieu of wholesale adoption, notably the command and control elements of the management process that reinforced a paramilitary model of policing, rather than utilizing the creative problem-solving approach intended to be the practice’s hallmark (Weisburd et al., 2003). In many agencies, there was an over-reliance reliance on crude measures of enforcement and arrests, which compounded problems of militaristic command and control (Eterno & Silverman, 2006).

In making sense of this problem, researchers described CompStat as being implemented in ways that strayed from the core innovative features that made it effective (Dabney, 2010). Implementation scientists would diagnose this as a failure to identify the constructs in a police agency’s inner context that prevented CompStat from being implemented with fidelity, ultimately limiting its effectiveness, and they would seek to understand why critical aspects of the program were lost in translation between settings.

De-escalation

More recently, de-escalation has become the focus of innovation and reform in an effort to reduce police use of force, with an evidence base that points toward promising practices and curricula (Engel, Corsaro, et al., 2020). Meaningful innovation in this regard will likely involve a two-step process. First, police leaders will need to identify effective de-escalation practices from among a range of possible techniques that emerged in a time of need. Then, with these techniques in hand, they will have to determine ways to implement them that are institutionally feasible, that rank and file officers find acceptable, and that integrate with the inner context of police departments in ways that establish de-escalation in the firmament of police practice.

These are not trivial concerns: one refrain is that little change in practice is necessary because skilled officers have been intuitively practicing de-escalation all along, and another is that de-escalation as envisioned by reformers is not a feasible practice because it is not in touch with the dangerous realities of policing. Given that legal precedent typically allows for the use of more force than de-escalation techniques strive for, only careful implementation that accounts for the inner and outer contexts of police use of force is likely to yield the results police innovators and the public are hoping for.

The missing science of de-implementation in policing

A critical part of the EBP process is one that receives little attention—the de-implementation of ineffective, unnecessary, or harmful practices. It is the idea that there is an evidence-informed way to go about effectively ceasing practices that do not work as intended, have outlived their usefulness, or that cannot be justified because they do more harm than good. De-implementation is therefore more than excising what doesn’t work (although that should be a goal), it is also about extricating practices from systems of policing that don’t work well enough, have run their course, or that are no longer justified given scarce resources and competing demands. In light of the many reasons for de-implementation, Wang et al. (2018) “describe a typology of de-implementation that represents four types of change: partial reduction, complete reversal, substitution with related replacement and substitution with unrelated replacement of existing practice.” The thought is that regardless of the extent of de-implementation for a given practice, it too requires a systematic approach to be effective.

Believing that police departments can summarily de-implement practices at the discretion of chiefs or elected officials relies on assumptions that have been proven inaccurate time and time again. In terms of IS, it assumes that the outer setting of policing, where the public, elected officials, and the media exert influence, is capable of ending an unnecessary or problematic practice regardless of how entrenched and resilient it may be in an agency’s inner setting. Consider the typical variance between “the law on the books” and “the law on the streets” (Arredondo et al., 2018), or the concern that some police leaders won’t consider evidence-based de-implementation because the internal barriers seem too great. A police department can instruct officers to desist in a practice, but its cultural or bureaucratic value may lead them to resort to similar practices or resort to ones that have the same net effect. These challenges arise from the considerable level of discretion officers report having when taking action and enforcing laws (del Pozo, Sightes, et al., 2021). In an example in dentistry, given the nation’s opioid overdose crisis, a recent study looked at ways to ensure oral surgeons de-implement the practice of unnecessarily prescribing opioids after tooth extractions. It found that even when offered decision support resources that would help them evaluate the pain management needs of patients more accurately, surgeons were reluctant to deviate from their long-established prescribing habits (Gryczynski et al., 2023). The study highlighted the challenges of de-implementation when practices have become entrenched.

De-implementation challenges abound in policing, from resistance to prohibitions on shooting at vehicles (Lopez, 2017), to the reduction of the use of no-knock search warrants except under genuinely exigent circumstances (Dungca & Abelson, 2022). In both cases, mounting evidence suggests de-implementation would be safer and more beneficial for both the public and police, but more research is necessary to understand how officers can come to embrace these changes as feasible and acceptable. In another example, a recent study concluded that de-implementing the K9 unit in a large municipal police department by order of the city’s mayor in the wake of a highly controversial deployment was not associated with increases in officer injury or suspect resistance during felony arrests (Adams et al., 2023). Assuming such findings are replicated, they may make the case for de-implementation of K9 units in other agencies, but people with knowledge of police culture might quicky realize that making such a change would depend on much more than evidence, given how much police and the public love dogs.

Knowing which variables have entrenched a practice is therefore critical when an agency desires to end it. As IS grows as a field, it has taken on the challenges of de-implementation. Police practitioners and researchers should keep abreast of these developments to leverage insights into how to successfully de-implement programs and practices as necessary.

Discussion

Policing is ripe for new methods to examine how to change organizations and how to assess the adoption, implementation, and sustainability of evidence-driven reforms in police settings. De-escalation, procedural justice, hot spot policing, focused deterrence, and virtually any other body of evidence-based practices lend themselves to studying the constructs that ensure they can be implemented with enough fidelity to be effective and sustainable. To assist with this transition, we have consolidated the principal resources discussed in this paper into a table for easier reference (see Table 3).

However, some researchers have cautioned against an overly scientific approach to policing, foremost among them David Thacher. In an arc that began shortly after Sherman’s clarion call for EBP (Thacher, 2001), and proceeded to a critique of a landmark report on proactive policing by the National Academy of Sciences, Engineering, and Medicine (Thacher, 2019), he reminds researchers and practitioners that policing is a contested practice precisely because it is not enough to simply determine what “works.” We must first understand what police are trying to accomplish, and what the broader consequences and tradeoffs are, both practical and moral. The goals of policing are open to debate, and its outcomes go far beyond the proximate ones that police agencies typically strive for and measure. This indicates that scientific evidence, which inescapably derives from data generated by value-laden theories (Ward, 2021), cannot, without proper context, set the operational agenda for a police department.

Police leaders therefore need an acute awareness of their inner setting (agency specific), especially their cultural and organizational values, if they expect to succeed at implementation. They will need it not only to understand internal barriers to innovation and reform, but to expose gaps between what an agency views as a useful strategy and what their communities expect of them. They will also need to understand broader societal variables, such as public support for an EBP, before it can be implemented. A few recent examples of failed or faltering implementation go far in illustrating Thacher’s concerns. One need only consider new and evolving technologies such as license plate readers, facial recognition, drones, the collection and analysis of DNA, acoustic gunshot detection sensors, gang databases, and the algorithmic use of police administrative data to identify crime patterns and predict future hotspots. These methods may help solve cases, raise clearance rates, reduce pattern crimes, prevent harms, and even save lives, but they have been met with serious resistance by actors in the outer context on grounds of racial profiling, poor police-community relationships, procedural injustices, and other threats to the legitimacy of policing. In some cases, they have been reduced or eliminated (Charles, 2020), sometimes by the passage of injunctive statutes. Taking this essential contestability seriously requires moving beyond consequentialist evaluations of a practice’s potential to reduce crime or produce an isolated measure of public safety. In this context, implementation strategies that focus on cross-systems or interorganizational collaboration and communication (e.g., communities of practice, learning collaboratives, change teams, cross-training), all features of bridging and address external stakeholder doubts or concerns, can reduce friction between police and other constituencies and promote aligned priorities and goals.

In illustrating these cases, we can see that concerns about norms and values go beyond whether a police practice is constitutional or not, or whether it is effective. Indeed, many of the practices that have been met with criticism and pushback are lawful on their face, or raise new questions that will only be resolved by years of analysis and litigation. The technique of hot spot policing offers a useful example. Broadly, it is the idea of using timely and accurate crime data to deploy police resources to a targeted area to reduce crime without displacing it. What precisely the police do there, however, varies by how leaders and their officers construe the program. It may consist of the simple presence of officers, or targeting a small number of repeat suspects who drive most of the crime. It can also involve implementing an aggressive, zero-tolerance dragnet at the location, or involve providing social services and economic supports to ameliorate some of the conditions that motivate crime. In some cases it may involve as little as showing up on a given block intermittently, for 15 minutes at a time. All are types of hot spot policing, but each is implemented differently, has different effects on crime and the wider community, and has varying levels of acceptability by police and/or the community and feasibility in terms of sustaining the effort over time in different jurisdictions.

It is therefore insufficient to only use legality as the principal constraint or guardrail in deciding what practices to implement, and how to do so. In fact, it is these contextual inner and outer setting factors that need attention to ensure that the police leadership and/or staff can support the initiative. In laying out a mandate for police leaders who advocate for evidence-based proactive policing, Thacher therefore observes that “their task is not to decide whether proactive policing ‘works.’ It is to decide how to carry it out in their own environment, and how to monitor and evaluate whether their efforts (and not a generic representation of those efforts) have been successful and appropriate” (Thacher, 2019). It is precisely the unique features of any given jurisdiction that makes IS a valuable pursuit for proponents of EBP, since it allows practitioners to explicitly consider the full range of factors that shape police innovation and to account for them in the inner and outer settings as implementation is planned and executed. This way, instead of giving preference to the scientific evidence, the process of IS serves to marry practitioner experiences with the evidence to produce an intervention that is relevant to an environment.

IS provides a framework for creating a learning organization where police practices can evolve over time based on the inner and outer setting factors. The evaluation of implementation outcomes, service, and distal outcomes illustrates a continuous evaluation process where an established practice may become less relevant as new contextual factors emerge or crises resolve. When its diminished relevance no longer justifies its costs, a practice should be de-implemented, or halted in a deliberate and minimally disruptive way. IS promotes the use of metrics to assess how different police practices influence various outcomes. For police leadership, this provides valuable data about the organization. Given that the senior police leadership in an organization changes every five years on average, it provides new leaders with critical data to understand how the agency engages in innovation, and provides perspectives about the agency’s inner and outer settings. This can have an impact on implementation and sustainability, since the police organization is being responsive to its setting instead of having its new leadership scrambling to understand the organization.

Conclusion

In introducing the field of IS to police researchers and practitioners, this article makes a call for IS to help resolve the fidelity and implementation issues related to EBP in police cultures. The timing is right: policing has experienced a tumult since 2020, when the murder of George Floyd by Minneapolis police officers brought fervent, nationwide calls for the improvement of police services, accompanied by concerns that prevailing practices were unmoored from evidence, creating harms that are either unnecessary, or that outweigh their positive effects. The extent to which this is true is open to debate, but it is nonetheless clear that police should expand their metrics of success to include outcomes that go beyond arrests, the crime rate, and contraband seized (del Pozo, Goulka, et al., 2021). Aside from calls for defunding the police—many of which failed because of their haste and their punitive tone—an alternative to the status quo is to better align policing’s focus with the goals of public health and safety where safety refers to the broader range of community relations that affect the quality of policing. Insofar as public health’s goal is to reduce injury, disease, and mortality for entire communities, such an alignment with policing seems natural. Progress on such an axis would then open up the pursuit of public safety to a new realm of metrics and methods, including those of implementation science.

Accordingly, agencies (and researchers) that seek to integrate implementation science into their work can be guided by these principles and recommendations:

• Effectiveness and implementation go hand in hand. When an evidence-driven agency is considering an organizational change, the critical question is typically whether the change will be effective. Planners might assess the results at other agencies, or launch a pilot at their own to test the waters. IS reminds us that implementation determines the effectiveness of an initiative as much as its inherent efficaciousness. Results from other agencies should be viewed in this light, and pilots should be designed with this in mind. Success requires assessing both effectiveness and implementation concurrently.

• Implementation is a cyclical process. To bring a new initiative from conception through sustainment consists of cycles of interwoven work, from initial exploration and planning through successful implementation, integrating the changes in a sustainable way, all the while testing for effectiveness, exploring and implementing the necessary adaptations, and paving the way for future innovations. Police agencies therefore need to think cyclically, rather than take a linear view of their work. For example, as a successful initiative grows in scale, takes on a new problem, comes under scrutiny in the aftermath of a crisis, sustains a budget cut, or faces substantial change in the wake of a new technology, a new cycle of implementation research can help navigate the associated challenges.

• Allocate adequate resources not just to a proposed initiative, but to the implementation process itself. Experimenting with different ways to successfully implement change, building out the training, policies and practices that make for successful implementation, and doing all of this by engaging the full range of stakeholders—both inside and outside the agency—takes considerable resources. The budgeting and planning process should not only allocate agency time and effort to the initiative, but also to the process of implementing it. This means allocating the funding for high quality implementation research and practice, building the internal and external partnerships to execute it, setting the necessary staff time, and rewarding the people in an agency who are succeed at implementation and sustainment.

• Involve implementation stakeholders broadly, and early. Successful organizational change is at root a human process, but policing’s hierarchical, top-down approach to many of its operations can hamper successful implementation by ignoring this. Involving police and community stakeholders at all ranks and positions in the exploration and preparation phases of implementation, and asking them to assess the specific variables identified as barriers and facilitators to change, will result in more effective plans, greater buy-in, and reduce the possibility and effects of resistance.

• Actively cultivate partnerships between practitioners and implementation scientists. The opportunity to conduct experimental implementation science in policing will hold great appeal to researchers working in adjacent fields of health care and correctional services delivery. Forming research-practice partnerships during exploration of new ideas, or as a standard feature of agency planning, will make implementation science research feel like an organic part of an agency’s work. Providing access to data, people, and resources will motivate research-practitioner partnerships, but dedicating the funding that allows researchers to devote their time and effort to an implementation science project in policing will provide the most compelling incentive.

• Hire implementation scientists and train staff. Large police departments with established planning sections should consider hiring implementation scientists. While it is likely that only the largest agencies have the resources for such positions, the complexity and pace of their operations would justify the expense. Smaller agencies could send their existing staff to implementation science seminars and boot camps. As these researchers develop a body of implementation techniques specific to policing, this knowledge can be disseminated through the rest of the profession for tailoring and use.

Ultimately, IS, with its measures, outcomes, interventions, hybrid trials and focus on feasibility, acceptability, scalability, and sustainability, can foster a virtuous cycle between research and practice. Not only should practice be informed by emerging evidence, but research questions and frameworks should be designed with practitioner participation and input to maximize the resonance of the results and to better integrate findings into existing systems (Green, 2008). In this way, IS seeks to do two things that make research relevant. The first is to show how scientific findings can be brought into practice with fidelity and success by taking the inherent challenges of implementation seriously and addressing them with rigorous effort. The second is to ensure the research itself is more likely to meet the needs of both the police department and the communities it serves while improving public safety and public health over the long term. The lessons learned from implementation, if they are identified in the organized and structured way that IS provides, cannot help but be generalized to inform future efforts. This overdue marriage of evidence and implementation can more fully yield the era of evidence-based policing that Sherman called for over 25 years ago.

References

Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and policy in mental health and mental health services research, 38(1), 4-23. https://doi.org/10.1007/s10488-010-0327-7

Adams, I. T., Mourtgos, S. M., McLean, K., & Alpert, G. P. (2023). De-fanged. Journal of Experimental Criminology. https://doi.org/10.1007/s11292-023-09553-1

American Hospital Association. (2022). Fast facts on US Hospitals. Retrieved January 1, 2023 from https://www.aha.org/statistics/fast-facts-us-hospitals

Arredondo, J., Gaines, T., Manian, S., Vilalta, C., Bañuelos, A., Strathdee, S. A., & Beletsky, L. (2018). The law on the streets: Evaluating the impact of Mexico’s drug decriminalization reform on drug possession arrests in Tijuana, Mexico. International Journal of Drug Policy, 54, 1-8. https://doi.org/https://doi.org/10.1016/j.drugpo.2017.12.006

Bauer, M. S., Damschroder, L., Hagedorn, H., Smith, J., & Kilbourne, A. M. (2015). An introduction to implementation science for the non-specialist. BMC Psychology, 3(1). https://doi.org/10.1186/s40359-015-0089-9

Becan, J. E., Bartkowski, J. P., Knight, D. K., Wiley, T. R. A., DiClemente, R., Ducharme, L., Welsh, W. N., Bowser, D., McCollister, K., Hiller, M., Spaulding, A. C., Flynn, P. M., Swartzendruber, A., Dickson, M. F., Fisher, J. H., & Aarons, G. A. (2018). A model for rigorously applying the Exploration, Preparation, Implementation, Sustainment (EPIS) framework in the design and measurement of a large scale collaborative multi-site study. Health & justice, 6(1), 9-9. https://doi.org/10.1186/s40352-018-0068-3

Becan, J. E., Fisher, J. H., Johnson, I. D., Bartkowski, J. P., Seaver, R., Gardner, S. K., Aarons, G. A., Renfro, T. L., Muiruri, R., Blackwell, L., Piper, K. N., Wiley, T. A., & Knight, D. K. (2020). Improving Substance Use Services for Juvenile Justice-Involved Youth: Complexity of Process Improvement Plans in a Large Scale Multi-site Study. Adm Policy Ment Health, 47(4), 501-514. https://doi.org/10.1007/s10488-019-01007-z

Belenko, S., Dembo, R., Knight, D. K., Elkington, K. S., Wasserman, G. A., Robertson, A. A., Welsh, W. N., Schmeidler, J., Joe, G. W., & Wiley, T. (2022). Using structured implementation interventions to improve referral to substance use treatment among justice-involved youth: Findings from a multisite cluster randomized trial. J Subst Abuse Treat, 140, 108829. https://doi.org/10.1016/j.jsat.2022.108829

Belenko, S., Knight, D., Wasserman, G. A., Dennis, M. L., Wiley, T., Taxman, F. S., Oser, C., Dembo, R., Robertson, A. A., & Sales, J. (2017). The Juvenile Justice Behavioral Health Services Cascade: A new framework for measuring unmet substance use treatment services needs among adolescent offenders. J Subst Abuse Treat, 74, 80-91. https://doi.org/10.1016/j.jsat.2016.12.012

Belenko, S., Visher, C., Copenhaver, M., Hiller, M., Melnick, G., O’Connell, D., Pearson, F., Fletcher, B., & the, H. I. V. S. W. o. C. (2013). A cluster randomized trial of utilizing a local change team approach to improve the delivery of HIV services in correctional settings: study protocol. Health & justice, 1(1), 8. https://doi.org/10.1186/2194-7899-1-8

Berardi, L., Bucerius, S., Haggerty, K. D., & Krahn, H. (2021). Narcan and Narcan't: Implementation factors influencing police officer use of Narcan. Social Science & Medicine, 270, 113669. https://doi.org/https://doi.org/10.1016/j.socscimed.2021.113669

Berk, R. A. (2005). Randomized experiments as the bronze standard. Journal of Experimental Criminology, 1(4), 417-433. https://doi.org/10.1007/s11292-005-3538-2

Bradley, D., & Nixon, C. (2009). Ending the ‘dialogue of the deaf’: evidence and policing policies and practices. An Australian case study. Police Practice and Research, 10(5-6), 423-435. https://doi.org/10.1080/15614260903378384

CFIR. (2023). Consolidated Framework for Implementation Research: Updated CFIR constructs. Center for Clinical Management Research. Retrieved November 24 from https://cfirguide.org/constructs/

Charles, J. B. (2020, July 6). Baltimore Hits Pause on Gun Violence Command Centers. The Trace. https://www.thetrace.org/2020/07/baltimore-hits-pause-on-gun-violence-command-centers/

Curran, G. M. (2020). Implementation science made too simple: a teaching tool. Implementation Science Communications 1(1): 27.

Dabney, D. (2010). Observations Regarding Key Operational Realities in a Compstat Model of Policing. Justice Quarterly, 27(1), 28-51. https://doi.org/10.1080/07418820902855370

Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation Science, 4(1), 50. https://doi.org/10.1186/1748-5908-4-50

Damschroder, L. J., Reardon, C. M., Opra-Widerquist, M. A., & Lowery, J. (2022a). Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum. Implementation Science, 17(1), 7. https://doi.org/10.1186/s13012-021-01181-5

Damschroder, L. J., Reardon, C. M., Opra-Widerquist, M. A., & Lowery, J. (2022b). The updated Consolidated Framework for Implementation Research based on user feedback. Implementation Science, 17(1), 75. https://doi.org/10.1186/s13012-022-01245-0

Deaton, A., & Cartwright, N. (2018). Understanding and misunderstanding randomized controlled trials. Social Science & Medicine, 210, 2-21. https://doi.org/https://doi.org/10.1016/j.socscimed.2017.12.005

del Pozo, B., Goulka, J., Beletsky, L., & Kleinig, J. (2021). Beyond Decriminalization: Ending the War on Drugs Requires Recasting Police Discretion through the Lens of a Public Health Ethic. The American Journal of Bioethics, 21(4). https://doi.org/10.1080/15265161.2021.1891339

del Pozo, B., Sightes, E., Goulka, J., Ray, B., Wood, C. A., Siddiqui, S., & Beletsky, L. A. (2021). Police discretion in encounters with people who use drugs: operationalizing the theory of planned behavior. Harm Reduction Journal, 18(1), 132. https://doi.org/10.1186/s12954-021-00583-4

Ducharme, L.J., Chandler, R.K. & Wiley, T.R. (2013) Implementing drug abuse treatment services in criminal justice settings: introduction to the CJ-DATS study protocol series. Health Justice 1, 5. https://doi.org/10.1186/2194-7899-1-5

Dungca, N., & Abelson, J. (2022, April 15). No-knock raids have led to fatal encounters and small drug seizures. The Washington Post. https://www.washingtonpost.com/investigations/interactive/2022/no-knock-warrants-judges/

Eccles, M. P., & Mittman, B. S. (2006). Welcome to Implementation Science. Implementation Science, 1(1), 1. https://doi.org/10.1186/1748-5908-1-1

Engel, R. S., Corsaro, N., Isaza, G. T., & McManus, H. D. (2020). Examining the Impact of Integrating Communications, Assessment, and Tactics (ICAT) De-escalation Training for the Louisville Metro Police Department: Initial Findings. International Association of Chiefs of Police.

EPIS. (2023). The EPIS Framework. Retrieved November 27 from https://episframework.com/

Estabrooks, P. A., et al. (2018). "Dissemination and Implementation Science for Public Health Professionals: An Overview and Call to Action." Prev Chronic Dis 15: E162.

Eterno, J. A., & Silverman, E. B. (2006). The New York City Police Department's Compstat: Dream or Nightmare? International Journal of Police Science & Management, 8(3), 218-231. https://doi.org/10.1350/ijps.2006.8.3.218

Fleddermann, K., Jacobson, N., Horst, J., Madden, L. M., Haram, E., & Molfenter, T. (2023). Opening the “black box” of organizational coaching for implementation. BMC Health Services Research, 23(1), 106. https://doi.org/10.1186/s12913-022-08948-6

Gleicher, L. (2017). Implementation Science in Criminal Justice: How Implementation of Evidence-based Programs and Practices Affects Outcomes. Illinois Criminal Justice Information Authority. Retrieved September 15 from https://icjia.illinois.gov/researchhub/articles/implementation-science-in-criminal-justice-how-implementation-of-evidence-based-programs-and-practices-affects-outcomes

Goulka, J., del Pozo, B., & Beletsky, L. (2021). From public safety to public health: Re-envisioning the goals and methods of policing. Journal of Community Safety and Well-Being, 6(1). https://doi.org/10.35502/jcswb.184

Green, L. W. (2008). Making research relevant: if it is an evidence-based practice, where's the practice-based evidence? Family Practice, 25(suppl_1), i20-i24. https://doi.org/10.1093/fampra/cmn055

Greenhalgh, T., Howick, J., & Maskrey, N. (2014). Evidence based medicine: a movement in crisis? BMJ : British Medical Journal, 348, g3725. https://doi.org/10.1136/bmj.g3725

Gryczynski, J., Mitchell, S. G., Asche, S. E., Truitt, A. R., Worley, D. C., & Rindal, D. B. (2023). De-Implementing Opioids for Dental Extractions (DIODE): a multi-clinic, cluster-randomized trial of clinical decision support strategies in dentistry. Implementation Science, 18(1), 5. https://doi.org/10.1186/s13012-023-01262-7

Gupta, K. (2011). A practical guide to needs assessment. John Wiley & Sons.

Handley, M. A., Schillinger, D., & Shiboski, S. (2011). Quasi-experimental designs in practice-based research settings: design and implementation considerations. J Am Board Fam Med, 24(5), 589-596. https://doi.org/10.3122/jabfm.2011.05.110067

Hollywood, J. S., McKay, K. N., Woods, D., Lauland, A., & Zhang, Y. (2018). Tips for Implementing Strategies. The Rand Corporation. https://www.rand.org/pubs/tools/TL261/better-policing-toolkit.html

Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. The National Academies Press. https://doi.org/doi:10.17226/10027

Ivers, N., Jamtvedt, G., Flottorp, S., Young, J. M., Odgaard‐Jensen, J., French, S. D., O'Brien, M. A., Johansen, M., Grimshaw, J., & Oxman, A. D. (2012). Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews(6). https://doi.org/10.1002/14651858.CD000259.pub3

Kirchner, J. E., Smith, J. L., Powell, B. J., Waltz, T. J., & Proctor, E. K. (2020). Getting a clinical innovation into practice: An introduction to implementation strategies. Psychiatry Research, 283, 112467. https://doi.org/https://doi.org/10.1016/j.psychres.2019.06.042

Knight, D. K., Belenko, S., Dennis, M. L., Wasserman, G. A., Joe, G. W., Aarons, G. A., Bartkowski, J. P., Becan, J. E., Elkington, K. S., Hogue, A., McReynolds, L. S., Robertson, A. A., Yang, Y., & Wiley, T. R. A. (2022). The comparative effectiveness of Core versus Core+Enhanced implementation strategies in a randomized controlled trial to improve substance use treatment receipt among justice-involved youth. BMC Health Services Research, 22(1), 1535. https://doi.org/10.1186/s12913-022-08902-6

Knight, D. K., Belenko, S., Wiley, T., Robertson, A. A., Arrigona, N., Dennis, M., Bartkowski, J. P., McReynolds, L. S., Becan, J. E., Knudsen, H. K., Wasserman, G. A., Rose, E., DiClemente, R., Leukefeld, C., & the, J. J. T. C. (2016). Juvenile Justice—Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS): a cluster randomized trial targeting system-wide improvement in substance use services. Implementation Science, 11(1), 57. https://doi.org/10.1186/s13012-016-0423-5

Landes, S. J., McBain, S. A., & Curran, G. M. (2019). An introduction to effectiveness-implementation hybrid designs. Psychiatry Research, 280, 112513. https://doi.org/https://doi.org/10.1016/j.psychres.2019.112513

Leukefeld, C. G., Cawood, M., Wiley, T., Robertson, A. A., Fisher, J. H., Arrigona, N., Donohue, P., Staples-Horne, M., Harris, P. W., Dembo, R., Roysden, J., & Marks, K. R. (2017). The Benefits of Community and Juvenile Justice Involvement in Organizational Research. J Juv Justice, 6(1), 112-124.

Lopez, G. (2017, May 8). Police have known for 45 years they shouldn’t shoot at moving cars. But they still do it. Vox. https://www.vox.com/policy-and-politics/2017/5/8/15533536/police-shooting-moving-cars-jordan-edwards

Lum, C., Koper, C. S., & Telep, C. W. (2022). Evidence-Based Policing Matrix. Retrieved January 15, 2023 from https://cebcp.org/evidence-based-policing/the-matrix/realms-of-effectiveness/

Madon, T. , Hofman, K. J. , Kupfer, L. & Glass, R. I. (2007). Implementation Science. Science, 318 (5857), 1728- 1729. doi:10.1126/science/1150009.

McCarty, D., Gustafson, D. H., Wisdom, J. P., Ford, J., Choi, D., Molfenter, T., Capoccia, V., & Cotter, F. (2007). The Network for the Improvement of Addiction Treatment (NIATx): Enhancing access and retention. Drug and Alcohol Dependence, 88(2), 138-145. https://doi.org/https://doi.org/10.1016/j.drugalcdep.2006.10.009

McIsaac, J. L., et al. (2018). The application of implementation science theories for population health: A critical interpretive synthesis. AIMS Public Health 5(1): 13-30.

Morris, Z. S., Wooding, S., & Grant, J. (2011). The answer is 17 years, what is the question: understanding time lags in translational research. Journal of the Royal Society of Medicine, 104(12), 510-520. https://doi.org/10.1258/jrsm.2011.110180

Panzano, P. C., & Roth, D. (2006). The Decision to Adopt Evidence-Based and Other Innovative Mental Health Practices: Risky Business? Psychiatric Services, 57(8), 1153-1161. https://doi.org/10.1176/ps.2006.57.8.1153

Police Executive Research Forum. (2022). Police chiefs compensation and career pathways: PERF’s 2021 survey. https://www.policeforum.org/assets/ChiefsCompensation.pdf

Pope, C. (2003). Resisting Evidence: The Study of Evidence-Based Medicine as a Contemporary Social Movement. Health, 7(3), 267-282. https://doi.org/10.1177/1363459303007003002

Powell, B. J., Waltz, T. J., Chinman, M. J., Damschroder, L. J., Smith, J. L., Matthieu, M. M., Proctor, E. K., & Kirchner, J. E. (2015). A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project. Implementation Science, 10(1), 21. https://doi.org/10.1186/s13012-015-0209-1

Proctor, E., Silmere, H., Raghavan, R., Hovmand, P., Aarons, G., Bunger, A., Griffey, R., & Hensley, M. (2011). Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Administration and policy in mental health and mental health services research, 38(2), 65-76. https://doi.org/10.1007/s10488-010-0319-7

Proctor, E. K., Bunger, A. C., Lengnick-Hall, R., Gerke, D. R., Martin, J. K., Phillips, R. J., & Swanson, J. C. (2023). Ten years of implementation outcomes research: a scoping review. Implementation Science, 18(1), 31. https://doi.org/10.1186/s13012-023-01286-z

Proctor, E. K., Landsverk, J., Aarons, G., Chambers, D., Glisson, C., & Mittman, B. (2009). Implementation Research in Mental Health Services: an Emerging Science with Conceptual, Methodological, and Training challenges. Administration and policy in mental health and mental health services research, 36(1), 24-34. https://doi.org/10.1007/s10488-008-0197-4

Prudente, T. (2020, December 1). An oft-tried plan to curb violent crime in Baltimore resurfaces. City leaders say better leadership will bring better results. The Baltimore Sun. https://www.baltimoresun.com/news/crime/bs-md-ci-cr-baltimore-police-focused-deterrence-20201201-mwsnkm25bndiddsiemx7grpx5m-story.html

Robertson, A. A., Gardner, S., Dembo, R., Dennis, M., Pankow, J., & Wilson, K. J. (2023). Impact of implementation interventions to improve substance use service delivery on recidivism among justice-involved youth. Health & justice, 11(1), 12. https://doi.org/10.1186/s40352-023-00210-3

Rudd, B. N., et al. (2020). Integrating implementation science in clinical research to maximize public health impact: a call for the reporting and alignment of implementation strategy use with implementation outcomes in clinical research. Implementation Science 15(1): 103. https://doi.org/10.1186/s13012-020-01060-5

Santos, R. G., & Santos, R. B. (2019). A four-phase process for translating research into police practice. Police practice & research, 20(6), 585-602. https://doi.org/10.1080/15614263.2019.1657629

Santos, R. G., & Santos, R. B. (2022). A four-phase process for translating research into police practice. In D. Weisburd & The George Mason Police Research Group (Eds.), Translational criminology in policing: Advances in Police Theory and Practice (First Edition. ed.). Routledge, Taylor & Francis Group.

Shea, C. M., et al. (2014). Organizational readiness for implementing change: a psychometric assessment of a new measure. Implementation Science 9(1): 7. https://doi.org/10.1186/1748-5908-9-7

Sherman, L. (1998). Evidence-based policing (Ideas in American Policing, Issue. https://www.policinginstitute.org/wp-content/uploads/2015/06/Sherman-1998-Evidence-Based-Policing.pdf

Shortell, S. M., O'Brien, J. L., Carman, J. M., Foster, R. W., Hughes, E. F., Boerstler, H., & O'Connor, E. J. (1995). Assessing the impact of continuous quality improvement/total quality management: concept versus implementation. Health services research, 30(2), 377-401.

Stalker, K. C., Brown, M. E., Evans, C. B. R., Hibdon, J., & Telep, C. (2020). Addressing Crime, Violence, and Other Determinants of Health through Community-Based Participatory Research and Implementation Science. Am J Community Psychol, 66(3-4), 392-403. https://doi.org/https://doi.org/10.1002/ajcp.12438

Smith, J. D., et al. (2020). The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implementation Science 15(1): 84. https://doi.org/10.1186/s13012-020-01041-8

Thacher, D. (2001). Policing is Not a Treatment: Alternatives to the Medical Model of Police Research. Journal of Research in Crime and Delinquency, 38(4), 387-415. https://doi.org/10.1177/0022427801038004003

Thacher, D. (2019). The Aspiration of Scientific Policing. Law & Social Inquiry, 44(1), 273-297. https://doi.org/10.1111/lsi.12367

Todak, N., & Huey, L. (2021). Busting Myths About Evidence-Based Policing. Police Quarterly, 25(1), 3-6. https://doi.org/10.1177/10986111211037655

Tucker, J. A., & Roth, D. L. (2006). Extending the evidence hierarchy to enhance evidence-based practice for substance use disorders. Addiction, 101(7), 918-932. https://doi.org/10.1111/j.1360-0443.2006.01396.x

Wagner, K. D., Bovet, L. J., Haynes, B., Joshua, A., & Davidson, P. J. (2016). Training law enforcement to respond to opioid overdose with naloxone: Impact on knowledge, attitudes, and interactions with community members. Drug Alcohol Depend, 165, 22-28. https://doi.org/10.1016/j.drugalcdep.2016.05.008

Wang, V., Maciejewski, M. L., Helfrich, C. D., & Weiner, B. J. (2018). Working smarter not harder: Coupling implementation to de-implementation. Healthcare, 6(2), 104-107. https://doi.org/https://doi.org/10.1016/j.hjdsi.2017.12.004

Ward, Z. B. (2021). On value-laden science. Studies in History and Philosophy of Science Part A, 85, 54-62. https://doi.org/https://doi.org/10.1016/j.shpsa.2020.09.006

Weisburd, D., Mastrofski, S. D., Greenspan, R., & Willis, J. J. (2004). The Growth of Compstat in American Policing (Police Foundation Reports, Issue. https://www.policinginstitute.org/publication/the-growth-of-compstat-in-american-policing/

Weisburd, D., Mastrofski, S. D., McNally, A. M., Greenspan, R., & Willis, J. J. (2003). Reforiming to preserve: CompStat and strategic problem solving in American policing [https://doi.org/10.1111/j.1745-9133.2003.tb00006.x]. Criminology & Public Policy, 2(3), 421-456. https://doi.org/https://doi.org/10.1111/j.1745-9133.2003.tb00006.x

Yamada, J., Shorkey, A., Barwick, M., Widger, K., & Stevens, B. J. (2015). The effectiveness of toolkits as knowledge translation strategies for integrating evidence into clinical care: a systematic review. BMJ Open, 5(4), e006808.

Zielinski, M. J., et al. (2020). Making change happen in criminal justice settings: leveraging implementation science to improve mental health care. Health & justice 8(1): 21.