Tidying Up A Few Misconceptions Around Evidence-Based Policing: Reply to Staller and Koerner (2021)

Abstract:

In this piece we reply to a commentary from Staller and Koerner (2021) on our work entitled, #Defund or #Re-Fund? Re-Examining Bayley’s Blueprint for Police Reform. In short, we agree on the necessity of reflexivity within policing research and the area of evidence-based policing more specifically, but also see this reply as an opportunity to clarify some misconceptions around evidence-based policing and what it means to be ‘evidence-based’. More specifically, we touch upon the flexibility of evidence-based policing to be implemented in tandem with other reform approaches, the value of experiential knowledge and qualitative methods within evidence-based policing, and the confounding of evidence and evidence-based. We conclude on the point of reflexivity and put out a call for follow-up studies that examine the implications of evaluated police practices.

Keywords: Policing; Reform; Evidence-Based Policing; Reflexivity

Corresponding Author: Jacek Koziarski – [email protected]

This is a pre-copyedited, author-produced version of an article accepted for publication in the International Journal of Comparative and Applied Criminal Justice, following peer review. The version of record, Koziarski, J. & Huey, L. (2021). Tidying Up A Few Misconceptions Around Evidence-Based Policing: Reply to Staller and Koerner (2021). International Journal of Comparative and Applied Criminal Justice, is available online at: https://doi.org/10.1080/01924036.2021.1949620. When citing, please cite the version of record.

We would like to thank Staller and Koerner (2021) for not only reading our work entitled, #Defund or #Re-Fund? Re-Examining Bayley’s Blueprint for Police Reform (Koziarski & Huey, 2021), but for engaging with it as well. While we do indeed agree with them on the importance of reflexivity within policing, we believe that Staller and Koerner’s (2021) comment also provides us with the opportunity to address some misconceptions around evidence-based policing and what it means to be ‘evidence-based’. As such, the present reply will be structured as follows. First, we will briefly touch upon what we believe to be a slight misinterpretation of our work by Staller and Koerner (2021). Next, we will discuss and address misconceptions around evidence-based policing and then conclude on a point of agreement: the need for reflexivity within policing research and the arena of evidence-based policing more specifically.

To begin, in their abstract Staller and Koerner (2021) write, “While we generally agree with the argument that police reform is an ongoing need of professional policing, we disagree that solely ‘evidence-based policing’ is the solution within this approach.” While we do indeed make the case for evidence-based policing as having the best chance of generating meaningful, long-term change within policing – especially when you consider how complex the police mandate can be given the continual shifting of responsibility for a myriad of social issues onto the policing mandate – we did not state anywhere that evidence-based policing is the sole approach to much-needed police reform. In fact, we believe that evidence-based policing can be broadly applicable to any policing context, no matter the reform approach taken. If one were to, for example, defund a police service and shift funding toward upstream solutions as some have suggested, it would be pertinent to not only evaluate the impacts of upstream programs relative their objectives, but also to consider effects on local crime and disorder. Has the footprint of the police been reduced in marginalized communities as some have suggested? Have mental health calls to the police decreased? Have there been any consequences as a result of defunding the police? In any case, no matter the approach, science – and thus evidence-based policing – is an important piece to any police reform endeavour as it can help provide answers to vital questions over the short- and long-term.

With respect to other misconceptions around evidence-based policing, there are a few things that we would like to address on this front. The first relates to the matter of ‘craft’, or experiential, knowledge. More specifically, Staller and Koerner (2021) quote us directly on the fact that evidence-based policing is meant to “replace experience, institution, and craft-based thinking that is often used as the central – if not only – tool in contemporary police decision-making” (as cited in Koziarski & Huey, 2021, p. 7) in order to make a point on epistemological and ontological hegemony. They, however, ignore our point later in that same paragraph that states the objective of evidence-based policing is not to completely omit these sources of knowledge. Indeed, many agree that experiential knowledge is a key piece of the evidence-based policing puzzle (Ratcliffe, 2014), and thus more closely aligns with the progressive approach to evidence-based practice outlined in the very source cited by Staller and Koerner (2021) to make the point about hegemonic epistemology and ontology (Petr & Walter, 2009). A further hole in the epistemological and ontological hegemony argument is that many evidence-based policing scholars recognize the importance of both qualitative and quantitative methods (Mitchell, 2018), which is another point made by Petr and Walter (2009). Under the evidence-based policing paradigm – and indeed within scientific inquiry more generally – they have different purposes: quantitative methods allow us to determine what works (or does not), whereas qualitative methods provide insight on why something worked (or did not).

Yet another misconception we would like to address here is the confounding of evidence and evidence-based. In their argument for reflexivity, Staller and Koerner (2021) cite a study that examines the ‘Tueller drill’, also known as the ‘21-foot rule’ to adjust for the reactionary gap between officer and subject in situations where the subject may have a deadly weapon (Sandel et al., 2020). While Staller and Koerner (2021) correctly point to this study as evidence that the 21-foot rule is an ineffective distance for officers to draw and fire their weapon at a charging subject but officer survival may increase if they are moving while shooting – evidence which we note is welcome given the largely anecdotal nature on which this ‘rule’ has been based on for many decades (Sandel et al., 2020) – we must stress here that even though there may be evidence in support of a particular practice, engaging in said practice does not mean that one is engaging in an evidence-based practice. The reason for this is two-fold. First, there are both qualitative (Huey et al., 2021) and quantitative (Farrington et al., 2002) hierarchies for empirical evidence. Indeed, some may criticize these hierarchies as contributing to epistemological and ontological hegemony, but the scientific reality is that there are some methods which bring us closer to understanding a phenomenon than others. This is vitally important because we can all envision a scenario in which a practitioner comes across the findings of a poorly designed cross-sectional study that provides evidence that X has an impact on Y, puts X into practice, and claims they are being ‘evidence-based’. This is, of course, simply not the case as the study design may not be sound enough to indeed determine that X has an impact on Y. A second, but related factor which differentiates evidence from evidence-based are the scientific concepts of replication and reproduction. That is, in order for a practice to become evidence-based, the findings must be replicated and/or reproduced outside of the original scientific investigation (Greenhalgh, 1996; Huey & Bennell, 2017). With each replication/reproduction preformed that confirms the findings of the original study, not only does our certainty around the effectiveness of the intervention (policy, practice, etc.) in question becomes more certain, but we build a base of evidence as well. In this sense, evidence-based policing is not merely about “producing the evidence justifying certain policing approaches” as Staller and Koerner (2021) suggest. Instead, it is about putting policing under the microscope of scientific scrutiny in order to examine whether practices or policies are achieving their intended objectives and if there are any adverse effects. In fact, the reason we discuss the practices of hot spots policing, problem-oriented policing, and pulling-levers policing (otherwise known as focused deterrence) in our original piece is because they are the only policing practices that can be currently considered as being ‘evidence-based’. That is, these particular practices have not only been rigorously evaluated, but findings that these approaches are effective at reducing crime have been replicated countless times. The most recent systematic review of hot spots policing by Braga and colleagues (2019), for example, included 65 studies.

As an aside, we do wish to express some concern over how Staller and Koerner (2021) discuss these particular practices within their comment. While we do indeed acknowledge their point is to account for the ‘bigger picture’ under the guise of reflexivity, it is inappropriate to seemingly hot swap measured study outcomes – or as Staller and Koerner (2021) put it, reference points – for outcomes that were not explicitly measured. In other words, the outcome of interest for most existing studies on hot spots policing and the other practices discussed is crime reduction, not “working towards a fair and social society.” Practices such as hot spots policing may in fact be welcome in communities that are ravaged by high rates of violence and can have few detrimental impacts if implemented properly. We note that the two studies cited by Staller and Koerner (2021) that supposedly provide insight into the harm caused by these particular practices are in fact unrelated – one of which discussed predictive policing (Browning & Arrigo, 2021) and the other implicit bias (Spencer et al., 2016). We also observe that studies which have explicitly examined the potential consequences of hot spots policing find little-to-no evidence of backfire or detrimental effects (Kochel & Weisburd, 2017; Ratcliffe et al., 2015; Weisburd, 2016; Weisburd et al., 2011).

Finally, to conclude on a point of agreement: reflexivity. As a scientific endeavour, we view evidence-based policing as already encompassing reflexivity and not as two separate entities as Staller and Koerner (2021) posit. Nonetheless, we do agree that more can be done with respect to reflexivity when it comes to training, workshops, and knowledge dissemination around policing research and evidence-based policing more specifically. Indeed, given that crime reduction and prevention is on the top of the priority list for many police leaders, we see how evidence-based policing can be employed in practice with a very narrow objective and little regard for any implications. It is, therefore, important to teach police practitioners not only about science and evidence-based policing, but the ‘bigger picture’ so-to-speak. Furthermore, we also stress the importance for more research that examines other areas of police practice. That is, as others have noted, evidence-based policing, to-date, has largely been focused on frontline crime reduction approaches to the detriment of other areas of policing that are in dire need of research (Koziarski & Lee, 2020; Telep, 2016). However, beyond this, we also need to have follow-up research that examines the broader implications of evaluated practices. Doing so can provide insight into the ‘bigger picture’ and encourage police practitioners to think about the broader impacts of their work.

References

Braga, A. A., Turchan, B. S., Papachristos, A. V., & Hureau, D. M. (2019). Hot Spots Policing and Crime Reduction: An Update of an Ongoing Systematic Review and Meta-Analysis. Journal of Experimental Criminology, 15(3), 289–311. doi:10.1007/s11292-019-09372-3

Browning, M., & Arrigo, B. (2021). Stop and Risk: Policing, Data, and the Digital Age of Discrimination. American Journal of Criminal Justice, 46(2), 298–316. doi:10.1007/s12103-020-09557-x

Farrington, D. P., Gottfredson, D. C., Sherman, L. W., & Welsh, B. C. (2002). The Maryland Scientific Methods Scale. In D. P. Farrington, D. L. MacKenzie, L. W. Sherman, & B. C. Welsh (Eds.), Evidence-Based Crime Prevention (pp. 13–21). Routledge.

Greenhalgh, T. (1996). “Is My Practice Evidence-Based?” BMJ, 313(7063), 957. doi:10.1136/bmj.313.7063.957

Huey, L., & Bennell, C. (2017). Replication and Reproduction in Canadian Policing Research: A Note. Canadian Journal of Criminology and Criminal Justice, 59(1), 123–138. doi:10.3138/cjccj.2016.e09

Huey, L., Mitchell, R., Kalyal, H., & Pegram, R. (2021). Implementing Evidence-Based Policing: A How-To Guide for Police Organizations. Policy Press.

Kochel, T. R., & Weisburd, D. (2017). Assessing Community Consequences of Implementing Hot Spots Policing in Residential Areas: Findings from a Randomized Field Trial. Journal of Experimental Criminology, 13(2), 143–170. doi:10.1007/s11292-017-9283-5

Koziarski, J., & Huey, L. (2021). #Defund or #Re-Fund? Re-Examining Bayley’s Blueprint for Police Reform. International Journal of Comparative and Applied Criminal Justice, 1–16. doi:10.1080/01924036.2021.1907604

Koziarski, J., & Lee, J. R. (2020). Connecting Evidence-Based Policing and Cybercrime. Policing: An International Journal, 43(1), 198–211. doi:10.1108/pijpsm-07-2019-0107

Mitchell, R. (2018). A Light Introduction to Evidence-Based Policing. In R. Mitchell & L. Huey, Evidence-Based Policing: An Introduction (pp. 3–14). Policy Press.

Petr, C. G., & Walter, U. M. (2009). Evidence-Based Practice: A Critical Reflection. European Journal of Social Work, 12(2), 221–232. doi:10.1080/13691450802567523

Ratcliffe, J. H. (2014). Reducing Crime: A Companion for Police Leaders. Routledge.

Ratcliffe, J. H., Groff, E. R., Sorg, E. T., & Haberman, C. P. (2015). Citizens’ Reactions to Hot Spots Policing: Impacts on Perceptions of Crime, Disorder, Safety and Police. Journal of Experimental Criminology, 11(3), 393–417. doi:10.1007/s11292-015-9230-2

Sandel, W. L., Martaindale, M. H., & Blair, J. P. (2020). A Scientific Examination of the 21-Foot Rule. Police Practice and Research, 22(3), 1–16. doi:10.1080/15614263.2020.1772785

Spencer, K. B., Charbonneau, A. K., & Glaser, J. (2016). Implicit Bias and Policing. Social and Personality Psychology Compass, 10(1), 50–63. doi:10.1111/spc3.12210

Staller, M. S. & Koerner, S. (2021). Evidence-Based Policing or Reflexive Policing: A Commentary on Koziarski and Huey. International Journal of Comparative and Applied Criminal Justice.

Telep, C. W. (2016). Expanding the Scope of Evidence‐Based Policing. Criminology & Public Policy, 15(1), 243–252. doi:10.1111/1745-9133.12188

Weisburd, D. (2016). Does Hot Spots Policing Inevitably Lead to Unfair and Abusive Police Practices, or Can We Maximize Both Fairness and Effectiveness in the New Proactive Policing? University of Chicago Legal Forum, 2016(16). http://chicagounbound.uchicago.edu/uclf/vol2016/iss1/16

Weisburd, D., Hinkle, J. C., Famega, C., & Ready, J. (2011). The Possible “Backfire” Effects of Hot Spots Policing: An Experimental Assessment of Impacts on Legitimacy, Fear and Collective Efficacy. Journal of Experimental Criminology, 7(4), 297–320. doi:10.1007/s11292-011-9130-z