European Journal on Criminal Policy and Research (2018)
The contemporary policing literature contains numerous examples of partnerships between academic researchers and police agencies. Such efforts have greatly contributed to evidence-based policing by increasing the knowledge base on effective strategies. However, research has demonstrated that successful collaboration between researchers and practitioners can be a challenge, with various organizational and inter-agency factors presenting difficulties at various stages of the process. Additionally, applied research can oftentimes face implementation challenges when the time comes to convert research into practice. The current study contributes to the literature by discussing research/practitioner partnerships and program implementation in the context of a multi-city risk-based policing project in the United States. We conceptualize police interventions as contingent on four distinct phases: 1) problem analysis, 2) project design, 3) project implementation, and 4) project evaluation. In this project, the research partners were able to successfully complete each phase in certain cities while the project experienced difficulty at one or more phases in other cities. We discuss these disparate experiences, identifying factors that facilitate or impede upon successful completion of each step. Policy implications and recommendations for future risk-based policing interventions are discussed.
Piza, E., Kennedy, L., and Caplan, J. (2018). Facilitators and Impediments to Designing, Implementing, and Evaluating Risk-Based Policing Strategies Using Risk Terrain Modeling: Insights from a Multi-City Evaluation in the United States. European Journal on Criminal Policy and Research, 24(4): 489-513.
Within evidence-based policing, partnerships between academic researchers and police practitioners have taken on heightened importance as they provide a vehicle for the design and evaluation of promising procedures and practices. The mutual benefits provided by researcher/practitioner collaborations have been documented over the years, as many contemporary police practices were conceived and developed within the framework of such arrangements (Caplan & Kennedy, 2016; Eck & Spelman, 1987; Kelling & Coles, 1996; Kennedy, 1997; Sherman & Weisburd, 1995). While there exists a thorough knowledgebase regarding the effectiveness of contemporary policing strategies, much less is known about the practical processes necessary for their implementation. Research has shown that police organizations can be stubbornly resistant to innovation due to resistance from rank-and-file officers (Leigh et al., 1996; Read & Tilley, 2000) and that newly adopted programs frequently take a much more simplistic form than originally envisioned (Sparrow, 2016). Challenges to introducing new practices can emerge for a variety of reasons. Therefore, understanding the procedural aspects of police practices, as well as the facilitators and impediments of successful implementation, can inform the replication of evidence-based strategies (Berman & Fox, 2010; Cissner & Farole, 2009).
The current study focuses on the opportunities and difficulties inherent in applied partnerships between researchers and police practitioners. We discuss such opportunities and difficulties in the context of our work on risk-based policing, conducted in partnership with 7 police departments in the United States. Each project used Risk Terrain Modeling (RTM: Caplan et al., 2011; Caplan & Kennedy, 2016) to diagnose the spatial attractors of criminal behavior and forecast the micro-level places at highest risk of hosting future crime incidents. These analyses were informed by the emerging body of research demonstrating RTM to be a valid crime forecasting tool across a wide variety of crime types, including aggravated assault (Drawve & Barnum, 2017; Kennedy et al., 2016), burglary (Caplan et al., 2015; Moreto et al., 2014), carjacking (Lersch, 2017), gun violence (Caplan et al., 2011; Drawve et al., 2016; Kennedy et al., 2011), motor vehicle theft and recovery (Piza et al., 2016), robbery (Barnum et al., 2017), and public drug selling (Barnum et al., 2016). These projects helped to further develop this body of research by using RTM findings to design police intervention in a manner that directly target spatial risk factors that contribute to the emergence and persistence of crime hot spots (Kennedy et al., 2016) rather than simply identifying micro-places to serve as target areas of interventions. The use of RTM in such a manner required each partnering police department to think creatively to consider not just where to deploy their crime prevention resources but what precisely their various operational units should do to address the criminogenic spatial influence presented by the spatial risk factors (Caplan, 2011). As such, the current study is particularly relevant to researchers/practitioner partnerships that involve the design and implementation of applied crime control strategies.
The unique scope of these risk-based policing projects required us to develop a working relationship with our police partners that differed in nature from researcher/practitioner partnerships focused primarily on the evaluation of existing practices. As will be discussed, risk-based policing was more easily implemented in certain jurisdictions than others. In the projects discussed here, 4 of the 7 agencies implemented risk-based policing in its entirety, carrying out each stage from problem analysis to program evaluation: Colorado Springs, CO (CSPD), Glendale, AZ (GPD), Kansas City, MO (KCPD), and Newark, NJ (NPD). Three of the 7 departments did not fully implement risk-based policing, experiencing difficulties at one or more of the stages of the process. As to not negatively reflect upon these agencies, we anonymize their names, referring to them as PD-A, PD-B, and PD-C throughout the manuscript.1 The main purpose of this study is the documentation of factors associated with both successful and unsuccessful implementations of risk-based policing. We begin with a review of relevant literature highlighting pertinent themes in researcher/practitioner partnerships and program implementation.
Review of relevant literature
Secret et al. (2011) identify key models of researcher/practitioner partnerships. Of the identified models, Secret et al. (2011) advocate the co-learning approach, noting that it provides the opportunity for a mutually beneficial collaboration by affording both parties the opportunity to contribute to the project in a manner that best meets their needs. Such a co-learning approach has been exemplified by the increased use of the action research model in criminal justice. Action research emphasizes the creation of problem-solving collaborations between researchers and practitioners whereby the two sides jointly contribute to problem identification, strategy development, and strategy implementation (Lewin, 1947). The action research model has been embraced by the U.S. government (Mock, 2010), as exemplified by the Department of Justice’s commitment to programs such as Project Safe Neighborhoods and the National Institute of Justice (NIJ) funding a wide range of research partnerships, such as the risk-based policing programs that are the focus of this article.
The emphasis on action research has greatly contributed to the knowledge base of “what works” in promoting public safety. Despite this occurrence, scholars have frequently noted a main barrier to evidence-based policing is the tendency for policy makers to put political and other considerations ahead of research evidence when designing crime control programs (Papachristos, 2011; Visher & Weisburd, 1998). A body of research has begun to emerge suggesting that the typical process of evidence generation, primarily led by academic scholars, may also present challenges to strategy development (Sparrow, 2011). A consistent theme in the literature is the inherent divide between academic researchers and the police agencies their work is meant to inform, with academics placing a premium on methodology and statistical analysis rather than the policy implications of the study (Buerger, 2010). While such emphasis regularly produces high-quality science, it may not always translate into research that is policy relevant (Wellford, 2009).
Despite such challenges, the practical utility of researcher/practitioner partnerships can be maximized when designed in a manner that is mutually beneficial for both parties (Braga, 2010, 2016). There are many prior examples of such collaboration. Strategies such as hot spots policing (Sherman & Weisburd, 1995), problem-oriented policing (Eck & Spelman, 1987; Goldstein, 1979, 1990), focused deterrence (Kennedy, 1997), broken windows (Kelling & Coles, 1996), and, particularly pertinent to the current study, risk-based policing (Caplan & Kennedy, 2016) were originally conceived by academic scholars based upon insights from scientific research. However, the development of these academic concepts into practice required the willingness of police agencies to dedicate the necessary resources to deploy and test said strategies. This is no small task. Instituting new programs requires leadership to actively put into place distinct processes and effectively manage numerous moving parts in a manner that focuses effort towards a singular goal.
The active process of strategy development is nicely captured in Welsh and Harris’s (2016) conceptualization of planned change. As is evident from the name, planned change involves “planning,” meaning that a person or group has explicitly thought about a problem and developed a specific solution (Welsh & Harris, 2016, p. 3). To help avoid knee jerk reactions to public safety problems and facilitate more carefully executed interventions, Welsh and Harris (2016) developed a 7-stage model for planned change: 1) analyzing the problem; 2) identifying goals and objectives; 3) program design; 4) action planning; 5) program implementation; 6) evaluating outcomes, and; 7) reassessment and review. Welsh and Harris’s model demonstrates that the process of designing and implementing interventions is dynamic, requiring the work of multiple actors at each step. For example, problem analysis requires the collection and analysis of data from multiple sources with results ideally being discussed and disseminated amongst a range of stakeholders. Program design and action planning require cooperation amongst numerous actors with responsibilities for addressing different dimensions of a specific problem. Evaluation, reassessment, and review require personnel trained in sophisticated statistical data analysis and program evaluation techniques. In ideal circumstances, these persons must also work to disseminate research findings and convert technical language into a form more accessible to practitioners.
It should not be taken for granted that programmers can seamlessly work through this process in all instances. Implementation challenges should be expected. Unfortunately, the importance and complexities of program implementation are often glossed over in the literature. For example, in discussing problem-oriented policing (POP), Scott (2010) noted that an unintended consequence of the popular S.A.R.A. model (Eck & Spelman, 1987) is the fact that several distinct processes of implementation are artificially conflated in the single “response” stage. As argued by Scott (2010), this does not accurately reflect the complexity of program implementation. Such issues are not unique to policing, as evaluations of criminal justice programs as a whole rarely include information on the implementation process (Hagan, 1989; Johnson et al., 2015; Klofas, et al., 2010). Nonetheless, a body of knowledge has begun to emerge, highlighting common challenges to program implementation.
Cissner and Farole (2009) conducted a multi-faceted process evaluation into failed experiments undertaken by the Center for Court Innovation (CCI) and Bureau of Justice Assistance (BJA). A common theme throughout many of the projects was an inability to establish clear data collection processes at the outset of the project. Projects that included robust data collection plans were able to easily designate project goals and objectives and readily measure progress towards these ends. However, such foresight is rare, as most of the programs reviewed by Cissner and Farole (2009) emphasized getting the program up and running over establishing data collection systems. Berman and Fox (2010) noted a similar shortcoming in the St. Louis Police Department’s Consent to Search program, an innovative strategy in which police, in response to community referrals, would request parents’ permission to search their homes when their (typically teen-aged) children were suspected of being in possession of an illegal firearm. In exchange for the consent to search, police agreed not to make any arrests if they found an illegal firearm (or any other illegal contraband), emphasizing the seizure of guns over the punishment of offenders. Police supervisors in charge of this project focused their efforts on establishing partnerships with the community and creating legally sound consent forms. Data collection of program outputs, and outcomes largely did not occur. This lack of data proved costly when the project managers were promoted to another assignment, as the new supervisor had little-to-no information on the procedural aspects of the project. This led the project to take a much different form than intended, with the unit coming to prioritize arrests of offenders over seizing guns.
Welsh and Harris (2016) demonstrated how success or failure of an intervention can be largely determined by project’s “change agents,” the persons responsible for coordinating, planning, developing, and implementing a new program. At the outset of a program, a change agent must first generate the necessary support for an agency to find a program promising enough to dedicate the time and resources necessary for its development. Change agents must then successfully identify and recruit the necessary stakeholders to the project. Given the fact that relevant stakeholders may not have always seen eye-to-eye, Cissner and Farole (2009) note that timing is important regarding stakeholder recruitment. For example, the Brooklyn Youthful Offender Domestic Violence Court did not engage defense attorneys during the planning stages, under the assumption they would object to the program. This decision later backfired, as defense attorneys strongly advised their clients against entering the program due to their lack of familiarity with the specific terms of participation.
After recruitment to the project, stakeholders in turn play important roles in ensuring the success of the program. At this stage, the importance of leadership in managing varying project personnel becomes key, specifically in regard to establishing clear lines of authority and working effectively with all stakeholders. While this may seem self-evident, Cissner and Farole (2009) found that ineffective and, in certain cases, nonexistent leadership was a common source of failure in the programs included in their evaluation. Selecting leadership can be complicated, especially in the case of multi-agency collaboration. In light of these concerns, programs may forestall making tough leadership decisions or bypass instituting formal leadership all together. With such a leadership void, key program processes and procedure can fall through the cracks.
Research also suggests that supervision of front-line staff, specifically in terms of the performance of mid-level managers and supervisors, is key to program implementation. Rengifo et al. (2017) noted that agency supervisors involved in the Kansas Offender Risk Reduction and Reentry Plan verbalized challenges to the need, feasibility, and success of the newly implemented program, sending a message to front-line participants that the newly developed strategy was not worthwhile. The effect of mid-managers on adherence to newly formed programs has also been observed in policing. In the Scotland Community Engagement Trial (ScotCET), front-line officers reported receiving a set of instructions from managers that provided minimal and/or incorrect information about the project’s purpose and objectives. Other officers reported being told that the program was “nothing new” as the officers already acted in a procedurally just manner during traffic stops (MacQueen & Bradford, 2017). In another example, the implementation of CompStat, a particularly heralded innovation in policing, also demonstrates the potential effect of mid-level managers. In their national study, Weisburd et al. (2003) found that CompStat reinforces the bureaucratic, paramilitary model of police organizations rather than foster the development of new, innovative strategies. A follow-up study found that mid-level police managers rarely communicated the problem-solving activities of CompStat meetings to front-line officers (Willis et al. 2007), which may help explain why bureaucratic adherence to traditional practices usurped the innovative strategy reforms CompStat was meant to promote.
Lastly, agency culture, specifically in terms of long-standing practices of the agency, can present challenges to policy makers interested in starting new programs. Sparrow (2008) argued that public service agencies tended to address problems through tool-based solutions whereby existing processes and strategies are leveraged, regardless of their “fit” with the problem at hand. Sparrow (2008) contrasts this method with a task-based approach, whereby the agency organizes activities around the specific problem that needs to be rectified, often requiring the creation of new processes that were not previously part of their “toolbox.” Sparrow (2008) argues the task-based approach as the more effective, given its emphasis on designing operations for the explicit purpose of solving specific problems rather than fitting into the existing organizational structure of the agency. However, agency culture and preference for familiarity often causes stubborn adherence to tool-based approaches.
The Current Study
The current study seeks to contribute to the literature on researcher/practitioner partnerships as well as program implementation. As stated earlier, we will review our experiences with risk-based policing projects conducted in partnership with 7 police departments throughout the United States. Our relationship with each agency prior to the start of this project varied from site to site. Crime analysts at three of the agencies (CSPD, GPD, PD-A) had previously conducted RTM analyses on behalf of their agencies. Two agencies directly partnered with us on research previously (KCPD, NPD), with results of RTM analyses reported in peer reviewed journal articles (Caplan et al., 2012; Kennedy et al., 2011). At NPD, we also had direct connections to the Chief of Police due to the primary author’s previous employment as a crime analyst with that agency. Our contact with PD-B similarly resulted from the agency’s Chief knowing the lead author from his work at NPD. In PD-C, we did not have any previous professional contact with the agency. Rather, the agency’s Director of Research and Evaluation reached out to us for the purpose of partnering on a risk-based policing project.
In 2012, we secured an NIJ grant in response to the “Testing Geospatial Police Strategies and Exploring Their Relationship to Criminological Theories” solicitation. This award funded risk-based policing partnerships with 6 of the 7 aforementioned police departments. In 2013, we partnered with PD-C on a follow-up project funded as part of NIJ’s “Testing Geospatial Predictive Policing Strategies” program. This project sought to replicate the 6-city study, with an additional component added to the problem analysis. In addition to conducting RTM, this study aimed to determine how the effect of various police officer enforcement actions varied depending on whether the activity occurred within a high-risk area. We (both the authors and PD-C) felt that the findings of this analysis would help refine the intervention strategy by emphasizing police tactics demonstrated to work best within high-risk places.
In considering these applied research projects, we were informed by the planned change model of Welsh and Harris (2016). In reflecting on our experiences, we truncated their 7-step model to 4 phases: 1) problem analysis, 2) project design, 3) project implementation, and 4) project evaluation.2 In the following section, we discuss our experiences in each of these phases, specifically focusing on factors that, in hindsight, seem to relate to successful and unsuccessful implementation.
Phase 1: Problem Analysis
The first phase of each risk-based policing project involved an RTM of various crime types in each jurisdiction, which was successfully conducted at each site. Our direct contact with crime analysts likely played a key role in the wide spread success of the problem analysis. As noted by Kennedy et al. (2011, p. 351-352), RTM requires access to significantly more data than traditional geospatial techniques, such as kernel density mapping. The need to collect, clean, and utilize such disparate data can present hardships in certain instances. Our close interaction with crime analysts, who are typically the sole police employees whose jobs revolve almost entirely around working with data (Shane, 2007), provided the precise data necessary to seamlessly conduct the RTM analyses. It should also be noted that the use of a researcher/practitioner partnership likely played a role in the successful completion of the problem analysis. Given the wide range of responsibilities that are typically assigned to crime analysts, it can sometimes be difficult for them to find sufficient time to conduct in-depth analysis for a new project (Brown, 2010, p. 48). The involvement of our research team, and the fact that we handled the bulk of the problem analysis, reduced the burden on crime analysts and made timely completion of the problem analysis more feasible.
Interestingly, the participation of a commercial partner also proved beneficial in terms of data access. One of the first steps of RTM is identifying a pool of potential risk factors for the crime in question (Caplan et al., 2011). We emphasized the input of each police department’s crime analysts and command staff in this process. In many cases, our discussions led to a great deal of brainstorming, resulting in the identification of risk factors that were not actively collected by the agency. As an example, NPD was interested in the effect of gas stations on gun violence, but had no internal mechanism for tracking such facilities because gas stations were licensed by the state rather than the city. In such cases, we were able to obtain the data from InfoGroup, a leading provider of residential and commercial data for reference, research, and marketing purposes.3
While the RTM analyses were seamlessly conducted in each instance, in PD-C we experienced difficulty with the second component of the problem analysis, which sought to measure how the effect of police activities differed across spatial contexts. PD-C was able to readily provide data for traditional enforcement actions, such as arrests, summonses, and pedestrian stops, as these incidents were readily captured within their internal data systems. However, the command staff was adamant that these actions did not fully reflect their crime prevention mission, with focused activities such as directed automobile patrols, foot patrols and team-policing units being emphasized by the new leadership. The leadership was more concerned with officers providing conspicuous presence at high crime places through these tactics rather than with whether or not they enacted enforcement actions while on duty. Unfortunately, PD-C had no established means for collecting such data. Since any analysis that did not include such activities would lack content validity, PD-C opted against conducting the second portion of the problem analysis.
To be clear, this lack of data should not be considered a failure of PD-C, as modern records management systems (RMS) primarily house data on official enforcement actions conducted by police. However, recent scholarship has advanced the notion that police could prevent crime by deemphasizing formal enforcement in favor of conspicuous presence and more informal community engagement (Ariel et al., 2016; Caplan and Kennedy, 2016; Nagin et al., 2015). Given the interest is such officer actions, police should strive to create processes to more readily reflect these less invasive activities. For example, Piza (in press) measured informal “guardian actions” (business checks, citizen contacts, bus checks, and taxi inspections) from after-action reports submitted by patrol officers at the end of each shift. To measure general police presence, rather than police enforcement, Ariel and Patridge (2016) used GPS devices to track officer movement across high crime bus stops. Integrating such alternative data sources into analytical products may be necessary for researchers to more readily measure non-enforcement police actions.
Phase 2: Project Design
Following the completion of the problem analysis, we conducted ACTION meetings (see Caplan & Kennedy, 2016, Chapter 7) with each agency to discuss the findings for the purpose of designing the intervention. In PD-A we were unable to advance to this stage due to an extremely high level of turnover at the agency. Our initial contact at PD-A was the supervisor of the Crime Analysis unit, who retired during the problem analysis phase. After completion of the problem analysis, we spoke with the new Crime Analysis supervisor to describe the next steps of the project. However, this individual was soon transferred to another unit, requiring us to introduce yet another new supervisor to the project. During this time, the Chief of Police also retired before the agency was able to officially proceed from the problem analysis to the project design phase. This required the Crime Analysis unit to start over in securing support for the project from agency leadership. By the time the necessary support was secured, there was not enough time to realistically design, implement, and evaluate the intervention.
In 5 of the remaining 6 cities, our meetings with the police agencies occurred fairly seamlessly. ACTION meetings typically took place over a workday or two, with attendees including the research team, crime analysts, members of the police department’s command staff, and representatives from any outside units that the PD anticipated may play a role in the intervention. In our strategy meetings, we followed presentation of the RTM findings with a discussion regarding the agency’s perception of the findings and capacity to address the significant risk factors. As an example, Table 1 shows the results of the RTM analysis for CSPD. In consultation with CSPD crime analysts and command staff, we identified 19 risk factors and tested their relation to motor vehicle theft. The RTMDx Utility, the software used to automate the RTM process (see Caplan & Kennedy, 2013), found a significant RTM for motor vehicle theft that included 6 of the risk factors. In interpreting the findings, we paid particular attention to the Relative Risk Value (RRV), exponentiated coefficients that act as a weighted value that can be used to compare the effect of risk values with one another (see Heffner, 2013 for a more detailed description of the statistical procedures of RTMDx). These findings helped to frame our discussion with CSPD personnel, as it highlighted the factors that should be considered for intervention. The composite RTM map, highlighting areas of the city at increased risk of motor vehicle theft, helped to further refine our discussion as we considered potential target areas for the risk-based intervention (see Figure 1).
Table 1. Colorado Springs RTMDx Findings
Relative Risk Value
In the Final RTM
Disorder calls for service
Multifamily housing units
Tested but not in the final RTM
Gas stations w/convenience stores
Hotels & motels
Parking stations & garages
Figure 1. Colorado Springs RTM Map for Motor Vehicle Theft
In many instances, police articulated the mechanisms they believed generated the criminogenic spatial influence of the risk factors, often providing examples in support of their observations. This process was typified by an example from GPD, in which convenience stores were found to be a significant risk factor for street robbery. When discussing the RTM findings in preparation for the ACTION meeting, we intuitively thought this was due to convenience stores acting as crime generators, with high numbers of pedestrians (i.e. potential victims) frequently traveling to/from the vicinity of convenience stores. However, an officer provided a much different explanation, stating that many convenience stores placed automated cell phone return kiosks in their businesses, where customers could dispose of old cell phones for cash. The officer felt that this provided offenders a way to earn fast cash for cell phones taken during robberies. The crime analysts were able to provide empirical support for this view, with cell phones being taken much more frequently in robberies occurring in close proximity of convenience stores than robberies at other locations in the city.
As the prior paragraph illustrates, our discussion with police personnel during ACTION meetings helped identify risk factors that should be targeted in the intervention. Somewhat to our surprise, ACTION meetings also frequently led to very candid discussions regarding the scope of the agency’s influence. Each agency was forthcoming in determining which risk factors they could readily affect as well as those they could not. For example, in each city, foreclosed properties were identified as a particularly powerful risk factor. In each instance, police leadership stated that addressing the spatial influence of foreclosures was beyond the reach of their agency. Thus, foreclosures were not a targeted risk factor in any of the interventions.
At PD-C, the project design phase did not proceed successfully. The problem was not a lack of ability to meet with command staff representatives to discuss the project findings. Rather, the problem was the lack of a mechanism to move these discussions away from the problem analysis findings and towards the development of an applied intervention. This may have been at least partially due to the piecemeal fashion by which PD-C decided to approach the project design. Rather than hold 1 meeting with all involved parties present, as the other agencies did, PD-C held a series of separate meetings with different agency representatives. As is natural in applied research, the research team commonly had to convince at least certain members of the agency that the project would be beneficial. Indeed, we had to do some version of this with each of the partnering agencies during the early stages of the project. However, the unique meeting structure at PD-C complicated this process. Each meeting with a new group of stakeholders put pressure on the research team to frequently “sell” the project as worthwhile. Even after securing the necessary initial support for the project, the multiple-meeting format hindered the project design. At the different meetings, attendees emphasized different risk factors for intervention. Also, as occurred in a number of the other cities, questions posed by officers led the research team to conduct follow-up analyses for the purpose of clarifying and building upon key points of the RTM analysis, which informed the intervention. However, at PD-C, the disparate meetings meant that these follow up analyses were very varied in nature and did not collectively speak to any overarching themes. Therefore, while they satisfied the curiosity of the requesting parties, these analyses did not ultimately have much practical value for the project.
In contemplating the lack of successful project design in PD-C, we noted the differing relationship we had with them compared to the other project partners. As previously discussed, many of the departments involved in the original 6-city study had some level of experience with RTM and/or a previous working relationship with the authors. PD-C, conversely, worked with RTM and the authors for the first time on this project. This unfamiliarity may have hindered our ability to move the project from problem analysis to program design. It should also be noted that our contact in this agency was with the Office of Research and Evaluation, not with any crime analysis personnel as in the other cities. This office was staffed with primarily civilian personnel, including the Director. While crime analysts are also primarily civilians, they are involved in the day-to-day functions of policing, contributing the analytical products necessary for a range of contemporary strategies (Santos, 2014). The Office of Research and Evaluation, on the other hand, was primarily involved in more macro-level projects focused on overarching policy that did not overlap directly with daily police functions. Therefore, this office may have lacked the working relationship with sworn personnel to effectively generate support for the project.
Lastly, PD-C did experience some turnover, albeit not near the level of PD-A discussed earlier, that may have negatively impacted the project. In about the sixth month of the project, the Director of Research and Evaluation, who initiated the project, left the agency. A replacement was not hired for several months. From there, it took another few months for us to establish reliable contact with the new office commanders, brief them about the project, garner their support, and re-conduct the RTM analysis (to account for the adjusted “pre-intervention” time period). This obviously affected the timeline for the intervention, as the problem analysis phase lasted about three times longer than anticipated. The effect that this had on the design of the intervention was less clear. During the preparation of the grant application, the original Director committed the agency to participating in the intervention portion of the project. However, upon assuming the position, the new Director stated that field operations were well outside the scope of the Research and Evaluation Office and did not similarly commit to the intervention. Instead, the new Director offered us the opportunity to garner support from the agency personnel responsible for patrol operations. In our view, the result of the lack of verbal commitment from the Research Director gave the appearance to the operations unit that we (the research team) were outside academics requesting support for our own pet project rather than members of an already existing partnership actively funded by NIJ. This was likely damaging, as new projects, specifically innovative projects not previously attempted by the agency, can gravely suffer if no clear “champion” emerges from inside the host agency (Bowers & Johnson, 2010). However, we acknowledge it is difficult to determine precisely how much this contributed to the project’s failure.
Phase 3: Project Implementation
The individualized results of the RTM analysis at each site, coupled with each agency’s unique mission and organizational structure, resulted in applied interventions that greatly differed in scope. CSPD selected motor vehicle theft as their priority crime and designed their intervention strategies to address incidents of social disorder. An array of activities was performed by various CSPD units, including Code Enforcement property inspections, Community Service Officer Neighborhood Cleanups, Community Meetings, Proactive Police Enforcement against disorder offenses, Proactive Traffic Enforcement, and the deployment of License Plate Recognition (LPR) devices for the purpose of identifying stolen motor vehicles.
NPD selected gun violence as their priority crime, and designed their intervention strategies to generate location checks and manager contacts at three business types: Restaurants, Food Take Outs, and Gas Stations. Each day during the intervention, a task force comprised of 3 officers under the supervision of a lieutenant visited businesses located within the target area. Upon visiting the business, officers were required to meet with the on-duty manager and have them sign a log in sheet to ensure that proper contact was established.
KCPD selected aggravated violence as their priority crime and designed their intervention strategies to address nightclubs, suspicious person with a weapon calls-for-service, weapon offending parolees and probationers, drug sales, packaged liquor stores, and liquor licensed retailers.4 Intervention activities included Code Enforcement, Directed Patrols, Licensing and Inspection checks, meet-and-greets with known offenders juxtaposed with social service referrals/support, CPTED inspections, Pedestrian Checks, Area Presence, Residence Checks, Traffic Violations, and Building Checks.
GPD selected street robbery as their priority crime and designed their intervention strategy to address all 7 significant risk factors identified in their RTM: Drug-related Calls for Service, Convenience Stores, Take Out Restaurants, Apartment Complexes, Gang Member Residences, Liquor Stores, and Bars. GPD intervention activities included Directed Patrols, distribution of flyers to pedestrians advising them to take caution when using their personal electronic devices in public, Community Meetings, Proactive Stops, and Proactive Arrests.
As each city deployed their risk-based intervention, it was interesting to note the different management structures necessary for implementation. In certain cases, the focused scope of the intervention involved a small number of participants which seemed to facilitate management. For example, NPD’s effort, comprising a single 4-person task force, was managed directly out of the Chief of Police’s office. The same lieutenant led the task force each tour of duty to ensure consistency in the treatment delivery. Officers were selected for the task force on an overtime basis, with those interested in the assignment notifying the Chief’s office in writing. The pool of interested officers participated in the task force on a rotating basis. From our perspective, the NPD was able to manage this program with a minimal amount of hardship due to clear identification of a project leader (the lieutenant) and relatively small number of officers from which to select from. Prior research supports this view, as projects requiring the coordination of multiple entities from different units are typically at higher risk of implementation failure than less complicated projects (Bowers & Johnson, 2010).
Each of the other agencies designed interventions that involved a wider array of personnel from a number of different units. In certain cases, the management of the program was somewhat simplified by the designation of street segments encompassed within a single precinct as the target area, which meant that a Police supervisor (typically Captain or Major) was already in charge of operations in the area. This was the case with the KCPD, which selected the Metro division to receive the risk-based intervention. This meant that the commanders directly had at their disposal personnel to address the targeted risk factors through a combination of patrol, investigative, and code enforcement activities.
Conversely, the designation of the Sand Creek division as the target area in Colorado Springs did not seem to simplify project management significantly. In contrasting CSPD to KCPD, this may have been due to the different number of risk factors targeted by the respective interventions. KCPD sought to mitigate the spatial influence of 7 separate risk factors. This likely maximized opportunity for multiple entities within the precinct to contribute to the intervention. CSPD’s intervention was singularly focused on social disorder, the top risk factor identified in the RTM. Given this singular focus, the Sand Creek commanders were challenged with leveraging all available resources that related to this precise issue. Resources were pulled from various units within CSPD, as valuable tools to combat social disorder resided outside of the Sand Creek division. In addition to the patrol officers and detectives from Sand Creek, CSPD’s intervention included the community outreach unit, the major crimes investigative unit, the city’s code enforcement unit, and the city’s sanitation department. From our vantage point, this required much more managerial effort that the interventions involving only resources from a single command or precinct. Nonetheless, the project was effectively managed, with the disparate entities holistically contributing to the intervention at each phase of the project.
GPD’s risk-based intervention was heavily patrol focused, with patrol officers expected to carry out the bulk of the project strategies. Rather than focus intervention strategies within a single patrol division, the approach taken by CSPD and KCPD, GPD selected clusters of high-risk street segments in the south-eastern portion of the city for intervention. The NPD similarly focused the intervention at micro-units spread throughout the city and successfully ensured treatment integrity as officer activity did not stray from the target areas (see Figure 2 and Figure 3). GPD, conversely, was not able to ensure that intervention activities were confined to the target areas. Approximately 9% (175 of 1850) of activities occurred outside of the target area, with 59 prospective control street segments being exposed to intervention activities. This led us to re-configure our original research design, with each street segment that experienced at least 1 intervention action as the "target areas" for the evaluation rather than the street segments originally selected to comprise the target area (see Figure 2).
Figure 2. Risk-Based Policing Target Areas: Colorado Springs and Kansas City
Figure 3. Risk-Based Policing Target Areas: Glendale and Newark
In hindsight, we cannot concretely state why GPD was the only agency to struggle keeping intervention activities confined to the originally identified target area. Patrol officers on a whole were briefed of the project and informed to conduct intervention activities when within the geographic target area. It is possible that the lack of an individual set of supervisors to ensure daily treatment fidelity led officers to become overzealous in choosing where to conduct intervention activities. Prior research has suggested that individual police officers often stray from pre-determined intervention boundaries for the purpose of seeking out additional problems to rectify (Sorg et al., 2014). Place-based policing may be particularly susceptible to such a mindset because target area boundaries are typically selected without input of patrol officers and the importance of adhering to boundaries is often not clearly explained by supervisors (Sorg et al., 2014). Conversely, NPD’s intervention included officers under the supervision of a lieutenant at all times of the intervention. Armed with a list of pre-identified businesses to visit, the lieutenant may have been better positioned to ensure officer activity was constrained to the target areas than GPD supervisors who were somewhat detached from the daily patrol activity.
Similar to CSPD, PD-B focused on a single risk factor (problem buildings). However, rather than design a completely new intervention, they used the RTM findings to inform an existing program. PD-B bolstered this effort by creating a computerized dashboard that notified building inspectors of newly designated problem buildings falling within high-risk areas, as diagnosed by RTM. These buildings would become the new focus of the intervention efforts. From a program design perspective, PD-B was able to focus its attention fully towards the creation of the problem buildings dashboard and training personnel in its use given the pre-existing building inspection program.
Phase 4: Project Evaluation
Project evaluation required that the research team be provided with the necessary data to conduct the analysis. At a minimum, we needed data on program outcomes (i.e. the crime of interest) and outputs (i.e. the activities that occurred as part of the intervention). Each of the 5 agencies that successfully implemented an intervention were able to provide accurate outcome data due to their use of a modern RMS. RMS also plays a role in the measurement of outputs, as traditional officer enforcement activities, such as arrests and citations, are readily captured within these databases. However, the vast array of activities incorporated in the risk-based policing strategies meant that officers often conducted activities that were not easily captured within RMS (see Table 2 for overview of intervention activities conducted by each agency). Therefore, measuring officer outputs required additional effort on the part of the police agencies. For example, the aforementioned business manager sign-in sheets used by NPD each tour of duty were provided to the research team for digitizing and geocoding for the evaluation. Conversely, GPD tracked patrol officer flyer distribution by creating a new code in their Computer Aided Dispatch (CAD) system to reflect this specific type of activity. Each time an officer interacted with a community member during flyer distribution, he/she would radio dispatch to create a new CAD assignment reflecting this activity. This made flyer distribution as measurable as the more traditional officer actions typically captured within data systems.
Table 2. Police Department Intervention Activities
COLORADO SPRINGS PD
Code enforcement checks
Community service officer neighborhood cleanups
KANSAS CITY PD
Quality of life summonses
CSPD was able to provide incident-specific data for each intervention activity except the LPR deployment. We were told that LPR units were deployed each day of the intervention period within the target area. However, no information was provided on the locations, times, or number of stolen motor vehicles detected by the LPRs. Similar to our observations regarding PD-C, the use of additional data technologies could have benefitted CSPD’s analysis efforts. Had patrol units been equipped with Automated Vehicle Locator devices, researchers could have readily identified the street segments LPR units traveled through each day. Nonetheless, CSPD did not experience any data collection difficulties with any of their other project outputs, despite the array of activities and units involved. Given that these other activities were emphasized in the intervention more than the LPRs, we were confident that the bulk of CSPD’s output activity was captured.
Unfortunately, PD-B was unable to provide output data in a usable format, which prevented us from conducting an evaluation. Rather, we were only provided with the total counts of building inspections and summonses issued during the intervention. The precise, dates, times, and locations of the outputs were unknown to us. The lack of location data was particularly problematic, as the intervention target area was expected to take shape organically as the program progressed. Thus, we were not only unable to measure treatment fidelity but were also unable to determine exactly where treatment was expected to be delivered in the first place. This may have been an effect of the overarching organizational culture of PD-B. With the appointment of a new Chief in 2011, PD-B instituted a rigorous Compstat process alongside their pre-existing inter-agency crime analysis meetings. As part of weekly meetings, police commanders and representatives from other city agencies were required to provide counts of their unit’s crime control actions. This reflects the limitations of tool-based strategy development (Sparrow, 2008), with the pre-existing agency tool (i.e. Compstat-style activity reports) insufficient for the task at hand (i.e. documenting risk-based policing outputs). To their credit, PD-B analysts contacted various parties at the Mayor’s office in an attempt to obtain the necessary detailed data, unfortunately to no avail. Looking back, having an analyst more involved in the day-to-day aspects of the intervention may have allowed for better output measurement.5
Table 3. Summary of Project Phases
In this article, we presented an honest accounting of our risk-based policing partnerships with 7 police agencies. In certain cases, we were able to fully implement each step of the process while insurmountable difficulties emerged during our work with other agencies (see Table 3). We feel that the findings have a number of implications for policing. For one, our experiences suggest that crime analysts can be important drivers of innovative police practices. Crime analysts have long been considered valuable “translators” of research for police officers and commanders, communicating analysis findings in a manner more accessible to practitioners (Lum & Koper, 2017). While crime analysts fulfilled this role in our risk-based policing projects, they were also oftentimes drivers of the project within their agencies. This suggests that crime analysts can potentially play a larger role in evidence-based policing than has traditionally been envisioned (Lum & Koper, 2017; Piza & Feng, 2017). Unfortunately, the role of crime analysts can be hindered by a police culture and organizational hierarchy that takes little notice of civilian staff (Santos & Taylor, 2014; Taylor et al., 2007), given that crime analysts are primarily staffed by non-sworn personnel. Keay and Kirby (2017) noted that police agencies in the UK have traditionally undermined crime analysts by not fully recognizing the value of analysts and poorly leveraging analyst skills. Nonetheless, Keay and Kirby (2017) argued that the increased implementation of evidence-based policing can be an evolutionary step in firmly establishing crime analysts as true law enforcement professionals by making their work products central to effective police practice (also see Santos, 2014). Therefore, expanded commitment to evidence-based policing may naturally lead to a situation where crime analysts play the type of active role that we witnessed in our projects.6
Our experience on this project also highlights the importance of a localized version of “tight-coupling,” a phenomenon previously associated with successful crime control programs (Klofas et al., 2010; Welsh & Harris, 2016). While the literature primarily discusses coupling as an inter-agency phenomenon, our experience suggests this concept can be applied to individual agencies as well. Police agencies are comprised of various units and functions that oftentimes adhere to their own internal procedures, goals, and objectives, which may not easily translate to other units (Mastrofski & Willis, 2011). Therefore, it is noteworthy that agencies focused the work of disparate units, such as patrol, investigations, and code enforcement, towards a singular goal. Indeed, outside of the NPD, all agencies leveraged the work of multiple internal units in maximizing the efficacy of RTM in addressing their identified risk factors and the occurrence of their priority crime.
Of course, the multi-pronged nature of the interventions was informed by the RTM analysis identifying multiple spatial risk factors for crime. In recognizing this fact, we feel that our experience has implications for data collection activities of police departments. As mentioned earlier, many risk factors of interest were not contained within police department databases, leading us to purchase such data from InfoGroup. While the InfoGroup data allowed us to analyze the risk factors of interest, police may benefit from collecting such data on their own. In particular, police may be able to collect variables not accessible by third party companies that may help refine RTM analyses. For example, while pawn shops have been shown to put nearby residences at risk of burglary by providing easy opportunities for burglars to “fence” illegally obtained goods (Moreto et al., 2014; Wright & Decker, 1994), individual pawn shops may greatly differ in terms of the frequency at which they purchase stolen property (Comeau et al., 2011). Isolating such facilities may help increase the predictive capacity and practical utility of RTM. Therefore, we feel that police should place greater emphasis on the frequent collection of spatial risk data so that such information is as accessible as crime and officer activity data, echoing the recommendations made by Kennedy et al. (2011).
In addition to improving data collection activities, we strongly recommend that police invest in the necessary training and resources for their analysts to conduct the research and evaluation tasks performed in this project. This would help improve the sustainability of projects that emerge from researcher/practitioner partnerships by ensuring that crime analysts can take the lead on research efforts after the academic partners are no longer involved in the project. In hindsight, there is more the authors could have done during the project to help towards this end. As previously discussed, the research team exclusively handled the problem analysis and program evaluation phases so that project milestones were achieved in a timely manner. However, had we planned for it at the outset of the project, crime analysts could have played a more direct role in these portions of the project. Piza and Feng (2017) recommend that researcher/practitioner partnerships embrace the knowledge-exchange feature of action research, which would directly expose crime analysts to the procedural aspects of rigorous research and evaluation. Ideally, this could lead to crime analysts “developing skills they can employ in their day-to-day duties” and allow them to “disseminate these newfound skills within her or his agency” (Piza & Feng, 2017, p. 22). More directly involving crime analysts in the problem analysis and evaluation stages of risk-based policing may help sustain such projects well after the conclusion of funding periods.
In conclusion, while we believe that this account of our prior risk-based policing projects can be helpful for those interested in replicating this kind of work, the issues of program implementation did not come into focus for us until we moved towards the deployment phase of the program. While we realized the importance of this issue early on, we recommend that policing scholars rigorously document factors related to program implementation as part of a priori process evaluations. By doing so, researchers will ensure that practitioners have access to analytical tools, such as RTM, and information necessary for successful replication of evidence-based programs (Johnson et al., 2015).
Ariel, B. & Partridge, H. (2016). Predictable policing: Measuring the crime control benefits of hotspots policing at bus stops. Journal of Quantitative Criminology, DOI: 10.1007/s10940-016-9312-y.
Ariel, B., Weinborn, C., & Sherman, L. (2016). “Soft” policing at hot spots—do police community support officers work? A randomized controlled trial. Journal of Experimental Criminology, 12(3), 277–317.
Barnum, J., Caplan, J., Kennedy, L., and Piza, E. (2017). The crime kaleidoscope: A cross-jurisdictional analysis of place features and crime in three urban environments. Applied Geography, 79: 203-211.
Barnum, J., Campbell, W., Trocchio, S., Caplan, J., and Kennedy, L. (2016). Examining the environmental characteristics of drug dealing locations. Crime & Delinquency, DOI: 10.1177/0011128716649735.
Berman, G. & Fox, A. (2010). Trial & error in criminal justice reform. Learning from failure. Washington, DC: Urban Institute Press.
Bowers, K. & Johnson, S. (2010). Implementation failure and success: Some lessons from England. In Knutsson, J. and Clarke, R. (eds.) Putting Theory to Work. Implementing Situational Prevention and Problem-Oriented Policing. Crime Prevention Studies: 163-198. Volume 20. Boulder, CO. London, UK: Lynne Rienner Publishers.
Braga, A. (2016). The value of “pracademics” in enhancing crime analysis in police departments. Policing: A Journal of Policy and Practice, 10(3), 308-314.
Braga, A. (2013). Embedded criminologists in police departments. Ideas in American Policing. Police Foundation: Washington, D.C.
Braga, A. (2010). Setting a higher standard for the evaluation of problem‐oriented policing initiatives. Criminology & Public Policy, 9(1), 173-182.
Brown, R. (2010). The role of project management in implementing communty safety initiatives. In Knutsson, J. and Clarke, R. (eds.) Putting Theory to Work. Implementing Situational Prevention and Problem-Oriented Policing. Crime Prevention Studies. Volume 20: 37-64. Boulder, CO. London, UK: Lynne Rienner Publishers.
Buerger, M. (2010). Polcing and research: Two cultures separate by an almost-common languague. Police Practice and Research, An International Journal, 11(2): 135-143.
Caplan, J. & Kennedy, L. (2016). Risk terrain modeling. Crime predictions and risk reduction. Oakland, CA: University of California Press.
Caplan, J. and Kennedy, L. (2013). Risk terrain modeling diagnsotics utility (version 1.0). Newark, NJ: Rutgers Center on Public Security.
Caplan, J., Kennedy, L., & Baughman, J. (2012). Kansas City’s violent crime initiative: A place-based evaluation of location-specific interventon activites during a fixed time period. Crime Mapping, 4(2): 9-37.
Caplan, J., Kennedy, L., & Miller, J. (2011). Risk terrain modeling: Brokering criminological theory and GIS methods for crime forecasting. Justice Quarterly, 28(2): 360–381.
Caplan, J., Kennedy, L., Barnum, J. and Piza, E. (2015). Risk terrain modeling for spatial risk assessment. Cityscape: A Journal of Policy Development and Research, 17(1): 7-16.
Cissner, A. & Farole, D. (2009). Avoiding failures of implementation. Lessons from process evaluations. New York: Center for Court Innovation. Washington, DC: Bureau of Justice Assistance.
Comeau, M., Duda, J., Petitti, N., & Klofas, J. (2011). Analysis of 2010 Rochester-City pawn shop transactions. Working paper: 2011-03. Rochester, NY: Center for Public Safety Initiatives, Rochester Institute of Technology.
Drawve, G. & Barnum, J. (2016). Place-based risk factors for aggravated assault across police divisions in Little Rock, Arkansas. Journal of Crime and Justice, DOI: 10.1080/0735648X.2016.1270849.
Drawve, G., Moak, S., & Berthelot, E. (2016). Predictability of gun crimes: A comparison of hot spot and risk terrain modelling techniques. Policing and Soceity, 26(3): 312-331.
Eck, J. & Spelman, W. (1987). Problem-solving: Problem-Oriented policing in Newport News. Washington, DC: Police Executive Research Forum.
Goldstein, H. (1990). Problem-Oriented policing. New York: McGraw-Hill.
Goldstein, H. (1979). Improving policing: A problem-oriented approach. Crime and Delinquency, 25: 236-258.
Hagan, J. (1989). Why is there so little criminal justice theory? Neglected macro- and micro-level links between organization and power. Journal of Research in Crime and Delinquency, 26(2): 116-135.
Heffner, J. (2013). Statistics of the RTMDx utility. In Caplan, J., Kennedy, L., and Piza, E. (eds.) Risk Terrain Modeling Diagnostics (RTMDx) Utility user manual (Version 1): 26-34. Rutgers Center on Public Security: Newark, NJ.
Huey, L., & Mitchell, R. J. (2016). Unearthing hidden keys: Why pracademics are an invaluable (if underutilized) resource in policing research. Policing: A Journal of Policy and Practice, 10(3), 300–307.
Innes, M., Fielding, N, & Cope, N. (2005). ‘The appliance of science?’ The theory and practice of criminal intelligence analysis. British Journal of Criminology, 45: 39-57.
Johnson, S., Tilley, N., & Bowers, K. (2015). Introducing EMMIE: An evidence rating scale to encourage mixed-method crime prevention synthesis reviews. Journal of Experimental Criminology, 11(3): 459-473.
Keay, S. & Kirby, S. (2017). The evolution of the police analyst and the influence of evidence-based policing. Policing, A Journal of Policy & Practice. DOI: 10.1093/police/pax065.
Kelling, G. & Coles, C. (1996). Fixing broken windows: Restoring order and reducing crime in our communities. New York: Simon and Schuster.
Kennedy, L., Caplan, J., Piza, E., & Buccine-Schraeder, H. (2016). Vulnerability and exposure to crime: Applying risk terrain modeling to the study of assault in Chicago. Applied Spatial Analysis and Policy, 9(4): 529-548.
Kennedy, L., Caplan, J., & Piza, E. (forthcoming). Risk-based policing: Crime risk governance using Risk Terrain Modeling. Oakland, CA: University of California Press.
Kennedy, L., Caplan, J., & Piza, E. (2011). Risk clusters, hotspots, and spatial intelligence: Risk terrain modeling as an algorithm for police resource allocation strategies. Journal of Quantitative Criminology, 27(3): 339-362.
Kennedy, D. (1997). Pulling levers: Chronic offenders, high-crime settings, and a theory of prevention. Valparaiso University Law Review, 31: 449-484.
Klofas, J., Hipple, N. & McGarrell, E. (eds.) (2010). The New Criminal Justice. American Communities and the Changing World of Crime Control. Routledge: New York and London.
Leigh, A., Read, T., & Tilley, N. (1996). Problem-Oriented policing. Paper 75. Crime Detection and Prevention Series. Police Research Group, Home Office: London.
Lersch, K. (2017). Risky places: An analysis of carjackings in Detroit. Journal of Criminal Justice, DOI: 10.1016/j.jcrimjus.2017.07.011.
Lewin, K. (1947). Group decision and social change. In Newcomb, T. & Hartley, E. (eds.) Readings in Social Psychology (p. 202-203). New York: Holt and Company.
Lum, C., & Koper, C. (2017). Evidence-Based policing: Translating research into practice. Oxford, UK: Oxford University Press.
MacQueen, S. & Bradford, B. (2017). Where did it all go wrong? Implementation failure—and more—in a field experiment of procedural justice policing. Journal of Experimental Criminology, 13(3): 321-345.
Mastrofski, S. & Willis, J. (2011). Police organization. In Tonry, M. (ed.) The Oxford Handbook of Crime and Criminal Justice: 479-508. Oxford, UK: Oxford University Press.
Mock, L. (2010). Action research for crime control and prevention. In Klofas, J., Hipple, N. & McGarrell, E. (eds.) The New Criminal Justice. American Communities and the Changing World of Crime Control. Routledge: New York and London.
Moreto, W., Piza, E., and Caplan, J. (2014). A plague on both your houses? Risks, repeats, and reconsiderations of urban residential burglary. Justice Quarterly, 31(6): 1102-1126.
Nagin, D., Solow, R., & Lum, C. (2015). Deterrence, criminal opportunities, and police. Criminology, 53(1): 74–100.
Papachristos, A. (2011). Too big to fail. The science and politics of violence prevention. Criminology & Public Policy, 10(4): 1053-1061.
Piza, E. (in press). The effect of various police enforcement actions on violent crime: Evidence from a saturation foot-patrol intervention. Criminal Justice Policy Review, DOI: 10.1177/0887403417725370.
Piza, E. & Feng, S. (2017). The current and potential role of crime analysts in evaluations of police interventions: Results from a survey of the International Association of Crime Analysts. Police Quarterly, DOI: 10.1177/1098611117697056.
Piza, E., Feng, S., Kennedy, L. and Caplan, J. (2016). Place-based correlates of motor vehicle theft and recovery: Measuring spatial influence across neighbourhood context. Urban Studies. DOI: 10.1177/0042098016664299: 1-24.
Ratcliffe, J. (2005). The effectiveness of police intelligence management: A New Zealand case study. Police Practice and Research, 6(5): 435-451.
Read, T. & Tilley, N. (2000). Not Rocket Science? Problem-Solving and Crime Reduction. Crime Reduction Research Series, Paper 6. Home Office, Policing and Reducing Crime Unit: London.
Rengifo, A., Stemen, D., & Amidon, E. (2017). When policy comes to town: Discourse and dilemmas on implementation of a statewide reentry policy in Kansas. Criminology, DOI: 10.1111/1745-9125.12146.
Sanders, C., Weston, C., & Schott, N. (2015). Police innovations, ‘secret squirrels’ and accountability: Empirically studying intelligence-led polcing in Canada. British Journal of Criminology, 55: 711-729.
Santos, R. B. (2014). The effectiveness of crime analysis for crime reduction: Cure or diagnosis?. Journal of Contemporary Criminal Justice, 30(2), 147–168.
Santos, R. B. (2013). Crime analysis with crime mapping. 3rd edition. SAGE: Los Angeles.
Santos, R. B., & Taylor, B. (2014). The integration of crime analysis into police patrol work: Results from a national survey of law enforcement agencies. Policing : An International Journal of Police Strategies & Management Article Information, 37(3), 501–520.
Secret, M., Abell, L., & Berline, T. (2011). The promise and challenge of practice-research collaborations: Guiding principles and strategies for initiating, designing, and implementing program evaluation research. Social Work, 56(1), 9-20.
Scott, M. (2010). Implementing crime prevention: Lessons learned from problem-oriented policing projects. In Knutsson, J. and Clarke, R. (eds.) Putting Theory to Work. Implementing Situational Prevention and Problem-Oriented Policing. Crime Prevention Studies. Volume 20: 9-36. Boulder, CO. London, UK: Lynne Rienner Publishers.
Shane, J. (2007). What every chief executive should know: Using data to measure police performance. Looseleaf Publications: Flushing, NY.
Sherman, L. & Weisburd, D. (1995). General deterrent effects of police patrol in crime "hot spots": A randomized controlled trial. Justice Quarterly, 12(4): 625-648.
Sorg, E., Wood, J., Groff, E., and Ratcliffe, J. (2014). Boundary Adherence During Place-Based Policing Evaluations: A Research Note. Journal of Research in Crime and Delinquency, 51(3): 377-393.
Sparrow, M. (2016). Handcuffed. What Holds Policing Back, and the Keys to Reform. Washington, DC: Brookings Institution Press.
Sparrow, M. K. (2011). Governing science. New Perspectives in Policing. Harvard University Executive Session on Policing and Public Safety. Cambridge, MA: Harvard Kennedy School of Government, Program in Criminal Jusitce Policy and Management Executive Session on Policing and Public Safety. Washington, DC: US Department of Justice, Office of Justice Programs, National Institute of Justice.
Sparrow, M. (2008). The character of harms. Operational challenges in control. Cambridge University Press: New York, NY and London, UK.
Taylor, B., Kowalyk, A., & Boba, R. (2007). The integration of crime analysis into law enforcement agencies: An exploratory study into the perceptions of crime analysts. Police Quarterly, 10(2), 154-169.
Visher, S. & Weisburd, D. (1998). Identifying what works: Recent trends in crime prevention strategies. Crime, Law and social change, 28: 2230242.
Weisburd, D., Mastrofski, S., McNally, A., Greenspan, R., & Willis, J. (2003). Reforming to preserve: COMPSTAT and strategic problem solving in American policing. Criminology & Public Policy, 2(3): 421-456.
Wellford, C. (2009). Criminologists should stop whining about their impact on policy
and practice. In N. A. Frost, J. D. Freilich, & T. R. Clear (Eds.), Contemporary issues in criminal justice policy (1st ed., pp. 17-24). Belmont, CA: Wadsworth Cengage Publishing.
Welsh, W. & Harris, P. (2016). Criminal justice policy and planning. 5th Edition. New York, NY. London, UK: Routledge.
Willis, J., Mastrofski, S., & Weisburd, D. (2007). Making sense of COMPSTAT: A theory-based analysis of organizational change in three police departments. Law & Society Review, 41(1): 147-188.
Wright, R. & Decker, S. (1994). Burglars on the job. Boston, MA: Northeastern University Press.