Abstract

This paper evaluates the impact of the “drop 1” implementation of the CONNECT system by the Metropolitan Police Service (MPS) on November 29, 2022, using a synthetic control method to compare crime trends with data from other police forces. While statistically significant changes were observed in several crime categories, robustness checks indicate caution in attributing these changes solely to CONNECT. The most robust findings include a decrease in vehicle crime (contrasting with an increase in the control), an increase in ‘Other crime,’ and a decrease in the detection rate for ‘Other theft.’ However, high prediction interval violations for some crime types suggest limitations in the model’s ability to fully capture complex crime trends. For categories like ‘Theft from the person’ and drug offences, large changes were observed, but robustness checks indicate these shifts may not be uniquely attributable to the CONNECT system.

This paper examines the impact of “drop 1” of the CONNECT system, implemented by the Metropolitan Police Service (MPS) on November 29, 2022, on crime rates and detection. Using a synthetic control method with data from comparable police forces, we analysed changes in various crime types before and after the system’s implementation.

While we observed statistically significant changes in several crime categories, our robustness checks suggest that not all of these changes can be confidently attributed to the implementation of CONNECT.

The most robust findings appear to be:

1. A decrease in vehicle crime, which contrasts with an increase in synthetic control.

2. An increase in ‘Other crime’, which showed good pre-implementation fit and a low placebo test p-value.

3. A decrease in the detection rate for ‘Other theft’, which demonstrated good pre-implementation fit and moderate robustness in our checks.

However, even for these findings, we must exercise caution in interpretation. The high prediction interval violation rates for some crime types (e.g., ‘Other crime’ at 0.824) suggest that our model may not fully capture the complexity of crime trends.

For other crime types, such as ‘Theft from the person’ and drug offences, the large observed changes were accompanied by robustness check results that suggest these changes may not be uniquely attributable to the CONNECT system.

1. Introduction

The accurate recording and analysis of crime data are crucial for effective policing and policy-making. In recent years, many police forces have implemented new IT systems to improve these processes. On November 29, 2022, the Metropolitan Police Service (MPS) launched CONNECT, a new system for Case, Custody, and Property management, known as Drop 1. While
such implementations are expected to impact policing practices and potentially crime rates, the precise effects are often unclear.

This study aims to quantify the impact of the CONNECT system on crime rates and detection in the MPS area. We hypothesise that the implementation of CONNECT may have affected both the actual incidence of crime and the recording practices, potentially leading to changes in reported crime statistics.

The importance of this research lies in its potential to inform policy decisions regarding the implementation of new technologies in law enforcement. By rigorously analysing the effects of the CONNECT system, we can provide valuable insights into the potential benefits and challenges associated with such technological upgrades. Furthermore, this study contributes to the broader literature on the impact of information technology on policing and crime statistics.

2. Methods

2.1 Data Source and Preparation

We obtained crime data from data.police.uk, covering the period from February 2019 to April 2024. This dataset includes crime counts for all major crime types and outcomes, allowing us to calculate positive outcome rates. We defined positive outcomes as:

• Local resolution

• Offender given a caution

• Awaiting court outcome

• Offender given penalty notice

• Offender given a drugs possession warning

The data was aggregated on a monthly basis for each police force and crime type. We excluded ‘Anti-social behaviour’ from our analysis due to its classification as an incident rather than a crime in UK policing.

2.2 Synthetic Control Method

To assess the impact of the CONNECT system, we employed the synthetic control method (Abadie et al., 2010). This approach creates a synthetic counterpart for the MPS using a weighted combination of control units (other police forces) that closely resembles the MPS in terms of crime patterns before the intervention.
We selected the following police forces as our control group, based on similarity to the MPS as defined by Her Majesty’s Inspectorate of Constabulary and Fire & Rescue Services (HMICFRS), with some additions due to data availability issues:

• West Midlands Police

• Merseyside Police

• Thames Valley Police

• West Mercia Police

• West Yorkshire Police

• Avon and Somerset Constabulary

The synthetic control was constructed for each crime type separately, using pre-intervention data to find the optimal weights for the control forces that best replicated the MPS’s pre-intervention crime trends. This method allows us to create a counterfactual scenario, estimating what would have happened in the MPS if the CONNECT system had not been implemented.

2.3 Seasonality Adjustment

To account for seasonal variations in crime rates, we applied a seasonal decomposition using the STL (Seasonal and Trend decomposition using Loess) method. This approach separates the time series into seasonal, trend, and residual components, allowing us to focus on the underlying trends without seasonal confounds. The seasonality-adjusted data were used in all subsequent analyses.

2.4 Statistical Analysis

For each crime type, we compared the actual post-intervention crime rates in the MPS with those predicted by the synthetic control. We calculated the following metrics:

1. Percentage changes in crime counts and detection rates for both actual and synthetic data

2. T-statistics and p-values to assess the statistical significance of differences between actual and synthetic data

3. Effect sizes using Cohen’s d to quantify the magnitude of the differences

To account for multiple comparisons and reduce the risk of Type I errors, we applied the Bonferroni correction to the p-values. This conservative approach ensures that our findings are robust against the increased risk of false positives due to multiple testing.

2.5 Robustness Checks

To ensure the reliability of our findings and address potential limitations of the synthetic control method, we conducted several robustness checks:

1. Placebo Tests: We ran the synthetic control analysis with randomly chosen placebo implementation dates to assess whether the observed effects could have occurred by chance. This helps to evaluate the specificity of our findings to the actual implementation date.

2. Sensitivity Analysis: We evaluated how sensitive the results were to the exclusion of individual predictor variables. This check helps to identify whether any single control unit is driving the results, ensuring the stability of our synthetic control.

3. Pre-implementation Fit Assessment: We calculated the Root Mean Square Error (RMSE) between the actual and synthetic control data in the pre-implementation period to assess the quality of the fit. A lower RMSE indicates a better fit and increases confidence in the synthetic control.

4. Prediction Interval Violation Rate: We computed how often the actual post-implementation values fell outside the predicted intervals to assess the model’s accuracy. This check helps to evaluate the reliability of our synthetic control predictions.

These robustness checks provide a comprehensive assessment of the validity and reliability of our findings, allowing us to draw more confident conclusions about the impact of the CONNECT system.

3. Results

3.1 Crime Counts

Our analysis revealed several notable changes in crime patterns coinciding with the implementation of the CONNECT system. Here, we discuss the most significant findings, their robustness, and potential interpretations.

3.1.1 Other Crime

The category of ‘Other crime’ showed a substantial increase following the implementation of CONNECT. The actual crime count increased by 27.4% compared to a 12.5% increase in the synthetic control. This difference was statistically significant (p = 0.0013) even after Bonferroni correction.

Robustness Check: The placebo test yielded a p-value of 0.02, suggesting that the change is unlikely to have occurred by chance. The pre-implementation fit (RMSE = 53.75) was among the best in our analysis, indicating a good match between the actual and synthetic data before the
intervention. However, the prediction interval violation rate was high at 0.824, suggesting some uncertainty in the post-implementation predictions.

Interpretation: The increase in ‘Other crime’ could potentially be explained by improved recording practices or easier reporting mechanisms facilitated by the new system. The robust placebo test result strengthens this interpretation, but the high prediction interval violation rate suggests caution in attributing the entire change to CONNECT.

3.1.2 Theft from the Person

‘Theft from the person’ showed a dramatic increase of 77.9% in the actual data, compared to a 20.4% increase in the synthetic control. This difference was highly statistically significant (p < 0.001) after Bonferroni correction.

Robustness Check: This crime type showed higher sensitivity (0.171) in our robustness checks, indicating that the results are somewhat sensitive to the exclusion of individual predictors. The placebo test p-value was 0.95, suggesting that the observed effect could potentially have
occurred by chance. The pre-implementation fit (RMSE = 584.38) was relatively high, indicating some discrepancy between the actual and synthetic data before the intervention.

Interpretation: While the increase in ‘Theft from the person’ is substantial, the robustness checks suggest we should be cautious in attributing this change directly to the CONNECT system. The high sensitivity and placebo test p-value indicate that this finding may not be as reliable as others in our analysis.

3.1.3 Vehicle Crime

Vehicle crime showed a 3.9% decrease in the actual data, contrasting with a 9.3% increase in the synthetic control. This difference was statistically significant (p < 0.001) after Bonferroni correction.

Robustness Check: The placebo test p-value for vehicle crime was 0.14, which is relatively low, suggesting that the observed effect is less likely to have occurred by chance. The pre-implementation fit (RMSE = 747.76) was relatively high, indicating some discrepancy
between the actual and synthetic data before the intervention. The prediction interval violation rate was moderate at 0.647.

Interpretation: The decrease in vehicle crime, contrasting with an increase in the synthetic control, is one of our more robust findings. This could suggest that the CONNECT system has enabled more effective prevention or detection strategies for this crime type. However, the relatively high RMSE in the pre-implementation period warrants some caution in this interpretation.

3.1.4 Drugs

Drug offences showed a 22.3% decrease in the actual data, compared to a 4.9% decrease in the synthetic control. This difference was statistically significant (p < 0.001) after Bonferroni correction.

Robustness Check: This crime type showed higher sensitivity (0.190) in our robustness checks, indicating that the results are somewhat sensitive to the exclusion of individual predictors. The
placebo test p-value was 0.68, suggesting that the observed effect could potentially have occurred by chance. The pre-implementation fit (RMSE = 331.37) was moderate.

Interpretation: The substantial decrease in drug offences could indicate changes in policing strategies or recording practices for these offences with the new system. However, the high sensitivity and placebo test results suggest that this finding should be interpreted with caution. Other factors, such as changes in drug policy or shifts in police priorities, could also be contributing to this trend.

3.1.5 Shoplifting

Shoplifting showed a 59.2% increase in actual data compared to a 5.1% increase in synthetic control, which was statistically significant (p < 0.001) after Bonferroni correction.

Robustness Check: The placebo test p-value (0.79) suggests this change may not be directly attributable to the CONNECT system. The sensitivity was very low (-0.0002), indicating that the result is stable across different predictor combinations. The pre-implementation fit (RMSE = 243.59) was relatively good.
Interpretation: While the increase in shoplifting is substantial and statistically significant, the high placebo test p-value suggests that this change might not be uniquely associated with the CONNECT system implementation. Other factors, such as changes in retail patterns or economic conditions, might be contributing to this trend.

3.2 Crime Detection Rates

In addition to crime counts, we analysed changes in detection rates following the implementation of CONNECT. Here, we discuss the most notable findings.

3.2.1 Drugs

The detection rate for drug offences showed a 39.3% decrease in the actual data, compared to a 12.1% decrease in the synthetic control. This difference was statistically significant (p < 0.001) after Bonferroni correction.

Robustness Check: The placebo test p-value was 0.69, suggesting that the observed effect could potentially have occurred by chance. The pre-implementation fit (RMSE = 0.0398) was moderate for detection rates.
Interpretation: The substantial decrease in drug offence detection rates, coupled with the decrease in drug offence counts, could indicate a shift in how drug-related crimes are being handled or recorded in the new system. However, the high placebo test p-value suggests caution in attributing this change directly to CONNECT. It’s possible that other factors, such as changes in policing priorities or resource allocation, could be contributing to this trend.

3.2.2 Other Theft

The detection rate for ‘Other theft’ showed a 45.7% decrease in the actual data, compared to a 11% decrease in the synthetic control. This difference was statistically significant (p < 0.001) after Bonferroni correction.

Robustness Check: The placebo test p-value was 0.15, and the sensitivity was 0.048, indicating moderate robustness. The pre-implementation fit (RMSE = 0.0006) was very good for detection rates.

Interpretation: The decrease in detection rates for ‘Other theft’ could potentially be related to changes in how these crimes are recorded or prioritised in the new system. The lower placebo
test p-value and good pre-implementation fit suggest that this finding is more reliable than some others in our analysis.