Description
Shared with Dropbox
This report analyses the effects of LED streetlight conversions on nighttime firearm crimes in Dallas, Texas. Using data from 2020 through May of 2022 on reported firearm crimes and LED conversions, using a differences-in-differences strategy (comparing trends in daytime firearm crimes compared to those at night), I find no evidence that LED conversions result in decreased nighttime firearm crimes in this sample. Code and Data to replicate the report findings can be obtained here.
In 2019, amid rising crime rates, Mayor Eric Johnson of Dallas, Texas convened a task force to suggest potential non-criminal justice focused interventions to potentially reduce violent gun crime (Mayors Task Force Report, 2019). One of the suggested interventions was expanding outdoor lighting, based on findings for an experiment of expanding lighting outside of public housing developments (Chalfin et al., 2022).
In response the city of Dallas did not expand lighting, but instead has replaced existing street lamps with brighter and more cost-efficient LED (light emitting diode) bulbs. While deviating from the original proposal, LED light expansion is theoretically expected to decrease crime for the same reasons as light expansion (Kaplan & Chalfin, 2021). Areas with brighter lights can potentially act as a deterrent for someone interested in committing crime hidden by darkness (Wood, 1981).
Figure 1 displays the monthly number of LED light conversions (both for the city of Dallas, as well as a local electric company ONCOR), so one can see that although historical conversions took place, such as a large spike in early 2019, there were an additional large number of conversions starting in mid 2020.
Figure 1: LED light conversions per month in Dallas
This analysis finds that such expansion did not result in appreciable firearm crime reductions in Dallas post 2020. What follows is a description of the empirical methods used to assess that effect, as well as further discussion of potential limitations of those results.
The data used in the study include the publicly disseminated crimes from Dallas PD, reported from 1/1/2020 through 6/5/2022 (we subsequent aggregate to months, so only include crimes through May 2022). These are broken down by the time period that they were reported into daytime/nighttime based on the initial time the incident occurred (not reported). Daytime events are calculated based on each month’s average sunrise/sunset times in Dallas, e.g. in January events between 7:30 and 17:30 are considered occurring during the day, while events in July the time period is between 6:23 and 20:39 are considered as daytime crimes. While indeterminate crime reporting times can potentially cause issues, this is mostly limited to property crimes in which the victim (the owner of the property) may not be physically present (Ratcliffe, 2002). Violent crimes in which an individual was victimized tend to not have as much uncertainty.
As the task force was focused on firearm crimes, so is the analysis. While I initially considered an analysis of all crimes, as I will later discuss the empirical design (differences in differences), the all crimes sample failed to follow the parallel trends assumption. As such I only consider firearm crimes, as reported via a firearm weapon used in the offense. In the Dallas data over this time period, this includes 4,162 assaults, 2,719 robberies, and 166 homicides (NIBRS offense categories).
The second data source are street light locations, as well as LED conversions, provided by the Child Poverty Action League for the analysis. This dataset, which includes geocoded street lamp locations for ONCOR (n = 71,862) and the city of Dallas (n = 13,541), as well as dates of LED conversions. We estimate per grid cell the number of LED conversions post 2020 per month.
We then aggregate these events to quarter square mile grid cells (n=970), month (n=29), and daytime/night (n=2). Resulting in a total of 56,260 observations. While there are more grid cells than 970 in Dallas, we eliminate grid cells that had 0 firearm crimes in the entire study period. These grid cells are totally uninformative to subsequent difference in difference estimates we will generate. Table 1 displays the summary statistics used in the subsequent regression models. We incorporate the cumulative number of lights converted per grid cell into the model, as this models the effect of light conversions as a permanent effect, not transitory only effecting crime in the month it occurred. (If the effect is of the former, it is unlikely reducing crime via the lights themselves, but potentially via the construction of lights themselves, similar to what is reported in Kim & Wo, 2021 for housing demolitions).
Table 1: Descriptive Statistics
Variable | Mean | Std. Dev | Min | Max | % Zeroes |
---|---|---|---|---|---|
Firearm Crimes | 0.12 | 0.40 | 0 | 8 | 89% |
Cumulative Lights | 0.78 | 6.07 | 0 | 247 | 94% |
Nighttime | 0.50 | 0.50 | 0 | 1 | 50% |
N | |||||
Grid Cells | 970 | ||||
Months | 28 | ||||
Total Observations | 56,260 |
|
|
|
|
Initially I considered a research design similar to the analysis in Chalfin et al. (2021), a micro level buffer analysis, examining changes in nighttime crimes for street lights that were converted. But, after examining the clustering conversions, I rejected this as a possibility. Figure 2 displays the clusters of light conversions post starting in 2020 in Dallas.
Figure 2: LED Light conversions post June-2020 in Dallas
These effectively cover much of the city, including areas of high crime (Wheeler & Reuter, 2021; Wheeler & Steenbeek, 2021). With a micro level design, there would be few locations left for controls that also had appreciable counts of firearm crimes. As such, I switched to an empirical design that could reasonably identify effects of the intensity of the lights, not just the effect of a single light.
To do this, I reduced Dallas proper (eliminating lochs that are typically included in the city boundary), and created a set of 1/4 square mile grid cells across the city. While quarter square mile cells are larger than micro units (Wheeler & Steenbeek, 2021) and are ad-hoc, firearm crimes tend to be fairly rare events, and so call for slightly larger spatial aggregations (Drake et al., 2022). Figure 3 shows the intensity of light conversions (starting January 2020 through May 2022) aggregated to grid cells.
Figure 3: Map of quarter square mile grid cells and LED conversion intensity
Breaking down firearm crimes to quarter square mile grid cells, months, and daytime/nighttime units of analysis, over 89% of the units of analysis had 0 firearm crimes, and the max number of firearm crimes was 8. Lower base rates tend to result in reduced power (Hinkle et al., 2013), and so I did not create even smaller grid cells for analysis.
An additional empirical modelling strategy that is plausible, but I did not take advantage of, was a regression discontinuity design at nighttime (Doleac & Sanders, 2015; Dominguez & Asahi 2017). This would require identifying many micro level discontinuities, and then seeing if LED expansion reduced the discontinuity effect. I leave such a potential analysis to a future date or another researcher to conduct.
The design I did take advantage of was a difference in difference strategy, using daytime crimes as the reference point to identify parallel trends. The equation I fit is specifically:
Where we are predicting the logarithm of the expected value of firearm crimes for grid cell j, month t, and day/night period n. On the right-hand side of the equation, are treatment effect of interest is
For a simple example, imagine that control areas (areas with no LED conversions), had on average 10 crimes in the daytime, and 20 at nighttime. Now consider a treated area, with only one LED conversion. If the treated area had 15 crimes on average during the daytime (this difference in baseline is captured via the grid cell fixed effect), in the Poisson model the parallel trends assumption, extrapolating from the control area, would predict the treated area should have 30 crimes (double) at nighttime. So, if the treated area has fewer than 30 crimes, say 25 crimes, that would result in a negative estimate for the
One may ask, why the term
The main benefit of this design, is that by comparing each grid cell to itself (daytime vs nighttime), it likely controls for other exogenous factors that may impact the estimates. For example, general trends due to Covid are unlikely to confound the treatment estimate in this design (Ashby, 2020). Only in the case that some exogenous factor impacts daytime or nighttime crimes differentially in an area would the treatment effect estimate here be biased. Spatial exogenous factors that may change over time do not directly impact the estimate.
The reason I use a Poisson model with a log link is because the parallel trends assumption between daytime and nighttime crimes appears to follow a constant ratio (as opposed to a constant linear change). Figure 4 shows in the aggregate, that the natural logarithm of daytime and nighttime crimes in Dallas over the first six months of 2020 (which had only a few light conversions over the study area, so can effectively be considered a pre time period).
Figure 4: Parallel trends in cumulative daytime vs nighttime firearm crimes in Dallas over six months
While the aggregate data does not guarantee a similar micro grid cell level constant ratio, it seems quite plausible given other longitudinal studies of micro level crime data (Wheeler et al., 2016). The parallel trends being on a ratio scale suggests a difference in difference design using Poisson regression is appropriate (Wilson et al., 2022), as opposed to a linear difference in difference estimator (Wheeler & Ratcliffe, 2018).
Finally, since the two-way fixed effects estimator for the difference in difference design is known to potentially be biased in the case of treatment effect heterogeneity, we additionally examine grid cell specific effects (Baker et al., 2022; Circo et al., 2022). We use the same equation as above, except for the treatment effect
Table 2 presents the results of the global treatment effect estimate for the effect of cumulative LED light conversion on firearm crimes. This shows the point estimate is positive, opposite of what we might expect, although is quite small. The standard error of the estimate is reasonably small as well, although does not reach typical statistical significance at a 0.05 level. At these small of values, the linear coefficient can be interpreted as a multiplier, e.g. one additional light conversion results in an increase in firearm crimes of 0.2% per month. Although again this effect is so small it seems reasonable to fail to reject the null hypothesis that light conversions do not impact firearm crime rates.
Table 2: Poisson regression model, estimating effect of cumulative light conversions on firearm crimes
| Beta | Standard Error | p-value |
---|---|---|---|
Cumulative Light Effect | 0.002 | 0.001 | 0.059 |
Grid Cell Fixed Effects | X | ||
Nighttime*Month Fixed Effects | X |
|
|
The model fit is excellent, for example there are observed 89.5% zeroes in the sample, and the model predicted density is 89.2% zeroes (Long, 1997). As such, there does not appear to be obvious reasons to pursue other alternatives to Poisson, such as quasi-Poisson or negative binomial models (Berk & MacDonald, 2008; Wilson, 2022).
For analyzing treatment effect heterogeneity, we estimate a treatment effect of light conversions for each individual grid cell. Figure 5 displays a caterpillar plot of those effects, eliminating several cells which have explosive effects (we eliminated cells that had standard errors larger than 2). None of the individual grid cells have a 95% point-wise confidence interval that the upper bound does not cover 0 (a few cells have positive effects, but due to multiple comparisons this is as likely due to noise in the estimates as to a real positive increase in firearm crimes).
In total, both the aggregate treatment effect estimates as well as the micro grid cell estimates do not provide evidence that LED conversions result in reduced firearm crimes at nighttime. While the micro level estimates are noisy (and so may lack power), the small standard error for the global estimate suggests the study design is reasonably powered to detect small effects.
Figure 5: Estimates of Treatment Effect Het. for grid cell LED conversions. Shown are estimates that have a standard error less than 2, ordered by the log incident rate ratio. Shown are point-wise 95% confidence intervals.
For subsequent robustness checks, we conducted three additional analyses. One, we estimate treatment effect heterogeneity using a random effects model (which shrinks estimates towards the global effect, which is useful here to prevent the noisy estimates). This results in smaller and more consistent standard errors, but results in effectively the same inferences – treatment effects are centered around 0 and the pointwise confidence intervals for individual grid cell effects clearly cover 0.
A second robustness analysis we undertake is to identify whether light conversions have a diminishing effect. To test this, we evaluate the treatment effect via the logarithm of cumulative lights in a grid cell (with zero lights still defined as 0). This again results in substantively very similar estimates as the main results reported (a small positive point estimate).
One might additionally undertake robustness checks in terms of different standard error estimates (e.g. clustered standard errors, or other robust covariance estimators). These will not change point estimates, and likely would only increase standard errors. As such I do not provide any additional covariance estimators for this data, as it would not change inferences of the null effect of light conversions on nighttime firearm crimes.
A third robustness check is evaluating whether increasing the size of the grid cells results in a different overall estimate. To do this, I generate grid cells at square miles and repeat the analysis. I find the same point estimate for the cumulative light effect rounded to the third decimal (0.002) as I did for the main analysis in Table 2.
While it is important to conduct empirical analysis to assess whether a particular strategy is effective in reducing crime, one simultaneously needs to be aware of potential factors that may bias the findings. For either making LED conversions seem less (or more) successful in improving public safety then they are in reality.
But before discussing potential limitations of the analysis at hand, LED light conversions have additional benefits beyond increased public safety. For example, they result in cheaper energy consumption (Pagden et al., 2020), as well as they tend to reduce fear of crime (Painter, 1996), and can be preferred by individuals using the space (Kaplan & Chalfin, 2021). These benefits alone may justify the conversions from a cost-benefit perspective, irrespective whether they also decrease crime. That being said, one should not justify the expense of LED conversions based on purported crime reductions if the empirical evidence does not support that they actually reduce crime.
For specific description of the empirical design and its results focused on firearm crime reductions, we categorize potential reasoning why the expansion of lights did not result in crime reductions into two categories; empirical reasons and theoretical reasons.
For empirical reasons, it is possible that the design is underpowered to detect an effect. The original task force report estimated even in the best-case targeted scenario, that it would only reduce a total of 38 firearm crimes per year. Given the additional geographic spread of the light conversions, even examining firearm events over a long period may not be sufficient to detect an effect in this design (Wheeler & Ratcliffe, 2018). While examining the effect of street lighting on other crimes is at times mixed, e.g., lighting appears to increase thefts from motor-vehicles (Tompson, et al., 2022), it may be fruitful to examine the crime reduction effect for a wider variety of incidents beyond just firearm events. Although with this in mind one would still need to take into account the severity of those crimes in any subsequent cost-benefit analysis (Davies & Farrington, 2020; Wheeler & Reuter, 2021).
Additionally, there may be confounds we are unaware of that impacted the design. But that is partially the strength of the quasi-experimental difference in difference design – such a confound would have to only effect nighttime crimes or daytime crimes singularly. Additional potential modelling limitations of the empirical design (such as not accounting for temporal or spatial residual autocorrelation in the models), would only increase the variance of the estimates. As such they would not explain the null effects found.
Going back to theoretical reasons, while on average it appears expanding street lights reduces crimes (Welsh et al., 2022), converting those lights to LED does not have as rigorous an empirical baseline. Improving illumination in an area that already has it may not be as overall effective (Bonner & Stacey, 2021). But, one of the findings from those prior analyses suggests that improved lighting not only decreases crime at night, but also during the daytime (Welsh & Farrington, 2008; Xu et al., 2018). This is potentially due to areas encouraging more social engagement and having increased levels of informal social control. If that is the case here, the research design, using daytime crimes as the counter-factual, would fail to identify a treatment effect.
But I believe this is not a fatal flaw in the research design, Kaplan and Chalfin (2021) find in a survey find that poor lighting may not change routine activities use of nighttime space in reference to LED conversion. As such it seems unlikely (although admittedly possible) that adding LED lights improves collective efficacy of the area. One should collect data though to assert that relationship, not simply presume it occurs (Linning et al., 2022).
A second theoretical aspect (although additionally related to the research design), is that displacement may account for the null effects (Tompson et al., 2022). The nature of the grid cell design was intentional (as I did not believe I could do an effective micro level study). But ultimately if displacement is what occurred, the aggregated treatment effect is still net zero (Wheeler, 2015). While potentially interesting from a theoretical perspective, it results in similar cost-benefit analysis calculations in the end from the perspective of city planning.
In total, these potential confounds do not invalidate the original task force proposal, using mobile street lights in very specific targeted areas (for an extended period of time), are still likely as valid now as they were then. If one re-examines Figure 3, the highest density of the conversions in central Dallas does not correspond with the recommendations for focused treatment areas by the task force. While these locations may have been chosen for other reasons, it makes conducting analysis of the crime reducing effects of other areas more difficult.
While this observational study does not confirm the efficacy of LED expansion, it is possible with a more targeted intervention (Deryol & Payne, 2018; Lawson et al., 2018), Dallas (or other cities) will see better returns on their investment in terms of reduced crimes.
Ashby, M. P. (2020). Initial evidence on the relationship between the coronavirus pandemic and crime in the United States. Crime Science, 9(1)
Baker, A.C., Larcker, D.F., & Wang, C.C. (2022). How much should we trust staggered difference-in-differences estimates? Journal of Financial Economics, 144(2), 370-395.
Berk, R., & MacDonald, J. M. (2008). Overdispersion and Poisson regression. Journal of Quantitative Criminology, 24(3), 269-284.
Bonner, H., & Stacey, M. (2021). The effectiveness of increased lighting on crime reduction and calls for service in a single jurisdiction. Crime Prevention and Community Safety, 23(1), 39-55.
Chalfin, A., Kaplan, J., & LaForest, M. (2021). Street light outages, public safety and crime attraction. Journal of Quantitative Criminology, Online First.
Chalfin, A., Hansen, B., Lerner, J., & Parker, L. (2022). Reducing crime through environmental design: Evidence from a randomized experiment of street lighting in New York City. Journal of Quantitative Criminology, 38(1), 127-157.
Circo, G., McGarrell, E. F., Rogers, J. W., Krupa, J. M., & De Biasi, A. (2022). Assessing causal effects under treatment heterogeneity: an evaluation of a CCTV program in Detroit. Journal of Experimental Criminology, Online First.
Davies, M. W., & Farrington, D. P. (2020). An examination of the effects on crime of switching off street lighting. Criminology & Criminal Justice, 20(3), 339-357.
Deryol, R., & Payne, T. C. (2018). A method of identifying dark-time crime locations for street lighting purposes. Crime Prevention and Community Safety, 20(1), 47-62.
Doleac JL, Sanders NJ (2015) Under the cover of darkness: how ambient light influences criminal activity. Review of Economics and Statistics 97(5):1093–1103
Dominguez P, Asahi K (2017) Crime time: how ambient light affect criminal activity. Available at SSRN 2752629
Drake, G., Wheeler, A. P., Kim, D. Y., Phillips, S. W., & Mendolera, K. (2022). The impact of COVID-19 on the spatial distribution of shooting violence in Buffalo, NY. Journal of Experimental Criminology, Online First.
Hinkle, J. C., Weisburd, D., Famega, C., & Ready, J. (2013). The problem is not just sample size: The consequences of low base rates in policing experiments in smaller cities. Evaluation Review, 37(3-4), 213-238.
Kaplan, J., & Chalfin, A. (2021). Ambient lighting, use of outdoor spaces and perceptions of public safety: evidence from a survey experiment. Security Journal, Online First
Kim, Y. A., & Wo, J. (2021). A spatial and temporal examination of housing demolitions on crime in Los Angeles blocks. Journal of Crime and Justice, 44(4), 441-457.
Lawson, T., Rogerson, R., & Barnacle, M. (2018). A comparison between the cost effectiveness of CCTV and improved street lighting as a means of crime reduction. Computers, Environment and Urban Systems, 68, 17-25.
Linning, S. J., Olaghere, A., & Eck, J. E. (2022). Say NOPE to social disorganization criminology: the importance of creators in neighborhood social control. Crime Science, 11(1).
Long, J.S. (1997). Regression models for categorical and limited dependent variables. Sage.
Mayor’s Task Force on Safe Communities (2019). Obtained from https://childpovertyactionlab.org/safe-communities.
Pagden, M., Ngahane, K., & Amin, M. S. R. (2020). Changing the colour of night on urban streets-LED vs. part-night lighting system. Socio-Economic Planning Sciences, 69, 100692.
Painter, K. (1996). The influence of street lighting improvements on crime, fear and pedestrian street use, after dark. Landscape and Urban Planning, 35(2-3), 193-201.
Ratcliffe, J. H. (2002). Aoristic signatures and the spatio-temporal analysis of high volume crime patterns. Journal of Quantitative Criminology, 18(1), 23-43.
Tompson, L., Steinbach, R., Johnson, S. D., Teh, C. S., Perkins, C., Edwards, P., & Armstrong, B. (2022). Absence of Street Lighting May Prevent Vehicle Crime, but Spatial and Temporal Displacement Remains a Concern. Journal of Quantitative Criminology, Online First.
Welsh, B. C., & Farrington, D. P. (2008). Effects of improved street lighting on crime. Campbell Systematic Reviews, 4(1), 1-51.
Welsh, B. C., Farrington, D. P., & Douglas, S. (2022). The impact and policy relevance of street lighting for crime prevention: A systematic review based on a half‐century of evaluation research. Criminology & Public Policy, Online First
Wheeler, A.P. (2015). What we can learn from small units of analysis. Dissertation, SUNY Albany.
Wheeler, A.P., & Ratcliffe, J.H. (2018). A simple weighted displacement difference test to evaluate place based crime interventions. Crime Science, 7(1).
Wheeler, A. P., & Reuter, S. (2021). Redrawing hot spots of crime in Dallas, Texas. Police Quarterly, 24(2), 159-184.
Wheeler, A. P., & Steenbeek, W. (2021). Mapping the risk terrain for crime using machine learning. Journal of Quantitative Criminology, 37(2), 445-480.
Wheeler, A. P., Worden, R. E., & McLean, S. J. (2016). Replicating group-based trajectory models of crime at micro-places in Albany, NY. Journal of Quantitative Criminology, 32(4), 589-612.
Wilson, D.B. (2022). The relative incident rate ratio effect size for count-based impact evaluations: When an odds ratio is not an odds ratio. Journal of Quantitative Criminology, 38(2), 323-341.
Wood, D. (1981). In Defense of Indefensible Space. From Environmental Criminology, P 77-95, Paul J Brantingham and Patricia L Brantingham, ed.
Xu, Y., Fu, C., Kennedy, E., Jiang, S., & Owusu-Agyemang, S. (2018). The impact of street lights on spatial-temporal patterns of crime in Detroit, Michigan. Cities, 79, 45-52.
Shared with Dropbox