Skip to main content
SearchLoginLogin or Signup

Automation and Artificial Intelligence in Police Body-Worn Cameras: Experimental Evidence of Impact on Perceptions of Fairness Among Officers

Published onFeb 10, 2024
Automation and Artificial Intelligence in Police Body-Worn Cameras: Experimental Evidence of Impact on Perceptions of Fairness Among Officers
·

Abstract

Objectives: Explore officers' perceptions of the fairness of monitoring with systematic variations in activation (manual/automatic) and auditing (on-demand/supervisor random/artificial intelligence) policy regimes for body-worn cameras (BWCs).

Methods: This study uses a survey experiment with a national probability sample of officers wearing BWCs (n=258) to assess the perceived fairness of BWC monitoring under varying activation and audit policies. Participants were randomly assigned one of six vignettes, each incorporating one of two BWC activation policies and one of three BWC footage review policies. The analysis involves a 2x3 experimental design to assess main and interaction effects.

Results: Automatic BWC activation and artificial intelligence enabled auditing of footage cause declines in perceived fairness of monitoring. Officers perceive the most unfairness in monitoring when they lack control over the initiation of recording and when the resultant footage is outside of their supervisors' immediate control.

Conclusions: The findings underscore potential adverse effects on officers' perceptions of monitoring fairness under varying BWC policy conditions. As this technology gains traction, the potential impact of officers’ concerns on program implementation and fidelity should be considered.

Introduction

Body-worn cameras diffused throughout US policing remarkably quickly (Nix et al., 2020; White & Malm, 2020). As BWCs did not prove reliable in attaining goals of reduced police use-of-force and public complaints (Lum et al., 2019), scholars began to pay attention to the activation of the cameras, and review of the resulting footage, as a possible interfering factor. Through the lens of deterrence, BWCs are predicted to dissuade officers from engaging in bad behavior, as “the degree of deterrence increases, officers are less likely to use force” (Ariel et al., 2017, p. 2). However, this raises two potential mechanisms of interference. First, if officers intentionally avoid BWC recording by failing to activate the camera, then the deterrent effect is not achieved. Failure to activate the cameras, intentionally or not, limits our ability to capture potential benefits of the technology (Adams et al., 2021; Lawrence et al., 2019). Some believe the most straight-forward solution to the problems of non-activation is to automate the process. By triggering activation according to preset times, locations, and behaviors, the burden of activation is lifted from the individual officer (Fan, 2017).

A second possibility for BWCs failing to attain their predicted effects is that the chance of bad behavior being detected is so small that officers are able to maintain their normal activity. In most cases, detection in this regard requires that a public complaint is lodged, though in some agencies front-line supervisors are encouraged to proactively review footage. The realities of overwhelming amounts of BWC-generated footage make this possibility more likely. In order to review footage, the agency must store the footage, as well as devote considerable resources to reviewing it. In short, an hour of video may require at least an hour of a human reviewer’s time to comprehensively audit. In a country creating petabytes of video footage, therefore, no human-only auditing solution is tenable if we set out to comprehensively review the officer activity documented across all BWC footage.

This study tests the effects of BWC activation and review policies on officer perceptions of monitoring fairness by using full factorial vignette survey data from a stratified random sample of US law enforcement officers. Participants were randomly assigned one of six vignettes (2x3 design) which contained a common context, one of two BWC activation policies, and one of three BWC footage review policies. OLS regression models were estimated to test the influence and effect size estimates for both experimental factors and their interaction. The results demonstrate the potential negative effects that varying policy conditions can have on perceived fairness of monitoring.

Literature Review

Emerging technologies in the BWC context are automatic activation of the cameras (Axon Signal Sidearm, 2021; White & Malm, 2020), and the automated auditing and review of BWC footage with artificial intelligence solutions (Axon AI, 2021; Graham et al., 2024; Makin et al., 2020; Shastry, 2022). To date no published research has investigated how these changes will be perceived by or impact the officers who wear the cameras. This study places BWCs in the context of a large literature concerned with the electronically monitored workplace, in order to test hypotheses that predict BWC activation and AI review of BWC footage will affect how officers perceive the fairness of monitoring.

Alder conceives of employee reactions to electronic workplace monitoring as resulting from organizational culture (Alder, 2001; Alder et al., 2007; Alder & Ambrose, 2005a). Alder suggests that organizations can improve employee reactions to increased monitoring by including them in the design of the monitoring system, and by restricting the monitoring to specific job functions. Both of these elements are largely missing from the decision to adopt BWCs in the US. That is, line-officers are charged with wearing the cameras but had “little or no input on the decision to adopt” the technology (Adams & Mastracci, 2019a, p. 3).

Beyond the lack of control in deciding to wear a camera, the decision to activate the camera is set by policy at the administrative level, and there is a current of distrust between line-officers and administrative levels that has been noted across policing scholarship (Bittner, 1984; Engel, 2000; Nix & Wolfe, 2017). Further, “job performance” is difficult to define in the police BWC context. The rapid adoption of the cameras in the US has been motivated by a number of factors, including hoped-for reductions in use-of-force and external complaints, as well as less tangible benefits such as more transparency, better relationships with the community, and public demand (Christodoulou et al., 2019; Lum, Stoltz, Koper, & Scherer, 2019; Smith, 2019).

The costs associated with review of BWC footage are high and pose a tempting target for technological fixes. Axon, Inc. is the largest manufacturer of BWCs in the world, and has been actively developing solutions within their “Evidence.com” platform (Axon Evidence, 2021), while other private entities also develop their unique approach (Shastry, 2022). These technologies promise to employ artificial intelligence to review BWC footage automatically, and alert supervisors of any concerns (Axon AI, 2021; Shastry, 2022). And while civil libertarians have raised concerns about how artificial intelligence in the BWC market will impact the rights of civilians (Porter & Ogden, 2023; Yakowicz, 2017), to date there has been no detectable discussion of how these algorithms will impact the work lives of officers.

Control of Activation

Reliance on manual activation of BWCs has caused some to worry that officers are intentionally not recording their public interactions, in order to undermine monitoring. Even if the cameras are capable of substantively altering officer bad behavior, if officers deliberately fail to activate their BWCs in order to avoid the surveillant gaze of agencies and communities, then the harmful behavior will not be deterred. Not all failures to activate are unintentional, but it is often difficult for the public, agencies, and courts to distinguish between technological failure and perceived officer intent to not record (Fan, 2017).

As agencies begin moving towards automatic-activation policies (Axon Signal Sidearm, 2021), it is possible officers will appreciate off-loading the physical act of activating the BWC (Fan, 2017). Officers are highly attentive to how quickly a situation can turn dangerous (Sierra-Arévalo, 2021; Sierra-Arévalo et al., 2023). In these dynamic, fluid, and chaotic situations (Graham v. Connor, 1989) officers can feel like the added burden of worrying about the camera and its status contributes to less physical safety (Newell, 2021).

Beyond the possible costs in physical reaction time, some research suggests a cognitive cost incurred from offloading memory storage to BWCs. Blaskovits and Bennell (2019) review how cameras can both benefit and harm officers’ accuracy in event recall. They review literature from both outside and inside policing research, and define the sources of the detrimental effect as cognitive offloading, retrieval-induced forgetting, and misinformation-type effects.

Considerations of activation type should be considered a form of control within the electronic performance monitoring literature (McNall & Stanton, 2011; Ravid et al., 2020; Stanton & Barnes-Farrell, 1996), and is a critical feature within organizational justice, specifically procedural justice, as well (Greenberg, 1987; Thibaut & Walker, 1975). Two decades of evidence establish a clear positive relationship between control and perceptions of fairness. Enhanced employee control over monitoring is associated with greater perceived fairness (Behrend et al., 2019; Douthitt & Aiello, 2001; McNall & Stanton, 2011; Stanton, 2000; Zweig & Webster, 2002), and an early meta-analysis found consistent evidence for that relationship as well (Colquitt et al., 2001). Based on this long line of findings, we should expect that manual activation is results in greater perceived fairness compared to automatic activation.

H1: Individuals without control over the monitoring device will report lower monitoring fairness compared to individuals with control over the monitoring device.

Feedback Source

It is difficult to develop solid hypotheses given the lack of literature in this area, but the typology and framework proposed by Ravid et al. (2020) provides some guidance. The authors consider this area a largely unexplored area of research that is in need of more empirical evidence (pp. 113-114): “It may not be true that individuals receive and perceive highly synchronous automatized feedback in the same manner as when delivered by a peer or supervisor.” This study provides an experimental test of the proposition, as a vignette is experimentally varied to present the respondent with three nominal categories of feedback delivery. In the first, the baseline mechanism is that BWC footage is reviewed by a direct supervisor only when there is an external complaint or significant use-of-force. This is the method by which the vast majority of agencies are currently reviewing BWC footage (White & Malm, 2020). In the second and third conditions, a random selection of BWC videos is reviewed by either a direct supervisor or artificial intelligence algorithm. The vignette holds random constant between these two conditions, while varying who is doing the review. As feedback moves from relatively known conditions (complaint, use-of-force) to random selection, I expect officers to perceive less fairness (synchronicity).

The literature base for understanding supervisor versus computer-mediated feedback within electronic performance monitoring is threadbare. However, in a presentation to the National Academy of Management, Alder and Ambrose (2000) suggest that fairness of feedback in electronic performance monitoring systems should be affected by whether the source of the feedback is a supervisor or computer. As noted in Ravid et al. (2020), this is simply an area where we must theorize without solid empirical grounding. Therefore, I expect that review of BWC footage by a supervisor will be perceived as fairer than that done by artificial intelligence.

H2: Footage review upon demand will be judged more fair than random review by a supervisor, and both will be judged as more fair than automated, random selection and review.

Finally, little is known about the interactive effects of variation across multiple modes of electronic performance monitoring, and “there is a need for greater explorations of the interactive effects of EPM characteristics on individual reactions” (Ravid et al., 2020, p. 121). Therefore, the following non-directional hypothesis is constructed.

H3: The interaction effect of variation in activation and review policies will have a statistically significant effect on perceptions of fairness (non-directional).

Sample, Methods & Procedure

Data

This study uses data from electronic self-administered surveys, distributed to law enforcement officers in the US. The survey was administered through Qualtrics over fourteen days beginning on May 20, 2020. I conducted an a priori power analysis (Cohen, 1988) using the G*Power 3.1 software (Faul et al., 2007) to compute the required sample size for this experiment, given the following parameters: six groups (2x3 full factorial), capable of detecting a medium effect size (Cohen’s d = 0.25), with a standard probability of error (α=0.05), and a desired power level of at least 0.8. With those parameters, the required sample size to detect a main effect for Activation Policy (two levels, numerator df = 1) is n=128; the required sample size to detect a main effect for Review Policy (three levels, numerator df = 2) is n=158; and the required sample size to detect an interaction effect (Activation*Review, numerator df = 2) is n=158. The sample sizes attained in this survey surpassed the sample size required to achieve a reasonably powered experiment. At the close of the survey, the sample size prescribed by the a priori power analysis had been achieved.

Sample

The survey was distributed between May 20 and June 4, 2020, to a random sample of US based law enforcement agencies, drawn from four size strata. The study intentionally oversampled from the largest strata of US law enforcement agencies (100+ sworn officers). Stratification by agency size is an important design consideration because of findings that while policy positions of agencies in regards to BWCs are largely consistent, where odd policy exists it is typically “extra small agencies” who hold heterodox positions (White et al., 2020). BWC diffusion differs by agency size and type as well, with larger agencies less likely to equip every officer, but more likely to have implemented a BWC program generally.

The survey was held open for two weeks and collected responses from 515 respondents. A total of 32 (6.2%) of the observations were removed for excessive non-response and persistently missing values (> 80%). Overall, this resulted in an effective sample of 483 respondents. Of the effective sample, 258 were officers who are equipped with BWCs, forming the analytic sample. I intentionally restricted the experiment to officers already wearing BWCs due to the known gap between the perceptions of officers who have worn BWCs and those who have not (Gaub et al., 2016). As the vignette design asks respondents to address a hypothetical policy environment (as the technology was not yet available at the time of the survey), asking non-BWC officers to add a second level of abstraction was unlikely to provide useful information. BWC-equipped officers, who on average are relatively positive regarding the cameras once they have experience with them, are more likely to be attuned to the policy and fairness ramifications of moving between various conditions of BWC activation and footage review.

The sample was 87.5% male and 12.5% female. The sample identified as predominantly White (89.4%) and heterosexual (92.6%). 29% of the sample had worn a BWC for over four years, 17.1% between three and four years, 22.1% between two and three years, 19% between one and two years, and 12.8% under one year. The oversampling decision in the sampling design can be seen in the agency demographics, with 33% of the sample coming from the large agency strata (100+). The descriptive statistics for the analytics sample are reported in Table 1. Compared to the national law enforcement population at the state and local level (Hyland & Davis, 2019), the sample is slightly less racially diverse, but representative of the reported proportions for sex.

Table 1: Sample Descriptive Statistics (n = 258)

N

Column %

Mean

Std Dev

Min

Max

BWC Experience

Formal Education

Sex

Race

Sexual Orientation

Union or Labor Membership

Agency Size

Years in Law Enforcement

256

0.000

11.395

9.418

0.0

46.0

Age

217

0.000

39.318

10.156

21.0

70.0

Less than one year

33

0.128

.

.

.

.

Between one and two years

49

0.190

.

.

.

.

Between two and three years

57

0.221

.

.

.

.

Between three and four years

44

0.171

.

.

.

.

More than four years

75

0.291

.

.

.

.

High School Graduate

61

0.281

.

.

.

.

Associate Degree

58

0.267

.

.

.

.

Bachelor's Degree

76

0.350

.

.

.

.

Master's Degree

19

0.088

.

.

.

.

Doctoral or Professional (JD, MD)

3

0.014

.

.

.

.

Male

189

0.875

.

.

.

.

Female

27

0.125

.

.

.

.

White

194

0.894

.

.

.

.

Non-white

23

0.106

.

.

.

.

Heterosexual (straight)

200

0.926

.

.

.

.

Homosexual (gay)

8

0.037

.

.

.

.

Bisexual

3

0.014

.

.

.

.

Prefer not to say

5

0.023

.

.

.

.

No

55

0.252

.

.

.

.

Yes

163

0.748

.

.

.

.

Less than 25

32

0.147

.

.

.

.

25-49

54

0.248

.

.

.

.

50-100

60

0.275

.

.

.

.

More than 100

72

0.330

.

.

.

.

Design

The design uses a between-subject randomized experimental design to investigate whether differences in BWC Activation Policy and Review Policy influence how officers perceive the fairness of BWC monitoring. The choice of whether to use a between- or within-subjects design is a key decision for any vignette factorial design (Aguinis & Bradley, 2014, p. 360). In a within-subjects design, respondents are rating each of the possible vignettes. For example, these designs are useful for understanding how the public makes decisions about who deserves a certain immigration status, or how public administrators assign social benefits within their discretion (Møller, 2016; Pedersen et al., 2018). In a between-subjects design, respondents are exposed to just one random vignette comprised of a random selection of factor levels. These might be useful in determining the effect of officer race on public judgments about the fairness of police (Riccucci et al., 2018), or the influence of officer gender on public judgements about a police agency’s fairness (Riccucci et al., 2014). While both approaches have value, the between-subject design minimizes respondent fatigue and carelessness (Stolte, 1994), a key consideration when deploying surveys in the policing context where response rates are already low (Nix et al., 2019).

Experimental vignettes are an especially useful method when it is difficult to manipulate the independent variables in an ethical manner, or when manipulation of the factors is simply not likely (Aguinis & Bradley, 2014). This is the case here, where policy around activation and footage review are difficult to manipulate within a single agency, let alone in a nationally representative random sample of agencies. Though the study employs a probability sample, vignette experiments can achieve results consistent with random samples through crowdsourced samples, such as those available through Amazon MTurk (Weinberg et al., 2014), and Coppock (2018) replicates experimental work more broadly in the crowdsourced environment.

Restricting the number of vignettes to a between-subjects design also makes the scenario more realistic for law enforcement officer, an important consideration in vignette construction (Aguinis & Bradley, 2014). This realism reflects the lived experience of officers, who are not offered a choice of monitoring policies in the real world, but instead directed to review a policy that has already been decided. BWC researchers have acknowledged that line officers should “play a role in the policy development process and should provide input on key issues such as activation, officer discretion to deactivate, and supervisory review of BWC footage” (White & Malm, 2020, p. 124), and that failure to include these officers often results in officers perceiving “lower levels of organizational justice and be more likely to resist the technology” (White & Malm, 2020, p. 21). Despite this, even in the rare case where some line officers have direct input into policy selection, for the vast majority, they will simply be expected to adhere to the policy. To reflect that reality, and capture how officers then perceive the fairness of the resulting monitoring scheme, a between-subjects design is used.

The experimental design allows analysis of the influence of each factor, as well as the interaction of those factors, on the outcome variable. This type of design is categorized as a “policy capturing” study by scholars documenting experimental vignette best practices for organizational studies (Aguinis & Bradley, 2014). That is, it focuses on respondents’ explicit responses to hypothetical scenarios, to assess the causal relationship (if any) between the effects of the independent variables on implicit judgements. Experimental vignettes have been used to study leadership, executive behavior, organizational citizenship behavior, and ethics (see Aguinis & Bradley, 2014 for a modern review of relevant scholarship). Most relevant for this study, this method has been used to explore causal antecedents of organizational unfairness violations (Skarlicki & Turner, 2014), and how police actions affect citizen perceptions of procedural versus distributive justice (Reisig et al., 2018).

The survey experiment design uses vignettes, or fictional descriptions, a common method used widely across disciplines (Jasso, 2006). Phillips (2020b) has recently used the method to investigate the formation of suspicion among officers, as well as officer’s view of de-policing in a separate effort (2020a). Harrits (2019) recommends vignette studies specifically for researching street-level bureaucracy (Lipsky, 1983), such as law enforcement officers. Designing vignettes that reflect the world encountered by street-level bureaucrats allows the researcher to “get close to studying behavior and meaning-making” (Harrits, 2019, p. 392). Stritch and Pedersen (2019) produce a design most similar to that used here, testing the effects of the locus of decision making on employee perceptions of fairness in public organizations through the lens of organizational justice principles developed by Leventhal (1976, 1980).

Measures

This study makes use of a 2x3 experimental design, with each of the two factors interacted with each level of the other. The dependent variable captures a respondent’s perception of monitoring fairness, and is drawn from other studies interested in workplace monitoring practices and policies (Alder & Ambrose, 2005a, 2005b; Alge, 2001; McNall & Stanton, 2011). The independent variables capture the actual policy variance either in use (White & Malm, 2020), or in active development (Axon AI, 2021; Makin et al., 2020; Shastry, 2022). The study uses variation in how BWC footage is reviewed or audited, and tests for effects of that variation on respondent perceptions of fairness.

This design of the independent variables offers a theoretical foothold to test how policy variance influences perceptions of monitoring fairness, because there are three ways in which BWC footage is reviewed – reactively, proactively (supervisor), and proactively (artificial intelligence). These first two ways form the current policy for the vast majority of US agencies (White & Malm, 2020), but there is significant effort from both BWC manufacturers and academics to develop complex models of social interaction involving police (Makin et al., 2020), which would allow for computer-mediated feedback to officers. This is an active area of technology development and implementation, with agencies in the US already experimenting with AI review of BWC footage (Shastry, 2022).

Dependent Variable

The dependent variable in this study is Monitoring Fairness (n = 258, Mean = 4.41, SD = 1.79, range: [1, 7]. This is based on a single survey item adopted from Alder and Ambrose (2005a, 2005b), who propose a unified framework of fairness judgements associated with electronic monitoring. The question prompt reads: “Imagine working for the Palgrave Police Department above. How would you feel about the following? ‘I think the body-worn camera procedures used by this agency are fair.’” The dependent variable is a seven-point Likert scale, from Strongly Disagree (1) to Strongly Agree (7).

This approach aligns with that of Stritch and Pedersen (2019) in selecting a single item measure of fairness, which is appropriate to both studies given the level of abstraction necessary for the vignette scenario. Normative judgements of “fairness,” like the dependent variable in this study, are a very common focus of experimental survey research (McNall & Stanton, 2011; Riccucci et al., 2014, 2018; Shaughnessy et al., 2016; Skarlicki & Turner, 2014; Stritch & Pedersen, 2019; Yost et al., 2019). Normative judgements of fairness are generally among the largest category of these types of studies.1 Evaluations of fairness are a subset of these normative judgements, and Jasso’s use of factorial vignettes to investigate fairness of earnings (Jasso & Webster, 1997) is considered formative in the method.

My decision to use a single item is well supported within literature measuring judgements of fairness, but comes with tradeoffs compared to multi-dimensional measurement structures, which generally have psychometric properties that generate less measurement error. Scholars have also argued that the multi-dimensional concept of organizational justice is perhaps too complex a rendering of the process by which employees come to judgements (van den Bos et al., 1998). The process can become especially challenging in highly complex organizational environments, including policing. There are tremendous vulnerabilities associated with the exchange of physical safety from the employee on behalf of perceived pecuniary and professional benefits from the organization (Wolfe, 2021). Instead, the fairness heuristic theory argues that employees resolve the complexity of making judgements about the trustworthiness of supervisors and organizations by using a mental shortcut akin to an immediate judgment of “fairness.” Perceptions of fairness matter a great deal to public employees broadly (Harrington & Lee, 2015). With the great uncertainty present in policing, fairness judgements operate as a proxy for more complex judgements of trustworthiness and justice, while also serving as a coping mechanism for officers (Wolfe, 2021).

In its simplest form, organizational justice is the expectation that an employee will be treated with fairness and respect (Greenberg & Colquitt, 2013). Applying the concept in the criminal justice context, Wolfe (2021, p. 2) argues that the “primary metric employees use to temper feelings of uncertainty or vulnerability are evaluations of the level of justice in their supervisor’s actions and decisions…fair supervisor behavior is an indication that they have their employees best interests in mind.” In its more complex academic formulation (see Colquitt, 2001 for an measurement and statistical validation), organizational justice is the attempt to “capture employees’ views of the procedural, distributive, informational, and interpersonal justice of their supervisors” (Wolfe, 2021, p. 1). In the current moment, with calls for policing reform from both within and without, the relationship between organizational justice and a variety of important outcomes lends the concept even more importance (Wolfe & Lawson, 2020). Rapid adoption of artificial intelligence technology in the workspace may negatively impact workers generally (Nazareno & Schiff, 2021), by impacting their workplace freedom, sense of meaning, cognitive load, external monitoring, and insecurity. Some of these channels of influence have been documented previously in the BWC context, with the cameras associated with increased burnout (Adams & Mastracci, 2019b) and greater perceived intensity of monitoring by internal and external audiences (Adams & Mastracci, 2019a). Given that backdrop, attending to officers’ concerns about automation and artificial intelligence in the police workplace is a critical evaluative step.

Independent Variables

A review of the electronic performance monitoring literature identified several monitoring characteristics to serve as predictors of perceived fairness. The two independent variables are factors that structure the experimental conditions of interest. The factors and their conditions are fully reported in Table 2. The Activation Policy variable features two conditions that vary the amount of control over monitoring the officer would have. In the manual activation condition, which remains the current default in US policing, officers physically turn the camera on and off as needed. In the automatic activation condition, activation of the camera is done without manual intervention, though officers are able to turn off the camera when policy allows. This automatic condition represents a trend in US BWC deployment, as vendors continue to offer new methods of automatic activation (White & Malm, 2020, p. 109).

The Review Policy features three conditions that vary when BWC footage is reviewed. Two of the conditions (complaint and use-of-force, and supervisor random) vary the types of auditing recommended by researchers (White & Malm, 2020, p. 110), while the third involving review by artificial intelligence algorithms is a rapidly-changing area under active development (Axon AI, 2021; Axon, Inc., 2020; Makin et al., 2020), and already seeing limited use in real-world policing contexts (Shastry, 2022). In the complaint and use-of-force condition, which represents the majority of actual BWC auditing policies in the US, certain categories of calls are automatically flagged for review, such as those involving a use-of-force, Alder and Ambrose (2005b, p. 161) find that “supervisor mediated feedback was associated with higher levels of monitoring fairness than was computer mediated feedback.”

This is a double-blinded survey experiment. The survey respondents in this study were not aware of the experimental manipulation in this study, nor which treatment group to which they have been randomly assigned, and I was not aware of which treatment group a respondent would be assigned to. Treatment assignment was blinded through Qualtrics administration. Qualtrics is a web-based survey administration suite that allows for randomization of survey respondents within the survey itself. The respondent pool was randomly distributed within the treatment framework, and each respondent was supplied with one of six information treatments regarding the Activation Policy (two levels), and the Review Policy (three levels). The text of each condition in the two factors is given in Table 2.

Comparing across treatment groups, the randomization procedure was successful, and respondents were randomly assigned such that the assigned groups do not differ in any systematic way. Appendix Tables A1 and A2 are balance tables and report the effective distribution of treatment factors (two levels of Activation Policy and three levels of Review Policy) across respondent characteristics.

Experimental Vignette Construction

Each respondent was randomly assigned a treatment constructed of the full factorial of Activation Policy x Review Policy, following a vignette stem that was consistent across treatments. A vignette design ought to reflect the reasonable experiences of the respondents, and when researching sensitive areas to simultaneously ensure the vignette intentionally creates some space between the respondent and the “vignette actor” being portrayed (Bradbury-Jones et al., 2014). To those ends, the vignette stem points the respondent to imagine working as a patrol officer, in a fictional police department. The patrol officer role was selected intentionally to reflect the lived experience of law enforcement officers, who as a rule all begin their careers as a patrol officer. This helps ensure that even a late-career officer, who may not currently be working as a front-line patrol officer, still has a reasonable life experience to connect to the vignette.

The fictive police department name was selected because it does not exist, and therefore avoids naming a department that the random sample may have overlapped, or any department that a respondent would have pre-formed judgements about. The vignette stem also does not vary whether the respondent (in their imaginary role) would wear a camera – all patrol officers in the fictional department do. This is a realistic policy scenario, given that in departments that do not equip all officers, it is common to prioritize patrol officers with BWCs. Finally, the vignette stem gives a common BWC policy practice, that all encounters with the public will be recorded, save for certain situations such as those involving a juvenile. Though there is no single consistent policy framework in the US, the selected one is a common one (White & Malm, 2020).

The vignette stem reads: “Imagine you work for Palgrave Police Department as a patrol officer. The department has decided to equip all patrol officers with a body-worn camera. All interactions with the public will be recorded by policy, except in certain situations such as interviews with juvenile victims.” Following that stem, the Qualtrics platform randomly assigns a level of Activation Policy and Review Policy, and displays that information to the respondent.

Table 2: Factor Conditions Description

Factor Level Name

Factor Level Text

Activation 1 (Manual)

Officers will activate the cameras manually by pressing the “record” button on the body-worn camera, as required by policy. When appropriate according to the policy, officers are allowed to stop the recording.

Activation 2 (Automatic)

The body-worn cameras will record automatically, by turning on when an officer, or any officer within 100 yards, leaves their vehicle (with a door sensor), draws their weapon (with a holster sensor), or turns on their vehicle’s emergency lights or siren. When appropriate according to the policy, officers are allowed to stop the recording.

Review 1 (Complaint & Use-of-Force)

Body camera footage will be reviewed by a supervisor when needed, such as public complaints and in use-of-force incidents.

Review 2 (Supervisor Random)

Supervisors will review five randomly selected body-camera recordings every month for every officer, in order to monitor for policy violations (such as unprofessional conduct and unreported use-of-force). Public complaints and reported use-of-force will also be reviewed by a supervisor as needed.

Review 3 (Artificial Intelligence Random)

An artificial intelligence computer program will review five randomly selected body-camera recordings every month for every officer, in order to monitor for policy violations (such as unprofessional conduct and unreported use-of-force). Footage that is flagged by the program will be forwarded to the officer’s supervisor for review. Public complaints and reported use-of-force will also be reviewed by a supervisor as needed.

Results

Participants’ perceptions of monitoring fairness were first modeled with a 2x3 ANOVA, with Activation Policy and Review Policy as between-subjects factors. ANOVA was conducted using the afex package in R (R Core Team, 2020; Singmann et al., 2020). Comparisons and contrasts were completed with the “emmeans” package in R (Lenth, 2020). Effect size interpretation below is guided by Field’s (2013) classifications for ANOVA, using partial omega square statistics: small (.01), medium (.06), and large (.14). I report pairwise effect size estimates in Table 6, and while different sources recommend different measures for these estimates, they are consistent to one another in these analyses. The Type III sums-of-squares was computed for the ANOVA. Type III models have the advantage over Type I and Type II because in the presence of an interaction between predictor variables, the main effect of each variable is still meaningful. Therefore, in this analysis, the effect of Activation Policy is evaluated after the effects of both Review Policy and Activation Policy x Review Policy. The same pattern repeats for each of the main effects and the interaction effect.

Analysis then proceeded with an ordinary least-squares regression, testing the equation:

Fair=β0(ReviewPolicy)+β1(ActivationPolicy)+β2(ReviewPolicy×ActivationPolicy)+β3()+ϵFair = \beta_{0}(ReviewPolicy) + \beta_{1}(ActivationPolicy) + \beta_{2}(ReviewPolicy \times ActivationPolicy) + \beta_{3}() + \epsilon

Main and Interaction Effects

The results of the analysis are reported in Table 4. All three tested hypotheses find support, as both main and interaction effects have a statistically significant negative relationship with perceptions of fairness. The main effect of Activation Policy (two levels) is significant (F(1, 226) = 21.04, p < .001) and can be considered as having a medium effect size, reported as partial omega squared (η²p = 0.085). The main effect of Review Policy (three levels) is significant (F(2, 226) = 3.49, p=0.032) and can be considered as small (η²p = 0.030). The interaction between Review Policy and Activation Policy is significant (F(2, 226) = 3.48, p=0.033) and can be considered as small (η²p = 0.030). Variance in each group is the same (Bartlett test, p=0.542), and error variance appears to be homoscedastic (p= 0.084).

Table 4: Analysis of Variance, Activation and Review on Perceptions of Fairness

Response: Perceived Fairness

DF

Sum of Squares

Mean Sq

F Value

Pr(>F)

Activation Policy

1

62.07

62.07

22.01

<.0001*

BWC Review Policy

2

19.12

9.56

3.39

0.0354*

BWC Review Policy*Activation Policy

2

19.62

9.81

3.48

0.0325*

Residuals

226

637.46

2.82

A linear model was fitted (estimated using OLS) to predict Perceived Fairness with Activation Policy and Review Policy. The model explains a statistically significant and moderate proportion of variance (R2 = 0.14, F(5, 226) = 7.15, p < .001, adj. R2 = 0.12). Standardized parameters were obtained by fitting the model on a standardized version of the dataset. The model's intercept, corresponding to Activation Policy = Manual Activation and Review Policy = Complaint and Use of Force, is at 5.43 (95% CI [4.93, 5.93], t(226) = 21.45, p < .001). Within this model, the effect of Activation Policy [Automatic Activation] is statistically significant and negative (beta = -1.33, 95% CI [-2.05, -0.61], t(226) = -3.63, p < .001; Std. beta = -0.74, 95% CI [-1.15, -0.34]). The effect of Review Policy [Supervisor Random] is statistically significant and negative (beta = -1.05, 95% CI [-1.81, -0.29], t(226) = -2.74, p = 0.007; Std. beta = -0.59, 95% CI [-1.01, -0.16]). The effect of Review Policy [AI Random] is statistically non-significant and negative (beta = -0.59, 95% CI [-1.32, 0.14], t(226) = -1.59, p = 0.114; Std. beta = -0.33, 95% CI [-0.74, 0.08]). The interaction effect of Review Policy [Supervisor Random] on Activation Policy [Automatic Activation] is statistically significant and positive (beta = 1.14, 95% CI [0.09, 2.20], t(226) = 2.14, p = 0.034; Std. beta = 0.64, 95% CI [0.05, 1.23]). The interaction effect of Review Policy [AI Random] on Activation Policy [Automatic Activation] is statistically non-significant and negative (beta = -0.20, 95% CI [-1.26, 0.86], t(226) = -0.36, p = 0.716; Std. beta = -0.11, 95% CI [-0.70, 0.48]). Full model results for both are reported in Table 5.

Table 5: OLS Regression Results

Model 1

Main

Model 2

Interaction

Activation Policy (Automatic)

-1.03 ***

-1.33 ***

-0.22

-0.37

Review Policy (Supervisor Random)

-0.45

-1.05 **

-0.27

-0.38

Review Policy (AI Random)

-0.68 *

-0.59

-0.27

-0.37

Activation (Automatic) * Review (Supervisor Random)

1.14 *

-0.54

Activation (Automatic) * Review (AI Random)

-0.2

-0.54

(Intercept)

5.29 ***

5.43 ***

-0.21

-0.25

n

232

232

R2

0.11

0.14

Figure 1 displays both the main and interaction effects of the model. While the main effects are clear, the interaction effects produced between activation and review policies are not. In every interaction with varying review policies, moving from the manual to automatic activation scheme results in statistically significantly lower average perceived fairness. However, activation and review policies have a statistically significant interaction, meaning the effect of review varies over different values of activation. This is most clear in the AI random review condition, which does not significantly vary from traditional review when there is a manual activation policy in place. However, AI review paired with automatic activation produces the largest negative and statistically significant effect in the model. Interestingly, while Supervisor random review produced a statistically significant negative effect on fairness in the manual activation scheme, there is not a significant effect moving to the automatic activation scheme. Reporting for nineteen pairwise effects can be found in Appendix Table A3.

Figure 1: Main and Interaction Effects of Activation and Review Conditions on Respondent’s Perceived Fairness of Monitoring

Discussion

This study's experimental manipulation of BWC Activation and Review policies provides critical insights into how these policies influence officers' perceptions of monitoring fairness. Echoing Alge and Hansen's (2014) assertion, our findings affirm that the implications of electronic monitoring extend beyond its mere implementation to the nuances of its application. The aversiveness associated with electronic monitoring, as we have shown, is significantly influenced by the manner of its deployment rather than its existence per se.

Ravid et al. (2020) have underscored the necessity for deeper exploration into the interactive effects of electronic performance monitoring characteristics on individual reactions. Our research directly responds to this call, bridging gaps in understanding the complex dynamics of electronic performance monitoring. We reveal that control mechanisms and feedback sources are pivotal in shaping fairness perceptions, with control over monitoring processes emerging as a particularly critical factor. Automatic activation policies, when evaluated independently, are perceived as diminishing fairness, underscoring the significant role of procedural justice within the framework of electronic performance monitoring (McNall & Stanton, 2011, p. 20; Stanton & Barnes-Farrell, 1996; Thibaut & Walker, 1975), and the narrower context of organizational justice literature focused on criminal justice employees (Wolfe & Lawson, 2020). This integration of organizational justice theories with the current study emphasizes the profound influence of participatory control on fairness perceptions in the context of public sector employment, situating BWCs squarely within the domain of electronic performance monitoring.

The employment of algorithms for performance review introduces complex ethical and practical considerations. Edwards et al. (2018) have highlighted the potential risks associated with this technology in terms of employee privacy. The findings here corroborate these concerns, suggesting that algorithmic performance review is perceived as manifestly unfair, if not outright detrimental. This perception is particularly salient in law enforcement, where BWCs could be perceived as instruments for undue surveillance rather than tools for accountability and transparency. Moreover, the interactive effects observed in our study emphasize the importance of gradual and considered policy evolution. Abrupt transitions from traditional to fully automated BWC systems are likely to provoke significant resistance among officers. Such findings underscore the necessity for policy makers and law enforcement executives to carefully consider the human and organizational impacts of technology adoption in law enforcement settings.

By extending the discourse on electronic performance monitoring to law enforcement officers, this research enriches the existing body of knowledge with new empirical evidence. The findings challenge the assumption that automated feedback, such as that from artificial intelligence, is equivalent in impact to that derived from human supervisors. Instead, the study demonstrates that AI-mediated feedback is perceived more negatively, highlighting the nuanced interplay between technology, policy, and human perception within the sphere of organizational justice.

In sum, this study not only reaffirms the relevance of organizational justice theories to the discussion of electronic performance monitoring in the public sector but also introduces novel insights into the specific challenges and opportunities presented by the integration of advanced technologies like BWCs in law enforcement. As the landscape of workplace monitoring evolves, our findings offer a foundation for future research and policy development aimed at balancing technological advancement with fairness, privacy, and ethical considerations in policing.

Limitations and Future Research

The experimental findings presented in this manuscript offer significant contributions to various literatures, particularly in understanding the internal validity of body-worn camera (BWC) activation modes and review types on officers' perceptions of fairness. However, these results are not without their limitations. The primary limitation stems from the research design—an experimental survey—which excels in exploring attitudes and preferences but is less effective in predicting actual behavior. This design inherently focuses on momentary judgments rather than enduring beliefs (Gaines et al., 2007), which may fluctuate over time, especially in rapidly evolving fields like law enforcement and technology use. Given the relatively quick changes in officer attitudes towards BWCs over time (Miethe et al., 2019; White et al., 2018), it is likely that the individual-level concept of “fairness” in relationship to BWCs is also subject to change.

External validity is always a concern, meaning “we cannot be sure the found effects would be exactly the same in a nonexperimental” context (Andersen & Jakobsen, 2017, p. 60). This weakness is a known vulnerability of experimental settings, which prioritize internal validity. Stepping outside the experimental setting, the external validity concerns are perhaps even stronger (Barabas & Jerit, 2010). In the nonexperimental setting, officers confronted with these activation and review policies are not confronted with orthogonal policy choices, and policy comes alongside a host of agency-level characteristics that are impossible to fully model within a survey. This external validity concern is somewhat muted by a successful randomization procedure and the use of a national probability sample. Still, further research is required to continue to improve our understanding of how public employees experience workplace monitoring.

Further, while experimental surveys can illuminate the impact of specific policies on perceptions, they fall short of explaining why these policies influence fairness judgments. This gap underscores the need for theoretical development and further empirical investigation. Additionally, the timing of data collection, which coincided with the onset of national protests following George Floyd's death, introduces a unique context that may or may not have influenced the study's outcomes. Although the protests began after the majority of data was collected, this timing suggests the importance of replicating this study in varied temporal and professional contexts to ensure the robustness of the findings.

Another consideration is the potential for demand effects, where participants might guess the research's aims, potentially biasing their responses. In other words, did respondents correctly infer that the study was interested in BWC policy shifts away from the most-common policy features (manual activation with review upon complaint or use-of-force)? However, the between-subjects design of this study likely mitigates such effects, as each participant was exposed to only one vignette scenario without a basis for comparison across different policy implementations.

Looking forward, future research should broaden its scope to consider different sampling frames and the impact of organizational features and contexts. The strong professional identity among police officers, for instance, may significantly influence their perceptions of fairness and justice, suggesting a rich area for further exploration. Additionally, examining the role of multi-dimensional dependent variables could provide a more nuanced understanding of fairness perceptions, potentially offering insights into specific aspects of organizational justice most affected by policy changes in BWC activation and review.

Conclusion

The advent of artificial intelligence (AI) for reviewing police behavior has transitioned from speculative fiction to a tangible reality, with law enforcement agencies actively exploring its implementation (Shastry, 2022). This technological evolution, however, is being pursued without fully considering the perspectives of those it directly affects—law enforcement officers—regarding the changes it heralds for their work environment. Historical resistance to technological change in professional settings is well-documented, with both private and public sector organizations facing challenges in integrating new technologies (Mitchell et al., 2012). It is crucial, though, to differentiate between genuine reluctance and a more nuanced ambivalence towards workplace surveillance technologies. The hesitancy of officers to accept new forms of surveillance is rooted in experiences that have been either overlooked or misinterpreted in scholarly work, as the push for greater control often overshadows considerations of fairness (Backman & Löfstrand, 2021).

Enhancing perceptions of organizational justice within law enforcement can yield multiple benefits, improving the well-being of employees, the efficiency of the agency, and the quality of community service (Wolfe & Lawson, 2020). Research indicates that fostering organizational justice and perceived organizational support can mitigate resistance to technological changes in the workplace (Baran et al., 2012; Magni & Pennarola, 2008; Mitchell et al., 2012; Nazareno & Schiff, 2021). However, this body of research has predominantly focused on the private sector, leaving a gap in understanding how these dynamics play out within public sector contexts, particularly in law enforcement. Initial resistance to BWCs was seen as a significant barrier to their adoption, yet current attitudes among officers towards the cameras are largely positive. Nonetheless, as this manuscript reveals, there are emerging BWC policies perceived as unfair by officers. As law enforcement agencies move forward with the next generation of BWC technologies, it is imperative to consider how officer perceptions might influence the successful adoption and efficacy of these systems. Given the significant financial investment required for BWC programs, ensuring that the introduction of new monitoring technologies takes into account the views of those being monitored is not only cost-effective but crucial for the overall success of these initiatives.

In essence, the progression towards AI-based surveillance in policing demands a holistic approach that carefully balances technological advancement with ethical considerations and the well-being of law enforcement personnel (Nazareno & Schiff, 2021). This study's findings accentuate the need to integrate considerations of fairness and organizational justice into the deployment of new surveillance technologies, fostering a work environment that supports innovation while respecting the values and operational realities of the law enforcement profession.

References

Adams, I. T., & Mastracci, S. H. (2019a). Police Body-Worn Cameras: Development of the Perceived Intensity of Monitoring Scale. Criminal Justice Review, 44(3), 386–405. https://doi.org/10.1177/0734016819846219

Adams, I. T., & Mastracci, S. H. (2019b). Police Body-Worn Cameras: Effects on Officers’ Burnout and Perceived Organizational Support. Police Quarterly, 22(1), 5–30. https://doi.org/10.1177/1098611118783987

Adams, I. T., Mourtgos, S. M., & Mastracci, S. H. (2021). High-Stakes Administrative Discretion: What Drives Body-Worn Camera Activations? Public Administration Review, 81(4), 691–703. https://doi.org/10.1111/puar.13339

Aguinis, H., & Bradley, K. J. (2014). Best Practice Recommendations for Designing and Implementing Experimental Vignette Methodology Studies. Organizational Research Methods, 17(4), 351–371. https://doi.org/10.1177/1094428114547952

Alder, G. S. (2001). Employee reactions to electronic performance monitoring: A consequence of organizational culture. The Journal of High Technology Management Research, 12(2), 323–342.

Alder, G. S., & Ambrose, M. L. (2005a). An examination of the effect of computerized performance monitoring feedback on monitoring fairness, performance, and satisfaction. Organizational Behavior and Human Decision Processes, 97(2), 161–177.

Alder, G. S., & Ambrose, M. L. (2005b). Towards understanding fairness judgments associated with computer performance monitoring: An integration of the feedback, justice, and monitoring research. Human Resource Management Review, 15(1), 43–67. https://doi.org/10.1016/j.hrmr.2005.01.001

Alder, G. S., & Ambrose, M. L. (2000). Feedback’s influence on the perceived fairness of computer performance monitoring: An empirical investigation. National Academy of Management Meetings, Toronto.

Alder, G. S., Schminke, M., Noel, T. W., & Kuenzi, M. (2007). Employee Reactions to Internet Monitoring: The Moderating Role of Ethical Orientation. Journal of Business Ethics, 80(3), 481. https://doi.org/10.1007/s10551-007-9432-2

Alge, B. J. (2001). Effects of computer surveillance on perceptions of privacy and procedural justice. Journal of Applied Psychology, 86(4), 797–804. https://doi.org/10.1037/0021-9010.86.4.797

Alge, B. J., & Hansen, S. D. (2014). Workplace monitoring and surveillance research since 1984: A review and agenda. In M. D. Coovert & L. F. Thompson (Eds.), The Psychology of Workplace Technology. Routledge, New York.

Andersen, S. C., & Jakobsen, M. (2017). Policy Positions of Bureaucrats at the Front Lines: Are They Susceptible to Strategic Communication? Public Administration Review, 77(1), 57–66. https://doi.org/10.1111/puar.12584

Ariel, B., Sutherland, A., Henstock, D., Young, J., & Sosinski, G. (2017). The deterrence spectrum: Explaining why police body-worn cameras ‘work’or ‘backfire’in aggressive police–public encounters. Policing: A Journal of Policy and Practice, 12(1), 6–26.

Axon AI. (2021, June 5). [Products/Software]. https://www.axon.com/company/axon-ai-research

Axon Evidence. (2021, June 5). [Products/Software]. https://www.axon.com/products/axon-evidence

Axon, Inc. (2020, December 3). Sprint for Justice Feature Review Webinar. https://www.axon.com/resources/webinars-events/sprint-4-justice-q-a

Axon Signal Sidearm. (2021, May 26). https://www.axon.com/products/axon-signal-sidearm

Backman, C., & Löfstrand, C. H. (2021). Representations of Policing Problems and Body-Worn Cameras in Existing Research. International Criminal Justice Review, 10575677211020813. https://doi.org/10.1177/10575677211020813

Barabas, J., & Jerit, J. (2010). Are Survey Experiments Externally Valid? American Political Science Review, 104(2), 226–242.

Baran, B. E., Shanock, L. R., & Miller, L. R. (2012). Advancing organizational support theory into the twenty-first century world of work. Journal of Business and Psychology, 27(2), 123–147.

Behrend, T., White, J., & Ravid, D. (2019). Moderating Effects of Person and Job Characteristics on Digital Monitoring Outcomes. Current Opinion in Psychology. https://doi.org/10.1016/j.copsyc.2019.07.042

Bittner, E. (1984). The 2 Cultures of Policing—Street Cops and Management Cops—Reussianni,e. American Bar Foundation Research Journal, 1, 206–213.

Blaskovits, B., & Bennell, C. (2019). Exploring the Potential Impact of Body Worn Cameras on Memory in Officer-Involved Critical Incidents: A Literature Review. Journal of Police and Criminal Psychology. https://doi.org/10.1007/s11896-019-09354-1

Bradbury-Jones, C., Taylor, J., & Herber, O. R. (2014). Vignette development and administration: A framework for protecting research participants. International Journal of Social Research Methodology, 17(4), 427–440. https://doi.org/10.1080/13645579.2012.750833

Christodoulou, C., Paterson, H., & Kemp, R. (2019). Body-worn cameras: Evidence-base and implications. Current Issues in Criminal Justice, 1–12.

Cohen, J. (1988). Statistical Power Analysis for the Behavioural Sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.

Colquitt, J. A. (2001). On the dimensionality of organizational justice: A construct validation of a measure. Journal of Applied Psychology, 86(3), 386–400. https://doi.org/10.1037/0021-9010.86.3.386

Colquitt, J. A., Conlon, D. E., Wesson, M. J., Porter, C. O., & Ng, K. Y. (2001). Justice at the millennium: A meta-analytic review of 25 years of organizational justice research. Journal of Applied Psychology, 86(3), 425.

Coppock, A. (2018). Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach. Political Science Research and Methods, 1–16. https://doi.org/10.1017/psrm.2018.10

Douthitt, E. A., & Aiello, J. R. (2001). The role of participation and control in the effects of computer monitoring on fairness perceptions, task satisfaction, and performance. Journal of Applied Psychology, 86(5), 867.

Edwards, L., Martin, L., & Henderson, T. (2018). Employee Surveillance: The Road to Surveillance is Paved with Good Intentions (SSRN Scholarly Paper ID 3234382). Social Science Research Network. https://papers.ssrn.com/abstract=3234382

Engel, R. S. (2000). The effects of supervisory styles on patrol officer behavior. Police Quarterly, 3(3), 262–293.

Fan, M. D. (2017). Missing Police Body Camera Videos: Remedies, Evidentiary Fairness, and Automatic Activation. Ga. L. Rev., 52, 57.

Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behavior Research Methods, 39(2), 175–191. https://doi.org/10.3758/BF03193146

Field, A. (2013). Discovering Statistics Using IBM SPSS Statistics (4th ed.). Sage.

Gaines, B. J., Kuklinski, J. H., & Quirk, P. J. (2007). The Logic of the Survey Experiment Reexamined. Political Analysis, 15, 1–20.

Gaub, J. E., Choate, D. E., Todak, N., Katz, C. M., & White, M. D. (2016). Officer perceptions of body-worn cameras before and after deployment: A study of three departments. Police Quarterly, 19(3), 275–302.

Graham, B. A. T., Brown, L., Chochlakis, G., Dehghani, M., Delerme, R., Friedman, B., Graeden, E., Golazizian, P., Hebbar, R., Hejabi, P., Kommineni, A., Salinas, M., Sierra-Arévalo, M., Trager, J., Weller, N., & Narayan, S. (2024). A Multi-Perspective Machine Learning Approach to Evaluate Police-Driver Interaction in Los Angeles (arXiv:2402.01703). arXiv. https://doi.org/10.48550/arXiv.2402.01703

Graham v. Connor, 490 US ___ (Supreme Court 1989).

Greenberg, J. (1987). A Taxonomy of Organizational Justice Theories. Academy of Management Review, 12(1), 9–22. https://doi.org/10.5465/amr.1987.4306437

Greenberg, J., & Colquitt, J. A. (2013). Handbook of Organizational Justice. Psychology Press.

Harrington, J. R., & Lee, J. H. (2015). What Drives Perceived Fairness of Performance Appraisal? Exploring the Effects of Psychological Contract Fulfillment on Employees’ Perceived Fairness of Performance Appraisal in U.S. Federal Agencies. Public Personnel Management, 44(2), 214–238. https://doi.org/10.1177/0091026014564071

Harrits, G. S. (2019). Using vignettes in street-level bureaucracy research. In Research Handbook on Street-Level Bureaucracy. Edward Elgar Publishing.

Hyland, S., & Davis, E. (2019). Local Police Departments, 2016 (NCJ 252835). Bureau of Justice Statistics. https://www.bjs.gov/index.cfm?ty=pbdetail&iid=6706

Jasso, G. (2006). Factorial Survey Methods for Studying Beliefs and Judgments. Sociological Methods & Research, 34(3), 334–423. https://doi.org/10.1177/0049124105283121

Jasso, G., & Webster, M. (1997). Double Standards in Just Earnings for Male and Female Workers. Social Psychology Quarterly, 60(1), 66–78. JSTOR. https://doi.org/10.2307/2787012

Lawrence, D. S., McClure, D., Malm, A., Lynch, M., & La Vigne, N. (2019). Activation of Body-Worn Cameras: Variation by Officer, Over Time, and by Policing Activity. Criminal Justice Review, 44(3), 339–355. https://doi.org/10.1177/0734016819846228

Lenth, R. (2020). emmeans: Estimated Marginal Means, aka Least-Squares Means [Computer software]. https://CRAN.R-project.org/package=emmeans

Leventhal, G. S. (1976). What Should Be Done with Equity Theory? New Approaches to the Study of Fairness in Social Relationships. Dr. https://eric.ed.gov/?id=ED142463

Leventhal, G. S. (1980). What Should Be Done with Equity Theory? In K. J. Gergen, M. S. Greenberg, & R. H. Willis (Eds.), Social Exchange: Advances in Theory and Research (pp. 27–55). Springer US. https://doi.org/10.1007/978-1-4613-3087-5_2

Lipsky, M. (1983). Street-Level Bureaucracy: The Dilemmas of the Individual in Public Service. Russell Sage Foundation.

Lum, C., Stoltz, M., Koper, C. S., & Scherer, J. A. (2019). Research on body-worn cameras: What we know, what we need to know. Criminology & Public Policy, 18(1), 93–118.

Magni, M., & Pennarola, F. (2008). Intra-organizational relationships and technology acceptance. International Journal of Information Management, 28(6), 517–523. https://doi.org/10.1016/j.ijinfomgt.2008.01.002

Makin, D. A., Willits, D. W., & Brooks, R. (2020). Systematic social event modeling: A methodology for analyzing body-worn camera footage. International Journal of Social Research Methodology, 0(0), 1–14. https://doi.org/10.1080/13645579.2020.1766775

McNall, L. A., & Stanton, J. M. (2011). Private Eyes Are Watching You: Reactions to Location Sensing Technologies. Journal of Business and Psychology, 26(3), 299–309. https://doi.org/10.1007/s10869-010-9189-y

Miethe, T. D., Lieberman, J. D., Heen, M. S. J., & Sousa, W. H. (2019). Public Attitudes About Body-Worn Cameras in Police Work: A National Study of the Sources of Their Contextual Variability. Criminal Justice Review, 44(3), 263–283. https://doi.org/10.1177/0734016819846241

Mitchell, J. I., Gagné, M., Beaudry, A., & Dyer, L. (2012). The role of perceived organizational support, distributive justice and motivation in reactions to new information technology. Computers in Human Behavior, 28(2), 729–738. https://doi.org/10.1016/j.chb.2011.11.021

Møller, M. Ø. (2016). “She isn’t Someone I Associate with Pension”—A Vignette Study of Professional Reasoning. Professions and Professionalism, 6(1), Article 1. https://doi.org/10.7577/pp.1353

Nazareno, L., & Schiff, D. S. (2021). The impact of automation and artificial intelligence on worker well-being. Technology in Society, 67, 101679. https://doi.org/10.1016/j.techsoc.2021.101679

Newell, B. C. (2021). Police Visibility: Privacy, Surveillance, and the False Promise of Body-Worn Cameras. Univ of California Press.

Nix, J., Pickett, J. T., Baek, H., & Alpert, G. P. (2019). Police research, officer surveys, and response rates. Policing and Society, 29(5), 530–550. https://doi.org/10.1080/10439463.2017.1394300

Nix, J., Todak, N., & Tregle, B. (2020). Understanding Body-Worn Camera Diffusion in U.S. Policing. Police Quarterly, 23(3), 396–422. https://doi.org/10.1177/1098611120917937

Nix, J., & Wolfe, S. E. (2017). The Impact of Negative Publicity on Police Self-legitimacy. Justice Quarterly, 34(1), 84–108. https://doi.org/10.1080/07418825.2015.1102954

Pedersen, M. J., Stritch, J. M., & Thuesen, F. (2018). Punishment on the Frontlines of Public Service Delivery: Client Ethnicity and Caseworker Sanctioning Decisions in a Scandinavian Welfare State. Journal of Public Administration Research and Theory, 28(3), 339–354. https://doi.org/10.1093/jopart/muy018

Phillips, S. W. (2020a). Exploring law enforcement’s views of de-policing: A vignette research design. Criminal Justice Studies, 33(2), 178–191. https://doi.org/10.1080/1478601X.2019.1699557

Phillips, S. W. (2020b). The formation of suspicion: A vignette study. International Journal of Police Science & Management, 1461355720929038. https://doi.org/10.1177/1461355720929038

Porter, J., & Ogden, S. (Directors). (2023, March 29). ACLU Colorado opposes AI review of Aurora PD body camera video. Denver7. https://www.youtube.com/watch?v=IfzZcYV45oo

R Core Team. (2020). R: A Language and Environment for Statistical Computing (4.02) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org/

Ravid, D. M., Tomczak, D. L., White, J. C., & Behrend, T. S. (2020). EPM 20/20: A Review, Framework, and Research Agenda for Electronic Performance Monitoring. Journal of Management, 46(1), 100–126. https://doi.org/10.1177/0149206319869435

Reisig, M. D., Mays, R. D., & Telep, C. W. (2018). The effects of procedural injustice during police–citizen encounters: A factorial vignette study. Journal of Experimental Criminology, 14(1), 49–58. https://doi.org/10.1007/s11292-017-9307-1

Riccucci, N. M., Van Ryzin, G. G., & Jackson, K. (2018). Representative Bureaucracy, Race, and Policing: A Survey Experiment. Journal of Public Administration Research and Theory, 28(4), 506–518. https://doi.org/10.1093/jopart/muy023

Riccucci, N. M., Van Ryzin, G. G., & Lavena, C. F. (2014). Representative Bureaucracy in Policing: Does It Increase Perceived Legitimacy? Journal of Public Administration Research and Theory, 24(3), 537–551. https://doi.org/10.1093/jopart/muu006

Shastry, T. (2022). 36% Reduction in Use of Force after Implementation of Training and Body-Worn Camera Analytics [Internal pilot study]. Truleo.

Shaughnessy, B., Braun, S., Hentschel, T., & Peus, C. V. (2016). Diverse and just? The role of quota-based selection policies on organizational outcomes. European Journal of Social Psychology, 46(7), 880–890. https://doi.org/10.1002/ejsp.2208

Sierra-Arévalo, M. (2021). American Policing and the Danger Imperative. Law & Society Review, 55(1), 70–103. https://doi.org/10.1111/lasr.12526

Sierra-Arévalo, M., Nix, J., & Mourtgos, S. M. (2023). The “war on cops,” retaliatory violence, and the murder of George Floyd*. Criminology, n/a(n/a). https://doi.org/10.1111/1745-9125.12334

Singmann, H., Bolker, B., Westfall, J., Aust, F., & Ben-Shachar, M. S. (2020). afex: Analysis of Factorial Experiments [Computer software]. https://CRAN.R-project.org/package=afex

Skarlicki, D. P., & Turner, R. A. (2014). Unfairness begets unfairness: Victim derogation bias in employee ratings. Organizational Behavior and Human Decision Processes, 124(1), 34–46. https://doi.org/10.1016/j.obhdp.2013.11.004

Smith, J. J. (2019). To Adopt or Not to Adopt: Contextualizing Police Body-Worn Cameras Through Structural Contingency and Institutional Theoretical Perspectives. Criminal Justice Review, 44(3), 369–385. https://doi.org/10.1177/0734016819847267

Stanton, J. M. (2000). Reactions to Employee Performance Monitoring: Framework, Review, and Research Directions. Human Performance, 13(1), 85–113. https://doi.org/10.1207/S15327043HUP1301_4

Stanton, J. M., & Barnes-Farrell, J. L. (1996). Effects of electronic performance monitoring on personal control, task satisfaction, and task performance. Journal of Applied Psychology, 81(6), 738.

Stolte, J. F. (1994). The Context of Satisficing in Vignette Research. The Journal of Social Psychology, 134(6), 727–733. https://doi.org/10.1080/00224545.1994.9923007

Stritch, J. M., & Pedersen, M. J. (2019). The Apparent Locus of Managerial Decision Making and Perceptions of Fairness in Public Personnel Management. Public Personnel Management, 48(3), 392–412. https://doi.org/10.1177/0091026018819017

Thibaut, J. W., & Walker, L. (1975). Procedural justice: A psychological analysis. L. Erlbaum Associates.

van den Bos, K., Wilke, H. A. M., Lind, E. A., & Vermunt, R. (1998). Evaluating outcomes by means of the fair process effect: Evidence for different processes in fairness and satisfaction judgments. Journal of Personality and Social Psychology, 74(6), 1493–1503. https://doi.org/10.1037/0022-3514.74.6.1493

Wallander, L. (2009). 25 years of factorial surveys in sociology: A review. Social Science Research, 38(3), 505–520. https://doi.org/10.1016/j.ssresearch.2009.03.004

Weinberg, J. D., Freese, J., & McElhattan, D. (2014). Comparing Data Characteristics and Results of an Online Factorial Survey between a Population-based and a Crowdsource-recruited Sample. Sociological Science, 1.

White, M. D., Flippin, M., & Malm, A. (2020). Does Agency Size Matter? Key Trends in Body-Worn Camera Policy and Practice.

White, M. D., & Malm, A. (2020). Cops, Cameras, and Crisis: The Potential and the Perils of Police Body-Worn Cameras. NYU Press.

White, M. D., Todak, N., & Gaub, J. E. (2018). Examining body-worn camera integration and acceptance among police officers, citizens, and external stakeholders. Criminology & Public Policy, 17(3), 649–677.

Wolfe, S. E. (2021). Doing Organizational Justice. In E. R. Maguire, H. Giles, & S. Hill (Eds.), The Rowman & Littlefield Handbook of Policing, Communication, and Society (pp. 35–46). Rowman & Littlefield Publishers. https://www.google.com/books/edition/The_Rowman_Littlefield_Handbook_of_Polic/pCUgEAAAQBAJ?hl=en&gbpv=0

Wolfe, S. E., & Lawson, S. G. (2020). The organizational justice effect among criminal justice employees: A meta-analysis*. Criminology, 58(4), 619–644. https://doi.org/10.1111/1745-9125.12251

Yakowicz, W. (2017, October 18). LAPD to Use Artificial Intelligence to Analyze Body Camera Footage. Inc.Com. https://www.inc.com/will-yakowicz/axon-artificial-intelligence-software-lapd-body-camera-footage.html

Yost, A. B., Behrend, T. S., Howardson, G., Badger Darrow, J., & Jensen, J. M. (2019). Reactance to Electronic Surveillance: A Test of Antecedents and Outcomes. Journal of Business and Psychology, 34(1), 71–86. https://doi.org/10.1007/s10869-018-9532-2

Zweig, D., & Webster, J. (2002). Where is the line between benign and invasive? An examination of psychological barriers to the acceptance of awareness monitoring systems. Journal of Organizational Behavior, 23(5), 605–633. https://doi.org/10.1002/job.157

Appendix

Balance Tables

Table A1: Respondent Characteristics by Activation Policy (balance test)

Activation Policy

Manual

Automatic

%

N

%

N

Total

df

Chi Square

p Value

Sex

216

2

0.029

0.9857

Race

217

4

2.516

0.6417

Formal Education

217

4

1.260

0.8682

BWC Tenure

234

4

0.247

0.9930

Agency Type

218

5

4.996

0.4163

Union/Labor Member

218

1

1.033

0.3095

Male

44.91%

97

42.59%

92

Female

6.48%

14

6.02%

13

White

45.62%

99

43.78%

95

Black or African American

1.38%

3

1.84%

4

Asian

0.92%

2

0.46%

1

Native Hawaiian or Pacific Islander

0.46%

1

1.38%

3

Other

2.76%

6

1.38%

3

High School Graduate

13.82%

30

14.29%

31

Associate Degree

12.90%

28

13.82%

30

Bachelor's Degree

18.89%

41

16.13%

35

Master's Degree

5.07%

11

3.69%

8

Doctoral or Professional (JD, MD)

0.46%

1

0.92%

2

Less than one year

7.26%

17

6.41%

15

Between one and two years

9.40%

22

9.40%

22

Between two and three years

10.26%

24

11.11%

26

Between three and four years

8.55%

20

8.55%

20

More than four years

14.96%

35

14.10%

33

Municipal Police

33.49%

73

31.65%

69

Sheriff's Department

9.63%

21

11.93%

26

State Police

1.83%

4

0.46%

1

Corrections

0.00%

0

0.46%

1

Campus

5.96%

13

3.67%

8

Other

0.46%

1

0.46%

1

No

11.47%

25

13.76%

30

Yes

39.91%

87

34.86%

76

Table A2: Respondent Characteristics by Review Policy (balance Test)

BWC Review Policy

Complaint & UoF

Supervisor Random

AI Random

% of Total

N

% of Total

N

% of Total

N

Total N

df

Chi Square

p Value

Sex

216

2

0.029

0.9857

Race

217

8

2.510

0.9613

Formal Education

217

8

5.380

0.7163

BWC Tenure

234

8

7.875

0.4458

Agency Type

218

10

12.793

0.2355

Union/Labor Member

218

2

0.178

0.9149

Male

31.48%

68

28.70%

62

27.31%

59

Female

4.63%

10

4.17%

9

3.70%

8

White

31.34%

68

29.95%

65

28.11%

61

Black or African American

1.84%

4

0.46%

1

0.92%

2

Asian

0.46%

1

0.46%

1

0.46%

1

Native Hawaiian or Pacific Islander

0.92%

2

0.46%

1

0.46%

1

Other

1.84%

4

1.38%

3

0.92%

2

High School Graduate

9.22%

20

11.06%

24

7.83%

17

Associate Degree

9.68%

21

8.29%

18

8.76%

19

Bachelor's Degree

12.44%

27

11.06%

24

11.52%

25

Master's Degree

4.15%

9

1.84%

4

2.76%

6

Doctoral or Professional (JD, MD)

0.92%

2

0.46%

1

0.00%

0

3.85%

9

3.42%

8

6.41%

15

Between one and two years

7.69%

18

5.13%

12

5.98%

14

Between two and three years

8.97%

21

7.69%

18

4.70%

11

Between three and four years

5.98%

14

6.84%

16

4.27%

10

More than four years

9.83%

23

8.97%

21

10.26%

24

Municipal Police

22.02%

21.56%

47

21.56%

47

Sheriff's Department

10.09%

22

5.50%

12

5.96%

13

State Police

0.46%

1

1.83%

4

0.00%

0

Corrections

0.46%

1

0.00%

0

0.00%

0

Campus

2.75%

6

3.67%

8

3.21%

7

Other

0.46%

1

0.46%

1

0.00%

0

No

9.63%

21

7.80%

17

7.80%

17

Yes

26.61%

58

25.23%

55

22.94%

50

Pairwise Effects Analysis

Pairwise effects information is crucial for practitioners and law enforcement leaders contemplating policy shifts in BWC activation and review policies. This is because not every policy shift is expected to generate a statistically significant effect, and not every statistically significant effect is expected to have the same effect size. Some agencies will be considering a brand-new BWC implementation, where analysis of only main and interaction effects makes sense. However, with upwards of three-quarters of agencies in my survey already having at least some BWCs, it is very likely that many agencies will be considering moving from one BWC policy scheme to another. In that case, practitioners should be interested in contrasting policy levels and their effects on perceived fairness. For example, the first four rows report the levels contrast for main effects – that is, what happens if an agency is considering just one (either activation, or review) policy shift? In the fourth row, we see the main effect of moving from manual to automatic activation. As previously noted, this is perceived quite negatively, with mean fairness values dropping over a point, and a statistically significant large effect size (d=.409). Though perhaps not realistic, the same effect size, but opposite direction, would be predicted for an agency moving from automatic activation to a manual activation policy.

Comparing the first three rows, we see that going from the traditional use-of-force or complaint review model to review by algorithm is expected to result in a large, statistically significant negative effect to perceived fairness. However, the same agency moving from use-of-force and complaint-based review to supervisor random review, or from supervisor to algorithmic review, would not expect to see a statistically significant shift in perception, though the overall direction is still negative. This result highlights just how impactful a (perceived) “large” move away from traditional monitoring to algorithmic monitoring can be, but contextualizes how to think about policy shifts. In other words, a policing organization carefully attending to organizational justice perceptions, but still under pressure to move towards algorithmic review, might consider that officers are less likely to perceive great injustice in a slower pace of change. Moving to a supervisor-based review, and then later considering moving from there to algorithmic review, would be less likely to face significant decreases in perceived fairness among officers.

The analysis of rows 5 thru 19 in Table 7 is intended to consider the interaction effects of moving from one (Activation*Review) policy to another (Activation*Review) policy. This is a realistic policy consideration for agencies, especially those who are Axon customers, as that manufacturer is actively developing and selling activation and review products that can be packaged together (Axon AI, 2021; Axon Evidence, 2021; Axon Signal Sidearm, 2021). Several important combinations should be noted.

First, as noted, by far the most common policy scheme in US agencies today is manual BWC activation with review done in the aftermath of a use-of-force incident or complaint (White & Malm, 2020). Therefore, the majority of agencies considering policy change in this area will be moving from this scheme. For shorthand, I’ll refer to this as “traditional,” though acknowledging there is a very short history to hang upon that word. Rows 5, 7, 8, 10, and 15 begin with this traditional starting point, and calculate the effect of moving to another scheme. Of this group, all but row 15 are statistically significant effects. Row 5 shows the largest effect size of all (Δ Fair = 2.118, d = 1.261, p < .001), as we move from traditional to the most technology-centric policy of automatic activation combined with artificial intelligence review. This policy shift is equivalent to dropping a full two points on the seven-point measure of fairness, or in other terms dropping a respondent’s view from Agree (6) to Neither Agree nor Disagree (4), or from there to Disagree (2). This is a very large negative effect. Similarly, row 7 moves from traditional but only changes activation to automatic (Δ Fair = 1.332, Cohen’s d = 0.793, p < .001). Comparing rows 5 and 7 shows the substantial difference an organization might make by moving only one policy lever at a time. While still a significant negative hit to fairness perceptions, the size of the effect is substantially smaller when only changing activation type, rather than both activation and review. Reviewing the entire pairwise effect list makes clear that activation type is the primary driver of negative perceptions in these interactions. In order to better visualize how the type of policy shift, and whether it is paired with a simultaneous policy shift, affects the presence and quality of a shift in fairness perceptions, all pairwise effects are ranked in Appendix Figure A1. For example, Row 5 discussed above (moving from traditional to fully automated) is the largest, and therefore first, pairwise effect depicted.

Chart Description automatically generated

Figure A1: Pairwise Effect Sizes, Interaction Levels

Table A3: Pairwise Power Comparisons for Full Factorial ANOVA:

Effect

Level

- Level

Diff.

Std Err Diff.

Lower CL

Upper CL

p-Value

t-Ratio

Cohen's d

1

BWC Review Policy

Complaint & UoF

AI Random

0.688

0.269

0.158

1.218

0.011**

2.557

0.409

2

BWC Review Policy

Complaint & UoF

Supervisor Random

0.477

0.268

-0.050

1.004

0.076

1.783

0.284

3

BWC Review Policy

Supervisor Random

AI Random

0.211

0.277

-0.335

0.756

0.448

0.761

0.125

4

Activation Policy

Manual Activation

Automatic Activation

1.016

0.221

0.579

1.452

0.000*

4.587

0.605

5

Review* Activation

Complaint & UoF, Manual Activation

AI Random, Automatic Activation

2.118

0.380

1.368

2.867

0.000*

5.567

1.261

6

Review* Activation

AI Random, Manual Activation

AI Random, Automatic Activation

1.528

0.393

0.752

2.303

0.000*

3.883

0.910

7

Review* Activation

Complaint & UoF, Manual Activation

Complaint & UoF, Automatic Activation

1.332

0.367

0.609

2.055

0.000*

3.630

0.793

8

Review* Activation

Complaint & UoF, Manual Activation

Supervisor Random, Automatic Activation

1.237

0.365

0.518

1.955

0.001*

3.392

0.736

9

Review* Activation

Supervisor Random, Manual Activation

AI Random, Automatic Activation

1.068

0.404

0.271

1.865

0.009*

2.641

0.636

10

Review* Activation

Complaint & UoF, Manual Activation

Supervisor Random, Manual Activation

1.049

0.383

0.294

1.805

0.007*

2.737

0.625

11

Review* Activation

Supervisor Random, Automatic Activation

AI Random, Automatic Activation

0.881

0.387

0.119

1.642

0.024*

2.279

0.524

12

Review* Activation

Complaint & UoF, Automatic Activation

AI Random, Automatic Activation

0.786

0.389

0.020

1.552

0.044*

2.021

0.468

13

Review* Activation

AI Random, Manual Activation

Complaint & UoF, Automatic Activation

0.742

0.380

-0.008

1.492

0.052

1.951

0.442

14

Review* Activation

AI Random, Manual Activation

Supervisor Random, Automatic Activation

0.647

0.378

-0.098

1.392

0.088

1.711

0.385

15

Review* Activation

Complaint & UoF, Manual Activation

AI Random, Manual Activation

0.590

0.372

-0.143

1.323

0.114

1.586

0.351

16

Review* Activation

AI Random, Manual Activation

Supervisor Random, Manual Activation

0.460

0.396

-0.321

1.241

0.247

1.160

0.274

17

Review* Activation

Supervisor Random, Manual Activation

Complaint & UoF, Automatic Activation

0.282

0.392

-0.490

1.054

0.472

0.721

0.168

18

Review* Activation

Supervisor Random, Manual Activation

Supervisor Random, Automatic Activation

0.187

0.390

-0.580

0.955

0.631

0.481

0.111

19

Review* Activation

Supervisor Random, Automatic Activation

Complaint & UoF, Automatic Activation

0.095

0.373

-0.640

0.831

0.799

0.255

0.057

Comments
0
comment
No comments here
Why not start the discussion?