Review 5 of "Conducting Systematic Social Observations of Body-Camera Footage: Methodological and Practical Insights"

Vote: Publish pending minor changes

[For votes to count, referees must reasonably explain why they voted as they did. Thus, please explain your vote. If you voted to publish pending minor changes, specify each change, why it is needed, and, possibly, how it should/could be done.]

Original contribution:

The article brings together SSO, a prominent approach in criminology, and VDA, an emerging framework for analyzing video data in the social sciences, and situates the use of body-worn cameras (BWC) within this nexus. BWC data are already an important source of information in criminology, and will only increase in importance with the expected further dissemination of the technology. The methodological literature on VDA, SSO, and use of BWC data needs articles like the one presented here, which tackle specific strengths and limitations of video-based research, using concrete research projects as examples. I’m convinced this article will be highly useful for researchers conducting similar studies in the future.

Well executed:

The article has a clear line of argumentation and is written in a straightforward, accessible fashion. It presents and reflects on relevant literature in an illuminating way, uses a systematic methodological approach and present it in a transparent fashion. The article also discusses paths forward for qualitative and quantitative research in criminology.

[Please put additional info below, as/if you see fit.]

P. 3, second paragraph from the bottom, last sentence: should be “conducTing”

Nassauer & Legewie 2018 has now been published in print (2021). I’m not sure, but maybe it makes sense to update the references and page numbers so that readers can more easily find the sections?

Ethics are discussed a bit on p. 6, and VDA is argued to provide advantages over traditional qualitative methods. The points made are well taken, but I would encourage a deeper engagement with the topic. For instance, BWC data often do contain direct identifiers (people’s faces), and researchers have to think about how to handle this information. If, as was the case in this study, facial expressions are not part of the analysis, it may be possible to run a face blurring algorithm over the data before researchers gain access, in order to further protect privacy. Another example is the notion of contextual integrity put forth by Nissembaum (2009): Taking data that was created in one context (here: police-citizen encounters) and use it in another (here, research) without consent clashes with people’s right to control over the flow of personal information. In other words, even though people were made aware of being filmed, this doesn’t necessarily waive their right to privacy, especially if it’s unclear whether they are in a position to deny officers recording them. I don’t, by any means, think that this makes the present unethical, since many considerations go into such an assessment. But I do think issues of research ethics merit more discussion here. Anne Nassauer and I have discussed some of these issues in two publications, in case those references help as a starting point (Legewie & Nassauer 2018; 2020).

I feel like the headers could provide a little more guidance for readers. Specifically, the repetition in sub-headers sometimes made it a bit harder than necessary to follow the text. Perhaps add some information to the header that locates it within the text?

Officers leaving the scene as an endpoint of the unit of analysis seems a bit like right-censored data in survey research: We know there’s more going on, and something important might still happen, but we don’t have the data. Maybe this could be mentioned briefly?

On the coding procedure, with conference calls in cases of unclear coding decisions, the authors could reference Campbell et al. 2013 “Coding In-depth Semistructured Interviews Problems of Unitization and Intercoder Reliability and Agreement.” The authors discuss agreement after discussion, which might be a useful point of reference.

I think an online appendix showing the full coding scheme would be very interesting and helpful for readers.

P. 16, endorsement of streaming videos for analysis: Would that still allow using data analysis software, such as Atlas.ti or Noldus? If not, that would be a drawback of streaming data, no?