Review of "Relative Openness in Peer Review: Current Practices in Criminology & Sociology Journals"

This is a review of Callie H. Burt’s (2024) preprint, “Relative Openness in Peer Review: Current Practices in Criminology & Sociology Journals.” I aim to provide suggestions for how the paper might be improved. My comments come in the order they arise in the paper, not their order of importance.

• Abstract (and throughout the paper): The use of the word “openness” suggests this paper will be about open access (OA). However, “whether the reviewer comments and editor decision are shared with reviewers” is not a matter of OA per se (see here). Perhaps I should add, I don’t think it’s “open science” either. Open access/science is a method of creating transparency and accountability in research, as is sharing reviews with reviewers. But unpublished reviews are “closed access.” By definition, for reviews to be truly OA, they have to be (1) published (2) with a CC license or a similar license. The review you’re reading now is an example. If I only sent it privately to someone, it wouldn’t be “open.” Hence, the use of the word “openness” in the paper is better represented with a word like “sharing.”1

• Page 2: The paper reads <“a ‘gold standard’ of scholarly communication,’>. That last quotation mark should be a double (”) not single (’).

• Page 2: Peer review typically serves, as well, a third main purpose: provide ideas on how to improve the paper.

• Page 3: For the paragraph on the EMBO Journal, the quotation marks aren’t quite right.

• Page 6: Regarding the bit, “For two non-responses, I had reviewed for the journal under the current editorial team. For those two cases, I recorded my experience with the sharing of reviews and decision.”: I think it’d be better to keep those cases as question marks than generalize from personal experience.

• Page 8: Burt concludes with an invitation to contact her with conflicting or additional information. For the Journal of Qualitative Criminology & Criminal Justice:

  • In the table, its title is missing “Journal of”.

  • The Publisher is listed as none but it’s the Southwestern Association of Criminal Justice.

  • I was surprised to see the journal is marked as not sharing. It publishes its reviews OA; see here. Perhaps what happens(?) is the editor doesn’t send the reviews and decisions to reviewers, specifically, because they’ll become available to everyone. Or maybe(?), the editor doesn’t share with the reviewers when the paper is rejected.