Abstract

Like other social sciences, criminology is experiencing a theory crisis as part of a larger replication crisis. Niemeyer and colleagues (2022) estimate the impact of weak criminological theory on false positive findings using the Ioannidis equation and find that likely at least 90% of findings are false positives. Weak criminological theory directly contributes to a methods-based crisis of flexible operationalization and poor reproducibility of findings. To better quantify the problem, we calculate the index of qualitative variation (IQV) for theoretical constructs in a sample of social disorganization empirical tests. The IQV empirically demonstrates the extent of researcher degrees of freedom through flexibility of theoretical constructs. It accounts for both the number of categories of operationalizations used for each construct and the distribution of use across those operationalizations. We found that the IQV for poverty, racial and ethnic heterogeneity, and residential stability were extremely high in our sample with values ranging from 0.84 to 0.99. This empirically identifies a theoretical contributor to the reproducibility crisis in criminology. We discuss the troubling implications of high theoretical flexibility for social disorganization theory and for broader replication efforts.

Disclaimer: The views expressed in this publication are those of the author and do not reflect the official policy or position of the US Air Force, Department of Defense, or the U.S. Government.

Funding Statement: This work was supported by the Air Force Office of Scientific Research, funding award FA9550-23-1-0453.

Conflict of Interest: This work contains no conflicts of interest for any contributing author.

KeyWords

social disorganization theory, credibility crisis, theory flexibility, researcher degrees of freedom, theory integration, index of qualitative variation.

Introduction

Since the mid 2000s, scientists in a number of fields have been examining their scientific findings to determine the prevalence rates of false-positive results (Forstmeier et al., 2017; Loken & Gelman, 2017; Maxwell et al., 2015; Niemeyer et al., 2022). Investigations of false-positive rates in criminology are more recent and focus on methodological concerns (e.g., statistical power or questionable research practices [QRPs]) (Barnes et al., 2020; Burt, 2020; Chin et al., 2023; Niemeyer et al., 2022). Recently, Niemeyer et al. (2022), however, investigated the role theory might play in generating false-positive results. They calculated false-positive rates for criminology using a series of Bayesian simulations that differentiated between theoretical and methodological sources of false-positives (Ioannidis, 2005). They found that even if methodological sources of false-positive results could be eliminated, shortcomings in theory played a substantial role in their continued existence, with only 1.13-10.22% of criminological research findings likely being true positive results.

A shortcoming of Niemeyer et al. (2022), however, was that they employed theoretically—not empirically—guided parameters for simulation values. The authors note: “Future efforts should identify ways to measure theoretical parameters more precisely to understand better how theory directly and indirectly (by its influence on researcher degrees of freedom) affects false-positive rates” (Niemeyer et al., 2022, p. 17). The purpose of this pilot study, therefore, is to empirically examine one potential source of researcher degrees of freedom relating to criminological theory: the presence of unstandardized theoretical concepts and operationalizations in the field (Ioannidis, 2005). We begin with an overview of the credibility crises in criminology and the role ambiguous theory plays in it. We then describe a pilot study that examines the flexibility of operationalizations used in empirical tests of social disorganization theory (Shaw & McKay, 1942/1972) and calculate an index of qualitative variation to measure discrepancies in the operationalizations of key concepts among studies testing the same theory. We conclude by discussing the implications of this study for work on the credibility crisis in criminology and criminological practice.

Credibility crisis in Criminology

The credibility crisis first emerged in psychology in the 2000s (Open Science Collaboration, 2015); central to this crisis was the discovery that positive research results fail to replicate, indicating the majority of their research findings may be false-positives (Klein et al., 2022; Open Science Collaboration, 2015; Simmons et al., 2011; Sotola, 2022). In general, investigations into the sources of the crisis have focused on: (1) publication biases in science; (2) methodological causes; and (3) issues associated with poor theories that guide empirical research. Unsurprisingly, evidence is beginning to accrue suggesting criminology is experiencing its own crisis of credibility (Barnes et al., 2020; Chin et al., 2023; Niemeyer et al., 2022; Pridemore et al., 2018). Methodological and practical sources of the replication crisis in criminology have been discussed in depth (see Chin et al. 2023; Pridemore et al. 2018; Schumm et al. 2023; Sweeten 2020; Wooditch et al. 2020); we focus on the theoretical sources.

Niemeyer et al.’s (2022) simulations examining possible contributions of weak theory to the field’s burgeoning credibility crisis represented the first effort by criminologists to link longstanding theoretical concerns in criminology with false-positive research findings. Specifically, they identified three ways theory can contribute to false-positives: the absence of theory falsification in the field (Bernard, 1990; Bernard & Snipes, 1996; Dooley & Goodison, 2020; Proctor & Niemeyer, 2019); the scientifically and theoretical fragmented nature of criminological research findings (Proctor & Niemeyer, 2019); and ambiguous research methods and conceptual definitions that give researchers numerous degrees of freedom, which can bias research findings (see also Ioannidis, 2005). Of these, two are particularly important for the field’s false-positive rate. First, the presence of multiple, competing perspectives—not all of which are scientific—means the field has multiple perspectives with contradictory assumptions generating contradictory, and therefore paradoxical, research findings (Niemeyer et al., 2022). Second, given the numerous, competing, and contradictory paradigms present in criminology, criminologists may be subject to few scientific social controls regarding their research practices.

Present Study

To understand the extent of weak theory in the field, this pilot study investigates key theoretical components in social disorganization theory. As a Chicago School theory, social disorganization theory (Shaw & McKay, 1942/1972) uses a processual approach to theorizing (Short, 1972), with origins in the work of Thomas and Znaniecki (1966) and Cooley (1909/2003). For Shaw and McKay ([1942] 1972), delinquency is the result of weak social controls that are generated through of a process of social disorganization that entails population growth, industrialization, the geographical concentration of people, racism, geographic segregation, and the formation of natural areas in urban environments. This dynamic processual explanation of delinquency differs significantly from empirical tests of the theory, which focus narrowly on a number of static constructs derived and operationalized from the theory; in particular: poverty, racial and ethnic heterogeneity, residential stability, and crime.

This reinterpretation of the theory from a processual account of delinquency into a construct-based explanation of crime (i.e., covering law approach) provides preliminary evidence that researchers possess some freedom when interpreting existing theories in criminology. A further demonstration of researcher degrees of freedom could potentially be established by examining how various empirical studies of social disorganization theory differ in their operationalizations of key constructs. Our study examines this form of flexibility by observing operationalizations of social disorganization theory constructs—poverty, racial and ethnic heterogeneity, residential stability, and crime—in peer-reviewed articles testing the theory. We further calculate indexes qualitative variation (IQVs) (Agresti & Agresti, 1978; Budescu & Budescu, 2012) for each construct to examine levels of qualitative variation in operationalizations to provide a quantitative measure for researcher degrees of freedom (i.e., flexibility).

Data

Our data consist of a purposive sample of 15 empirical journal articles testing social disorganization theory, with this form of sampling being a particularly useful starting point for the qualitative thematic analysis of documents (Morgan, 2022). Articles were selected for inclusion from the comprehensive reading list for crime prevention in the community and environmental crime segment of a prominent PhD program in Criminal Justice. This sampling frame included 39 items, twenty-four of which were dropped as they were books or nonempirical, leaving a final sample of 15 empirical social disorganization articles. These studies are identified in the reference section.

Method

Sample articles were coded using the MAXQDA qualitative software package (MAXQDA 2023). A codebook was created to capture variations in operationalizations of key social disorganization theory constructs. This process entailed the: (1) deductive operationalization of subcodes for each construct; (2) trial coding of a few articles, with new inductive subcodes for each construct added to the codebook as needed until saturation and completeness were reached; (3) coding of the full sample using the revised codebook (only variable names and operationalizations for the constructs were coded); and (4) the analysis of operationalizations for social disorganization theory by exporting MAXQDA coding data into Excel 16.2, and then importing the spreadsheet file into Stata 15.1 (StataCorp, 2017). The constructs coded from social disorganization theory included poverty, racial and ethnic heterogeneity, residential stability, and crime.

Poverty

As a processual approach, Shaw and McKay ([1942] 1972) viewed poverty as a means of fleeing the polluted and crowded of areas of the city where factories and commercial activities were located, leading to the formation of distinct natural areas—i.e., the central business district, zone of transition, working class zone, residential zone, commuter zone, and blackbelt—with each possessing different patterns of social organization. This process including poverty is commonly interpreted by covering law approaches as involving the construct of poverty—a static trait. Some studies operationalize poverty as percent below the poverty line (Messner & Blau, 1987; Moore & Sween, 2015; Roh & Choo, 2008; Warner & Rountree, 1997), while others use socioeconomic status (Bellair, 1997; Bursik, 1999; Lee, 2000; Miethe & McDowall, 1993; Sampson & Groves, 1989), female-headed households (Rice & Csmith, 2002), concentrated disadvantage (Baumer et al., 2003; Kubrin & Weitzer, 2003; Stewart et al., 2006; Stewart & Simons, 2006), or family disruption (Sampson & Wooldredge, 1987). The most frequent operationalization was socioeconomic status (33.33%). The full descriptive statistics are displayed in Table 1.

Racial and Ethnic Heterogeneity

In social disorganization theory Shaw and McKay ([1942] 1972), racial and ethnic heterogeneity were tied to immigration into the zone of transition that surrounded a city’s central business district. It contained the most affordable housing; consequently, it contained high levels of population turnover, leading to cultural heterogeneity as different nationalities came to live in the same crowded area. Empirically, the construct-based interpretation sees heterogeneity as a static property of an ecological unit. Baumer et al. (2003), Messner and Blau (1987), and Miethe and McDowall (1993) operationalize heterogeneity through racial demographics—while Bellair (1997), Bursik (1999), Moore and Sween (2015), Rice and Csmith (2002), Roh and Choo (2008), Sampson and Groves (1989), and Warner and Rountree (1997) use a diversity measure of ethnic heterogeneity. Diversity measures of ethnic heterogeneity were the most frequently used (70%). Notably, some articles do not include a measure of heterogeneity at all (Kubrin & Weitzer, 2003; Lee, 2000; Sampson & Wooldredge, 1987; Stewart et al., 2006; Stewart & Simons, 2006). This lack of inclusion is strange given the process’ centrality to the theory.

Stability

In Shaw and McKay ([1942] 1972), a process entailing the immigration of groups, largely of European national origins, into the city fueled residential turnover and mobility. Impoverished groups flooded to the zone of transition because of its cheap housing and proximity to employment. As one moved away from the central business district and through the zone of transition, housing density decreased, the housing shifted from dense residential apartments to the small homes, and home ownership increased. Homeownership brought with it residential stability. Consequently, as one moved away from the central business district, residential stability increased among residents.

The covering law empirical interpretation of community stability has been operationalized several ways. It has been operationalized as either residential stability (Bellair, 1997; Bursik, 1999; Kubrin & Weitzer, 2003; Sampson & Groves, 1989), home rental rates (Sampson & Wooldredge, 1987), or 5-year resident rates (Moore & Sween, 2015; Roh & Choo, 2008; Warner & Rountree, 1997). Operationalizations of residential stability were used in half (50%) of the sample. The other articles in the sample omitted a measure of stability (Baumer et al., 2003; Lee, 2000; Messner & Blau, 1987; Miethe & McDowall, 1993; Rice & Csmith, 2002; Stewart et al., 2006; Stewart & Simons, 2006). Again, the omission of a key component of social disorganization theory is concerning, particularly because some articles omitted measures of both stability and heterogeneity (Lee, 2000; Stewart et al., 2006; Stewart & Simons, 2006).

Crime

Shaw and McKay ([1942] 1972) were primarily interested in juvenile delinquency and the social processes causing it. However, the majority of social disorganization studies in the sample examined adult crime (Baumer et al., 2003; Bellair, 1997; Kubrin & Weitzer, 2003; Lee, 2000; Messner & Blau, 1987; Miethe & McDowall, 1993; Moore & Sween, 2015; Rice & Csmith, 2002; Roh & Choo, 2008; Sampson & Groves, 1989; Sampson & Wooldredge, 1987; Warner & Rountree, 1997), with only 14.29% of crime operationalizations examining juvenile violent crime (Stewart et al., 2006; Stewart & Simons, 2006). The most frequent operationalization of crime was auto theft (21.43%). Bursik (1999) did not include a measure of crime and instead used social disorganization constructs to predict measures of community cohesion. This substitution of crime for delinquency is important because it may impact replicability if the theory does not generalize beyond samples measuring juvenile crime.

Table 1: Descriptive Statistics and Indices of Qualitative Variation for Social Disorganization Theory Operationalizations of Key Constructs

Analysis

To assess construct flexibility, we calculated an Index of Qualitative Variation (IQV) for each construct across the sample (n=15). The IQV accounts for diversity both in number of categories and percent of responses that fall into each category (Agresti & Agresti, 1978; Budescu & Budescu, 2012). We used the normalized version of the index to allow comparison across constructs with differing numbers of categories (Budescu & Budescu, 2012). IQV values were calculated in Stata 15.1 (StataCorp 2017) using the user generated code “divcat” (Enzmann 2015). The equation used is:

where k is number of categories and p is the percent within each category in decimal form (Agresti & Agresti, 1978). The IQV ranges from 0 to 1 with 0 being perfect homogeneity and 1 being perfect heterogeneity (Budescu & Budescu, 2012).

Findings

Poverty was operationalized in five distinct ways in this sample (percent below the poverty line, socioeconomic status, female-headed household, concentrated disadvantage, and family disruption). The IQV value of 0.922 reflects the large amount of diversity both in number of categories of operationalization and in the distribution of operationalizations across these categories. Three of the categories—percent below the poverty line, socioeconomic status, and concentrated disadvantage—occurred with similar frequencies, indicating there is no consensus in the literature regarding the operationalization of poverty. Table 1 displays the IQV findings for each construct of interest.

Operationalizations of heterogeneity only included two categories in the sample: racial demographics (70%) and ethnic heterogeneity (30%) diversity measures. As a result of fewer categories and less dissensus, the IQV for heterogeneity is lower than poverty with a value of .84. Ten articles included operationalizations of heterogeneity.

The stability construct included three unique operationalizations: residential stability, home rental rates, and 5-year resident rates. Residential stability (50%) was the most prevalent operationalization, followed by 5-year resident rates (37.5%), and home rental rates (12.5%). An IQV value of 0.891 displays a level of variability between poverty and heterogeneity. Only eight articles included a measure of the stability construct, cutting the original sample size (n=15) almost in half.

Crime had the highest variability in operationalization (IQV .989). Measures included offenses like auto theft (21.43%), as well as aggravated assault (14.29%) and general property crime (14.29), for example. This IQV value is extremely close to the highest value of 1 representing perfect heterogeneity.

Discussion & Conclusion

Our findings reveal considerable heterogeneity in measures of key constructs for social disorganization theory. Poverty, racial and ethnic heterogeneity, residential stability, and crime all demonstrate high levels of qualitative variation in their operationalization—which serve as a potential source of false positives research findings (Ioannidis, 2005). This high variability suggests a deeper issue with the strength of the theory, a concern given Oberauer and Lewandowsky's (2019) contention that weak theories—theories lacking precision—are central to the reproducibility crisis in psychology. Among other problems, weak theory contributes to researcher degrees of freedom by allowing for various operationalizations of unclear theoretical concepts (Bringmann et al., 2022; Oberauer & Lewandowsky, 2019). The high IQV values for social disorganization theory constructs reveal this problem to be present in the pilot sample. This finding is scientifically problematic given psychology’s discovery that flexible theories are incapable of producing robust, replicable results (Scheel et al., 2021; Smaldino, 2019).

Problems associated with weak theories have led psychologists to advocate for theoretical reforms prior to methodological and statistical ones to decrease false positive rates (Rooij, 2019; Smaldino, 2019). Theory formalization serves as one solution to this problem (Navarro, 2021; Robinaugh et al., 2021; van Rooij & Baggio, 2021; for examples of formalism in criminology, see Gibbs, 1972, 1985; Proctor, 2010; Proctor & Niemeyer, 2019), with formal, precise definitions of constructs serving as a necessary precondition to establishing a logical derivation chain linking constructs and theoretical assumptions to operationalizations (Meehl, 1990; Scheel et al., 2021). At minimum, imprecise theoretical definitions and relationships likely produce inferential errors and scientific stagnation (Fried, 2020). Given the flexibility of social disorganization operationalization demonstrated in this pilot study, criminology should learn from the experiences in psychology and be mindful of the impact of theoretical flexibility in replication efforts.

Additionally, we identify another source of theoretical flexibility: The scientific paradigm being used to interpret the theory. As evident in Figure 1, the structural equation models employed by Sampson and Groves (1989) and Bursik (1999) are consistant with a covering law approach to science—which emphasize logical classes of phenomena that are abstracted into constructs from the properties of concrete things. Covering laws approaches then seek to explain relationships among constructs using propositions. When empirically tested, constructs are operationized as variables and their propositions serve as the basis for the hypotheses linking variables (Proctor and Niemeyer 2019). Key to a covering law approach is its emphasis on abstract constructs with causal powers. In other words, a neighborhood’s ‘heterogeneity,’ ‘poverty status,’ or ‘urbanization’ are causally related to its ‘crime.’ These abstractions operate like forces (e.g., gravity) in the physical sciences (Glennan 2017).

Figure 1: Changing Interpretations of Social Disorganization

This approach, however, differs siginificantly from the processual approach of Shaw and McKay ([1942] 1972), which emphasized concrete things and how their behavior—not the abstracted characteristics of things—serve as the source of causality in the world (Thomas & Znaniecki, 1966). Thus, in a processual view, heterogeneity is not a property of a community operating as a causal force. Instead, it is the product of individuals acting and interacting in response to selection pressures in their environments, such as neighborhood polution. Viewed statistically, heteregeneity is not a cause, but it is rather a byproduct or symptom that is the result of the actions of living people. Or as Thomas and Znaniecki (1966) stated:

“And since concrete social life is concrete only when taken together with the individual life which underlies social happenings, since the personal element is a constitutive factor of every social occurrence, social science cannot remain on the surface of social becoming, where certain schools wish to have it float, but must reach the actual human experiences and attitudes which constitute the full, live and active social reality beneath the formal organization of social institutions, or behind the statistically tabulated mass-phenomena which taken in themselves are nothing but symptoms of unknown causal processes and can serve only as provisional ground for sociological hypotheses” (13-14).

The slow, progressive reinterpretation of social disorganization theory as a covering law explanation of crime is problematic because criminology once possessed strong norms against integrating theories with contradictory assumptions (Bernard, 1989; Bernard & Snipes, 1996; Elliot, 1985; Elliot et al., 1979, 1985; Hirschi, 1979, 1989; Proctor & Niemeyer, 2019). Yet, we observe social disorganization theory slowly losing its processual character and being transformed into a covering law approach overtime. This finding not only demonstrates flexibility expanding to include entire paradigms, but also reveals the possible absence of social controls in criminology’s peer review system. This transformation of the theory necessarily creates problems related to logical coherence as the paradigmatic assumptions of covering law and processual approaches differ substantially.

As a pilot study, these findings should be regarded as tentative. This study only included 15 studies testing a single criminological theory. As a Chicago School theory of crime, social disorganization theory’s statement is ambiguous (Bursik, 1988; Turner & Turner, 1990), which itself facilitates flexibility in interpretating and operationalizing the theory. Other, more precisely stated, criminological theories would likely demonstrate less flexibility in operationalization. Future research should examine a census of empirical tests for social disorganization theory to determine how representative these our findings are for the theory at large. Moreover, other theories should be examined so criminologists have a better sense of the extent of theory flexibility that is contributing to the theory crisis and replication crisis in criminology.

References

Agresti, A., & Agresti, B. F. (1978). Statistical Analysis of Qualitative Variation. Sociological Methodology, 9, 204. https://doi.org/10.2307/270810

Barnes, J. C., TenEyck, M. F., Pratt, T. C., & Cullen, F. T. (2020). How Power ful is the Evidence in Criminology? On Whether We Should Fear a Coming Crisis of Confidence. Justice Quarterly, 37(3), 383–409. https://doi.org/10.1080/07418825.2018.1495252

*Baumer, E., Horney, J., Felson, R., & Lauritsen, J. L. (2003). Neighborhood disadvantage and the nature of violence. Criminology, 41(1), 39–72. https://doi.org/10.1111/j.1745-9125.2003.tb00981.x

*Bellair, P. E. (1997). Social interaction and community crime: Examining the importance of neighbor networks. Criminology, 35(4), 677–704. https://doi.org/10.1111/j.1745-9125.1997.tb01235.x

Bernard, T. J. (1989). A Theoretical Approach to Integration. In S. F. Messner, Marvin D. Krohn, & Allen Liska (Eds.), Theoretical Integration in the Study of Deviance and Crime (pp. 137–159). State University of New York Press.

Bernard, T. J. (1990). Twenty Years of Testing Theories: What Have We Learned and Why? Journal of Research in Crime and Delinquency, 27(4), 325–347. https://doi.org/10.1177/0022427890027004002

Bernard, T. J., & Snipes, J. B. (1996). Theoretical Integration in Criminology. Crime and Justice, 20, 301–348. https://doi.org/10.1086/449245

Bringmann, L. F., Elmer, T., & Eronen, M. I. (2022). Back to Basics: The Importance of Conceptual Clarification in Psychological Science. Current Directions in Psychological Science, 31(4), 340–346. https://doi.org/10.1177/09637214221096485

Budescu, D. V., & Budescu, M. (2012). How to measure diversity when you must. Psychological Methods, 17(2), 215–227. https://doi.org/10.1037/a0027129

Bursik, R. J. (1988). Social disorganization and theories of crime and delinquency: Problems and prospects. Criminology, 26(4), 519–552. https://doi.org/10.1111/j.1745-9125.1988.tb00854.x

*Bursik, R. J. (1999). The Informal Control of Crime Through Neighborhood Networks. Sociological Focus, 32(1), 85–97. https://doi.org/10.1080/00380237.1999.10571125

Burt, C. (2020). Doing better science: Improving review & publication protocols to enhance the quality of criminological evidence. The Criminologist, 45(4), 1–6.

Chin, J. M., Pickett, J. T., Vazire, S., & Holcombe, A. O. (2023). Questionable Research Practices and Open Science in Quantitative Criminology. Journal of Quantitative Criminology, 39(1), 21–51. https://doi.org/10.1007/s10940-021-09525-6

Cooley, C. H. (2003). Social organization: A study of the larger mind (3. printing). Transaction Publ. (Original work published 1909)

Dirk Enzmann. (2015). DIVCAT: Stata module to calculate five measures of diversity for multiple categories [Stata]. https://EconPapers.repec.org/RePEc:boc:bocode:s457956

Dooley, B. D., & Goodison, S. E. (2020). Falsification by Atrophy: The Kuhnian Process of Rejecting Theory in US Criminology. The British Journal of Criminology, 60(1), 24–44. https://doi.org/10.1093/bjc/azz026

Elliot, D. S. (1985). The Assumption that Theories Can Be Combined with Increased Explanatory Power: Theoretical Integrations. In R. F. Meier (Ed.), Theoretical Methods in Criminology (pp. 123–149). Sage Publications.

Elliot, D. S., Ageton, S. S., & Canter, R. J. (1979). An Integrated Perspective on Delinquent Behavior. Journal of Research in Crime and Delinquency, 16(3), 3–27.

Elliot, D. S., Huizinga, D., & Ageton, S. S. (1985). Explaining Delinquency and Drug Use. Sage Publications.

Forstmeier, W., Wagenmakers, E., & Parker, T. H. (2017). Detecting and avoiding likely false‐positive findings – a practical guide. Biological Reviews, 92(4), 1941–1968. https://doi.org/10.1111/brv.12315

Fried, E. I. (2020). Lack of theory building and testing impedes progress in the factor and network literature. Psychological Inquiry, 31(4), 271–288. https://doi.org/10.1080/1047840X.2020.1853461

Gibbs, J. P. (1972). Sociological Theory Construction. Dryden Press.

Gibbs, J. P. (1985). The Methodology of Theory Construction in Criminology. In R. F. Meier (Ed.), Theoretical Methods in Criminology (pp. 23–50). Sage Publication.

Hirschi, T. (1979). Separate and Unequal Is Better. Journal of Research in Crime and Delinquency, 16(1), 34–38. https://doi.org/10.1177/002242787901600104

Hirschi, T. (1989). Exploring Alternatives to Integrated Theory. In S. F. Messner, Marvin D. Krohn, & Allen Liska (Eds.), Theoretical Integration in the Study of Deviance and Crime (pp. 37–49). State University of New York Press.

Ioannidis, J. P. A. (2005). Why Most Published Research Findings Are False. PLoS Medicine, 2(8), e124. https://doi.org/10.1371/journal.pmed.0020124

Klein, R., Ratliff, K., Vianello, M., Adams, R., Bahník, Š., Bernstein, M., Bocian, K., Brandt, M., Brooks, B., Brumbaugh, C., Cemalcilar, Z., Chandler, J., Cheong, W., Davis, W., Devos, T., Eisner, M., Frankowska, N., Furrow, D., Galliani, E., … Nosek, B. (2022). Investigating Variation in Replicability: A “Many Labs” Replication Project. https://doi.org/10.17605/OSF.IO/WX7CK

*Kubrin, C. E., & Weitzer, R. (2003). Retaliatory Homicide: Concentrated Disadvantage and Neighborhood Culture. Social Problems, 50(2), 157–180. https://doi.org/10.1525/sp.2003.50.2.157

*Lee, M. R. (2000). Community Cohesion and Violent Predatory Victimization: A Theoretical Extension and Cross-National Test of Opportunity Theory. Social Forces, 79(2), 683. https://doi.org/10.2307/2675513

Loken, E., & Gelman, A. (2017). Measurement error and the replication crisis. Science, 355(6325), 584–585. https://doi.org/10.1126/science.aal3618

Maxwell, S. E., Lau, M. Y., & Howard, G. S. (2015). Is psychology suffering from a replication crisis? What does “failure to replicate” really mean? American Psychologist, 70(6), 487–498. https://doi.org/10.1037/a0039400

Meehl, P. E. (1990). Why Summaries of Research on Psychological Theories are Often Uninterpretable. Psychological Reports, 66(1), 195–244. https://doi.org/10.2466/pr0.1990.66.1.195

*Messner, S. F., & Blau, J. R. (1987). Routine Leisure Activities and Rates of Crime: A Macro-Level Analysis. Social Forces, 65(4), 1035. https://doi.org/10.2307/2579022

*Miethe, T. D., & McDowall, D. (1993). Contextual Effects in Models of Criminal Victimization. Social Forces, 71(3), 741. https://doi.org/10.2307/2579893

*Moore, M. D., & Sween, M. (2015). Rural youth crime: A reexamination of social disorganization theory’s applicability to rural areas. Journal of Juvenile Justice, 4(1), 47–63.

Morgan, H. (2022). Conducting a Qualitative Document Analysis. The Qualitative Report. https://doi.org/10.46743/2160-3715/2022.5044

Navarro, D. J. (2021). If mathematical psychology did not exist we might need to invent it: A comment on theory building in psychology. Perspectives on Psychological Science, 16(4), 707–716. https://doi.org/10.1177/1745691620974769

Niemeyer, R. E., Proctor, K. R., Schwartz, J. A., & Niemeyer, R. G. (2022). Are Most Published Criminological Research Findings Wrong? Taking Stock of Criminological Research Using a Bayesian Simulation Approach. International Journal of Offender Therapy and Comparative Criminology, 0306624X2211329. https://doi.org/10.1177/0306624X221132997

Oberauer, K., & Lewandowsky, S. (2019). Addressing the theory crisis in psychology. Psychonomic Bulletin & Review, 26(5), 1596–1618. https://doi.org/10.3758/s13423-019-01645-2

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. https://doi.org/10.1126/science.aac4716

Pridemore, W. A., Makel, M. C., & Plucker, J. A. (2018). Replication in Criminology and the Social Sciences. Annual Review of Criminology, 1(1), 19–38. https://doi.org/10.1146/annurev-criminol-032317-091849

Proctor, K. R. (2010). Social learning, social control, and strain theories: A formalization of micro-level criminological theories [PhD Thesis, UC Riverside]. https://escholarship.org/uc/item/20p8s027

Proctor, K. R., & Niemeyer, R. E. (2019). Mechanistic criminology (1 Edition). Routledge, Taylor and Francis Group.

*Rice, K. J., & Csmith, W. R. (2002). Socioecological Models of Automotive Theft: Integrating Routine Activity and Social Disorganization Approaches. Journal of Research in Crime and Delinquency, 39(3), 304–336. https://doi.org/10.1177/002242780203900303

Robinaugh, D. J., Haslbeck, J. M. B., Ryan, O., Fried, E. I., & Waldorp, L. J. (2021). Invisible hands and fine calipers: A call to use formal theory as a toolkit for theory construction. Perspectives on Psychological Science, 16(4), 725–743. https://doi.org/10.1177/1745691620974697

*Roh, S., & Choo, T. M. (2008). Looking inside Zone V: Testing social disorganization theory in suburban areas. W. Criminology Rev., 9, 1.

Rooij, I. van. (2019, January 18). Psychological science needs theory development before preregistration. Psychonomic Society Featured Content. https://featuredcontent.psychonomic.org/psychological-science-needs-theory-development-before-preregistration/

*Sampson, R. J., & Groves, W. B. (1989). Community Structure and Crime: Testing Social-Disorganization Theory. American Journal of Sociology, 94(4), 774–802. https://doi.org/10.1086/229068

*Sampson, R. J., & Wooldredge, J. D. (1987). Linking the micro- and macro-level dimensions of lifestyle-routine activity and opportunity models of predatory victimization. Journal of Quantitative Criminology, 3(4), 371–393. https://doi.org/10.1007/BF01066837

Scheel, A. M., Tiokhin, L., Isager, P. M., & Lakens, D. (2021). Why hypothesis testers should spend less time testing hypotheses. Perspectives on Psychological Science, 16(4), 744–755. https://doi.org/10.1177/1745691620966795

Schumm, W. R., Crawford, D. W., Lockett, L., AlRashed, A., & Bin Ateeq, A. (2023). Research anomalies in criminology: How serious? How extensive over time? And who was responsible? Accountability in Research, 0(0), 1–37. https://doi.org/10.1080/08989621.2023.2241127

Shaw, C. R., & McKay, H. D. (1972). Juvenile delinquency and urban areas: A study of rates of delinquency in relation to differential characteristics of local communities in American cities (Rev. ed., 2. impr). Univ. of Chicago Press. (Original work published 1942)

Short, J. (1972). Introduction to the Revised Edition. In C. R. Shaw & H. D. McKay (Eds.), Juvenile Delinquency and Urban Areas. University of Chicago Press.

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-Positive Psychology: Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Smaldino, P. (2019). Better methods can’t make up for mediocre theory. Nature, 575(7783), 9–10. https://doi.org/10.1038/d41586-019-03350-5

Sotola, L. K. (2022). How Can I Study from Below, that which Is Above? Comparing Replicability Estimated by Z-Curve to Real Large-Scale Replication Attempts [Preprint]. PsyArXiv. https://doi.org/10.31234/osf.io/qt34k

StataCorp. (2017). Stata Statistical Software: Release 15.1 [Computer software]. StataCorp LLC.

*Stewart, E. A., Schreck, C. J., & Simons, R. L. (2006). “I Ain’t Gonna Let No One Disrespect Me”: Does the Code of the Street Reduce or Increase Violent Victimization among African American Adolescents? Journal of Research in Crime and Delinquency, 43(4), 427–458. https://doi.org/10.1177/0022427806292338

*Stewart, E. A., & Simons, R. L. (2006). Structure and Culture in African American Adolescent Violence: A Partial Test of the “Code of the Street” Thesis. Justice Quarterly, 23(1), 1–33. https://doi.org/10.1080/07418820600552378

Sweeten, G. (2020). Standard Errors in Quantitative Criminology: Taking Stock and Looking Forward. Journal of Quantitative Criminology, 36(2), 263–272. https://doi.org/10.1007/s10940-020-09463-9

Thomas, W. I., & Znaniecki, F. (1966). Social Personality: Organization of Attitudes. In W. I. Thomas On Social Organization and Social Personality (pp. 11–36). University of Chicago Press.

Turner, S. P., & Turner, J. H. (1990). The impossible science: An institutional analysis of American sociology. Sage Publications.

van Rooij, I., & Baggio, G. (2021). Theory before the test: How to build high-verisimilitude explanatory theories in psychological science. Perspectives on Psychological Science, 16(4), 682–697. https://doi.org/10.1177/1745691620970604

VERBI Software. (2023). MAXQDA 24 [Mac OS 14.2.1]. VERBI Software. maxqda.com

*Warner, B. D., & Rountree, P. W. (1997). Local Social Ties in a Community and Crime Model: Questioning the Systemic Nature of Informal Social Control. Social Problems, 44(4), 520–536. https://doi.org/10.2307/3097221

Wooditch, A., Fisher, R., Wu, X., & Johnson, N. J. (2020). p-value Problems? An Examination of Evidential Value in Criminology. Journal of Quantitative Criminology, 36(2), 305–328. https://doi.org/10.1007/s10940-020-09459-5

*Denotes study included in our pilot study.