Skip to main content
SearchLoginLogin or Signup

Understanding Iatrogenic Effects for Evidence-Based Policy: A Review of Crime and Violence Prevention Programs

This paper has two main objectives. The first is to advance primary research on intervention studies that report iatrogenic effects. In short, we need to know a great deal more about why interventions cause harm. The second objective is to make the case that a body of ...

Published onNov 01, 2020
Understanding Iatrogenic Effects for Evidence-Based Policy: A Review of Crime and Violence Prevention Programs
·

Abstract

This paper has two main objectives. The first is to advance primary research on intervention studies that report iatrogenic effects. In short, we need to know a great deal more about why interventions cause harm. The second objective is to make the case that a body of knowledge on iatrogenic effects can play an important role in informing evidence-based policy. We view this knowledge as another key input in the policymaking process. Some primary research has been conducted on interventions with iatrogenic effects, and several studies with crime and violence outcomes are reviewed. Building on a number of important advances from this work, as well as from prior reviews on the subject, a research agenda is proposed to help enrich our understanding of interventions that cause harm and contribute to evidence-based policy in the area of crime and violence prevention. Directions for future research include carrying out longer follow-up assessments of interventions with iatrogenic effects and investigating the role of theory failure and implementation failure as potential explanations. In the same way that immediate action is needed to end interventions that cause harm, using the knowledge base on iatrogenic effects has the potential to provide policymakers with an important tool to help avoid causing harm in the first instance.

Keywords: Iatrogenic effects; Crime and violence prevention; Evidence-based policy; Theory failure; Implementation failure

1. Introduction

An evidence-based approach is premised on the idea that public policy should be rational and based on the best possible research evidence. With many other considerations and competing interests at stake in the development of public policy, an evidence-based approach is in many ways about making sure that research is at center stage in the decision-making process. As noted by Petrosino (2000, p. 636), “An evidence-based approach requires that the results of rigorous evaluation be rationally integrated into decisions about interventions by policymakers and practitioners alike.”

In the context of implementing evidence-based programs (Fagan, 2017; Fixsen et al., 2013), scaling up evidence-based programs for wider public use (Dodge, 2001; Fagan et al., 2019), or promoting these programs through federal and state registries (Fagan and Buchanan, 2016), intervention studies reporting iatrogenic effects are avoided at all costs. At face value this makes good sense. These are interventions for which the treatment group participants would have been better off had they not received anything. They are damaging on other fronts, including financial costs and erosion of public and government support.

There is also concern that bias exists in the reporting of iatrogenic effects of intervention studies. This can occur when studies or the findings of studies are buried altogether—the well-known ‘file-in-the-drawer’ problem (Rosenthal, 1979). This can also occur as a result of publication bias, which holds that journals are more likely to publish studies that report desirable intervention effects (Wilson, 2009). Eisner’s (2009) study of conflict of interest in the reporting of effects from intervention trials draws attention to the under-reporting of iatrogenic effects. This set of circumstances may in fact contribute to the limited knowledge on iatrogenic effects of interventions.

Some primary research has been conducted on crime and violence prevention programs with iatrogenic effects. Generally speaking, this has taken the form of analyses of data to investigate causal mechanisms or potential explanations related to iatrogenic effects or longer follow-up assessments to examine whether iatrogenic effects persist or change over time (or over different stages of the life-course). For example, researchers from the Oregon Social Learning Center used the latter approach in their effort to better understand the negative influence of peers on adolescent externalizing behavior problems and delinquency in the context of a group-based preventive intervention (Dishion et al., 1999; Poulin et al., 2001). In the Cambridge-Somerville Youth Study, a delinquency prevention experiment begun in 1939, a 72-year post-intervention follow-up assessment was recently completed to investigate whether iatrogenic effects on mortality observed in middle age persisted or changed in old age (Welsh et al., 2019).

The present paper draws inspiration from and attempts to build upon a number of reviews that have called for greater attention to interventions that cause harm (Lilienfeld, 2007; McCord, 2003; Rhule, 2005; Rubenson et al., 2020; Welsh and Rocque, 2014; Werch and Owen, 2002; Zane et al., 2016). Collectively, these reviews are also important because they made some progress toward three important goals: (a) estimating the prevalence of iatrogenic effects among some areas of intervention research; (b) identifying explanations for why interventions cause harm; (c) and clarifying some directions for future research and policy development. It is important to note that the review by Rubenson et al. (2020) also makes an important contribution by distinguishing between adverse effects that cause genuine harm to participants or are a function of “flawed or inappropriate evaluation methods” (p. 1).

Work on these fronts needs to continue. It is our view, however, that there are two intersecting points that can make this work even more formidable. The first is to promote primary research on interventions that report iatrogenic effects. As noted above, this can involve analyses of outcome and implementation data, as well as longer term follow-up assessments. The second point is to make clear that this is being done in the service of advancing evidence-based policy; that is, like the public benefit from using effective and worthwhile interventions, there is an equally pressing public benefit to ending the use of interventions that cause harm.

In this paper we argue that a program of research designed to study interventions with iatrogenic effects has the capacity to serve as another key input in the larger body of knowledge upon which evidence-based policy is developed. Key to this would involve examining the logic model or theoretical basis of interventions, implementation factors, key characteristics of interventions (e.g., dosage, delivery format), and methodological features related to evaluation (e.g., intent-to-treat analysis, differential attrition).

The paper has a special focus on the prevention of violent crime and crime in general. It reviews studies of preventive interventions with outcome measures of criminal and violent offending as well as criminal justice interventions. It is important to state in unequivocal terms that any intervention that causes harm needs to be shut down immediately. Our argument for continued study of interventions with iatrogenic effects begins after interventions have ceased operating and only if new research would not cause any further harm to study participants.

Section 2 provides some background on evidence-based policy. Section 3 describes the methodology used to review studies, and section 4 reports on crime and violence prevention studies with iatrogenic effects and longer follow-up assessments. Section 5 brings together some directions for future research to help enrich our understanding of interventions that cause harm and contribute to evidence-based policy. Section 6 offers some concluding remarks.

2. Evidence-Based Policy

Evidence-based policy has been defined as comprising two key components: “the application of rigorous research methods, particularly randomized controlled trials … to build credible evidence about ‘what works’ to improve the human condition; and the use of such evidence to focus public and private resources on programs, practices, and treatments (‘interventions’) shown to be effective” (Baron, 2018, p. 40). Although it may be implied in Baron’s (2018) definition of evidence-based policy, we would add a third key component: a measure of external validity. This often involves carrying out one or more replication experiments or effectiveness trials (Elliott and Mihalic, 2004; Fagan et al., 2019; see also Gottfredson et al., 2015). This could also involve carrying out a systematic review (with meta-analytic techniques) to assess the available research evidence on the intervention in question—using only the highest quality evaluation studies (as promoted by the Cochrane Collaboration and Campbell Collaboration; Chalmers, 2003).

The process of making evidence-based policy, or evidence-based policymaking, is in reality far more convoluted. One only need consult the research utilization literature, especially Carol Weiss’s pioneering work on pathways or routes for how evaluation research can exert influence on policy decisions: conceptual, instrumental, political, and imposed use (Weiss, 1998; Weiss et al., 2008). Of these four routes of policy influence, it is her latest coinage—imposed use—that holds the most relevance to evidence-based policy. Here, there is often a mandate of some sort that decision-makers must use the best available research evidence (e.g., from a government registry; see Fagan and Buchanan, 2016). Put another way, “Imposed use is not a brand-new concept but one that may occur in any field where a higher level of government with funds to disperse demands specific action on lower operating levels, based on evidence” (Weiss et al., 2005, p. 25).

Whether in the context of imposed use or under some other condition, evidence-based policymaking can be thought of as a result of a collection of inputs. These inputs include the strength of the research evidence of effectiveness of the intervention and the availability of financial resources (along the lines of Baron’s definition), in addition to government/political priorities of the day (i.e., not all priorities are given the same attention or even acted upon) and research evidence of economic efficiency of the intervention (Mears, 2007; 2010). As noted above, it is the hope of the evidence-based movement that research—rather than guesswork or political whims—will be at center stage in the policymaking process (Haskins, 2018; Sherman, 2003).

It is our view that another important input in evidence-based policymaking should be research on interventions with iatrogenic effects. Here, the key question is: What are the lessons that can be drawn from interventions with iatrogenic effects to aid in the development of evidence-based policy? This is more than the consideration of studies that report iatrogenic effects in, for example, an intervention trial or a systematic review of the intervention in question. This is the first step, and it is an important one because of the long-standing concern with non-reporting of studies that show harmful effects.

The next step involves the need to take into consideration the accumulated body of research on iatrogenic effects, drawn from a wide array of interventions and over time. Beyond the specific nature of the intervention, this body of research could provide insights on a wide range of important moderating variables, including key intervention characteristics (e.g., dosage, delivery format—individual- or group-based), implementation factors, setting, and even theoretical orientation. The idea here is that a general knowledge base could provide guidance for a specific intervention. One noteworthy example is deviant peer influence or deviancy training. This mechanism is especially relevant to group-based interventions that mix high-rate or chronic delinquents with more minor delinquent youths (Dishion et al., 1999; Dodge et al., 2005), and it is the focus of several studies profiled in section 4.

3. Methods

We carried out a review of the literature on crime and violence prevention studies that have (a) reported iatrogenic effects and (b) conducted longer follow-up assessments. Compared to the more rigorous systematic review method, the narrative review method is well suited to the aims of the paper. Based on a recent review of systematic reviews to identify crime and violence prevention programs with iatrogenic effects (Welsh and Rocque, 2014), it was known that few of these studies have conducted longer follow-up assessments. Moreover, it was important to be able to describe the rich details of the individual studies.

In addition to the above criteria for inclusion of studies, there were no restrictions placed on the location, year of reporting, or language of the studies. Both published and unpublished studies were eligible for consideration. This was especially important in an effort to try to address concerns about publication bias (Wilson, 2009). Two main search strategies were used to locate studies meeting the inclusion criteria: searches of electronic bibliographic databases (e.g., Criminal Justice Abstracts, National Criminal Justice Reference Service Abstracts, Google Scholar), and forward-citation searches of studies with iatrogenic effects. Because of the highly specific nature of our searches and limited number of studies identified, it was possible to screen each study by reviewing the full text.

4. Intervention Studies with Iatrogenic Effects

This section reports on the four studies that were identified from a review of the literature. Its purpose is to describe the nature of the different interventions, examine developmental or social processes for the reported iatrogenic effects, and discuss how continued research on the studies can inform public policy. This is presented in both tabular and narrative form. Table 1 summarizes key information on the studies, and the following provides additional details on and context to the studies. It is important to note that these four studies do not represent a true sub-group of studies that have reported iatrogenic effects. As discussed in the Methods section, this is a highly specialized sample of studies, especially owing to their longer follow-ups.

[Table 1 about here]

4.1 Adolescent Transitions Program

Based on an ecological model of antisocial behavior (Patterson et al., 1992), the Adolescent Transitions Program (ATP) focused on parenting and peer developmental processes through a multi-modal preventive intervention (Dishion and Andrews, 1995). The first component, the parent focus, targeted family management practices and communication skills. The second component, referred to as teen focus, involved developing prosocial goals and self-regulation while using peer reinforcement as a means to promote completion of various exercises and session activities. One hundred and nineteen high-risk youth (mean age = 12 years) and their families were randomly assigned to one of four conditions: parent focus only; teen focus only; combined parent and teen focus; or no-treatment control. The intervention was delivered in a group format and involved 12 weekly 90-minute sessions completed over a period of approximately 3 to 4 months.

The researchers hypothesized that the optimal intervention condition would be the combined one. Initially, this seemed to be the case. Dishion and Andrews (1995) found that the combined treatment condition resulted in reliable reductions in observed negative family interactions in comparison to the control condition immediately after termination of the intervention. However, one year following the families’ involvement in ATP, researchers found increased tobacco use and, according to teacher reports, increased prevalence of externalizing behavior (i.e., aggressive acts) among the teen focus condition and the combined parent and teen condition compared to the control condition (Dishion and Andrews, 1995). A subsequent three-year follow-up reported the persistence of iatrogenic effects among the teen focus condition compared to the control group, including escalations in self-reported smoking and teacher-reported delinquency (i.e., property and violent offenses) (Dishion et al., 2001).

Similar to the Cambridge-Somerville Youth Study (see below), peer contagion was posited as the strongest explanation for ATP’s iatrogenic effects (Poulin et al., 2001). Specifically, the researchers emphasized friendship formation, which shows how deviant talk embedded in positive affect can define a mechanism of “deviancy training” and may result in escalations of negative behaviors (see also Poulin et al., 1999). In addition, Dishion et al. (1999) offered two mechanisms to help explain the converging evidence from prevention and developmental science on peer influence. First, youth that are reinforced for deviant behavior through laughter, social attention, and interest are likely to escalate their behavior. Second, high-risk adolescents derive meaning from the deviancy training process and this process provides the cognitive basis for motivation to commit future delinquent acts.

4.2 Buddy System

Developed in the early 1970s, the Buddy System is a youth mentorship preventive intervention targeted at delinquency. The intervention consisted of 533 males and females, ages 11 to 17 years (mean age = 13.5), who lived in one lower-income area of Hawaii (O’Donnell and Williams, 2013). Participants were randomly assigned to receive the intervention or to a no-treatment control group. The intervention was based on a triadic model derived from the work of Tharp and Wetzel (1969). In this model, professional consultants supervised mentors, also known as “mediators” or “buddies,” who were often teachers or parents with preexisting relationships to adolescents considered high risk (O’Donnell, 1992). The mentors were trained by the consultants to utilize behavior modification techniques, and mentors received small monthly allowances for activities with the youth.

The intervention lasted for three years. An evaluation at the completion of the intervention found that participants in the treatment group had lower arrest rates than the control group, but only if they had an arrest in the year prior to referral (O’Donnell et al., 1979). For those without any criminal history, the treatment group participants had significantly higher arrest rates than those in the control group (22.5% vs. 16.4%). The authors noted that, “The results raise the specter of possible iatrogenic treatment effects of the Buddy System approach with youngsters with no record of prior major offenses” (Fo and O’Donnell, 1975, p. 524). These results were interpreted as a “peer network effect, created during the many opportunities for youth to meet one another and form friendships during participation” (O’Donnell and Williams, 2013, p. 55). This explanation was further substantiated by all of the youth (treatment and control groups) being recruited from the same local area.

O’Donnell and Williams (2013) conducted a 35-year follow-up to assess whether the iatrogenic effects persisted in adulthood. Among participants with an arrest in the year prior to program referral, it was found that the positive effects on criminal activity persisted in adulthood for the treatment group compared to the control group. Among participants with no arrest record before program referral, it was found that the negative effects on criminal activity persisted in adulthood, but only for female participants in the treatment group compared with their control counterparts (29.2% vs. 10.3%). Nearly two-thirds of all arrests (65%) were for serious offenses (felonies), with men accounting for a higher proportion of violent offenses (52.3% vs. 33.3%) and women accounting for a higher proportion of drug-related offenses (28.1% vs. 12.8%) (O’Donnell and Williams, 2013, p. 58).

The researchers emphasized the effects of participants’ peer networks and the impact of relationships on behavior to explain both the positive and negative program effects and their persistence over time. Explicitly, they suggested that participation in the Buddy System may have altered the peer network of some of the adolescents, which in turn impacted the trajectory of their early adult life, particularly in terms of their relationships (O’Donnell and Williams, 2013). It was also noted that this altered trajectory was consistent with activity theory (Vygotsky, 1981), which holds that relationships are formed in settings with people who engage in activities together. These shared experiences are intersubjective and in turn affect the cognition, emotion, and subsequent behavior of the people in each shared setting (O’Donnell et al., 1993). For high-risk youth, this could help explain the beneficial effect of the Buddy System, whereby contact with lower-risk youth may have introduced those participants to more conventional activities. These same processes may have exposed lower-risk adolescent females to criminal activities, where friendships that some females formed with arrested participants in the intervention facilitated a trajectory of relationships that supported criminal behavior (O’Donnell and Williams, 2013).

4.3 Cambridge-Somerville Youth Study

The Cambridge-Somerville Youth Study (CSYS) is a delinquency prevention experiment that began in 1939. Six hundred fifty underprivileged boys of average and difficult temperament (later reduced to 506), ages 5-13 years (median = 10.5), from Cambridge and Somerville, Massachusetts, were placed in matched pairs and one member of each pair was randomly assigned to the treatment group. Referred to as “directed friendship,” the preventive intervention involved individual counseling through a range of activities and home visits with the families. Counselors talked to the boys, took them on trips and to recreational facilities, tutored them in reading and arithmetic, encouraged them to participate in the YMCA and in summer camps, and encouraged them to attend church. The program lasted a mean average of 5.5 years, ending in 1945. The control group received no special services (Powers and Witmer, 1951).

In a 30-year post-intervention follow-up assessment, which included 94.9% of the participants (480 of 506), McCord (1978) reported iatrogenic effects of the intervention. Comparisons between the treatment and control groups indicated that the treatment men had not fared better on any measured outcome, and actually fared worse on seven outcomes. The treatment group men were significantly more likely to: commit more than one crime (as measured by convictions); suffer symptoms of alcoholism; manifest signs of mental illness; die at a younger age; suffer from at least one stress-related disorder, especially high blood pressure or heart trouble; have occupations with lower prestige; and report their work as unsatisfying (McCord, 1978). Just under one-fifth of all convictions were for violent offenses.

In later years, McCord (2003; see also Dishion et al., 1999) proposed a “peer deviancy” hypothesis, observing that peer contagion among treatment group youth who had attended summer camps appeared to explain much of the iatrogenic effects. According to McCord (2003), these camps likely allowed for a great deal of unstructured socializing, representing an ideal environment for deviancy training to take place (see Gottfredson, 2010). Analyzing the 30-year follow-up data, McCord (2003) found that for boys who attended summer camp only once (n=59), the odds ratio (OR) predicting undesirable outcomes was 1.33, which was significantly higher than the OR of 1.12 among treatment group boys who did not attend summer camp. For boys who attended camp more than once (n=66), the OR for undesirable outcomes was 10.0, meaning that participants were ten times as likely to experience undesirable outcomes as their matched mates. McCord (2002, p. 235) concluded: “I strongly suspect that the boys from the Youth Study tended to bond together, encouraging one another’s deviant values.” McCord’s construct theory of motivation—which argues that youth respond to situational cues in constructing motivation based on the way they perceive the choices of others—provided a theoretical explanation for why deviancy training takes place among high-risk youth in largely unsupervised settings (McCord, 2003; 2004).

In a 72-year post-intervention follow-up assessment of the CSYS, Welsh et al. (2019) investigated whether the iatrogenic effects on mortality observed in middle age persisted or changed in old age. Records were located for 96.4% of the participants (488 of 506), with a total of 446 participants confirmed deceased (88.1%) and 42 alive (8.3%). Matched-pairs analyses showed no significant differences for all outcomes of interest: mortality at latest follow-up; premature mortality (younger than 40 years); and cause of mortality (natural versus unnatural). A Cox proportional hazard regression analysis indicated no difference in time to death between the treatment and control group men. In not being able to detect iatrogenic effects on mortality, the main implication is that the iatrogenic effects on mortality experienced in middle age did not persist in old age (i.e., up to age 90).

An explanation for this change in intervention effects over time was not entirely clear. One view is that the observed iatrogenic effects on mortality in middle age “may have been a singular event, irrespective of their concordance with effects for a wide range of other outcomes” (Welsh et al., 2019, p. 8). In commenting on these findings, Farrington and Hawkins (2019) suggest that the earlier iatrogenic effects on mortality may have been rather weak, as indicated by tests of statistical significance and the magnitude of the effect. The next wave of this follow-up, which is focused on criminal offending over the full life-course, may provide further insight on an explanation for the study’s latest findings on mortality.

4.4 Milwaukee Domestic Violence Experiment

From 1987 to 1988, the Milwaukee Police Department assigned 36 police officers to handle all domestic disturbance calls during evening shifts. Upon dispatch to these calls, suspects were randomly assigned to receive one of three sanctions: “short arrest,” “long arrest,” or a warning. The arrested suspects were usually handcuffed and taken to the police station for 3-12 hours, with short arrests lasting a mean of 4.5 hours and long arrests lasting a mean of 11.1 hours after booking. Suspects who were warned were left at the scene.

An initial deterrent effect was found for both the short and long arrest conditions in comparison to the warning condition for up to 30 days (Sherman et al., 1992a). In a 1-year follow-up it was found that the short arrests and short and long arrests combined had produced an escalation effect, meaning that offenders in these conditions were more likely to have committed subsequent violence against their female partners. It was also found that different offenders reacted differently to arrest, with evidence of a specific deterrent effect of arrest among employed, but not unemployed suspects on subsequent violence toward their female partners. Specifically, suspects who were white, employed, married, and high school graduates were less likely to have an incident of repeat violence if they were arrested. In contrast, suspects who were African-American, unemployed, unmarried, and high school dropouts were more likely to have an incident of repeated violence if they had been arrested (Sherman et al., 1992a).

Sherman and colleagues (1992b, p. 681) hypothesized that suspects subjected to social control in jobs and marriages were more likely to be deterred by legal sanctions than those “without such stakes in conformity.”  Important to this explanation was consideration of interactions between legal and informal controls rather than viewing them in isolation. It was noted that these results support the conditional hypothesis of deterrence, which suggests that legal threats only deter offenders who are tied to conventional institutions and would have much to lose from the stigmatization of arrest (Sherman et al., 1992b).

In a 23-year follow-up, Sherman and Harris (2013) investigated the effects of arrest on the mortality rates of suspects. Although marginally significant, the authors found that suspects who received the sanction of arrest, compared to the warning condition, were almost three times as likely to have died of homicide. There was some support for a stake-in-conformity explanation for this finding. Specifically, ties to conventional society can also protect against provocative behavior with peers that could precipitate a homicide (Sherman and Harris, 2013).

Additionally, Sherman and Harris (2015) investigated the long-term effects of the intervention on victim mortality, finding that victims whose perpetrators were arrested had significantly higher death rates from all causes compared to victims whose perpetrators were warned. It was also found that African-American victims of domestic violence were disproportionately more likely to die after partner arrests relative to white victims.

5. A Research Agenda

In a review of potentially harmful treatments in psychotherapy, Lilienfeld (2007) makes a series of recommendations for future research, including a greater focus on “therapist variables” and “client variables” to unpack why therapies cause harm. To the first point, Lilienfeld (2007, p. 64) suggests that “potentially harmful therapists may ultimately prove to be even more important than the identification of potentially harmful therapies.” On the second point, he posits that client differences are essentially moderators of treatment effects, since the interaction between therapist and client cannot be assumed to be uniform. We think these insights have direct analogues in evidence-based policy, where factors relating to program implementation (i.e., therapist variables) and program participants (i.e., client variables) can influence the success of the intervention. In efficacy trials as part of the process of scaling up interventions, participants are under highly controlled conditions often directed by the developers of the intervention. Here, intervention effects are less likely to be moderated by these implementation characteristics. If iatrogenic effects are nonetheless discovered under these ideal conditions, there is good reason to believe that it may involve some fundamental issue with the underlying intervention (i.e., theory failure).

Theory failure represents an important concept in evidence-based policy because it highlights the need for a solid theoretical foundation on which to base any intervention (Rossi et al., 2004). Theory failure implies that the “basic idea or mechanism of prevention was unsound” (Ekblom and Pease, 1995, p. 594). Where an efficacy trial demonstrates iatrogenic effects, theory failure is a plausible explanation. This requires asking a series of questions, such as: What was the underlying theory of change in this program’s logic model? Is there a sound theoretical basis for this theory of change?

For example, in the context of crime and violence prevention, theory failure may be the best explanation for the iatrogenic effects associated with the Scared Straight program (McCord, 2003; Petrosino et al., 2003). Here, the underlying theory is that at-risk youth could be dissuaded from future offending via exposure to serious, adult criminals. If Scared Straight turned out to be ineffective, this would provide reason to doubt this theory moving forward. Iatrogenic effects tell us something more. Not only did the program fail to deter, it also appeared to inspire these youth to escalate their delinquent behaviors. The lesson becomes: Do not expose at-risk youth to serious, adult criminals—for this may create a “delinquency fulfilling prophecy” (Finckenauer, 1982, p. 169).

When we move beyond the efficacy trial, things become more complicated. In part, this is because both the delivery of the intervention and the population that is subject to the intervention could moderate the intervention and alter average effects—possibly even producing iatrogenic effects in what appeared to be an efficacious program. As Gottfredson and colleagues (2015, p. 895) observe, “the emphasis in Prevention Science has shifted more toward understanding how these EBIs [evidence-based interventions] can be implemented on a broader scale to produce larger impacts on entire populations.” In other words, the crucial component is now perceived as the translation of effective interventions into scaled-up, real-world conditions, and scholars have called for research that specifically examines the outcomes of these efforts (see Spoth et al., 2013).

This translation thus shifts the explanation of iatrogenic effects from theory failure to implementation failure. As Leadbeater and colleagues (2018, p. 859) suggest, “Ethical issues can arise in the tensions between the need for fidelity in implementation quality and real-world practice. For example, communities may not have the resources to scale-up an intervention with fidelity, and the impact of the community’s efforts may be negligible at best, or iatrogenic at worse, because of low implementation quality.” This could be due to dosage effects, poorly trained staff, or factors exogenous to the actual implementation. For example, deviancy training is often a plausible explanation for why a theoretically sound, group-based intervention causes harm to participants (e.g., Dodge et al., 2005). Here, peer contagion represents an unintended consequence of the implementation scheme, where negative peer effects are more powerful than the hypothesized positive intervention effects, causing the average effect to become null or even iatrogenic. As with the possibility of theory failure above, primary research on interventions with iatrogenic effects is needed to examine implementation failure.

This draws attention to an especially pertinent question: “Does the prevention scientist, who is also a program developer and purveyor of intervention, have a responsibility to ensure that community organizations monitor the implementation to ensure that it is delivered as intended to enhance benefits and reduce negative or iatrogenic effects?” (Leadbeater et al., 2018, p. 860). With an analogy to criminal law, Blackstone’s (2016) ratio posits that it is better for ten guilty men to go free than for one innocent to be convicted. We might similarly suggest that interventions that cause harm should be avoided, even at the cost of interventions not being scaled-up for wider public use until it can be determined that they are safe. For example, if an effective intervention produces iatrogenic effects upon wider implementation, this is not just regrettable but deeply problematic. The intervention should be abandoned at once, and efforts should be taken to investigate why the intervention caused harm. This approach should be adopted for any intervention with iatrogenic effects. In some cases, this might involve longer follow-up assessments to investigate whether the iatrogenic effects persist or change (e.g., O’Donnell and Williams, 2013; Sherman and Harris, 2013). In other cases where there exists a sizeable number of studies showing iatrogenic effects for a specific intervention (e.g., Scared Straight, military-style boot camps; see Welsh and Rocque, 2014), longer follow-up assessments may not be needed.

Identifying effective interventions is not sufficient for evidence-based policy moving forward; we must also draw attention to interventions with iatrogenic effects and try to understand what caused the harm. While it may be more newsworthy to show that an intervention “works” than to explain why it does not, we have argued that the latter is an ethical imperative for prevention research. The Society for Prevention Research refers to the reporting of iatrogenic effects in efficacy and effectiveness trials as a “desirable standard,” but we suggest that this should be upgraded to “standard” practice (Gottfredson et al., 2015). One of the goals of doing so would be to motivate primary research on interventions with iatrogenic effects.

In the medical sciences, the Consolidated Standards of Reporting Trials (CONSORT) statement has aimed at keeping up to date reporting guidelines that include minimum standards for describing the methods and results of intervention trials (Schulz et al., 2010). This statement makes unbiased reporting of results in medical trials an ethical standard and has recently been adapted to the social and behavioral sciences (Montgomery et al., 2018). The guidelines focus on how and why interventions work, for whom, and under what conditions. Adherence to guidelines that focus on transparent and detailed reporting of psychological and social intervention trials can also “minimise reporting biases and maximise the credibility and utility of this research evidence” (Montgomery et al., 2018, p. 2).

6. Conclusions

Interventions with iatrogenic effects present an uncomfortable reality in the context of evidence-based policy; namely, it is possible for interventions to do more harm than good (McCord, 2003). Given this possibility, and evidence that it is not nearly as remote as we might wish (see Welsh and Rocque, 2014), greater attention to interventions with iatrogenic effects seems deserving.

Two main objectives guided this paper. The first has to do with advancing primary research on intervention studies that report iatrogenic effects. In short, we need to know a great deal more about why interventions cause harm. The second aim is about making the case that a body of knowledge on iatrogenic effects can play an important role in informing evidence-based policy. We view this knowledge as another key input in the policymaking process.

A number of important advances have been made from continued research on interventions with iatrogenic effects. Most notably, this research has improved our understanding of deviancy training and peer contagion in the context of unstructured, group-based preventive interventions with high-risk adolescents (e.g., Dodge et al., 2005). Building on this knowledge, our proposed research agenda identifies several other areas of interest on and factors related to iatrogenic effects that could benefit from further research. Although desirable, this does not always require investigators to carry out longer follow-up assessments.

But it is not enough to simply draw lessons from interventions with iatrogenic effects. We need to make this body of knowledge another central element—another input variable—in the process of developing evidence-based policy. For example, safety guidelines could be drawn up for certain evidence-based interventions where the knowledge base on iatrogenic effects holds relevance. This could involve training practitioners on ways to lessen opportunities for unstructured socializing among program participants. Another possibility is to develop a series of safety checklists that could be communicated through registries of evidence-based programs. It would also be desirable to make recommendations specific to interventions designed to prevent aggression, violence, and crime in general. While our review was focused on crime and violence prevention programs, only one program specifically targeted violent crime. The other three programs reported on outcome measures of aggression and violent crime.

In no way should these safety guidelines or checklists become a barrier to innovation in the development or the timely delivery of evidence-based interventions. Much like how benefit-cost analyses or other types of economic analyses can provide guidance for funding and resource allocation decisions (Mears, 2010), the body of knowledge on iatrogenic effects is meant to provide some level of guidance on intervention content and delivery and even monitoring of implementation.

We think the stakes are high. In the same way that immediate action is needed to end (as well as defund) interventions that cause harm, expanding upon and using in a systematic and unbiased manner the knowledge base on iatrogenic effects has the potential to provide policymakers with an important tool to help avoid causing harm in the first instance.


Acknowledgments: We are grateful to the editor and the anonymous reviewers for insightful comments.

Declarations of interest: None.

Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.


References

Baron, J. (2018). A brief history of evidence-based policy. Annals of the American Academy of Political and Social Science, 678, 40-50.

Blackstone, W. (2016). Commentaries on the laws on England: Book II: Of the rights of things. S. Stern (Ed.). Oxford, UK: Oxford University Press.

Chalmers, I. (2003). Trying to do more good than harm in policy and practice: The role of rigorous, transparent, up-to-date evaluations. Annals of the American Academy of Political and Social Science, 589, 22-40.

Dishion, T. J., & Andrews, D. W. (1995). Preventing escalation in problem behaviors with high-risk young adolescents: Immediate and 1-year outcomes. Journal of Consulting and Clinical Psychology63, 538-548.

Dishion, T. J., McCord, J., & Poulin, F. (1999). When interventions harm: Peer groups and problem behavior. American Psychologist, 54, 755-764.

Dishion, T. J., Poulin, F., & Burraston, B. (2001). Peer group dynamics associated with iatrogenic effects in group interventions with high‐risk young adolescents. New Directions for Child and Adolescent Development91, 79-92.

Dodge, K. A. (2001). The science of youth violence prevention: Progressing from developmental epidemiology to efficacy to effectiveness to public policy. American Journal of Preventive Medicine, 20, 63-70.

Dodge, K. A., Dishion, T. J., & Lansford, J. E. (Eds.). (2005). Deviant peer influences in programs for youth: Problems and solutions. New York: Guilford Press.

Eisner, M. (2009). No effects in independent prevention trials: Can we reject the cynical view? Journal of Experimental Criminology, 5, 163-183.

Ekblom, P., & Pease, K. (1995). Evaluating crime prevention. In M. Tonry & D. P. Farrington (Eds.), Building a safer society: Strategic approaches to crime prevention (pp. 585-662). Chicago: University of Chicago Press.

Elliott, D. S., & Mihalic, S. (2004). Issues in disseminating and replicating effective prevention programs. Prevention Science, 5, 47-53.

Fagan, A. A. (2017). Illuminating the black box of implementation in crime prevention. Criminology & Public Policy, 16, 451-455.

Fagan, A. A., & Buchanan, M. (2016). What works in crime prevention? Comparison and critical review of three crime prevention registries. Criminology & Public Policy, 15, 617-649.

Fagan, A. A., Bumbarger, B. K., Barth, R. P., Bradshaw, C. P., Cooper, B. R., Supplee, L. H., et al. (2019). Scaling-up evidence-based interventions in US public systems to prevent behavioral health problems: Challenges and opportunities. Prevention Science, 20, 1147-1168.

Farrington, D. P., & Hawkins, J. D. (2019). The need for long-term follow-ups of delinquency prevention experiments. JAMA Network Open, 2(3), 1-3 (e190782).

Finckenauer, J. O. (1982). Scared straight and the panacea problem. Englewood Cliffs, NJ: Prentice-Hall.

Fixsen, D. L., Blase, K. A., Metz, A., & van Dyke, M. (2013). Statewide implementation of evidence-based programs. Exceptional Children, 79, 213-230.

Fo, W. S., & O’Donnell, C. R. (1975). The Buddy System: Effect of community intervention on delinquent offenses. Behavior Therapy6, 522-524.

Gottfredson, D. C. (2010). Deviancy training: Understanding how preventive interventions harm. Journal of Experimental Criminology, 6, 229-243.

Gottfredson, D. C., Cook, T. D., Gardner, F. E., Gorman-Smith, D., Howe, G. W., Sandler, I. N., et al. (2015). Standards of evidence for efficacy, effectiveness, and scale-up research in prevention science: Next generation. Prevention Science16, 893-926.

Haskins, R. (2018). Evidence-based policy: The movement, the goals, the issues, the promise. Annals of the American Academy of Political and Social Science, 678, 8-39.

Leadbeater, B. J., Dishion, T. J., Sandler I., Bradshaw, C. P., Dodge, K. A., Gottfredson, D. C., et al. (2018). Ethical challenges in promoting the implementation of preventative interventions: Report of the SPR task force. Prevention Science, 19, 853-865.

Lilienfeld, S. O. (2007). Psychological treatments that cause harm. Perspectives on Psychological Science, 2, 53-70.

McCord, J. (1978). A thirty-year follow-up of treatment effects. American Psychologist, 33, 284-289.

McCord, J. (1981). Consideration of some effects of a counseling program. In S. E. Martin, L. B. Sechrest, & R. Redner (Eds.), New directions in the rehabilitation of criminal offenders (pp. 394-405). Washington, DC: National Academy Press.

McCord, J. (2002). Counterproductive juvenile justice. Australian and New Zealand Journal of Criminology, 35, 230-237.

McCord, J. (2003). Cures that harm: Unanticipated outcomes of crime prevention programs. Annals of the American Academy of Political and Social Science, 587, 16-30.

McCord, J. (2004). Toward a theory of criminal responsibility. In J. McCord (Ed.), Beyond empiricism: Institutions and intentions in the study of crime (pp. 147-176). New Brunswick, NJ: Transaction.

Mears, D. P. (2007). Towards rational and evidence-based crime policy. Journal of Criminal Justice, 35, 667-682.

Mears, D. P. (2010). American criminal justice policy: An evaluation approach to increasing accountability and effectiveness. New York: Cambridge University Press.

Montgomery, P., Grant, S., Mayo-Wilson, E., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018). Reporting randomised trials of social and psychological interventions: The CONSORT-SPI 2018 Extension. Trials19:407, 1-14, https://doi.org.10.1186/s13063-018-2733-1.

O’Donnell, C. R. (1992). The interplay of theory and practice in delinquency prevention: From behavior modification to activity settings. In J. McCord & R. E. Tremblay (Eds.), Preventing antisocial behavior: Interventions from birth through adolescence (pp. 209-232). New York: Guilford Press.

O’Donnell, C. R., Lydgate, T., & Fo, W. S. (1979). The Buddy System: Review and follow-up. Child Behavior Therapy1, 161-169.

O’Donnell, C. R., Tharp, R. G., & Wilson, K. (1993). Activity settings as the unit of analysis: A theoretical basis for community intervention and development. American Journal of Community Psychology21, 501-520.

O’Donnell, C. R., & Williams, I. L. (2013). The Buddy System: A 35-year follow-up of criminal offenses. Clinical Psychological Science1, 54-66.

Patterson, G. R., Reid, J. B., & Dishion, T. J. (1992). Antisocial boys. Eugene, OR: Castalia.

Petrosino, A. (2000). How can we respond effectively to juvenile crime? Pediatrics, 105, 635-637.

Petrosino, A., Turpin-Petrosino, C., & Buehler, J. (2003). “Scared straight” and other juvenile awareness programs for preventing juvenile delinquency: A systematic review of randomized experimental evidence. Annals of the American Academy of Political and Social Science, 589, 41-62.

Poulin, F., Dishion, T. J., & Burraston, B. (2001). 3-year iatrogenic effects associated with aggregating high-risk adolescents in cognitive-behavioral preventive interventions. Applied Developmental Science, 5, 214-224.

Poulin, F., Dishion, T. J., & Haas, E. (1999). The peer influence paradox: Friendship quality and deviancy training within male adolescent friendships. Merrill-Palmer Quarterly, 41, 42-61.

Powers, E., & Witmer, H. L. (1951). An experiment in the prevention of delinquency: The Cambridge-Somerville Youth Study. New York: Columbia University Press.

Rhule, D. M. (2005). Take care to do no harm: Harmful interventions for youth problem behavior. Professional Psychology: Research and Practice, 36, 618-625.

Rosenthal, R. (1979). The ‘file drawer problem’ and tolerance of null results. Psychological Bulletin, 86, 638-641.

Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach, 7th ed. Thousand Oaks, CA: Sage.

Rubenson, M. P., Galbraith, K., Shin, O., Beam, C. R., & Huey, S. J. (2020). When helping hurts? Toward a nuanced interpretation of adverse effects in gang-focused interventions. Clinical Psychology: Science and Practice. DOI: 10.1111/cpsp.12321.

Schulz, K. F., Altman, D. G., Moher, D., & CONSORT Group. (2010). CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials. British Medical Journal, 340, 698-702.

Sherman, L. W. (2003). Preface: Misleading evidence and evidence-led policy: Making social science more experimental. Annals of the American Academy of Political and Social Science, 589, 6-19.

Sherman, L. W., & Harris, H. M. (2013). Increased homicide victimization of suspects arrested for domestic assault: A 23-year follow-up of the Milwaukee Domestic Violence Experiment (MilDVE). Journal of Experimental Criminology9, 491-514.

Sherman, L. W., & Harris, H. M. (2015). Increased death rates of domestic violence victims from arresting vs. warning suspects in the Milwaukee Domestic Violence Experiment (MilDVE). Journal of Experimental Criminology11, 1-20.

Sherman, L. W., Schmidt, J. D., Rogan, D. P., & Smith, D. A. (1992a). The variable effects of arrest on criminal careers: The Milwaukee Domestic Violence Experiment. Journal of Criminal Law and Criminology83, 137-169.

Sherman, L. W., Smith, D. A., Schmidt, J. D., & Rogan, D. P. (1992b). Crime, punishment, and stake in conformity: Legal and informal control of domestic violence. American Sociological Review, 57, 680-690.

Spoth, R., Rohrbach, L. A., Greenberg, M., Leaf, P., Brown, C. H., Fagan, A. A., et al. (2013). Addressing core challenges for the next generation of type 2 translation research and systems: The translation science to population impact (TSci) framework. Prevention Science, 14, 319-351.

Tharp, R. G., & Wetzel, B. J. (1969). Behavior modification in the natural environment. New York: Academic Press.

Vygotsky, L. S. (1981). The genesis of higher mental functions. In J. V. Wertsch (Ed.), The concept of activity in Soviet psychology (pp. 144–188). Armank, NY: Sharpe.

Weiss, C. H. (1998). Have we learned anything new about the use of evaluation? American Journal of Evaluation, 19, 21-33.

Weiss, C. H., Murphy-Graham, E., & Birkeland, S. (2005). An alternate route to policy influence: How evaluations affect D.A.R.E. American Journal of Evaluation, 26, 12-30.

Weiss, C. H., Murphy-Graham, E., Petrosino, A., & Gandhi, A. G. (2008). The fairy godmother—And her warts: Making the dream of evidence-based policy come true. American Journal of Evaluation, 29, 29-47.

Welsh, B. C., & Rocque, M. (2014). When crime prevention harms: A review of systematic reviews. Journal of Experimental Criminology, 10, 245-266.

Welsh, B. C., Zane, S. N., Zimmerman, G. M., & Yohros, A. (2019). Association of a crime prevention program for boys with mortality 72 years after the intervention: Follow-up of a randomized clinical trial. JAMA Network Open, 2(3), 1-11 (e190782).

Werch, C. E., & Owen, D. (2002). Iatrogenic effects of alcohol and drug prevention programs. Journal of Studies on Alcohol, 63, 581-590.

Wilson, D. B. (2009). Missing a critical piece of the pie: Simple document search strategies inadequate for systematic reviews. Journal of Experimental Criminology, 5, 429-440.

Zane, S. N., Welsh, B. C., & Zimmerman, G. M. (2016). Examining the iatrogenic effects of the Cambridge-Somerville Youth Study: Existing explanations and new appraisals. British Journal of Criminology, 56, 141-160.


Table 1. Summary of studies with iatrogenic effects and longer follow-up assessments

Study Name and Authors

Intervention Type

Intervention Features

Participants and Sample Size

First Measure of Iatrogenic Effects

Developmental or Social Processes

Effects at Latest Follow-Up

Explanation of Follow-Up Effects

Adolescent Transitions Program (Dishion & Andrews 1995; Poulin et al. 2001)

Parent training and child self-regulation with peer reinforcement

Parent and teen focus interventions delivered in group format in a clinical setting

High risk boys and girls (mean age = 12 years); N=119

1 year: increase in smoking and externalizing behavior (for teen and teen/ parent conditions)

Deviancy training through friendship formation (via group format in clinical sessions)

3 years post-intervention: increase in smoking and delinquency (for teen condition)

Learned deviance

Buddy System (Fo & O’Donnell 1975; O’Donnell & Williams 2013)

Individual mentoring

Trained mentors used behavior modification techniques and engaged in activities with youth

Boys and girls from low income area (mean age = 13.5 years); N=533

3 years: higher arrest rates (for youth with no prior arrests)

Deviancy training through “peer network effect” (via interactions in community)

35 years: higher arrest rates for females only (for females with no arrests at referral)

Earlier deviant friendships facilitated a “trajectory of relationships” in support of criminal behavior

Cambridge-Somerville Youth Study (McCord 1978; Welsh et al. 2019)

Individual counseling/ mentoring (“directed friendship”)

Home visits with families and engaged in activities with boys, including summer camp

Under-privileged boys (median age = 10.5 years); N=506

30 years post-intervention (mean age = 45 years): increase in criminal activity, pre-mature mortality, other

Deviancy training through bonding (among boys who attended summer camp)

72 years post-intervention (88.1% of participants deceased): no differences in mortality outcomes

Singular event (other outcomes not measured), earlier mortality effects were weak

Milwaukee Domestic Violence Experiment (Sherman et al. 1992a; Sherman & Harris 2013)

Police arrest for domestic violence

Police arrest at scene and detain at police station for 3-12 hours (short and long arrests)

Adult suspects (mean age = 32 years), 91% male, 55% un-employed, 50% prior arrest; N=1,200

 

1 year: increase in repeat violence by African- American, unemployed, suspects (for short and long arrests)

Suspects had no ties to conventional society

23 years: higher mortality rates from homicide (for short and long arrests)

Continuation of no ties to conventional society

Notes: All effects are reported as treatment versus control.

Comments
0
comment
No comments here
Why not start the discussion?