Can ADR improve Expert Evidence?
Associate to President Kingham, Land Court of Queensland from July 2018 to September 2019
Jason M. Chin
Sydney Law School, University of Sydney, Australia; Institute for Globally Distributed Open Research and Education (IGDORE), Indonesia
Nick Wray-Jones & Jason M Chin, “Can ADR improve expert evidence?” (2021) 95 The Australian Law Journal 467.
Courts have developed numerous procedures to regulate expert evidence. These procedures aim, in large part, to manage experts’ biases. Recently, the Land Court of Queensland created a new procedure called Court Managed Expert Evidence (CMEE). CMEE’s innovation is the addition of an alternative dispute resolution component – an individual who works with experts and lawyers to manage disputes regarding experts’ opinions. In this article, the authors evaluate this advance against the backdrop of the psychology of bias and ADR’s capacity to limit that bias, as well as an interview with President Kingham, who developed CMEE. They conclude CMEE provides some innovations that may counter expert bias in ways existing procedures cannot. Still, it is inherently limited due to the unconscious nature of many biases and the fact that it does not demand reliable knowledge. The authors suggest others consider implementing CMEE, especially in complex matters where the benefits outweigh the costs.
Australian courts rely on a variety of procedural mechanisms in an attempt to control protracted battles of expert witnesses and manage the biases they bring into the court. These reforms have been divisive, with some celebrating them as an end to the problems associated with expert witnesses, and others remaining sceptical.1 In 2018, the Land Court of Queensland added to this landscape, developing a procedure (Court Managed Expert Evidence or ‘CMEE’) that builds on existing ones by introducing a court-appointed ‘Convenor’ into the process.2 The Convenor uses alternative dispute resolution (ADR) techniques to supervise pre-trial meetings of experts and the drafting of joint expert reports (JERs).3 This procedure was developed by President Kingham, who currently presides over the Land Court. To date, some courts have used facilitators or moderators to manage expert evidence,4 but none have developed a comprehensive process analogous to the CMEE procedure. In this article, we will suggest that while CMEE carries many of the same limitations of other procedures, it also presents some distinctive advantages. Courts outside the land context may wish to consider adopting it.
This article will focus on the potential for CMEE to address the biases of expert witnesses. Expert witness bias is a complicated topic that has received a great deal of academic attention.5 we will seek to address through psychological research. As we will see, close management by a CMEE Convenor provides the chance of preventing or mitigating some biases, but will struggle against some more nuanced, unconscious psychological biases.
We will proceed following way: Part II provides an overview of courts’ concerns regarding bias in expert evidence, as well as the procedures some courts presently use to manage expert evidence. In Part III, we: (a) explain the CMEE process and its underlying philosophy; (b) consider whether existing psychological literature supports the claim that ADR reduces bias; and (c) consider examples of how ADR techniques might affect bias in CMEEs. In Part IV, we consider the limitations of the CMEE process. We suggest that the existing psychological literature indicates that CMEEs may reduce bias, although the process may be unable to undo biases entirely. Part V concludes.
Expert evidence is often crucial to the outcome of modern litigation.6 Courts, however, regularly express concern about biased experts.7 When courts refer to bias in the context of experts, they are typically referring to “adversarial bias” – that experts favour the party retaining them and thus discredit or disregard opposing views.8 Adversarial bias is composed of two distinct biases: selection and association bias. Selection bias arises because parties tend to select experts whose professional opinions support their case.9 In contrast, association bias arises when experts feel they are “part of the team” in an adversarial setting, especially because they are being paid by a specific party and may want to be retained again.10 For example, forensic psychologists tend to make findings which support their associated party, even if the experts are randomly selected.11 Association bias is very similar to “in-group” bias: the tendency to favour one’s own group while excluding others.12
However, adversarial bias is one part of a wider range of cognitive biases that potentially affect experts’ opinions. Bias is “any systematic error in reasoning and thinking that can alter an individual’s memory, perception, and decision making.”13 Therefore, when we refer to “bias” in this article, we are referring to cognitive biases generally. Without providing an exhaustive list, the following are some common biases.
Self-serving bias causes people to view themselves favourably. This bias warps people’s perception of their own abilities.14 Specifically, it causes people to perceive themselves as above average across a wide range of metrics, including ethics,15 professional competence,16 intelligence,17 and insight,18 even though it is impossible for more than 50% of people to be above average in any given field. Self-serving bias also causes people to overestimate their objectivity; they claim that other people are affected by self-serving biases, but that they are not.19 This is known as the “bias blind spot” and has been specifically demonstrated in forensic science experts and forensic psychology experts.20 Overall, self-serving bias causes experts (like everyone else) to overestimate their abilities and objectivity.
Fundamental attribution error causes people to explain others’ behaviour according to their disposition, rather than situational factors.21 Conversely, it causes people to explain their own behaviour based on their situation, rather than their disposition.22 For example, if two experts disagree, they may misattribute their counterpart’s behaviour. Each expert may think the other is stubborn, adversarial, or incompetent, rather than identifying differences in their methodology or interpretation that cause a difference of opinion.
Contextual bias arises from exposure to irrelevant or prejudicial contextual information, which biases people towards particular interpretations and conclusions.23 Studies have shown the ubiquitous effect of contextual bias on forensic scientific experts.24 For example, telling fingerprint examiners that sets of fingerprints were erroneously matched can skew them towards reaching the same conclusion, even if they previously declared the fingerprints to match.25
Confirmation bias is the tendency to search for things that support a desired outcome, while also ignoring or reinterpreting contradictory information.26 As Justice Paciocco explains, confirmation bias means that, “in a world of interpretation, we tend to find what we are looking for.”27 In the most extreme cases, this means people intentionally gather and interpret evidence to justify a predetermined position. In what is likely more common, however, confirmation bias operates subconsciously.28 This means that experts’ perspective is invariably coloured by their experience, ethics, and personal preferences – whether or not they find themselves in an adversarial setting.29
Noble cause distortion biases people towards ethically appealing conclusions.30 For example, noble cause distortion can lead police to abuse their powers in order to obtain convictions.31 Similarly, noble cause distortion may cause experts to adopt erroneous methods or reasoning to achieve an ethical goal, either consciously or subconsciously. For instance, “recovered memory syndrome” experts in the 1980s and 90s used unscientific methods to recover traumatic memories from the victims of child sexual abuse, which led to a number of wrongful convictions.32
Additionally, biases can spread throughout an expert’s analysis or to other experts because of “bias cascades” or “bias snowballs”.33 A bias cascade occurs when irrelevant information passes between stages of experts’ reasoning. For example, this may occur if an expert who collects evidence at a crime scene – and is exposed to prejudicial information – also interprets the evidence, or if they pass on the information to another expert who interprets the evidence. In contrast, a bias snowball occurs when a biased expert provides their analysis to another expert, who is affected by additional biases. This means the resulting analysis is influenced by both experts’ biases.
In response to the above wide-ranging concerns, courts have developed numerous procedures to manage expert evidence, including: codes of conduct, single and court-appointed experts, pre-trial meetings, and concurrent evidence. To supplement these procedures, lawyers cross examine experts to identify flaws in their reasoning, and judges and jurors then weigh experts’ evidence accordingly. Each of these has their own strengths and weaknesses in dealing with expert bias. We will now briefly review these established procedures as a backdrop for our discussion of the CMEE, which introduces an ADR component to the existing mix.
Codes of conduct remind experts that their primary duty is to the Court.34 Specifically, they aim to undo adversarial bias by promoting a culture of objectivity and independence among experts. However, some commentators argue that these codes have limited value in practice.35 For example, Chin, Lutsky, and Dror explain that many biases operate unconsciously. Because of this, people cannot necessarily overcome them through conscious effort, although they often believe that they can.36 Furthermore, even if expert codes can dampen association bias, they cannot undo selection bias. Indeed, selection bias is difficult to address without removing parties’ right to choose their experts.37 This leads to the second option for managing expert evidence: single and court-appointed experts.
Single experts (whom parties jointly nominate) and court-appointed experts mitigate the risk of adversarial bias by removing the retainer relationship between experts and individual parties. However, this model cannot account for a wider range of cognitive biases. For example, a single expert can still have preconceived views, based on their personal experience, which lead to confirmation bias. As Mark Livesey QC explains, “[t]here is a strong argument that the search for the ‘neutral’ expert is futile. Every expert… can be influenced. Expertise is subjective, and contingent on or aligned to various factors, whether they be professional, institutional, financial or merely personal.”38 Furthermore, having a single expert per field of expertise may deprive judges of competing views, especially where there is genuine disagreement between experts about a particular topic. In those cases, a well-structured debate between experts will be more valuable to a judge than a single, uncontested opinion which may not be representative of a field of expertise.39 Similarly, judges are not well-placed to appoint a single expert on their own initiative when different experts disagree about a particular topic.40
Pre-trial meetings give experts an opportunity to meet and discuss their views, usually before reducing their opinions to writing in a JER. This allows experts to consider the strengths and weaknesses of their preliminary views, as well as understand their counterpart’s position, and identify areas of agreement and disagreement.41 Ideally, pre-trial meetings counteract biases by forcing experts to consider a different point of view and make reasonable concessions where appropriate. Further, pre-trial meetings promote a “collegiate environment in which the experts can see their role more clearly as one of assisting the court rather than advocating for a party’s case.”42
However, this does not always occur in practice. Meetings can be unstructured and adversarial, especially if either expert does not participate in good faith. For example, as Andrew Ross explains, expert meetings can collapse when one or both experts do not participate constructively or professionally, demonstrate a lack of interest in agreeing on key issues, or focus on criticising their counterpart’s professionalism, rather than discussing the merits of the case.43 Similarly, experts from New South Wales indicate that, although some pre-trial meetings are successful, a larger number are problematic. Among other things, meetings sometimes degenerate because of disorganisation, overbearing personalities, lack of clarity about the relevant issues, and disagreement about the experts’ duties in the meeting process.44 However, the most common reason for meetings degenerating was the lack of a “settled, structured, closely managed and supervised set of procedures to facilitate the [meeting] and joint report process.”45 Additionally, some experts may only be able to meet by email and phone,46 even though studies show that these mediums tend to generate and aggravate disagreement and miscommunication.47
Referring to some of these problems, President Kingham also observes that “an expert may be overly confident in their methods or their own point of view” (which may be a result of self-serving bias), and that “[t]hey may not respect their counterpart or consider them to be their inferior” (which may lead to fundamental attribution error when disagreements arise).48 Further, her Honour explains that parties’ lawyers may “set off the experts on parallel tracks” by briefing them with different information, asking them to make different assumptions, or asking them to address different issues.49 In those circumstances, contextual bias and confirmation bias may undermine the meeting process because the experts have been prompted towards specific perspectives and conclusions by their retaining parties.
Concurrent evidence involves experts giving oral evidence at the same time. It is similar to pre-trial meetings, in that it allows experts to discuss their areas of agreement and disagreement, as well as test each other’s arguments (analogous to a system of peer review).50 This model arguably reduces bias because it allows experts to critique each other’s opinions in real time, while also allowing experts to reinforce colleagues’ opinions when they are appropriate.51 However, concurrent evidence may not be able to uncover more nuanced forms of bias, which influence experts’ opinions without making them untenable or visibly partisan.52 Rather, concurrent evidence may only force experts to avoid reasoning and conclusions which are evidently inaccurate or skewed. Furthermore, concurrent evidence potentially favours experts with strong personalities and strongly held beliefs, who can dominate discussions.53
In addition to these procedures, judges and jurors can give limited weight to experts whom they perceive as biased.54 This is not a procedure, but it is a way of accounting for the effect of expert bias, and is therefore worth considering. The rationale for this approach is that cross-examination exposes bias, meaning judges and juries can account for bias by giving an expert’s evidence limited weight. As a result, bias will not infect the judge or jury’s decision-making.55
However, cross-examination has serious limitations.56 In order to effectively cross-examine an expert witness, lawyers must understand the relevant field of expertise to some extent, even if it is extremely complex. Additionally, lawyers must ask the right questions to uncover bias. Neither of these is guaranteed. In a 2008 study of forensic scientists, more than 50% of participants reported that lawyers’ poor knowledge of forensic science had a detrimental impact on proceedings. Among other things, this meant that cross-examination did not uncover “abnormal or striking results”, and lawyers were unaware of how data could be consistent with different conclusions.57 Additionally, we suggest that lawyers have a narrow, legalistic view of bias, associating bias with the legal tests for actual and apprehended bias, rather than cognitive biases generally.58 This means they will focus on adversarial bias (eg the expert’s allegiances) when cross examining experts. Finally, it may be difficult to expose bias through cross examination because most biases operate subconsciously, meaning experts often cannot perceive their own biases.59
Even if cross-examination does uncover bias, judges and jurors may not appreciate the potential effects of bias on experts’ evidence. If decision-makers are not familiar with cognitive biases, they are unlikely to ascribe less weight to an expert’s opinion because cross examination revealed (for example) confirmation or contextual bias in an expert’s reasoning.60 Further, judges and jurors cannot necessarily cure bias by giving evidence limited weight because they are not well-placed to determine how and to what extent bias affects an expert’s cognition. They see a narrow and refined product of the expert’s cognition – their written report and oral evidence – which may not reveal the full extent of the expert’s reasoning or the factors that shape their interpretation of the facts. Finally, weighing evidence is an art, not a science. It involves a discretionary decision based on the perception of a witness and their evidence. This means judges and jurors cannot guarantee that their weighing of an expert’s evidence accurately reflects the extent of that expert’s bias.
The CMEE model builds on the above procedures. Specifically, it is an enhanced version of traditional pre-trial meetings.61 In a CMEE, a neutral, court-appointed Convenor supervises the meetings and JER-drafting process. The President of the Land Court appoints Convenors, who are either members62 of the Court or the Court’s Judicial Registrar. They convene case management conferences with the parties’ lawyers and (potentially) their experts, as well as pre-trial meetings of experts, to progress JERs. Convenors manage communications between parties and experts63 and assist experts to ensure their JERs are coherent and comprehensive.64 Because a Convenor’s role is procedural, they cannot decide any part of the case, and they can only make directions about the preparation of experts’ evidence if the parties consent. If an issue arises in the CMEE which requires directions, but the parties cannot agree, the President makes directions.65
Ideally, CMEEs retain the advantages of traditional pre-trial meetings, while avoiding their most common pitfalls.66 Convenors ensure that the dialogue between experts is productive and professional, and that their JER is comprehensive and clear. The result is a well-structured discussion between experts, captured in their JER, which has the most utility for the parties and the court. Indeed, experts from New South Wales have indicated that the factor most likely to improve pre-trial meetings and JERs is an “independent facilitator or chairperson” who could liaise with experts and lawyers to clarify relevant issues and questions and assist experts in preparing a useful JER.67
President Kingham reports, anecdotally, that successful CMEEs reduce the number of disputed issues and increase clarity regarding the issues which remain.68 Importantly, CMEEs also distinguish between issues which are important and unimportant for the resolution of a case.69 From her Honour’s perspective, this makes trials more manageable and judgment writing less challenging because the “pathway to a decision” is clearer for the Court and the parties.70
President Kingham suggests that the CMEE procedure reduces expert bias by reinforcing experts’ duty to the Court and ensuring that pre-trial meetings are organised, professional, and constructive.71 In doing so, Convenors use the common ADR skills of facilitating and managing communications between experts, lawyers, and the Court.72 By using these skills, Convenors aim to achieve similar goals to ADR: to “identify, reduce, resolve (if appropriate) and manage disagreements” and “prepare for mediation or a trial of limited issues”.73
In the next part, we expand on the psychology of ADR to determine its capacity to reduce bias. Following that, we consider how CMEE Convenors using ADR techniques may reduce bias in practice. Importantly, although President Kingham discusses ADR as a whole, we focus on mediation. This is because, among the various forms of ADR, the CMEE process is most analogous to a mediation. CMEEs involve a neutral third party (the Convenor) who facilitates discussion, attempts to improve decision-making, and assists experts to reach agreement where possible.74
More specifically, CMEEs are most analogous to facilitative mediation. Convenors maintain a constructive dialogue and encourage experts to reach agreement without directly brokering agreements.75 Unlike evaluative mediation,76 Convenors do not critique the merits of experts’ opinions; they must not influence experts to adopt certain opinions77 or simply tell experts what to write in their JERs.78 Instead, they remain neutral and bring a layperson’s perspective to the meeting and report writing process to ensure that JERs are comprehensible for the parties’ representatives and the Court.79 As President Kingham explains, the CMEE process “is not evaluative, except to the extent that suggestions about process might involve applying [the Convenor’s expertise regarding the Court’s procedure and jurisdiction] to help the parties move through process stalemates.”80
The psychological literature indicates that ADR may reduce bias. As Myers explains, most conflicts only involve a small amount of truly incompatible goals. The larger problem is misperception; each party misinterprets the other’s motives and goals because of biases (such as self-serving bias or fundamental attribution error).81 At the same time, both parties perceive themselves as free from bias because of the bias blind spot. ADR changes parties’ perspectives about each other and themselves, counteracting biases. This is because ADR involves direct contact and communication.
Direct contact between opposing sides tends to reduce conflict, so long as the contact is non-competitive, supported by authorities, and on equal footing.82 This is because direct contact provides information contradicting biases. For example, Tropp and Pettigrew’s 2006 meta-analysis shows that contact between groups reduces inter-group prejudice in a wide variety of circumstances.83 As Hewstone et al explain, direct contact reduces the anxiety people associate with outgroup members,84 as well as reducing the perception that the outgroup poses a threat.85 Additionally, direct contact increases empathy – which allows people to see the other’s perspective86 – as well as trust, which acts as a positive bias when interacting with the outgroup.87 Further, contact is associated with increased forgiveness, which allows parties’ relationships to move forward after real or perceived transgressions.88
Communication helps parties cooperate, discuss problems, and reach agreements.89 For example, communication – especially face-to-face communication90 – helps people produce mutually beneficial outcomes in social dilemmas (i.e. situations where self-interest makes it tempting not to cooperate, but the “collectively optimal” outcome requires mutual cooperation).91 According to Kerr and Kaufmann-Gilliland, this is because communication creates a sense of “commitment to cooperate”.92 Indeed, communication can make people more likely to cooperate even if: it will produce little or no benefit for their own group;93 or their counterpart will not know whether they in fact cooperated.94
The aggregate of these factors is that, through direct contact and communication, ADR may reduce bias. Specifically, ADR challenges parties’ perspectives, forcing them to consider each other’s point of view, as well as the weaknesses of their own position, which potentially exposes, challenges, and breaks down biases. For example, ADR reduces the negative perceptions people have about each other and a conflict (such as adversarial bias), while also forcing people to see each other’s perspective (which may counteract self-serving bias and fundamental attribution error). This change of perspective means people are more likely to understand why their counterpart disagrees with them or finds their reasoning weak or incomplete (potentially exposing confirmation and contextual bias). Based on this new perspective, parties can cooperate towards mutually beneficial outcomes. Therefore, based on the existing psychological literature, we suggest ADR has the potential to expose and counteract biases.
However, direct contact and communication may not always create this change of perspective. Direct contact may not be effective if: the status of the parties is unequal; there is no impetus for parties to cooperate or work towards common goals; or there is no institutional support to legitimise the contact.95 Similarly, parties can use communication to undermine and compete with one another, depending on the circumstances.96 As Krauss and Morsella explain, it is naïve to consider communication as a panacea for conflict. Rather, communication is a “neutral instrument – one that can be used to convey threats as well as offers of reconciliation, to put forth unreasonable offers as well as acceptable ones, to inflame a tense situation as well as to defuse it.”97 As a result, even if parties enjoy direct contact and communication, their interactions may still become disorganised or adversarial.
Mediators help resolve these problems. Specifically, mediators control the agenda, tone, and pace of discussions, ensuring that direct contact between parties does not become hostile, unprofessional, or simply unproductive.98 Mediators also promote constructive communication, which aims to increase clarity and understanding, as well as exposing and challenging biases, assumptions and interpretations.99 As a result, mediators help parties remove the illusions they may have about a conflict, each other, and their own perspective.100 They may also assist parties by proposing mutually beneficial solutions to resolve or narrow areas of disagreements. Indeed, parties are more likely to accept these proposals when they come from a mediator (a neutral third party), rather than each other.101 In doing so, mediators may remind parties that they each have an interest in a quick and less costly resolution.102
Here, we offer an analogy to journal-editor mediated peer review, one of the dominant ways of resolving disputes on topics of specialized knowledge in academic science. This process certainly has its shortcomings and cannot directly improve the quality of research submitted to journals.103 Facially neutral editors can, however, encourage authors to more readily acknowledge shortcomings in their research identified by a peer reviewer. They can also moderate criticisms that might ordinarily be presented in a non-constructive way as to elicit more forthcoming responses from the authors. The result, in some cases, may be claims that are more transparent and less likely to reflect the author’s preferred views (eg what they hypothesised prior to conducting the study).
As President Kingham explains, CMEE Convenors use the ADR techniques of facilitating and managing communication between experts, lawyers, and the Court to enhance JERs.104 In the following paragraphs, we focus on three specific functions of Convenors: chairing meetings of experts; facilitating communication between lawyers and experts; and managing disagreements between experts. We suggest that the use of ADR techniques in each of these functions has the potential to reduce expert bias.
When chairing meetings of experts, Convenors must help experts discuss and attempt to reach agreement on their evidence. If necessary, the Convenor may explain experts’ duty to the Court, as well as the Court’s expectations of experts in giving evidence. In all cases, the Convenor must ensure that experts engage in a comprehensive, professional discussion.105 If Convenors do these things, they may reduce bias – most notably, adversarial bias – because they will ensure that experts do not perceive the meeting process as a “win-lose” situation, where one expert’s view must prevail. Instead, Convenors promote a perception that pre-trial meetings are a collaborative exercise between colleagues.106 If the meeting process breaks down, Convenors can guide experts back towards a more respectful, productive form of dialogue. This aligns with feedback from experts in South Australia, who reported that the presence of a chairperson makes meetings more neutral, promotes order and fairness, and controls experts’ egos.107
Convenors facilitate communication between experts and lawyers once the expert meeting process has commenced.108 In doing so, Convenors ensure experts receive “the same information at the same time.”109 This substantially reduces the risk of contextual bias. By controlling the flow of information between lawyers and experts, Convenors reduce the risk that experts will be exposed to irrelevant or prejudicial information from their respective lawyers. Ideally, Convenors will be appointed before experts are briefed, so that they can fulfil this function during the briefing process.110
Furthermore, Convenors facilitate communication between groups of experts involved in the same case.111 This reduces the risk of experts sharing information which may lead to contextual bias, but also reduces the risk of bias cascades and snowballs. President Kingham’s comments in Cherwell Creek Coal Pty Ltd v BHP Queensland Coal Investments Pty Ltd & Ors (No 12)112 support this. Her Honour explained that:
The overlap of issues and experts presents a real risk to the integrity of the process. In order to prepare their report, one group of experts may want to clarify an issue addressed (or not addressed) in the joint expert report of another group of experts. There is a risk that an expert common to both groups could influence how the second group interprets the first group’s report, without all experts from the first group knowing what the expert common to both has conveyed.
As well as driving an efficient process and promoting comprehensive and well prepared reports, the CMEE Convenor can promote the integrity of the expert meeting process and ameliorate risks such as the one identified. Under the Practice Direction, the CMEE Convenor manages and facilitates communications between experts in different disciplines and keeps the parties informed.” (citations omitted, emphasis added)
Finally, Convenors manage disagreements between experts. Similar to a mediator, Convenors can help experts identify the real areas of disagreement between them (which may be much narrower than the experts first expect). Once the areas of disagreements are clear, Convenors can help experts understand why they disagree.113 Specifically, Convenors ensure that experts critically analyse each other’s assumptions, investigations, methodology, and conclusions.114 In doing so, Convenors “promote rational, rigorous and respectful discussions, encouraging the experts to critique ideas not people and to carefully consider each other’s point of view.”115
This process uncovers the reasoning underlying each expert’s position, which may counteract a range of biases. For example, if experts understand why their counterpart disagrees with them, they are less likely to commit the fundamental attribution error. Instead of concluding that “my counterpart is stubborn” or “they are incompetent”, experts are more likely to conclude that “they disagree with me because of [for example] a methodological difference.” Additionally, understanding another expert’s perspective can provide information contradicting confirmation bias. Considering a well-reasoned, alternative viewpoint will encourage experts to think critically about their own perspective; at a minimum, this shows theirs is not the only reasonable interpretation of the case. Further, if an expert understands why a colleague finds their reasoning weak or incomplete, they are less likely to succumb to self-serving bias because seeing flaws and limitations in one’s own work provides a source of reality testing.
In this part, we will review some of the limitations of the CMEE process. These include both the inherent limitations of the process and more practical ones. For instance, unconscious biases are not something even a well-trained convenor can visibly identify, and thus represent a particularly difficult challenge for any procedure. As to practical hurdles, CMEEs carry a cost that may not be justified in all cases.
While the above psychological literature suggests the CMEE process may address some of the biases present in an adversarial system, it should not be seen as a panacea. Notably, cognitive bias typically operates unconsciously. This makes it difficult for experts to overcome, even with the benefit of ADR. For instance, while Convenors may be able to prevent experts from being exposed to biasing material, they cannot undo that once it has occurred. As a result, CMEEs may not be able to reduce bias cascades affecting different stages of individual experts’ reasoning. If an expert is exposed to contextual information – for example, when conducting a site inspection – and transfers that contextual bias into the next stage of their analysis, Convenors may not be able to detect or undo the bias.
Further, it is unclear how CMEEs could mollify deeply ingrained biases like noble cause distortion. An expert who is convinced that they are fighting for a moral cause is unlikely to concede that their reasoning is weak, even if a Convenor and other experts critique their views. Convenors also cannot undo selection bias because parties can still select experts whose professional views support their case.
The effectiveness of CMEEs is largely reliant on Convenors’ actions and decision-making. In order to consistently reduce bias, Convenors must (at a minimum) understand and employ the ADR techniques outlined by President Kingham and follow the Court’s Practice Direction. However, this may not always occur. An example of a failed CMEE appears in Irvine v Bundaberg Regional Council.116 In that case, Member Stilgoe observed that, despite participating in a CMEE, two valuation experts did not engage with each other’s reasoning in their JER. Her Honour observed at  that:
It is not surprising that the valuers’ opinions about the appropriate figure for profit rent differed, but the absence of any detailed analysis of the competing views is disappointing, given that the experts participated in the Land Court’s Court Managed Expert Evidence process (CMEE). The CMEE is designed to identify and eliminate these sorts of problems before the production of the joint expert report. Rather than rejecting each other’s methodology out of hand, a diligent joint expert reporting process should have given me an assessment by each expert based on both their own and their opponent’s methodology
Because the experts did not engage with each other’s methodology in their JER, it is very unlikely that the CMEE process would have uncovered or mitigated bias as we suggest in Part III(c). In particular, the CMEE would not have reduced fundamental attribution error, confirmation bias, or self-serving bias because the experts did not critically engage with each other’s perspective and reasoning in their JER. Indeed, because their report lacked this analysis, the experts produced a further JER which included a comparative sales analysis after their first day of oral evidence.117 In Member Stilgoe’s words: “[t]his was a CMEE that did not meet its objectives.”118 This occurred despite  of the CMEE Practice Direction requiring Convenors to assist experts in the production of their JER to:
d) check they have each considered the underlying facts, assumptions, methodologies, and conclusions of any other expert witness included in the report; [and]
e) check that, to the extent they disagree on the matters in (d) above, they each explain–
i. why they disagree; and
ii. what their evidence would be if the Court accepted the evidence of the other expert on any of those matters
The Court has committed resources to training Convenors. Specifically, President Kingham notes that the Court has produced an internal practice note for Convenors and offers mentoring for Convenors who are less familiar with the procedure.119 Ideally, these measures will reduce the risk of similar failed CMEEs, but they cannot guarantee that CMEEs will always succeed.
In some cases, existing procedures may be as effective as the CMEE process in reducing bias. This is most likely to be the case if the numbers of experts and issues involved are relatively low, and the court makes detailed directions regarding pre-trial meetings and JERs. For example, r 429B of the Uniform Civil Procedures Rules 1999 (Qld) allows courts to: set the agenda for experts’ meetings; specify the matters which they must discuss; make directions about the form of JERs; and give any other directions the court considers appropriate. A court using these powers could require experts to meet in person, ensure they have the same information, discuss their methodologies, assumptions and reasoning in detail, identify the reasons for their disagreements (if any), and critically engage with one another’s point of view in their meetings and report writing. Ideally, this process counteracts biases in largely the same way as CMEEs. The experts would independently enjoy the benefits of direct contact and constructive communication as they progress through the agenda and draft their JER. Indeed, in cases involving experienced expert witnesses and detailed directions, the presence of a Convenor may make the meeting and report-writing process more complex without necessarily adding value for the parties or the court.
However, even with the benefit of detailed directions, pre-trial meetings may still collapse for the reasons outlined in Part II(b). Furthermore, as we explained in Part III(b), direct contact and communication may be ineffective without a mediator figure who can facilitate discussion. These problems are most likely to arise when the numbers of experts and issues involved in the meeting process are high, making the meeting process unwieldy, even with the benefit of detailed directions. CMEE Convenors can overcome these problems by supervising the meeting and reporting process and guiding experts away from common pitfalls in their discussions and report writing (as described in Part III(c)). Convenors have a distinct advantage over trial judges in this regard, as they are privy to meetings and the drafting of reports.
The CMEE Practice Direction reflects this distinction between simpler cases (with fewer experts and issues) and more complex cases, which are more likely to benefit from the CMEE process. Specifically, - of the Practice Direction states that the Court refers cases to CMEE on a case-by-case basis, taking into account the number of experts involved, the complexity of issues, the relationship between different areas of expertise, and whether there is a history of non-compliance with the Court’s rules or directions.
President Kingham agrees that, as a general observation, CMEEs are best suited to more complex cases.120 For instance, her Honour notes that cases involving a single area of expertise and a clear set of issues may not require a CMEE. However, if the facts of the case are disputed, so that the experts need to address a range of alternative scenarios in their JER, a Convenor can help organise their meeting and report writing.121 Similarly, if a case involves a relatively low number of fields of expertise, but the fields are interrelated (so that JERs rely on the outcome of others), a Convenor can help coordinate the process.122
Some have suggested Australia should adopt reliability as the criterion for the admissibility of expert evidence.123 That debate is largely beyond the scope of this article. However, it is worth noting that, even if CMEEs reduce bias, they will not necessarily increase the reliability of expert evidence. An unbiased expert can still produce poor quality evidence if their methods are unreliable. By encouraging experts to explain their assumptions and reasoning, as well as critiquing each other’s views, CMEEs may indirectly expose unreliable methodology, but whether this will occur in practice is uncertain. Therefore, it still falls to lawyers (who cross-examine experts), as well as members deciding cases, to test reliability.
Finally, CMEEs may increase the cost of litigation for parties, as well as demanding a large amount of courts’ resources. The process of Case Management Conferences, Meetings of Experts, and corresponding with Convenors can demand significant time and resources from lawyers and experts, whom parties typically pay by the hour. This may deter parties from pursuing litigation, especially if they are not well-resourced.124 From the Court’s perspective, CMEEs are also time and resource-intensive. They involve a large administrative workload for Convenors and associates, who have wide-ranging duties outside of CMEEs. This may deter other courts from adopting the CMEE process.
However, as we noted in Part III, the Court does not refer cases to CMEE as a matter of course. If a CMEE would be too costly, parties make submissions to that effect. Further, Convenors can only make directions regarding expert evidence by consent;125 they cannot unilaterally require experts to produce additional reports or undertake additional work. Finally, even if CMEEs increase the pre-trial cost of litigation, they will often pay dividends at trial. In particular, a rigorous pre-trial process – one that reduces areas of disagreement, makes JERS easier to understand, and forces experts to critically analyse each other and themselves – ensures JERs have maximum utility for parties and the Court during a trial. As President Kingham notes, such a process also tends to reduce the length and complexity of trials.126
The CMEE process builds on existing procedures for managing expert evidence by inserting a neutral Convenor into pre-trial meetings. We suggest that CMEE Convenors may improve pre-trial meetings and JERs by employing ADR techniques to reduce some forms of bias. To that extent, the CMEE process represents a novel way to address the issue of bias in expert evidence. At a minimum, it promotes organised and constructive pre-trial meetings, leading to high quality JERs, and therefore other Australian courts should consider implementing it for some matters. Indeed, President Kingham notes that her Honour is open to collaborating with other courts which may want to adopt the process.127
In criminal cases involving forensic scientific evidence, CMEEs may be employed to manage the information that is provided to forensic experts. Exposing forensic experts to biasing information (eg emotional case details) has a demonstrable effect on their decision making.128 This type of bias has contributed to numerous wrongful convictions. At the investigative stage, academics have proposed a case management system to control what experts are exposed to: “people at the various stages of the forensic investigation should determine which information is relevant and needed for the next stage. They will only convey that information while isolating any information that is irrelevant.”129 CMEE - in some cases - may provide a similar safeguard when forensic scientists are called in to be expert witnesses.
However, the CMEE process has limitations. Most importantly, although psychological literature indicates that Convenors using ADR techniques may reduce bias, they may not be able to cure bias in every case because of its subconscious nature. In light of this and other limitations, we suggest that courts should reserve the CMEE process for complex disputes involving numerous experts. The presence of a CMEE Convenor will be most useful in these cases. The Convenor can coordinate meetings and JERs which otherwise run a high risk of becoming disjointed.
In the future, President Kingham will engage an independent third party to evaluate the CMEE process. That evaluation will draw on the perspectives of Convenors, practitioners, and experts to identify the strengths and weaknesses of the process, as well as ways it could be improved.130 We suggest that, if possible, the evaluators should consider the potential effects of CMEEs on bias, as we have described.