•  
  •  
 

Abstract

“[A]d quæstionem facti non respondent judices; . . . ad quæstionem juris not respondent juratores.” Judges do not answer questions of fact; jurors do not answer questions of law.

The jury is a central institution in the American level infrastructure. The Sixth and Seventh Amendments elevate the jury trial right to constitutional status. Of course, when a judge presides at a jury trial, the question that naturally arises is the division of decision-making labor between judge and jury. In the past few decades, the Supreme Court’s Apprendi v. New Jersey line of authority has expanded the jury’s authority to decide facts that come into play during sentencing. The Court has announced that a fact must be decided by a jury if the fact would have the significant impact of increasing the maximum imposable punishment or triggering a higher minimum sentence.

While Apprendi has raised the visibility of the question of the jury’s factfinding power during sentencing, the advent of AI has placed a new focus on the division of labor between judge and jury over the decision of another kind of fact, namely, facts conditioning the admissibility of evidence. The early American courts followed the traditional English view that the judge decides all questions of fact determining the admissibility of evidence. However, later the Jacksonian Democrats presented a convincing critique that the traditional English view enabled trial judges to formally or virtually dictate the jury’s decision.

In the 1920s, to implement that critique leading American scholars developed a new binary procedure for determining preliminary facts. In one category in this procedure, competence, the trial judge follows the English practice. For example, in the case of privileges, the trial judge decides the preliminary facts. The judge listens to the foundational testimony on both sides, passes on credibility, and makes a final decision on the facts. Realistically, it would undermine privilege rules to assign the jury the task of deciding the preliminary facts. It is not that lay jurors are incompetent to decide whether a third party was so close to a client conversing with their attorney that the third party’s presence negated the client’s reasonable expectation of privacy. Rather, the point is that even if the jury were to decide that the evidence is technically inadmissible, there is a grave risk that the jury’s exposure to the evidence would taint their deliberations.

In the second, smaller category, conditional logical relevance, the judge plays only a limited, screening rule and leaves the real decision to the jury. In the case of the preliminary facts of lay witnesses’ personal knowledge and the authenticity of exhibits, it is safe to entrust the decision-making authority to the jury. If the jury decides that the witness “doesn’t know what he’s talking about,” or that the exhibit “isn’t worth the paper it’s written on,” lay common sense will naturally lead the jury to disregard the testimony during the balance of their deliberations.

The scholarly commentary is in agreement that the products of AI tools should not be admitted unless the proponent establishes that the embedded algorithm is reliable. However, the commentary is in conflict on the related issue of whether the judge or jury ought to decide the reliability question. Some commentary refers to Federal Rule of Evidence 104(a) (codifying the competence procedure) while others cite Rules 104(b) and 901(a) (codifying the conditional relevance procedure). Rule 901(b)(9) lists proof of the validity of a “process or system” as one of the preliminary facts subject to Rule 901(a).

This Article argues that for the most part, the question of the reliability of an AI tool should be treated like the question of the validity of run-of-the-mill scientific methodologies governed by Rules 104(a) and 702. While the jury has a simple, binary choice whether or not a witness saw an accident, validation testimony is usually probabilistic in nature. Moreover, while the proponent can often lay a foundation establishing a witness’s personal knowledge or a letter’s authenticity in less than a minute, the foundational testimony for scientific methodology is usually much lengthier, often consuming hundreds of pages of transcript. Finally, while jurors can draw on their fund of lay experience and knowledge to evaluate the personal knowledge and authenticity issues, they ordinarily have to exert much greater mental effort to understand scientific evidence. All three factors make it unrealistic to think that jurors can readily put scientific evidence completely out of mind even if, at a conscious level, they decide that the evidence is technically inadmissible. For that reason, the reliability of an AI tool should be classified as a 104(a) competence issue.

However, the application of Rule 104(a) should not be the end of the judge’s analysis. Some AI tools differ significantly from typical scientific methodologies and instruments in the sense that the tools are more autonomous. Some tools following the unsupervised machine learning model can collect and identify their own training examples. The tools can evolve operational rules that even their original developers neither know of nor fully understand. The tools’ lack of transparency makes it much harder for the jury to intelligently decide how much weight to ascribe to the evidence. Federal Rule of Evidence 403 empowers judges to exclude otherwise admissible evidence when the judge believes that the attendant probative dangers substantially outweigh the probative value of the item of evidence. One of those dangers is that the jury will overestimate the probative worth of the evidence and, for that reason, reach an erroneous verdict. AI tools employing unsupervised machine learning pose that danger to a much greater degree than ordinary scientific methodologies, and for that reason judges should apply Rule 403 more aggressively to such AI tools. Together Rules 104(a), 403, and 703 comprise a sound framework for evaluating the admissibility of proffers of AI evidence.

Download

Share

COinS