Dealing with hallucinations: a guide for international judges and arbitrators
TL;DR
A pragmatic, multi-stage framework specifically tailored for international courts and arbitral tribunals to identify, manage, and respond to AI-generated falsehoods is proposed, which emphasizes procedural fairness, a factor-based assessment of culpability and impact, and a graduated range of remedial measures available to adjudicators.
Abstract
The advent of generative AI (GenAI) has introduced a new challenge to legal practice: the proliferation of ‘hallucinations’, plausible-sounding yet fabricated legal content, including citations, facts, and legal rules. This article delves into the nature of artificial intelligence (AI)-driven hallucinations, examining their technical origins and the many reasons that can make legal professionals susceptible to these errors. It presents novel empirical data from a comprehensive database of adjudicated cases, charting the upward trajectory of this issue across diverse jurisdictions, and argues that hallucinations represent more than mere inaccuracies; they constitute a profound epistemic challenge. Far from being spared, international dispute settlement presents unique vulnerabilities to this challenge. Accordingly, the article proposes a pragmatic, multi-stage framework specifically tailored for international courts and arbitral tribunals to identify, manage, and respond to AI-generated falsehoods. This framework emphasizes procedural fairness, a factor-based assessment of culpability and impact, and a graduated range of remedial measures available to adjudicators. Ultimately, the article advocates for principled and cautious engagement with GenAI in international dispute resolution.