Information Asymmetry in the Legal Services Market: Creating a Ranking Using Machine Learning Methods

Abstract

How can you tell whether a defense attorney is any good? In this article, we describe a methodology for measuring the quality of legal representation by comparing machine learning predictions based on objective case characteristics with actual case outcomes. Our premise is that if an attorney's cases consistently end better than expected, they are probably doing their job well. Similar logic has long been used in other fields: it is how teachers are evaluated in the economics of education, surgeons in healthcare, and players in professional sports. We adapt these approaches to criminal proceedings. Such a predictive model can be trained on Russian court data from 2010–2025 and would account for the fact that a criminal case passes through several sequential procedural decision points. The resulting rating is adjusted for teamwork among multiple defense attorneys, the stage of proceedings, and small caseloads. We discuss the weaknesses of the approach: most importantly, the pretrial stage is invisible to us, published decisions are incomplete, and skilled attorneys may systematically select more difficult cases. Nevertheless, even in this form, such an evaluation tool can provide clients with a meaningful and objective benchmark when choosing a defense attorney, which could substantially improve the quality of information in the legal services market.