Can candidates trust a confident answer? Free and paid large language models on dental specialty examination questions

Abstract

Response-level dataset and analysis code for a study of stated confidence and repeated-query consistency in seven large language models answering the oral and maxillofacial radiology questions of the Turkish Dental Specialty Examination (DUS), 2012–2026. Seven models from four vendors, including three pairs available at different consumer access tiers, each answered 199 text-based questions five times as separate, history-free queries, reporting a confidence value of 0–100 with every answer (6965 completed responses, collected 21–24 September 2026). The deposit contains one row per response, the raw text returned by each model, the data-collection and analysis code, and the analysis output tables. The dataset contains no examination question text. The question stems, answer options and official answer keys are published by the Assessment, Selection and Placement Centre (ÖSYM) and are not redistributed here; questions appear only as identifiers S001–S199. No API credentials are included. Random seeds are fixed, so the analysis reproduces the reported tables exactly. See README.md for the column dictionary and instructions for re-running the analysis. Licence: data under CC BY 4.0; code under the MIT Licence (see LICENSE-CODE in the deposit).