For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. As models get stronger, standard benchmarks for machine translation are approaching saturation. Further, automatic translation metrics are unreliable, opaque, and vulne...
Vilém Zouhar, Niyati Bafna, Mukund Choudhary et al.· 0 citations
Extensions to XNLI support the source-retention and intervention findings on a harder task and two model families, showing that circuit-targeted adaptation provides a more controlled, intervention-supported alternative to global fine-tuning.
Khumaisa Nur'aini, Ayu Purwarianti, Alham Fikri Aji et al.· 0 citations
This work introduces EDRAC, the first large-scale benchmark for dialectal Arabic machine reading comprehension (MRC) and generative QA, covering five major dialects: Egyptian, Moroccan, Emirati, Syrian, and Saudi Arabic, and benchmarks Arabic-centric and multilingual LLMs on EDRAC using lexical and semantic metrics.
Noor Abo Mokh, K. Chirkunov, Teresa Lynn et al.· 0 citations
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.