From Messy Country Names to ISO 3166 Codes: A Small Model That Abstains, and When a Lexicon Is Enough

Abstract

Company registers and forms record countries as free text: in any language, misspelled, abbreviated, historical, or not a place at all. CountryCode-Decide is a 74.7 MB open model (mmBERT-small, two heads, 8-bit ONNX) that maps such a string, read with its field and register, to an ISO 3166-1 alpha-2 or ISO 3166-2:US code with a calibrated probability, and abstains when a requested error rate cannot be met; thresholds are certified by fixed-sequence testing on held-out register strings. Generated training data passed quality gates and a human label audit, after the first audit failed 9 of 10 releases. On two registers never used in development (361 strings, read once after a frozen pre-registration) the model is 88.6% accurate against 85.0% for a 35B LLM that is 29 times slower, and all pre-registered hypotheses were met. Post-hoc analyses, prompted by external questions, qualify this. An exact lexicon with the training context rules reaches 89.5% there, and on the strings a person had to label model, LLM and lexicon are level. The pre-registered 2% certificate counted repeated strings as independent; counted per register and key, only 5% is certified. Run as a cascade, lexicon first, the model raises the share of messy register strings answered automatically from 24.8% to 79.0% at 4.8% error, but its certificate does not hold on what the lexicon cannot read: on the fresh registers 5 of its 8 answers there were wrong. Code, weights and aggregate results are released under Apache-2.0.