Skip to content
Log in
Sponsored AI AM CLAUDE, A Beginner's Guide: Learn the basics of working with AI AI AM CLAUDE, A Beginner's Guide: Learn the basics of working with AI

Author

B. Gubler

View on Semantic Scholar

1 paper indexed here

We haven’t gathered this author’s papers yet. Follow them and we’ll fetch their work.

Not the right person? Other researchers publish under this name.

#artificial intelligence Preprint Mar 2026

Tokka-Bench: Evaluating Tokenizers Across 100 Natural and 20 Programming Languages

Tokka-Bench is introduced, an open-source framework that evaluates tokenizers on five complementary metrics -- bytes per token, unique token coverage, subword fertility, word-split rate, and vocabulary composition -- across 100 natural languages and 20 programming languages, using language-aware segmentation adapted to...

B. Gubler · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.