Skip to content

QQ: A Language Metadata Toolkit for Multilingual NLP

Feb 2026 · 1 citation · 36 references
Computer Science

TL;DR

This work presents QQ, a metadata toolkit and browser explorer that compiles language metadata sources into a graph of language varieties, scripts, regions, identifiers, names, and relations, and exposes it through a Python API, a command-line interface, and a browser-based explorer.

Abstract

Multilingual NLP research increasingly involves hundreds or thousands of languages across different datasets. Managing, discovering, and reporting language metadata becomes a common hurdle at these scales. We present QQ, a metadata toolkit and browser explorer. QQ compiles language metadata sources into a graph of language varieties, scripts, regions, identifiers, names, and relations, and exposes it through a Python API, a command-line interface, and a browser-based explorer. Users can normalize identifiers, retrieve metadata, traverse relations, and discover which external resources contain a language. We demonstrate QQ on three workflows: an audit of the HuggingFace Hub, linking resources that use different identifier systems, and generating reproducible language-reporting tables. QQ supports FAIR-oriented metadata practices through versioning, open formats, and reusable interfaces.

View source

Similar papers

2026

PolyglotQL: A Pipeline for Multilingual Text-to-SPARQL Dataset Generation

PolyglotQL provides an extensible and modular architecture that aggregates, normalizes, and augments heterogeneous question–SPARQL pairs from established text-to-SPARQL datasets, highlighting the benefits of structured context in multilingual semantic parsing.

J. Pérez, Fabio Barth, Georg Rehm · 0 citations
Open access 2026

TextLens & LeTTuce: Automated Corpus Annotation and Multilingual Tagging as a Service

We present TextLens , a web-based platform for automated linguistic annotation designed to lower technical barriers for researchers in digital humanities, linguistics and translation studies. Hosted by the Dutch Language Institute (INT), TextLens allows users to upload and annotate corpora in a variety of formats (.txt...

Cynthia Van Hee, Jonas Doumen, Vincent Prins et al. · 0 citations
Sep 2026

Natural-language processing (NLP) and indexing, Part 3. Semantic search

This article explores asymmetric extractive semantic search, using the large language model (LLM) SBERT. It first examines traditional exact, keyword, and fuzzy match search algorithms, discussing their features and limitations. It then explores how semantic search leverages the capacity of an LLM to capture query inte...

Donald Howes · 0 citations
Conference

TABERTA: A Language Model for Dataset Discovery

TABERTA is presented, a unified retrieval framework for dataset discovery that adapts a bi-encoder language model to relational data by fine-tuning it on serialized representations of tables by fine-tuning the language encoder to diverse structural patterns using optimized strategies that serialize schema and table con...

Enas Khwaileh, L. Traeger, Albert Gatt et al. · 0 citations
Review Open access Sep 2026

Accelerating metadata annotation in collaborative research centers: A hybrid AI workflow for biomedical entities

Collaborative Research Centers rely on FAIR-compliant, richly structured metadata, yet manual annotation is a major bottleneck. We implemented a search-augmented large language model (LLM) workflow within a local research data management system to pre-annotate biomedical entities, using human-in-the-loop verifica...

M. Watter, F. Engel, Aref Kalantari et al. · 0 citations

Related blog posts

MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

MIT News · Artificial Intelligence Sep 24, 2026

Estimating suicide risk from text

A new language-processing tool could help identify the highest-risk individuals from natural language, enabling swifter interventions.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.