World models learn state evolution from trajectories, making access to temporal context a central training requirement. Federated learning can use distributed records, while ownership boundaries within a trajectory restrict the examples each client can construct. Our study benchmarks this cross-time setting through hou...
Yi-Pan Wei, Zhao-Kun Yan, Zi-Ming Hong et al.· 0 citations
Methods for training language models on stale samples are judged by comparisons against importance-corrected baselines. We show that details of the experimental harness can reverse the observed ranking of methods, and we introduce PTH (Probe The Harness), a set of checks that makes the harness visible. Our case is a co...
We show that the causality of language models (LMs) may not be necessary nor optimal. This is the case when system behavior (denoted as $S$) is incorporated as a first-principle Bayesian feature. Here, $S$ refers to extra dominant factors beyond the data space, and they involve coupled effects. Despite being the de fac...
On-policy self-distillation (OPSD) supplies dense token-level feedback on trajectories sampled from the student's own policy, a richer training signal than the outcome-level rewards of reinforcement learning. This feedback comes from a teacher conditioned on a full reference solution unavailable to the student. The ref...
Rui Li, Li-Yang He, Zheng Zhang et al.· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
LLM compression reduces inference costs and memory requirements, but selecting a method and configuration remains largely empirical because comparable resource reductions can produce different capability losses. We systematically investigate capability scaling-down laws for LLM compression across pruning, quantization,...
Xue-Qi Cheng, Liang Wu, Kelly Wan et al.· 0 citations
This work trains a model over the simplest possible harness: a tool to call itself with any specified prompt and a tool to read tokens in a range from the input context over a diverse synthetic dataset.
When a large language model solves a mathematical problem, its reasoning is largely hierarchical, and the solution often branches at a few tokens where the next-token entropy is high. Such tree-like structure embeds in hyperbolic space with far lower distortion than in Euclidean space. Activation steering, however, usu...
Ze-Yong Zhang, Tung Sum Thomas Kwok, Teng-Fei Ma et al.· 0 citations
The results show that rich personalization can be supported primarily by reusable, conditionally composed shared capacity, while independent user adaptation remains confined to a tiny correction state.
Song-Yuan Sui, Srikanth Malla, Chi-Ho Choi et al.· 0 citations
As LLMs are increasingly deployed as autonomous adjudicators in games such as Call of Cthulhu (CoC), robust rule adherence becomes critical when user intent conflicts with system rules. However, as these models are trained to be helpful and compliant, they may be vulnerable to a class of manipulations we term Rhetorica...
Weiying Chen, Junlong Shen, Zhanyuan Guo et al.· 0 citations
Hallucination detection has become a pressing requirement for trustworthy AI deployment at scale. The most accurate detection methods depend on GPU-intensive inference, proprietary API calls, or white-box access to the generating model, putting them out of reach for resource-constrained researchers and practitioners. W...
Turkish is agglutinative: meaning is carried by morphemes, yet the subword tokenizers that drive modern language models split words by corpus statistics, fragmenting semantically loaded suffixes and -- in the case of WordPiece and rule-based analyzers -- failing to decode their output back to the original text. This pa...
This work constructs TypewriterCorpus, a 54B-token historical corpus with extensive temporal filtering, proposes lexically grounded instruction tuning that constrains all responses to vocabulary from historical source documents, and introduces History-Event, a benchmark of 2,344 events for evaluating both competence an...
Xiaoxi Luo, Zachary Shinnick, Niclas Griesshaber et al.· 4 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.