Pixel-based language models process rendered text without relying on a fixed subword vocabulary, offering a promising approach to noisy and multilingual inputs. However, their practical value depends on whether they can be reliably adapted to downstream tasks. To this end, we systematically examine this question throug...
Hao-Wen Tang, Wei Xia, Rao-Yuan Zhao et al.· 0 citations
On-policy distillation can transfer the reasoning capabilities of a strong teacher to a smaller student. However, distillation from a stronger teacher model faces two mismatches: its solutions may be difficult for a smaller student to imitate, and its token-level probabilities may be incompatible with, or even unavaila...
Run-Sheng Chen, Yihong Liu, Rao-Yuan Zhao et al.· 0 citations
It is found that embedding-based alignment metrics do not reliably indicate whether alignment will improve or degrade downstream performance, and that the compatibility between alignment and downstream objectives should be considered when designing/evaluating alignment methods.
Yana Veitsman, Yihong Liu, Hinrich Schütze· arXiv.org· 0 citations
The Last Translation Benchmark is introduced, a collection of human-authored and peer-reviewed examples that break leading machine translation models and a new evaluation approach: each example comes with handcrafted verification rules describing concrete failure cases on that example, therefore allowing reliable and a...
Vilém Zouhar, Niyati Bafna, Mukund Choudhary et al.· 0 citations
Findings show that successful MoE-block restoration does not necessarily imply localization to a single expert, as it is shown that successful MoE-block restoration does not necessarily imply localization to a single expert.
Yuetian Lu, Ali Modarressi, Yihong Liu et al.· arXiv.org· 0 citations
This work proposes a pipeline for automatically generating step-by-step linguistic reasoning traces from Universal Dependencies treebanks, dictionaries, and grammar-rule banks and shows that linguistic reasoning traces are most effective as inference-time guidance in ICL, which substantially improve translation perform...