Matryoshka Representation Learning (MRL) is a widely adopted approach for training text encoders so they provide useful text representations at various sizes, available by simply truncating the resulting vectors at sizes pre-determined at training time. Recent works have shown that randomly truncating text embeddings h...
Sotaro Takeshita, Yurina Takeshita, Simone Paolo Ponzetto et al.· 0 citations
This work introduces MulTaBench, a benchmark of 40 datasets, split equally between image-tabular and text-tabular tasks, designed to enable the research of novel architectures which incorporate joint modeling and target-aware representations, paving the way for the development of novel Multimodal Tabular Foundation Mod...
Alan Arazi, Eilam Shapira, Shoham Grunblat et al.· arXiv.org· 3 citations
Targeted data selection seeks training examples from a candidate pool that improve downstream task performance. While trajectory-based selectors effectively guide this process, existing approaches typically construct reference states by warming up on the candidate pool itself, thereby inheriting pool-dependent distribu...
Huitao Yang, Hengzhi He, Tung Sum Thomas Kwok et al.· 0 citations
Flux Attention is introduced, a context-aware framework that dynamically optimizes attention computation at the layer level by integrating a lightweight Layer Router into frozen pretrained LLMs, which adaptively routes each layer to FA or SA based on the input context.
Quantong Qiu, Zhiyi Hong, Yi Yang et al.· arXiv.org· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
When a model solves a problem on one attempt and fails it on the next, what separates the two is rarely the final answer token; it is the trajectory that reached it. Contrastive activation steering leaves that signal unused: CAA, SADI, RepE and ITI build their direction from experimenter-supplied text, recorded while t...
Xuanbo Su, Yingfang Zhang, Hao Luo et al.· 0 citations
As frontier AI systems are pretrained on web-scale data, test set contamination has become a critical concern for accurately assessing their capabilities. While research has thoroughly investigated the impact of test set contamination on discriminative evaluations like multiple-choice question-answering, comparatively...
Rylan Schaeffer, Joshua Kazdan, Baber Abbasi et al.· 0 citations
This work presents AutoSDT, an automatic pipeline that collects high-quality coding tasks in real-world data-driven discovery workflows and is the only automatically collected and the largest open dataset for data-driven scientific discovery.
Yifei Li, Hanane Nour Moussa, Ziru Chen et al.· Conference on Empirical Meth...· 11 citations· ⚡1
ICER is a black-box framework that addresses the gap in text-to-image models in harmful content generation through two components: an LLM-based rewriter that produces fluent, natural-language adversarial prompts, and in-context experience replay that accumulates successful jailbreaking patterns into a reusable prior.
Zhi-Yi Chin, Pin-Yu Chen, Wei-Chen Chiu et al.· 2 citations
Looped transformers have demonstrated promising parameter efficiency by reusing layers for latent computation. Prior studies compare looped and non-looped models at matched parameters or per-token FLOPs. However, to the best of our knowledge, whether looping improves test-time scaling as outputs grow longer remains und...
A language-model agent is jointly defined by its model and its harness, the executable program that organizes model calls, tool use, and information flow. Because different tasks call for different ways of organizing these operations, the harness needs to be adapted using feedback from the task at hand. We introduce ha...
Alvin Zhang, Xue-Chen Liu, Zi-Xuan Wang et al.· 0 citations
A CS-ASR synthetic benchmark and multidimensional evaluation framework tailored to enterprise domains, systematic evaluation of frontier ASR systems across 5 language pairs, and diagnostic analysis of the additional transcription errors that code-switching introduces across language pairs and models are proposed.
Shama Gupta, Hoang Nguyen, Chelsea Huang et al.· 0 citations
This investigation using an operator-level mechanistic analysis of attention and feed-forward blocks reveals that these blocks systematically ignore MA coordinates while reading, but not while writing; creating a read-write asymmetry that blocks corrective feedback while allowing continued accumulation.
Swagata Mukhopadhyay, Vishal Vivek Saley, Vraj Parikh et al.· 0 citations
Flint-interactive gives AI agents a language for creating interactive visualizations. Click a chart, filter data, highlight related content, and explore beyond the view the author originally chose. The post Flint-interactive: A language for creating interactive data visualizations appeared first on Microsoft Research.
