We introduce FlexTab, a flexible encoder-decoder architecture for in-context learning on tabular data that pairs a single, task-agnostic encoder with a suite of task-specific decoders. Unlike existing tabular in-context learners, which entangle feature representations with a specific prediction target, our design produ...
Marek Polewczyk, Maximilian Schambach, Marco Spinaci et al.· 0 citations
Heterogeneous graph neural networks (HGNNs) perform well on graphs with multiple node and relation types, but their robustness to attacks with restricted structural access remains insufficiently understood. We propose Blackknife, a query-free, strict black-box evasion attack that observes only the one-hop structures of...
Honglin Gao, Junhao Ren, Lan Zhao et al.· 0 citations
Lifelong continual learning remains an obstacle on the path to human-like intelligence. Modern transformers show sparks of intelligence with in-context learning. The quadratic nature of attention, however, prohibits transformers from performing this process on arbitrarily long sequences. In this work, we argue that ext...
Luke McDermott, Robert W. Heath Jr., Rahul Parhi· 0 citations
Learning accurate value functions plays a decisive role for reinforcement learning (RL) agents to solve long-horizon, complex tasks. Conventional temporal-difference (TD) learning objectives suffer from value-estimation bias that accumulates over the horizon, while extended-horizon modeling methods, such as n-step TD b...
This paper develops constrained diffusion models with primal-dual inference (PDI) to sample from optimal distributions of entropy-regularized optimization problems with \emph{average} constraints. We formalize constrained sampling in the Lagrangian dual domain, where the optimal distribution takes the form of a Gibbs d...
Reward models (RMs) are a key component of reinforcement learning from human feedback, enabling the alignment of large language models with human values such as helpfulness and harmlessness. However, these alignment goals can conflict, leading to interference when optimised jointly. We refer to this conflict as alignme...
Latent chain-of-thought compresses reasoning through continuous hidden-state recurrence, but existing approaches are difficult to optimize with standard on-policy reinforcement learning (RL) and hard to analyze mechanistically. We introduce SWITCH, a switchable latent reasoning framework that addresses both challenges...
Jiayu Yang, Chao Chen, Shengen Wu et al.· 0 citations
Analyses of grokking usually read hidden activations in one basis, most often the Fourier basis modulo $p$. We ask how much the answer to "has the network organised yet?" depends on that choice. The measure FSD$_{\rm gen}$ averages each unit's activation over inputs with the same answer and counts how many units share...
Post-training quantization (PTQ) is essential for efficient large language model (LLM) inference, but reliably quantizing activations remains challenging when weights, activations, and KV caches are all quantized to 4-bit precision. A key difficulty lies in massive activations, whose extreme values dominate the activat...
Zimo Zhao, Maolin Wang, Bowen Yu et al.· 0 citations
The IEEE P3109 draft standard defines a parameterized family of binary floating-point formats and associated operations, with a focus on facilitating machine learning. These formats allow efficient and consistent representation of values in a small number of bits. The defined formats are parameterized over width and pr...
Andrew Fitzgibbon, Christoph M. Wintersteiger, Jeffrey Sarnoff· 0 citations
Dynamic medical treatment requires deciding treatment intensity and intervention timing, while patient states evolve continuously and adverse events may occur between clinical interactions. Most existing treatment learning methods assume fixed schedules or enforce safety only at discrete decision points. We propose Int...
Yuepeng Wang, Ken Kawano, Akifumi Wachi et al.· 0 citations
Every tabular classification task requires sufficient labeled data. When labels are scarce, practitioners can collect more examples, encode domain expertise into features, or accept lower performance. Large language models offer an alternative by leveraging knowledge acquired during pretraining. This raises three quest...
Daria Grushina, Kseniia Kuvshinova, Alina Kostromina et al.· 0 citations
A machine-learning approach called CheMeleon provides a practical way to develop new drugs or other molecules with much less detailed, expensive experimentation.
Research Manager Mike Walker reflects on a career spanning hacker culture, cybersecurity, and AI, sharing lessons about trust, teamwork, leadership, and why solving difficult problems requires curiosity, judgment, agility—and an understanding of the rules. The post Learn the rules: The hacker mindset appeared first on Microsoft Research.
Learn how Google Antigravity and our Cheminformatics Skills accelerate drug discovery workflows, search compounds, and propose novel rigidified macrocycles.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.
