This work introduces a distributional framework for SAE analyses, in which each feature is represented not by a single decoder vector like in the literature, but by an activation-weighted distribution over the hidden states that express it, and provides a unified semantic metric for cross-layer feature comparison.
T. M. Cao, D. Nguyen, My T. Thai· arXiv.org· 2 citations
The proposed EDRM (Entropy Dynamics-based Reasoning Manifold), a lightweight routing framework whose default heuristic requires no router training, supports selective reasoning based on early decoding signals and demonstrates the potential of entropy-based control for efficient LLM inference.
Wei Xia, Haoqing Wang, Zhi-Hong Deng et al.· 2 citations
Learning permutations is fundamental to sorting, ranking, and matching, but existing differentiable methods based on entropy-regularized Sinkhorn produce a single softened solution and collapse under ambiguity. We present PermFlow, a conditional flow matching framework that operates directly on the affine subspace of m...
Sampling from constrained distributions has a wide range of applications, including in Bayesian optimization and robotics. Prior work establishes convergence and feasibility guarantees for constrained sampling, but assumes that the feasible set is connected. However, in practice, the feasible set often decomposes into...
Cornelius V. Braun, Tilman Burghoff, Marc Toussaint· 0 citations
Reach audiences
Advertise in front of researchers, engineers, and readers.
Unlearning specific concepts in text-to-image diffusion models has become increasingly important for preventing undesirable content generation. Among prior approaches, sparse autoencoder (SAE)-based methods have attracted attention due to their ability to suppress target concepts through lightweight manipulation of lat...
Hyeonjin Kim, Hangyeol Jung, Heechan Yun et al.· 0 citations
Machine Learning Interatomic Potentials play a fundamental role in computational chemistry and materials science, enabling applications from molecular dynamics simulations to drug design and materials discovery. While recent approaches can estimate inter-atomic forces with high precision, it remains unclear to what ext...
Amir Masoud Nourollah, Irtaza Khalid, Stefano Leoni et al.· 0 citations
Unsupervised pretraining has driven empirical advances in goal-conditioned reinforcement learning (GCRL), but its theoretical foundations remain poorly understood. In particular, an influential class of methods, mutual information skill learning (MISL), discovers behaviorally diverse skills that can later be used for d...
Alireza Modirshanechi, Benjamin Eysenbach, Peter Dayan et al.· 0 citations
We investigate whether an explicit learned memory graph can replace the Feed-Forward Network (FFN) sublayer in a decoder-only transformer while preserving the surrounding autoregressive architecture. In the proposed graph Memory Transformer (GMT), a memory cell routes token representations over a learned bank of centro...
Existing evaluation methods largely rely on clean, static benchmarks, which can overestimate true model performance by failing to capture the noise and variability inherent in real-world user inputs. This is especially true for language models, which can face human-generated text queries containing mistakes, typos, or...
Angelika Romanou, Mark Ibrahim, Candace Ross et al.· 0 citations
On-policy agents discard trajectories after each update, including rare high-return episodes. We propose Optimistic Policy Regularization (OPR), which turns those episodes into a closed-form reward bonus: we estimate an elite action prior from top-quartile episodes and add a \reviewblue{bounded transform of the log-rat...
Recent studies have shown that reinforcement learning with KL-regularized objectives can enjoy faster rates of convergence or logarithmic regret, in contrast to the classical $\sqrt{T}$-type regret in the unregularized setting. However, the statistical efficiency of online learning with respect to KL-regularized object...
Kaixuan Ji, Qingyue Zhao, Heyang Zhao et al.· 0 citations
A hybrid Benders solver with output-sensitive iteration bounds is developed, leveraging a Helly-based conflict compression theory to avoid generic exponential cut explosion and establishes an unconditionally polynomial Benders iteration complexity in network size.
Wen-Xin Li, Wen-Chao Liu, An-Zhe Jiang et al.· 0 citations
