GeoSteer is proposed, an optimization-based method for norm-preserving activation steering that consistently improves over state-of-the-art activation steering baselines and replaces predefined one-step edits with adaptive, geometry-aware optimization.
Abstract
Activation steering provides a lightweight way to control large language models (LLMs) by modifying their hidden activations at inference time. Among these approaches, norm-preserving steering aims to change model behavior without altering the activation norm, reducing the risk of representation collapse and degradation. However, existing norm-preserving methods are limited by predefined steering trajectories and by their reliance on one-step updates, which may fail to capture the complex structure of activation distributions. We propose GeoSteer, an optimization-based method for norm-preserving activation steering. GeoSteer formulates steering as a Riemannian optimization problem and updates activations through a sequence of small geodesic steps on the representation manifold. To avoid fixed steering directions, GeoSteer learns a nonlinear activation-space objective that distinguishes desired from undesired activations, and uses this function to adaptively guide each steering step. This multistep formulation yields smoother, more stable, and more consistent steering behavior while preserving the activation norm. Across TruthfulQA, RealToxicityPrompts, and UltraFeedback benchmarks, GeoSteer consistently improves over state-of-the-art activation steering baselines. These results suggest that norm-preserving steering can be made more effective by replacing predefined one-step edits with adaptive, geometry-aware optimization.
Steering Vector Dissection is proposed, a framework to explicitly isolate individual and semantically consistent features from these composite directions that tend to entangle multiple semantic and stylistic concepts into a single composite direction, leading to unpredictable steering effects.
Takeru Hiramatsu, Kyohei Atarashi, Koh Takeuchi et al.· 2 citations
This work proposes a disentanglement-based intervention framework, termed IB-ACT, that identifies both where and how to intervene by exploiting the layer-wise geometry of representation spaces to isolate behavior-relevant information without disturbing other dimensions, and introduces a layer-selection mechanism determ...
Toan Doan, Thin Nguyen, Sunil Gupta· Proceedings of the 32nd ACM...· 0 citations
LocUS (Localized Unembedding Steering), a method which grounds activation steering to the model's own output vocabulary subspace within the unembedding matrix, enforces a geometric constraint that restricts the steering transformation to a specific subspace and at the same time localizes its application to a sparse sub...
Irene Tallini, Lorenzo Basile, Valentino Maiorca et al.· 0 citations
It is found that distribution-driven methods recover human value topologies aligned with theoretical predictions, and better geometric alignment also leads to more human-consistent transfer across values: steering one value correctly lifts compatible values and suppresses opposing ones.
Mohammad Mahdi Abootorabi, Armin Saghafian, Ali Bazshoushtari et al.· Proceedings of the 2026 Conf...· 2 citations
Manifold-valued data, and consequently the distributions they induce, are prevalent across many domains, ranging from the locations of geospatial events, such as earthquakes, to biomolecular torsion angles that encode information about three-dimensional structure. While diffusion and flow-based generative models have b...
Ualibyek Nurgulan, Seungwoo Yoo, Prin Phunyaphibarn et al.· 0 citations
GeometricSPRINT (Geometric Step Pruning for Inference in Trajectories), a training-free framework for constructing non-uniform sampling schedules from the geometry of denoising trajectories, consistently improves over uniform DDIM (Denoising Diffusion Implicit Models) schedules at matched NFE budgets.
Arpita Joshi· 0 citations
Related blog posts
MIT News · Artificial Intelligence· news.mit.eduNov 17, 2026
By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.
Microsoft Research Blog· microsoft.comNov 16, 2026
AI coding agents are increasingly help developers fix bugs and build features largely on their own, but every interaction with their models consumes tokens. In each session, an agent sends instructions, tool descriptions, and session history to its model. Most efforts to reduce this cost focus on individual tasks. Yet developers return to the same […] The post TokenEconomizer: Learning from Experience to Make AI Coding Agents More Efficient appeared first on Microsoft Research.
MIT News · Artificial Intelligence· news.mit.eduNov 9, 2026
Assistant Professor Benjamin Lindquist studies how engineers deliberately injected randomness into early AI, giving rise to deep-learning models that sometimes hallucinate.
We use cookies to run the site and, with your consent, for analytics and to show ads.
See our Cookie Policy.