Skip to content

Efficient Reasoning Exploration via State-Conditioned Latent Steering with Progress Guidance

Sep 2026 · 0 citations · 51 references
Computer Science

TL;DR

SPS is a training-free latent steering framework that constructs a state-conditioned Direction Bank containing multiple progress-guided steering vectors for different prefix-state regions and applies it at high-uncertainty transitions to guide the next reasoning step toward meaningful progress.

Abstract

Best-of-$N$ is a widely used inference strategy for complex reasoning, whose effectiveness depends on whether sampled candidates can cover diverse and high-quality reasoning paths. However, post-trained reasoning models often suffer from \emph{exploration collapse}, where independent rollouts repeatedly follow similar reasoning paths and limit the gains from increasing the rollout budget. Existing methods alleviate this issue by promoting broader exploration, but do not explicitly guide exploration toward continuations that make meaningful progress, resulting in limited exploration efficiency. To address this, we propose \emph{\underline{S}tate-conditioned \underline{P}rogress-guided \underline{S}teering} (SPS), a training-free latent steering framework. Specifically, SPS constructs a state-conditioned Direction Bank containing multiple progress-guided steering vectors for different prefix-state regions. During online inference, SPS retrieves a suitable steering vector based on the current prefix state and applies it at high-uncertainty transitions to guide the next reasoning step toward meaningful progress. Extensive experiments across multiple model scales and benchmarks demonstrate that SPS consistently outperforms strong baselines. Further analyses validate the effectiveness of its key designs and offer valuable insights for future research. The code is available at https://github.com/rattlesnakey/SPS.

View source

Similar papers

#machine learning Preprint Sep 2026

Sampling via Decision-Flow: Training-Free Extraction of Improved Latent Reasoning Paths in Large Language Models

Decision-Flow Sampling (DF-Sample), a training-free, data-free inference-time framework that constructs a hierarchical reasoning tree, scores terminal nodes for quality, and back-propagates utilities to inform each intermediate branching decision, indicating substantial latent reasoning potential in pretrained base mod...

Zhen-Dong Mi, Shao-Yi Huang · 0 citations

Efficient Reasoning via Constrained Optimization in Latent Space

This paper proposes a novel training-free framework to achieve efficient reasoning that reduces token generation costs without sacrificing performance, and investigates the latent representations and proposes a novel training-free framework.

Zhi-Nan Hou, Xing-Chen Li, Ke-You You · 2 citations

The Model Knows Another Way: Strategy Switching for Effective RLVR Exploration

This work introduces Problem--Strategy Rollout Allocation (PSRA), which treats unguided and strategy-conditioned prompts as competing exploration arms and uses Bayesian sequential allocation to direct a fixed rollout budget toward arms most likely to yield informative, non-saturated groups.

Jin Cui, Xin-Yue Long, Bo-Ran Zhao et al. · 1 citation
#natural language process... Preprint Sep 2026

Explore Before Committing: Hypothesis-Guided Search for Deep Research Agents

HypoSearch is proposed, which generates lightweight hypotheses as soft search hints, explores them through bounded independent branches, and compares branch-level evidence before commitment, and consistently outperforms single-trajectory search and standard parallel baselines.

Ruo-Chen Zhou, Zheng-Zong Chen, Luan Zhang et al. · 1 citation

Learning from Think-Mode Advantage via On-Policy Distillation

ThinkOPD outperforms Uniform ThinkOPD in both same-model settings and both cross-model teacher-student pairs, and it exceeds representative rationale and self-distillation baselines in a controlled comparison.

Wan-Qi Ren, Jian-Xiang Wang, Dan-Xuan Liu et al. · 0 citations
#machine learning Preprint Sep 2026

Dense Process Supervision for Search Agents via Fact Utility Estimation

A dense process supervision method based on fact utility estimation, which models the reasoning process as the accumulation of discrete evidence facts, and converts the estimated fact utilities into dense step-level rewards to guide RL training.

Rongzhi Zhu, Xiang-Yu Liu, Yi Liu et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.