Skip to content

Jailbreaks for Black-Box Uncertainty Quantification in Large Reasoning Models

Sep 2026 · arXiv.org · Vol abs/2609.35350 · 0 citations · 63 references
Computer Science

TL;DR

J4U is proposed, a jailbreak-derived technique for UQ that empirically reproduces the behavioral signatures predicted by the relaxation theory and provides a practical tool for UQ in black-box LRM deployment.

Abstract

While Large Reasoning Models (LRMs) excel at complex reasoning, alignment through reinforcement learning often induces systemic overconfidence. In production environments, where logits may be unavailable, robust black-box uncertainty quantification (UQ) is essential for trustworthiness and safety. Focusing on question-answering for LRMs, we show that existing black-box methods, such as paraphrase-based self-consistency and confidence verbalization, offer little to no improvement over simple repeated sampling, suggesting that alignment suppresses useful output variability. We introduce prompt-level relaxation operators that broaden the model's effective output distribution by approximating the effect of an optimal policy obtained with a stronger KL-regularization parameter, hence closer to the reference model. Theoretically, we demonstrate that relaxation improves calibration. We propose Jailbreak for Uncertainty (J4U), a jailbreak-derived technique for UQ that empirically reproduces the behavioral signatures predicted by our relaxation theory. Across 3 datasets and 4 LRMs, including a closed-source production model, J4U's improvement over repeated sampling achieves statistical significance in up to 6 times more LRM-dataset-metric settings than the strongest black-box UQ state-of-the-art baseline we evaluate, with average ECE reductions up to 5 times larger. These results provide a practical tool for UQ in black-box LRM deployment.

View source

Similar papers

On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Models

It is revealed that off-the-shelf LRMs exhibit a confidence prior heavily concentrated on a few high values, which persists throughout RL, and it is proved that this concentration suppresses policy gradient updates for rarely sampled confidence values and inflates the lower bound on expected Brier risk.

Shuo-Yuan Wang, Bei-Er Luo, Hao Zeng et al. · 1 citation

Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models

This work introduces DSAR (Deceptive Safety Alignment Rate), a metric that jointly assesses reasoning traces and final answers to quantify their safety inconsistency, and proposes SARA (Safety-Aware Reasoning Alignment), an RL-based method that rewards both safety-aware reasoning and safe final answers.

Xiang-Yu Zhou, Saleh Zare Zade, Rafi Ibn Sultan et al. · 0 citations
#machine learning Preprint Oct 2026

Improving Reasoning with Rollout Agreement for Group-Based Policy Optimization

Recent methods for Reinforcement Learning with Verifiable Rewards (RLVR) use rollout diversity to improve reasoning performance or training efficiency. Despite these gains, the extent to which differences among correct-answer rollouts reflect useful alternative reasoning or unresolved errors remains poorly understood....

Sohyun Son, Yerim Hong, Jaeyun Park et al. · 1 citation
Preprint Nov 2026

Not All Reasoning Generalizes: A Case Study on Learning Generalizable Reasoning Strategies in RLVR

Reinforcement learning with verifiable rewards (RLVR) can enable generalizable improvement for large language model (LLM) reasoning. This paper studies how to incentivize such generalization via reasoning strategies. We study this question with a controllable equation-solving framework: finding multiple inputs x for eq...

Zi-Qi Pang, Yu-Xiong Wang · 0 citations
#machine learning Preprint Oct 2026

Process Reward Modeling with Reliability- and Conservatism-Aware Bayesian Belief

BayesianPRM is proposed, which treats the process reward as a random variable within a reward hypothesis space and maintains a Bayesian belief over it, and yields a reliability- and conservatism-aware Bayesian belief, whose expectation determines the process reward.

Zhong-Jian Qiao, Zhi-Xiang Zhou, Jiafei Lyu et al. · 1 citation

Related blog posts

MIT News · Artificial Intelligence Nov 17, 2026

Learning the language of machine learning

By studying the computational foundations of language learning, Associate Professor Jacob Andreas builds intelligent systems that can communicate effectively with humans.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.