Temperature Sampling is Entropy-Optimal: An Information-Theoretic Framework for LLM Decoding

Entropy-Guided LLM Decoding via Probabilistic Circuits

Chopthin-Consensus Power Sampling: A Diversity-Preserving Approach to LLM Decoding

Inference-time power sampling via Sequential Monte Carlo (SMC) can substantially improve large language model (LLM) reasoning without requiring post-training. However, many existing SMC approaches rely on equal-weight resampling, which can aggressively prune low-weight trajectories, discarding potentially correct reaso...

A Pairwise-Error-Probability Framework for One-Shot Information Theory

We develop a one-shot (finite-blocklength) channel-coding framework based on the pairwise error probability (PEP) of a decoder with randomized tie-breaking. The tie-breaking rule yields a probability-integral-transform identity: the induced error spectrum describes both random-coding achievability and exact fixed-code...

Multistage Rewinding Decoder for QLDPC Codes

In this paper, we propose a multistage decoding framework that leverages internal information produced by an underlying message-passing decoder. The proposed method targets the failure dynamics caused by both classical trapping sets and degenerate errors supported on symmetric stabilizers, which are among the primary l...

On the robustness of noisy solutions in non-convex neural networks

Using a finite energy message-passing algorithm, it is demonstrated numerically that thermal noise enables effective generalization in the regime of constraint densities where both recovering the teacher and finding a zero temperature solution are computationally hard.

SchurQuant: Groupwise Discrete Optimization for Layer-Wise LLM Quantization

SCHUROPT is introduced, which analytically eliminates the suffix's optimal continuous response, yielding an exact groupwise quadratic with Schur-complement curvature, and achieves the highest mean zero-shot accuracy among the evaluated backpropagation free PTQ baselines.