Changwen Zheng

GUPO: Gradient Uncertainty-aware Policy Optimization for Post-Training Large Language Models

Gradient Uncertainty-Aware Policy Optimization is proposed, which models each group gradient as a random variable under a Bayesian formulation and estimates its probability distribution and derives gradient uncertainty using a Dirichlet-based formulation and uses it to calibrate the contribution of each group gradient during aggregation.