PRO-Step: Step-level Process Reward Optimization for Retrieval-Augmented Generation
This work trains a generative PRM that evaluates both dimensions, employs PRM-guided value tree search to construct preference pairs contrasting valid steps against flawed ones, and optimize the policy via step-level Direct Preference Optimization.
Minkeon Kim, Namjun Lee, Jaekwang Kim
· 0 citations