Learning to Generalize Across Conditions: A PPO-Based Learning Approach to Multi-Condition Multi-Objective Optimization

Abstract

This study introduces sPPO-CR, a policy-gradient deep reinforcement learning (DRL) algorithm for multi-objective (MO) and multi-condition multi-objective (MCMO) optimization. The method builds on a simplified single-step variant of Proximal Policy Optimization (PPO) and incorporates a Chebyshev-based reproduction mechanism with a log-likelihood rejection filter to safely integrate synthetic off-policy samples (sPPO-CR). sPPO-CR is benchmarked against sTD3-CR, a single-step variant of TD3, and three evolutionary baselines (NSGA-II, NSGA-III, and MOEA/D) across mathematical benchmarks and an airfoil shape optimization task. Results show that while evolutionary algorithms remain effective in standard MO settings, DRL methods, particularly sPPO-CR, offer improved performance in certain higher-dimensional and condition-dependent problems through effective generalization across both the design and condition spaces. Moreover, sPPO-CR achieves similar or superior performance to sTD3-CR while requiring an order of magnitude fewer network updates. Overall, the findings demonstrate that DRL-based optimizers provide a scalable and computationally efficient alternative for solving complex multi-condition multi-objective problems.