LLM-enhanced Dynamic Fleet Planning with Hierarchical Multi-agent Reinforcement Learning Framework
TL;DR
The proposed hierarchical multi-agent proximal policy optimization framework can reduce total airlines' operational costs—including direct operating cost and capital cost and achieves a computation speedup in comparison with a conventional optimization baseline.
Abstract
Civil aviation fleet planning is a typical long-horizon operations research problem, for which the problem formulation and algorithms could play critical roles in either the computational efficiency or the solution quality. However, conventional meta-heuristic approaches have some inherent drawbacks: during the optimization process, the model is unable to "learn" efficiently from infeasible solutions and gradually concentrate on high feasibility regions, leading to an exponential increase in complexity with increasing planning horizon, and their randomness prevents an optimality guarantee on individual instances. In this paper, we introduce a hierarchical multi-agent proximal policy optimization framework to tackle these problems. To effectively handle ultralong episodes, our approach builds up the optimization architecture by reward shaping and environment designing within a base PPO, which is then boosted through multi-agent coordination and hierarchical decomposition. Large language model (LLM)-based agent workflows are deployed to automatically tune hyperparameters toward the specific domains' performances. Validated on realistic flight data from 2019 shared by our airline partner, our results show that the proposed method can reduce total airlines' operational costs—including direct operating cost and capital cost and achieves a computation speedup in comparison with a conventional optimization baseline.