Preference-conditioned deep reinforcement learning for dynamic scheduling in sustainable and robust manufacturing

Abstract

Modern manufacturing requires scheduling methods that adapt to changing order arrivals, machine disruptions, customer priorities, stakeholder preferences, and time-varying energy conditions. This paper proposes a preference-conditioned deep reinforcement learning (DRL) approach for dynamic scheduling in sustainable and robust manufacturing. The approach is embedded in a cyber-physical production system (CPPS)-oriented framework that links production states, machine availability, energy-related background data, simulation-based learning, performance monitoring, and decision support. Within this framework, a Double Deep Q-Network (DDQN) scheduler is developed for joint job sequencing, machine assignment, and start-time adjustment. The scheduler uses a candidate-based state representation for dynamic order arrivals, vector-valued Q-output for objective-specific value estimation, and a priority- and preference-aware reward design. Customer priorities are treated as order-level attributes, while stakeholder preferences are encoded as system-level objective weightings. This enables one policy to consider energy-related cost, carbon emissions, energy demand, and tardiness while adapting to different preference profiles. The concept is demonstrated in an on-demand manufacturing (ODM)-oriented parallel CNC machining case with heterogeneous orders, product-specific setup and processing requirements, hourly electricity prices, carbon-intensity signals, and curriculum-adaptive machine breakdowns. DDQN is compared with three dispatching rules and two DRL baselines under shared training and testing scenarios. The results show that DDQN achieves the lowest energy-related cost and carbon emissions in training and unseen testing while maintaining acceptable delivery performance. Overall, the study demonstrates the potential of CPPS-oriented and preference-conditioned DRL for adaptive, energy-aware, and robust scheduling in smart manufacturing systems.