Systematic Reward Shaping for Deep Reinforcement Learning Control of Adaptive Facades

Abstract

Buildings consume about 40% of global energy [1], with the building envelope being the major link between climatic conditions and energy consumption. Adaptive facades are dynamic shading elements for buildings that modulate the admission of daylight and solar gains. While adaptive facades are mature technologies, associated control systems are often overly simplified, ignoring climatic conditions or occupancy, which leads to sub-optimal operation. Deep reinforcement learning (DRL), a technique which self-trains a control model solely based on data observations, has the potential to substantially improve control systems. However, the formulation of a robust control objective, typically referred to as reward, remains challenging. The introduced FRADS-GYM is an open-source DRL environment for training agents on buildings with adaptive facades. An adaptive observation normalization layer aligns the forgetting horizon with seasonal physics. DRL control agents were trained on four ASHRAE climate zones and evaluated on these four climates, with zero-shot generalization assessed on an unseen fifth. Observation normalization is shown to be the dominant first-order factor for successful training, while the specific choice of forgetting horizon is a second-order decision. With normalization in place, the visual-comfort reward achieved a sevenfold reduction in glare violations against static clear glazing and incidental primary-energy savings of 11–29% against a rule-based controller. By contrast, a scalar energy reward collapses to the static-clear baseline in every climate due to thermal-mass lag, so composite rewards remain the prerequisite for DRL gains on the energy axis.