Deep reinforcement learning for simulated autonomous heat stress mitigation in dairy cattle

Abstract

Heat stress significantly constrains dairy cattle productivity and welfare, compromising milk production and fertility. Yet, despite advances in heat stress detection and prediction, conventional mitigation strategies still rely on fixed thresholds, overlooking inter-animal variability and cumulative thermal load. In this simulation-based proof-of-concept study, we evaluated deep reinforcement learning (DRL) for individualised heat-stress decision support. Using multi-modal sensor data from the MmCows dataset, we developed a Deep Q-Network (DQN) framework for simulated cooling decisions. Cooling interventions were not experimentally applied to animals; their expected effects were computationally modelled as action-specific reductions in effective thermal load with corresponding energy penalties. We trained and evaluated four DQN variants, namely Standard, Double, Duelling and Double Duelling DQN with Prioritised Experience Replay, on real-world sensor data from 10 Holstein dairy cows over a 14–day period. The environment incorporated core body temperature (CBT), Temperature-Humidity Index (THI), behavioural indicators and milk-yield. The Double DQN agent achieved the highest numerical reward (18.26 ± 3.89), but its learned policy selected no intervention in essentially all evaluation cases, and its reward was not significantly different from the No Action baseline (17.66 ± 5.10; Welch t-test, p = 0.510). This pattern was reproduced across five independent random seeds, with all agents converging on low- or no-intervention policies. The moderate heat-stress conditions and reward formulation favoured low-energy policies. These findings indicate that DRL is a promising framework for testing animal-centred control policies in precision livestock farming and developing field-validated autonomous cooling systems with safety constraints through controlled trials and reward-weight sensitivity analysis.