Blockchain-Anchored Detection and Recovery for Resilient Reinforcement Learning-Based Cyber-Physical Systems

Abstract

Reinforcement learning (RL) controllers in cyber-physical systems (CPS) assume that sensor observations describe the true plant state, yet replay, sensor-drift, and false-data-injection (FDI) attacks can corrupt the observation stream at inference time, and detection alone does not tell the controller what to use after an alarm. This study evaluates a proof-of-concept defense that combines a SHA-256 hash-linked ledger, which provides exact-match replay detection and a tamper-evident recovery anchor, with a Long Short-Term Memory autoencoder (LSTM-AE) for audit alarms and a fixed Proximal Policy Optimization (PPO) controller. Ten independently trained policies per environment are evaluated in paired episodes on CartPole-v1 and LunarLander-v3. Content-hash matching detects every injected bitwise replay with zero false matches in 379,396 clean steps. However, perfect detection does not guarantee safe recovery: exact-gated ledger hold raises LunarLander replay return from 178.4 to 264.6 (paired mean +86.2, 95% CI [64.8, 107.6]) but lowers CartPole from 411.1 to 376.2 (−34.9, not significant after Holm adjustment). A trigger-by-anchor factorial ablation shows that local and ledger anchors are control-equivalent under a fixed trigger, while LSTM-AE-gated hold is harmful in every tested cell. A 4,800-episode sensitivity study across three severity levels per attack shows that the recovery effect scales with severity in opposite directions: +5.8 to +225.0 on LunarLander and −4.6 to −60.3 on CartPole. The ledger contributes exact-replay lookup, provenance, and tamper evidence rather than an automatic control advantage; safe general recovery further requires dynamics-aware substitution and distributed, physical validation.