Hyperparameter optimisation of LSTM Networks for In-Field Low-Cost Air-Quality Sensor Qalibration

Abstract

Low-cost air quality sensors (LCS) offer an unprecedented opportunity to monitor urban pollution with high spatial resolution. However, their physical readings are highly susceptible to environmental cross-sensitivities, particularly hygroscopic growth effects during winter heating seasons. While Long Short-Term Memory (LSTM) networks have emerged as a powerful tool to model these complex temporal dependencies and calibrate raw sensor outputs against reference-grade instrumentation, their performance remains heavily dependent on the manual, often sub-optimal configuration of their structural and training hyperparameters. This work introduces a systematic, automated framework for the in-field calibration of a co-located network of low-cost sensors (PAQMON 1.0 mobile air quality monitor integrating the NOVA SDS011 optical sensor for PM2.5 and PM10 detection based on Mie light scattering, together with the AM2302/DHT22 module for temperature and relative humidity measurements) against a regulatory reference station (SEPA). To transcend standard trial-and-error tuning, we implement a modular, automated hyperparameter optimization pipeline using Optuna, fully tracked and orchestrated via MLflow as our MLOps engine. The optimization space is parameterized to simultaneously explore structural configurations, specifically the temporal lookback window length and recurrent cell dropout and training dynamics, including the initial learning rate and scheduler decay thresholds. Evaluating the methodology on a field-campaign dataset, the Optuna-guided search rapidly converged on optimal configurations, yielding highly accurate calibrations for fine particulate matter (PM2.5). The automated framework successfully isolated the ideal temporal window necessary to counteract humidity-induced biases while maintaining a computationally efficient network footprint. Conversely, coarse particulate matter (PM10) calibration remained challenging, exposing distinct environmental non-linearities that hyperparameter variations alone could not fully mitigate. By integrating MLflow, we demonstrate a reproducible, productionready MLOps toolchain capable of managing individual model registries for heterogeneous sensor nodes, establishing a robust path toward scaling high-fidelity edge-calibration in municipal sensor networks.