Privacy-Preserving Ambient Sensing for Activities of Daily Living: Multimodal Radar–Thermal Human Activity Recognition and Smart Plug Appliance Recognition

Abstract

Continuous monitoring of Activities of daily living (ADLs) requires sensing systems that are privacy-preserving, low-power, and robust to environmental variation. Ambient sensing technologies provide an alternative to RGB video and wearable devices, but individual sensing modalities exhibit characteristic limitations. Sparse mmWave radar provides strong motion sensitivity but limited posture detail, low-resolution thermal sensing preserves posture-related spatial information, and smart plug telemetry captures only appliance-mediated behavioural interaction. To address these limitations, this paper proposes a layered multimodal ambient-sensing framework comprising a sparse-track 24-GHz FMCW radar, a 32×24 low-resolution thermal sensor, and a Moko smart plug. It experimentally evaluates a radar–thermal HAR branch together with a separate smart plug appliance-recognition branch. The framework proposes three streams to enable continuous non-wearable monitoring while maintaining redundancy and reduced privacy exposure for intelligent-building and ambient assisted living environments. Radar and thermal streams are jointly evaluated on binary motion and four-class posture and activity recognition tasks collected across multiple environmental configurations using recording-grouped cross-validation, while the appliance stream is evaluated using per-plug telemetry from residential-grade appliances. The radar–thermal streams use a single-subject, fixed-placement dataset of binary-motion windows and four-class posture and motion windows collected across six furniture configurations. The separate intrusive load monitoring stream utilises smart plugs to classify appliances. Regarding binary motion recognition, radar (F1,Transformer=0.882±0.034) and thermal (F1,XGBoost=0.870±0.069) pipelines achieved similar macro F1 performance. On the four-class posture and activity recognition task, thermal features (F1,thermal=0.775±0.053) substantially outperformed radar (F1,radar=0.609±0.110). Weighted late fusion produced only modest descriptive gains. Separately, smart plug telemetry demonstrated strong appliance recognition performance using lightweight tree-based models suitable for constrained edge deployment. The results support a scoped redundancy argument. Sparse track-level radar carries gross motion, while low-resolution thermal sensing carries posture. The smart plug appliance monitoring extends the framework toward appliance-mediated instrumental activity of daily living (IADL) monitoring, with lightweight tree-based models achieving strong recognition performance under constrained edge deployment conditions. The findings support a layered multimodal sensing architecture for privacy-preserving ADL monitoring, where radar contributes motion-sensitive coverage, thermal sensing contributes posture-aware spatial context, and smart plug telemetry contributes appliance-level behavioural evidence within intelligent healthcare and ambient assisted living environments.