A Knowledge-Distilled Conformer Framework for On-Device Sleep Assessment

Abstract

Sleep disorders are prevalent across the population and, in older adults, are associated with cognitive decline, reduced functional capacity, and diminished quality of life. While polysomnography (PSG) provides detailed neurophysiological information for sleep assessment, its reliance on specialized expertise and high computational cost prevents the scalable, continuous monitoring needed in home and long-term care settings. We propose an edge-deployable, Conformer-based multimodal PSG framework for automated sleep staging. A high-capacity teacher model encodes electroencephalogram (EEG), electrooculogram (EOG) and electromyogram (EMG) through per-channel Conformers, a graph transformer and cross-attention fusion, with a temporal transformer capturing inter-epoch dynamics. To enable low-power operation, a dual knowledge-distillation strategy combining feature and logit-level transfer compresses the teacher into a compact, EEG-only student. The framework is evaluated across two clinical cohorts, the Multi-Ethnic Study of Atherosclerosis (MESA) and the Cleveland Family Study (CFS). The teacher achieves performance competitive with state-of-the-art methods on both cohorts, but distillation retention is cohort-dependent. On CFS, the student retains 92.7% of the teacher’s macro-F1, with above 90% retention for Wake, N1, N2, and N3, though REM retention is limited to 79.7%. The distilled student reduces parameters by 38.5× and latency by 11.3×, processing a 30-s epoch in 6.64 ms using 0.64 MB on an edge-class CPU. On the older, sparser MESA cohort, retention falls to 78.1% of the teacher’s macro-F1, with REM retention dropping to 52.1%. These results show that resource-efficient multimodal sleep staging is feasible for on-device deployment, while identifying robust minority-stage retention on sparser, older cohorts as the key remaining challenge for reliable cross-cohort operation.