Whispered Speech Recognition Enhancement Using Teager Energy Operator with PLP Features

Abstract

Whispered speech poses persistent challenges for automatic speech recognition (ASR) owing to the absence of vocal fold vibration and the dominance of turbulent airflow excitation, which substantially alters spectral and temporal energy distributions. Conventional front-end features, such as Perceptual Linear Predictive Cepstral Coefficients (PLPCC), exhibit limited robustness in these conditions, particularly under cross-mode (mismatched) training–testing scenarios. To address this, we propose Teager Energy–based Perceptual Linear Predictive Cepstral Coefficients (TEPLPCC), a novel feature extraction framework that integrates the nonlinear Teager Energy Operator (TEO) within the perceptually motivated PLP analysis. The proposed method exploits instantaneous energy variations to characterize the nonlinear aerodynamic behavior inherent in whispered speech. Extensive experiments were conducted on the Whi-Spe bimodal (normal/whisper) speech database using three recognition back ends—Dynamic Time Warping (DTW), Hidden Markov Models (HMM), and Convolutional Neural Networks (CNN)—under both matched (Normal/Normal, Whisper/Whisper) and mismatched (Normal/Whisper, Whisper/Normal) conditions. Feature normalization via Cepstral Mean Subtraction (CMS) was employed to mitigate channel and mode variability. Results demonstrate that TEPLPCC consistently surpasses baseline PLPCC and existing TEO-derived features (TELFCC and TEMFCC), achieving up to 13% relative improvement in mismatched conditions and approximately 10% higher recognition accuracy on average. These outcomes establish TEPLPCC as a perceptually consistent and energy-aware representation, significantly advancing the robustness of ASR systems for whispered speech.