Beneath the Noise: Uncovering and Distilling the Encoding Bottleneck of Quantum Models

Abstract

Quantum machine learning (QML) has attracted growing interest for its potential to match large classical models with far fewer parameters. This derives from encoding classical data representations into a Hilbert space exponentially larger than the number of qubits. In practice, however, noise mitigation limits these models to few qubits and shallow parameterized quantum circuits (PQCs), shrinking the accessible Hilbert space. This avoids noise but exposes a deeper problem of information loss that varies across samples: encoding compression. We uncover this bottleneck with the Hilbert-space encoding margin (HEM), partitioning inputs into a collapsed-encoding (CE) region and a preserved-encoding (PE) region, and find baseline accuracy on CE lagging PE by 43--58~pp across six setups. We propose quantum region-aware distillation (Q-RAD), a dual-PQC architecture that distills knowledge from a classical teacher separately on CE and PE samples. Under oracle routing, Q-RAD improves overall accuracy by up to +8.9/+8.1~pp over single-PQC and parameter-matched depth-doubled baselines. These results show that the encoding bottleneck can be exploited via its sample-level structure and architectural separation. Our code is available at https://github.com/skwonbb/Q-RAD/.