Skip to content
Open access

Domain-Randomized End-to-End Constellation Shaping for Impaired Fading Channels

Oct 2026 · International Journal on Advanced Science, Engineering and Information Technology · 0 citations

Abstract

Learned constellation shaping can beat classical modulation when the channel strays from the additive-Gaussian ideal, yet it is usually tested on idealized models where quadrature amplitude modulation (QAM) is already near-optimal, and where the reported gain turns out to depend heavily on how the receiver is modeled. We return to end-to-end geometric and probabilistic shaping, this time on fading channels that also carry the impairments of cheap transceivers: power-amplifier compression and oscillator phase noise. Three things come out of the study. The equalizer matters more than it might seem: an amplitude-only receiver, which is easy to build by accident, underestimates the achievable rate by about a factor of two and exaggerates how much learning helps relative to QAM. Once detection is genuinely coherent, an end-to-end jointly shaped constellation does beat both 16-QAM and the standard 16-APSK constellation under impairments, by 0.04–0.08 bits/channel use at 12 dB on both AWGN and Rayleigh channels, a small but steady margin that runs in the direction the impairment geometry predicts. A single model trained with domain randomization over impairment severity, rather than one model per severity, holds up better when pushed to severities it never saw during training. We then take the study further: the advantage grows with constellation order, roughly tripling between M = 16 and M = 64, and it carries through to channel coding only when the bit labeling is optimized together with the geometry. Under a naive labeling, the mutual-information gain is lost, or even reversed, in the coded block-error rate; under an optimized labeling, the coded BLER falls well below that of QAM. All results are reported across several conditions, with an additive-white-Gaussian-noise sanity check and fully converged baselines.

Read PDF

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.