2026· IEEE Open Journal of the Communications Society· Vol 7, pp. 9893-9905· 0 citations· 29 references
TL;DR
The analysis demonstrates that the RL-based approach, enabled by an effective neural network initialization strategy, surpasses traditional methods and ML-based DPD schemes such as DLA and ILA and provides a scalable and efficient solution for compensating pattern-dependent nonlinearities in high-speed optical communications.
Abstract
This paper proposes a reinforcement learning (RL)-based digital pre-distortion (DPD) method to mitigate pattern-dependent nonlinearities in optical transmitters. The RL agent is configured with an optimized symbol window size, allowing it to effectively model pattern-dependent nonlinearities introduced by the digital-to-analog converter (DAC) and power amplifier (PA). To assess its performance, the peak-to-peak output voltage ( $V_{pp}$ ) of the DAC is swept, and the results are compared with conventional DPD methods, including non–machine learning (ML) techniques such as linear DPD, Volterra series, and look-up tables, as well as ML-based approaches such as indirect learning architecture (ILA) and direct learning architecture (DLA). Our analysis demonstrates that the RL-based approach, enabled by an effective neural network initialization strategy, surpasses traditional methods and ML-based DPD schemes such as DLA and ILA. Furthermore, we incorporate transfer learning (TL) into the RL framework to reduce training complexity across different $V_{pp}$ levels without full retraining, achieving substantial complexity reduction. The results demonstrate that combining RL with TL provides a scalable and efficient solution for compensating pattern-dependent nonlinearities in high-speed optical communications.
Deep learning-based active noise control (ANC) algorithms demonstrate superior potential over traditional methods in addressing nonlinear distortion. Although recent deep learning approaches incorporating Volterra Neural Networks (VNNs) have been optimized to tackle nonlinearities, there remains room for further improvement: (1) utilizing element-wise addition in skip connections across different feature processing stages increases the risk of feature aliasing or suppression; (2) directly introducing higher-order terms of the Volterra series is prone to causing overfitting; and (3) models trained under a singular nonlinear condition struggle to adapt to real-world scenarios with varying nonlinearities. To address these issues, this letter proposes a novel time-domain ANC framework. While retaining the modeling capabilities of WaveNet and VNNs, the proposed framework integrates the U-shaped structure and incorporates an adaptive gating mechanism for the higher-order Volterra terms. The proposed method is compared with a state-of-the-art deep learning framework. Additionally, the model is evaluated under several controlled nonlinear conditions, and comprehensive ablation studies are conducted. Simulation results demonstrate that the proposed algorithm outperforms existing end-to-end representative Deep Neural Network (DNN) methods, and ablation studies confirm the effectiveness of the proposed modules.
Tianyi Ge, Liang An, Ning Han et al.· IEEE Signal Processing Lette...· 0 citations
This work presents a direct-learning architecture of a neural network (NN) digital predistortion (DPD) linearizer for a multiple-input multiple-output (MIMO) system. The array-propagated neural network (APNN) technique accounts for nonlinear cross-modulation distortion due to antenna coupling in MIMO systems while maintaining low complexity compared to a single-input single-output (SISO) system. The behavior of the MIMO antenna array is modeled using a NN to train the DPD NNs through memory backpropagation. The linearization methods were evaluated across two test cases using 65 nm complementary metal-oxide-semiconductor (CMOS) power amplifiers (PAs). The first case utilized two PAs under a worst case 4 dB coupling scenario, transmitting 20 MHz 802.11ac Wi-Fi signals with a 10 dB peak-to-average power ratio (PAPR) at a 1 GHz center frequency and an average power of 8.5 dBm. Results demonstrate an error vector magnitude (EVM) improvement from −15.1 to −40.6 dB and from −16.2 to −39.4 dB for the two chains, respectively, with computational complexity increases of 35% and 32% compared to the SISO model. The second test case employed a three-antenna array, in which mutual coupling was introduced according to element proximity. In this scenario, 80 MHz 802.11ax Wi-Fi signals with an 11 dB PAPR were transmitted at an average power of 9 dBm. The proposed approach yielded EVM improvements from −22.1 to −39.7 dB, from −19.1 to −40.4 dB, and from −20.5 to −40.2 dB. In this configuration, only cross samples from adjacent channels were included, demonstrating the scalability of the APNN method.
In this study, we examine and contrast the effectiveness of different artificial neural network (ANN) topologies for power amplifier (PA) digital pre-distortion (DPD). In particular, we investigate long short-term memory (LSTM) networks, gated recurrent units (GRU), recurrent neural networks (RNN), con-volutional neural networks (CNN), and fully connected neural networks (DNN). For training and assessment, a dataset comprising measured input and output signals from a commercial NXP Doherty PA working in the 3.6–3.8 GHz region with a 16-QAM OFDM signal is utilised. Normalised mean squared error (NMSE), adjacent channel power ratio (ACPR), and model complexity are used to evaluate the models. Simulation results show that while CNNs offer a favorable trade-off between linearization performance and model complexity, GRU and LSTM architectures achieve the best overall NMSE and ACPR improvements, albeit with a higher number of parameters. Power spectral density and AM/AM characteristic analyses further confirm the superior linearization performance achieved using recurrent gated structures.
Hafsa Laakouri, M. Ouadefli, A. Tribak et al.· EPJ Web of Conferences· 0 citations
Due to the continuous increase in communication bandwidth and the use of highly efficient yet nonlinear power amplifiers, Digital Predistortion (DPD) algorithms are becoming increasingly complex. In particular, neural network (NN) based DPD approaches using Phase-Normalized NN architectures often incur substantially higher computational costs than widely deployed polynomial-based methods, such as the Memory Polynomial (MP) and Generalized Memory Polynomial (GMP) models. To bridge this gap between research performance and practical implementation, we propose a low-complexity Feature Selection NN DPD architecture. The proposed method employs an offline feature-engineering pipeline based on the Least Absolute Shrinkage and Selection Operator (LASSO) and the Minimum Redundancy Maximum Relevance (MRMR) algorithm to construct a compact and informative input representation. Using measured wideband FR3 power amplifier datasets that are publicly released with this work, we demonstrate up to 30% reduction in computational complexity while maintaining comparable linearization performance.
Cel Thys, Rodney Martinez Alonso, A. Alsarraf et al.· 0 citations
This study introduces an effective optimization technique for identifying the optimal hyperparameters of a deep neural network (DNN) designed to model the behavior of power amplifiers (PAs). Hence, an innovative and efficient yield-analysis-based approach is proposed to improve both the modeling accuracy and the digital predistortion (DPD) performance of PAs. The method leverages a long short-term memory (LSTM) DNN architecture to capture the nonlinear and memory effects inherent in PA systems, thereby enhancing overall performance and linearization capability. To achieve optimal training, multiple optimization strategies are systematically applied to determine the most suitable hyperparameters, such as learning rate, network depth, and neuron configuration. To validate the effectiveness of the proposed method, a PA operating in the 1.8 GHz to 2.2 GHz frequency range is considered, for which extensive simulations and evaluations demonstrate that the optimized DNN achieves minimal modeling error while significantly improving DPD performance. The results confirm that the proposed framework offers a reliable and efficient solution for PA modeling and linearization, outperforming conventional techniques in terms of accuracy and consistency.
Lida Kouhalvandi, L. Matekovits, Sercan Aygün et al.· Signal Processing and Commun...· 0 citations
We propose a reinforcement-learning (RL) guided transmitter optimization framework for short-reach IM/DD free-space optical (FSO) links that jointly tunes probabilistic shaping (PS), geometric shaping (GS), and pre-equalization (Pre-EQ) filter coefficients using the soft actor-critic (SAC) algorithm. The agent adapts transmitter parameters in a closed-loop manner from measured link-quality feedback, removing reliance on explicit channel models. In a 30-m PS-PAM4 FSO testbed with a 7-tap FIR Pre-EQ at the transmitter and standard offline DSP at the receiver, the agent uses generalized mutual information (GMI) as the reward to coordinate PS/GS with Pre-EQ. Experiments show a peak achievable information rate (AIR) of 194.3 Gb/s at 110 Gbaud and receiver-sensitivity gains of ~1.5 dB over traditional Pre-EQ and ~3 dB over no Pre-EQ, demonstrating the efficacy of closed-loop, model-free transmitter learning for next-generation optical wireless access.
Ouhan Huang, Junhao Zhao, Yinjun Liu et al.· IEEE Photonics Technology Le...· 0 citations