Proximal Policy Optimization (PPO) is widely used in reinforcement learning because it provides stable on-policy learning with relatively simple implementation. However, standard PPO uses a fixed clipping threshold, applying the same trust-region width to all training samples even though the reliability of policy updates may vary across states and actions. This paper studies a family of adaptive-clipping PPO variants that replace the constant clipping parameter with sample-dependent thresholds derived from policy and return statistics. We introduce Dual-Adaptive Z-Score PPO (PPO-DAZ3), which dynamically adjusts the clipping interval using two signals: action confidence, derived from the probability of the selected action under the current policy, and update confidence, measured from normalized advantage magnitude. This formulation enables persample adaptation while preserving the standard PPO training pipeline. Experiments on three classic reinforcement learning benchmarks-LunarLander-v3, CartPole-v1, and Acrobotv1-show that adaptive clipping can improve learning performance in some environments. PPO-DAZ3 achieved the strongest results on LunarLander-v3, while the smooth advantage-based variant PPO-AZ3S performed best on Acrobotv1. On CartPole-v1, standard PPO remained competitive, indicating that adaptive clipping is most beneficial on more challenging control tasks.
Taha M. Mahmoud, N. Kaabouch· 2026 6th International Confe...· 0 citations
This paper presents a comparative benchmarking study of classical and quantum machine learning models for intrusion detection using three benchmark datasets: NSL-KDD, UNSW-NB15, and MQTTEEB-D2025. The study evaluates how preprocessing choices, feature selection strategies, and quantum encoding methods influence model performance across datasets with different levels of noise and complexity. A unified pipeline is adopted, incorporating normalization, imbalance handling, dimensionality reduction, and two feature selection approaches: Random Forest importance and a quantum-aware method based on Quantum Kernel Alignment with Mutual Information. Four models are assessed: Support Vector Machine, Random Forest, Quantum Support Vector Machine, and Pegasos Quantum SVM. Results show that classical models remain stable across datasets, while quantum models are more sensitive to feature representation and kernel alignment. Quantum performance improves significantly with quantum-aware feature selection, particularly on cleaner datasets, whereas heterogeneous datasets remain challenging. Pegasos Quantum SVM offers a favorable balance between accuracy and computational efficiency, highlighting the importance of preprocessing alignment for practical quantum intrusion detection.
Taha M. Mahmoud, N. Kaabouch· 2026 6th International Confe...· 0 citations