Accelerating HDAC8 inhibitor discovery through deep learning, cheminformatics, docking, and molecular dynamics simulations.
Abstract
Histone deacetylase 8 (HDAC8) is an emerging epigenetic target implicated in cancer, neurodegenerative disorders, and other human diseases, driving the urgent need for potent and selective inhibitors. In this study, we developed and validated a deep learning-based computational pipeline for the accurate prediction of HDAC8 inhibitor potency, integrating cheminformatics, neural networks, molecular docking, and molecular dynamics simulations. A large, curated dataset of 4583 HDAC8 inhibitors with experimental pIC50 values was retrieved from the ChEMBL database (target CHEMBL3192) and preprocessed using RDKit. Each molecule was represented as a 1033-dimensional feature vector combining nine physicochemical descriptors with a 1024-bit Morgan fingerprint. Through systematic hyperparameter ablation comparing eight neural network architectures, the optimal configuration comprising three hidden layers (512 → 256 → 64 units) with batch normalization and dropout regularization achieved a moderate but useful test set R2 of 0.617 and RMSE of 0.642 log units. Residual analysis confirmed model reliability across the central tendency of chemical space, though systematic regression-to-the-mean effects were observed for extreme potency values. Leveraging an extrapolation-trained model with SELFIES-based evolutionary generation, we identified ten novel HDAC8 inhibitor candidates with predicted pIC50 values ranging from 10.49 to 11.07, all satisfying Lipinski's Rule of Five. The highest-priority candidate (Compound 1, predicted pIC50 = 11.07) was subjected to molecular docking and 200 ns molecular dynamics simulations, revealing a stable binding mode (protein RMSD = 1.8-2.2 Å, ligand RMSD = 1.5-1.8 Å) with persistent hydrogen bonding interactions (GLY 131 occupancy = 27%, GLY 120 occupancy = 22%). This integrated computational pipeline demonstrates the power of deep learning for accelerated HDAC8 inhibitor discovery, providing a robust framework for identifying and optimizing potent, drug-like candidates. These computational predictions, while promising, should be interpreted as hypotheses that require rigorous experimental validation.