HYBRID QUANTUM OPTIMIZATION OF DISTILBERT FOR EFFICIENT NATURAL LANGUAGE PROCESSING

Abstract

The authors created a hybrid quantum-classical model for efficient sentiment classification based on Frozen Distilled Bidirectional Encoder Representations from Transformers (frozen dBERT), supervised dimensionality reduction, variational quantum learning, and residual integration. The binary Stanford Sentiment Treebank had a total of 67,349 training observations. Following contradiction, duplication and overlap removal, 66,971 observations were left, of which 60,274 were used for training, and 6,697 for internal validation. There were 872 observations on the official development partition that were not touched. The embeddings, which had a dimension of 768, were mean-pooled to 4, 6, or 8 dimensions. The six-dimensional bottleneck was chosen and represented in the variational circuit with six qubits. The selected circuit was composed of 12 completed trials and 14 pruned trials, and had 3 layers, 36 quantum parameters, 52 logical gates and a circuit depth of 16. The classical six-dimensional bottleneck achieved 0.8360 accuracy, 0.8427 harmonic-mean classification score, 0.9184 receiver operating characteristic area under the curve and 0.6724 Matthews correlation coefficient on the official development partition. It was able to lower the dimensionality of the representation by 99.219 percent and obtain a model-head latency of 0.000607 milliseconds per observation. The quantum-only classifier had an accuracy of 0.6663 and a harmonic-mean classification score of 0.6914, whereas the residual hybrid kept a zero-quantum contribution. The results show efficient classical compression with no quantum advantage for the simulation of six-qubit states.