A Deep Learning Model for Prediction of Unknown Gene Functionality: Gene Bio-BERT.

Abstract

The Human Genome Project (HGP) was a large international research effort that timelined between 1990 and 2003, marking the successful mapping of the entire human genome. Despite the promising performance of deep learning models, especially LLMs like Bio-BERT and ProtBERT on well-annotated datasets. Bio-BERT is a pre-trained language model designed for biomedical corpora, it is not able to independently predict gene functionality. A pre-trained model called ProtBERT uses protein sequences to learn features. It is unable to predict gene functionality directly, without more training data. In order to overcome these challenges, a Gene Bio-BERT based framework is proposed for automated gene function prediction utilizing deep learning methods in biomedical data analysis. This Gene Bio-BERT Framework is divided into 3 modules such as Data collection and Preprocessing, Gene Bio-BERT model training, Feature aggregation layer and prediction of functionality. The initial module focuses on data collection and preprocessing. Data is collected using entrez API from NCBI (National Center for Biotechnology Information) to retrieve human gene data. Preprocessing techniques like Tokenization and feature extraction are then used to handle the data. In the second module, a Gene Bio-BERT transformer encoder with an attention-based feature fusion layer and optimized hyperparameters is used to train the model and learn contextual embeddings. The third module generates results by using aggregated transformer representations to produce functional predictions for unknown genes. In predicting gene function, the proposed Gene Bio-BERT model attains an exceptional accuracy of 94.5% and F1 score of 0.87. Additionally, the model's predictive accuracy remains similar when tested on an unannotated gene which gives similarity score 0.84.