PLNKicongo: Analysis of models based on the BERT architecture for processing the Angolan language Kicongo

Abstract

Pre-trained contextual language models have demonstrated excellent performance in training with data from new languages and tasks. For this purpose, additional pre-training is necessary, since the lack of vocabulary of the language always tends to degrade the results. In this article, we present a procedure to deal with and treat unknown languages or languages without available resources such as Kicongo, a Bantu matrix language, commonly spoken in the northern regions of Angola, with greater incidence in rural regions and in the countries of central Africa (Democratic Republic of Congo, Republic of Congo and Gabon) and throughout the world. With the extension of Natural Language Processing (NLP) models based on the Bidirectional Encoder Representations from Transformers (BERT) architecture (BERT, RoBERTa and DistilBERT), the performance evaluation of the models was carried out using metrics such as accuracy, precision, recall and f1-score, achieving a performance rate higher than 97%.