Comparative Evaluation of Continual Learning Strategies and Their Impact on Catastrophic Forgetting in Artificial Neural Networks

Abstract

Traditional machine learning algorithms operate on datasets with low diversity, with most data used for training and the remainder for inference. But in realtime applications, it is not possible to forecast data diversity, the number of tasks, and classes upfront to train and build the model. In a practical scenario, data arrives continuously, enabling the model to learn continuously. Catastrophic forgetting is a perennial problem in Continual Learning. Building the model on different tasks, with a group of classes without forgetting, is a challenge in Classincremental Learning. In this proposed work, we present a comparison of existing approaches, including Replaybased, Regularization-based, Architecture-based, and Knowledge-Distillation-based. Various evaluation metrics, such as continual learning accuracy and catastrophic forgetting percentage, are presented. Various state-of-theart methods are compared, and their performance is evaluated on MiniImageNet. Finally, future research directions for improving continual learning methods and mitigating catastrophic forgetting are discussed.