From Perceptrons to Convolutional Neural Networks: A Practical Tutorial on Spatial Deep Learning

Abstract

This tutorial takes the reader on a historical and technical journey from the simple Perceptron (1958) to modern Convolutional Neural Networks (CNNs) that dominate spatial data processing (images and video). We start with the Perceptron’s linear classifier, then expose its inability to learn non-linear patterns (e.g., XOR), which motivates the Multi-Layer Perceptron (MLP) and the backpropagation algorithm. Next, we discuss the limitations of MLP when faced with structured data like images—parameter explosion, loss of spatial information, and lack of translation invariance—and use these limitations as a natural springboard to the core ideas of CNNs: local connectivity, weight sharing, and hierarchical feature learning. Throughout, we provide intuitive explanations, mathematical formulations, and step-by-step numerical examples (e.g., a complete forward and backward pass for a small network, and a manual 2D convolution). Clear graphical representations and examples help readers understand each concept. The tutorial concludes with a detailed walkthrough of influential CNN architectures (LeNet-5, AlexNet, VGG, GoogLeNet, ResNet, DenseNet, and EfficientNet) and also discusses more recent attention-based models (e.g., Vision Transformers and ConvNeXt), explaining why each was necessary and how it advanced the field. Aimed at students and practitioners with a basic knowledge of calculus and linear algebra, this tutorial connects foundational ideas to state-of-the-art deep learning, focusing on spatial data. It is designed for readers who want to understand why each architectural choice was made, not just what the final model looks like.