Developmental Analysis and Technical Report: Small Language Model (SLM) Training from Scratch
This report documents the ongoing development and empirical analysis of a 4.2-million-parameter Transformer-based small language model (SLM) trained from scratch by Novum Labs. The objective of this project is to evaluate the learning trajectory, convergence dynamics, semantic coherence, and dataset scaling characteris...