Yun-Qiong Shi

Analysis of Positional Encoding Methods and Their Mathematical Principles in Large Language Models

: Transformer architecture is an architectural phenomenon in natural language processing. The self-attention mechanism inherent to it does not consider the word order. Thus, positional encoding (PE) is required to provide the word vectors with sequential information. This essay is a review of the processing involved in...