Jun 2026· Advances in Engineering Technology Research· 0 citations· 26 references
TL;DR
This paper summarizes the basic theoretical framework of LLM technology, including its definition, key features, development history, and core technologies, and classifies its technical architecture, which is divided into pure decoder, encoder-decoder, and sparse hybrid expert.
Abstract
As an important trend in the development of artificial intelligence, the large language model (LLM) is committed to building a two-way human-computer interaction, which has excellent performance in dialogue, real-time feedback, task execution, and so on. Research LLM architectures and their trade-offs in efficiency and understanding. Based on the references, this paper summarizes the basic theoretical framework of LLM technology, including its definition, key features, development history, and core technologies. Then, the existing literature is quantitatively analyzed, and the research hotspots of LLM technology are analyzed by using Citespace bibliometric tools. Based on the LLM, this paper mainly classifies its technical architecture, which is divided into pure decoder, encoder-decoder, and sparse hybrid expert. To reflect its interactive ability, this paper supplements it from two aspects: dialogue depth and multimodal support, and makes a comprehensive comparison of several existing mainstream LLMs. LLM can deal with complex decision problems, and is an important support for intelligent decision technology by facing the human-computer interaction mechanism to realize dynamic adjustment and self-optimization.
Large language models (LLMs) are built on the classic Transformer architecture and have become a core driving force for the rapid development of modern artificial intelligence. This paper presents a systematic review of LLMs, elaborating on their fundamental working principles, mainstream open-source models, effective lightweight optimization methods, retrieval-augmented generation frameworks and key human-value-aligned technologies. Nowadays, LLMs have been widely applied in practice. Typical scenarios include intelligent text generation, professional knowledge-based question answering and automated code generation, delivering remarkable value to both industries and academia. However, their large-scale industrial application is still restricted by multiple challenges. The major issues involve content hallucination, poor model interpretability, excessive computing resource consumption, potential ethical risks and unsatisfactory multimodal integration capability. This paper also forecasts the future development directions of LLMs, such as lightweight deployment on edge devices, safety-focused human value alignment, in-depth cross-modal fusion and customized large models for vertical industries. Additionally, it collects a number of representative cases, which can offer solid references and practical guidance for relevant researchers and engineering practitioners to carry out further studies.
Comparison of GPT-4, BERT (bidirectional encoder representations from transformers), Gemini, and DeepSeek large language models (LLM), focusing on architectures, training methodologies, and real-world applications reveals GPT-4 excels in natural language generation and complex reasoning, supporting up to 128K tokens with moderate latency and higher costs making it effective for conversational artificial intelligence (AI).
Kavish Sanghvi, Aparna S. Sharma, Surbhi Hooda· Computer Science and Informa...· 0 citations
In recent years, large language models (LLMs) have achieved significant results in natural language processing. They are applied to various tasks, including text generation, question answering, automatic summarization, code generation, and complex reasoning. With the increasingly complex real scenarios, the length of input text that models need to deal with also grows. Thus, the long-context processing ability of language models has gradually become an important factor in evaluating the practicability of LLMs. This paper gives an introduction to the long-context processing ability of large language models. It first introduces the background of large language models and the basic concept of long-context processing. It then summarizes the main technical methods of long-context modeling, such as improving positional encoding, training stage expansion, inference-stage optimization, and architecture-level innovation. Third, the paper also discusses the use of long-context ability in long-document question answering, long-text summarization, multi-document integration, code understanding and long-context evaluation tasks. Then, summarize the current main challenges and prospects of research work. This paper argues that the ability of long context should not only come from increasing the context window, but also from the ability of the model to locate, integrate and reason about important information in long text.
Jun Wu· Applied and Computational En...· 0 citations
A comprehensive review of the evolution of NLP from traditional rule-based approaches to modern transformer models including BERT and GPT demonstrates that NLP continues to transform intelligent systems and is expected to play an increasingly significant role in the development of next-generation AI technologies.
P. Kalaiselvi· International Journal of Eme...· 0 citations
This survey reviews the evolution of language models from early statistical approaches to modern Transformer-based architectures and summarizes key developments, including attention mechanisms, scaling laws, alignment techniques, and efficient inference methods.
P. Peykani, V. Charles, Ali Emrouznejad et al.· Archives of Computational Me...· 0 citations
AutoDavis is introduced, a first-of-its-kind automatic and dynamic evaluation protocol that enables on-demand benchmarking of LVLMs across specific capability dimensions and shows effectiveness and reliability, offering a new paradigm for dynamic benchmarking of multimodal intelligence.
Han Bao, Yue Huang, Yanbo Wang et al.· Proceedings of the 32nd ACM...· 0 citations