Skip to content

Category

small language model

3,629 papers

#small language model Preprint Aug 2026

Beam Search, Self-Consistency, and the Limits of Inference-Time Scaling for Grammar-Constrained Text-to-SQL in Small Language Models

The constrained case of this"model size vs. inference compute"trade-off, in which the model outputs are constrained by a strict grammar at inference time, is examined, which demonstrates that the constrained trade-off behaves differently from the unconstrained trade-off.

Ty Chermsirivatana, John MacCormick · 0 citations
#small language model Preprint Aug 2026

RTLGuard: A Lightweight Teacher-Student Defense for Poisoned RTL Code Generation Models

RTLGuard leverages a teacher-student framework designed to sanitize compromised RTL generation models by fine-tuning a small-scale,"clean"teacher model on a limited set of trusted RTL data, and incorporating feature alignment and knowledge distillation to suppress malicious behaviors.

Mahshid Rezakhani, K. Azar, H. Kamali · 0 citations
#small language model Preprint Aug 2026

Spectral Allocation: Why Muon Outperforms Adam, and How to Improve Muon

Spectral-Aware Muon is introduced, which holds the head at the Muon scale and amplifies the bulk using a static spectral prior, and both variants outperform tuned AdamW and Muon (Scion implementation) baselines in all evaluated model-scale and batch-size configurations.

Xiaodong Wu, Wen-Yi Yu, Chao Zhang et al. · 1 citation
#small language model Preprint Aug 2026

Label-Free Foundational Model Selection for Medical Image Classification under Distribution Shift via Pseudo Label Discrepancy

This work proposes a label-free selection criterion built on SUDO, a framework for evaluating clinical AI systems without ground-truth annotations, and shows that AURCC can be used to rank a variety of vision-language models on chest X-ray classification across three inter-hospital shift scenarios, under zero-shot and...

J. Larrea, L. Mansilla, Enzo Ferrante · 0 citations
#small language model Preprint Aug 2026

Distinct dynamics of conceptual and referential disruptions in human reading and large language model processing

Linguistic meaning is grounded in conceptual content, from which reference to particular entities emerges as words enter discourse. To examine the processing dynamics associated with these two dimensions of meaning, we selectively disrupted conceptual or referential information in short narratives and traced the result...

Rui He, N. Altay, W. Hinzen · 0 citations
#small language model Preprint Aug 2026

MoganBert-TR: A Turkish Encoder Foundation Model Trained from Scratch with a CLM-to-MLM Curriculum

MoganBert-TR, a 149M-parameter Turkish encoder foundation model trained from scratch on a language-specifically filtered corpus, together with an embedding model derived from it (MoganBert-Embed) that ranks first among student models on the MTEB(Turkish) overall average and outperforms all compared Turkish tokenizers o...

Furkan Yilmaz, Habibe Aleyna Tasdemir, Muhammed Faruk Gozay · 2 citations
#small language model Preprint Aug 2026

Vulnerable Code Search: Transferable Attack for Code Language Models

This paper introduces a programming language-agnostic, transferable, adversarial attack that exploits this CLM vulnerability and demonstrates that this attack, even when computed using smaller code embedding models, is highly effective and transferable to larger, closed-source embedding models.

Kaicheng Wang, Li-Yan Huang, Jesse Thomason et al. · 0 citations
#small language model Preprint Aug 2026

Retrieve, Match, Escalate: Accurate and Scalable Product Linking with VLM-Distilled Cross-Encoders and Agentic VLMs

This work presents a production retrieve-then-match cascade that spends computation in proportion to difficulty: retrieval surfaces plausible matches, a lightweight text cross-encoder auto-resolves the high-confidence majority, and an agentic multimodal vision-language model settles the ambiguous remainder.

Jian Wang, Steven Xu, Sanjyot Thete et al. · 0 citations
#small language model Preprint Aug 2026

Belief Cascades Drive Persuasion in LLM Agent Networks

This work introduces a controlled testbed for studying how goal-directed persuaders shift elicited stances in networks of LLM agents grounded in real-world ego-network topologies, and argues for evaluating multi-agent persuasion as a trajectory- and exposure-level process.

Haoyi Qiu, Genglin Liu, P. Venkit et al. · 0 citations
#small language model Preprint Aug 2026

The Von-Neumann State-Space Transformer for neural decoding

A von-Neumann inspired hypothesis of efficient computation as an alternative for neural decoding, a memory-augmented Transformer whose feed-forward block is a low-rank instruction bank: a shared base operator plus a small set of learned low-rank instructions, from which a per-token code synthesizes the weight matrix ac...

Morteza Sarafyazd · 0 citations
#small language model Preprint Aug 2026

SHIFT-LLM: Distribution Shift Correction in Depth-Pruned LLMs

SHIFT-LLM, a training-free post-pruning correction framework that inserts a Linear Residual Adapter at each pruning site, consistently recovers accuracy lost to depth pruning across most configurations, achieving gains up to +15.7 points on Llama-3.1-8B-Instruct.

Ali Bahri, Hang Li, Hongliang Li et al. · 0 citations
#small language model Open access Aug 2026

Estimated Prevalence of Developmental Language Disorder Risk in West Virginia Schools.

PURPOSE Developmental language disorder (DLD) is linked to long-term academic and social difficulties. Despite these lasting impacts, state-level prevalence estimates are limited. This study estimated prevalence among early-elementary students in West Virginia who may be at risk for DLD and examined variation by grade,...

Megan Israelsen-Augenstein, Michelle Moore, Tracy Toman et al. · 0 citations

From tech blogs

See all →

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.