Skip to content

Category

small language model

3,243 papers

#natural language process... Preprint Oct 2026

CLUE: Uncovering Social Context for Pragmatic Understanding

Understanding a conversation requires connecting what is said to the speakers'relationships, expectations, and earlier exchanges. Prior work shows that supplying relevant context can improve models'social judgments, but much of this information may already be present in the conversation. This motivates training models...

Cheng-Fei Wu, Zhao-Qing Wu, Dan Goldwasser · 0 citations

Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling

Interactive agents repeatedly choose among a finite set of actions, labels, or ordinal outcomes. Generating a free-form response for each choice adds latency and complicates the interface between a model and its environment. We present Intern-Decision, a family of 0.8B, 2B, and 4B multimodal decision models that return...

Tong Zhu, Zhe Xu, Ze-Feng He et al. · 0 citations
#natural language process... Preprint Oct 2026

Identifying Effective Strategies in Prompt Evolution for Stress and Emotion Prediction

Early stress detection and risk prediction are crucial for timely mental health assessment and support. While large language models have recently shown promise for social media stress prediction, relying on accuracy can obscure false positives. Also, prompt revisions yield inconsistent gains across models and tasks, wi...

Seung-Jun Yi, Seung-Jun Yi, Sriram Sattiraju et al. · 0 citations
#machine learning Preprint Oct 2026

Patch where the text is new: a compression view of where byte-level models should spend compute

Byte-level language models such as the Byte Latent Transformer (BLT) run their large global model once per patch and start a patch where a small model's next-byte entropy is high. This rule rarely starts a patch at in-line computed results, such as the number after"="in a worked solution, and accuracy on them collapses...

Nicolás Vera Zúñiga · 0 citations
#machine learning Preprint Oct 2026

The Right Byte First: How UTF-16 Helps Multilingual Boundary Prediction

Many hierarchical byte-level language models group bytes into patches using a learned boundary predictor whose view of the next codepoint depends on encoding and lookahead. In UTF-8, codepoints of many scripts take three bytes: 8 of 24 bits mark length and continuation, and the first byte carries only 4 high codepoint...

Hyunsik Kim · 0 citations
#machine learning Preprint Oct 2026

HyCo: Hybrid Drafting for Cost-Efficient Mobile-Cloud Speculative Decoding

Device-cloud speculative decoding reduces server computation, but growing a deep draft tree with a small language model (SLM) requires repeated passes on a mobile neural processing unit (NPU). We present HyCo, which assigns depth to the server model's native multi-token prediction (MTP) head, executed on the device, an...

Hanbin Bae, Heejin Choi, Joohyung Lee et al. · 0 citations
#machine learning Preprint Oct 2026

Augment the Cache, Replace the Activation: A 2x2 Study of Internal-State Sharing

Multi-agent systems built from frozen language models communicate through text, which discards the internal state the sender already computed and makes the receiver pay to rebuild it. Work on internal channels avoids that cost, framing the problem as transfer: agent A sends information to agent B. Every agent pair that...

Adam Deng, Shi-Cheng Liu, Ya-Da Zhu · 0 citations
#machine learning Preprint Oct 2026

Ranking and Partitioning of Attention Heads for Sparse and Continual Reinforcement Learning Fine-tuning

We study sparse and continual reinforcement learning fine-tuning of language models through attention-head ranking and partitioning. For sparse learning, we rank attention heads by gradient norm to identify a small subset of heads that can be trained effectively. For continual learning, we partition heads across tasks...

Rui-Zhe Shi, Zai-Wen Yang, Qiu-Bo Wang et al. · 0 citations
#machine learning Preprint Oct 2026

PocketTSLM: Deploying an On-Device Soft-Prompt Time-Series Language Model for Privacy-Preserving Smartphone Reasoning

Consumer phones and wearables increasingly collect physiological time-series data, yet most mobile systems treat these streams as summaries or text rather than as temporal evidence. This representation is poorly matched to personal health systems that need to retrieve, route, or summarize sensor evidence while preservi...

Max Rosenblattl, Nikhil Krishnaswamy, Leon Nissen et al. · 0 citations
#machine learning Preprint Oct 2026

Can Capability Enhancement be Localized?

Transformer-based language models have shown strong capabilities across many tasks and domains. However, prior work shows that, for a given task, not all attention heads matter equally. Specifically, they suggest that many \textit{redundant} heads can be safely removed with little-to-no effect, while a small set of \te...

Anh Tuấn Nguyễn, Anna Bair, Zico Kolter · 0 citations
#machine learning Preprint Oct 2026

Resisting Harmful Instructions through Training-Free Weight Editing

Instruction-following language models can comply with requests that conflict with intended behaviors and safety alignment. We introduce a training-free weight-editing approach for reducing compliance with targeted undesirable instructions. Our method needs only forward passes on a small set of prompt pairs per target b...

Ali Ebrahimpour-Boroojeny, Xiao-Cong Yang, Samuel Schapiro et al. · 0 citations
#machine learning Preprint Oct 2026

Predicting Live Birth After Fresh Embryo Transfer in IVF/ICSI: Differential ICM/Trophectoderm Morphology Encoding and a Hybrid Attention--Logistic Regression Ensemble

Predicting live birth before fresh embryo transfer could support shared decision-making in IVF/ICSI, but routine clinical data are small, heterogeneous, and contain pathway-dependent variables prone to outcome leakage. We aimed to construct a curated single-centre IVF/ICSI dataset and develop a prediction model suited...

Abdul Raheem Shahzad, Jin-Xin Chen, Ming Zhang et al. · 0 citations

From tech blogs

See all →
MIT News · Artificial Intelligence Oct 15, 2026

AI model decodes the language cells use to communicate

A new computational approach reveals how cells communicate to assemble into tissues. This could accelerate stem cell engineering, improve disease models, and inform therapies.

Microsoft Research Blog Oct 15, 2026

Proof-based general-purpose computing with Magnetar

Zero-knowledge proofs can verify computing results without rerunning workloads or exposing sensitive data. Magnetar helps bring them into the mainstream with support for standard programming languages, major speed gains, and quantum-resistant security. The post Proof-based general-purpose computing with Magnetar appeared first on Microsoft Research.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.