Conference Open access

Distilling LLM Reasoning into Dense Encoders: Bridging the Accuracy-Efficiency Gap in Recommendation

D. Han Daeyoung Roh A. Y. Kim Hwanjun Song M. Yi

2026 · Annual Meeting of the Association for Computational Linguistics · pp. 22513-22531 · 0 citations · 42 references

Computer Science

TL;DR

Reasoning-to-Encoder Distilla-tion (R2END), a framework that establishes a text encoder as the optimal student architecture for scalable recommendation, and compresses the teacher’s reasoning into a dense vector space via a semantic alignment objec-tive, effectively capturing user-item dynamics.

Abstract

Large Language Models (LLMs) have shown remarkable potential in recommendation systems but suffer from prohibitive inference latency. Existing distillation approaches typically target Small Language Models (SLMs) or Conventional Recommendation Models (CRMs), yet face a critical trade-off between computational cost and semantic reasoning capacity. To bridge this accuracy-efficiency gap, we introduce Reasoning-to-Encoder Distilla-tion (R2END), a framework that establishes a text encoder as the optimal student architecture for scalable recommendation. Unlike meth-ods that mimic token generation, R2END compresses the teacher’s reasoning into a dense vector space via a semantic alignment objec-tive, effectively capturing user-item dynamics. Extensive experiments on four datasets demonstrate that R2END not only outperforms state-of-the-art baselines but also achieves drastically reduced latency, offering a sweet spot for recommendation.

DOI Semantic Scholar

Read PDF

Similar papers

Preprint Aug 2026

Empowering Compact LLMs with Fusion of Layer-wise Exits for Recommendation

The Fusion of Layer-wise Exits for Sequential Recommendation (FLEXRec), a discriminative framework that enhances compact LLMs while retaining scalable full-corpus ranking and achieves state-of-the-art accuracy among competing methods while remaining highly efficient.

Xurong Liang, Tong Chen, Q. Nguyen et al. · 0 citations

Book Open access Jul 2026

Think, But Don't Tell: Implicit Reasoning for LLM-based Sequential Recommendation via Multi-Teacher Distillation

I Reasoning via Multi-Teacher Distillation is proposed, a novel framework that 'compiles' the reasoning abilities of large teacher LLMs into a lightweight student Small Language Model (SLM), which significantly outperforms state-of-the-art baselines in both recommendation accuracy and inference efficiency.

Weihai Lu, Xiaoxi Cui, Chenke Yin · Annual International ACM SIG... · 0 citations

Jul 2026

The Case Against Generation for Retrieval: Discriminative Language Models as Effective Retrievers

Large Language Models (LLMs) have emerged as powerful assets for recommender systems. However, deploying them as generative recommenders or zero-shot rankers at web-scale remains bottlenecked by prohibitive computational overhead and grounding challenges. In this paper, we revitalize the classic, highly efficient two-t...

Zhe Xu, Prachi Agrawal, Kavosh Asadi et al. · arXiv.org · 0 citations

Preprint Sep 2026

SelfDR: Self-Distillation from Reasoning for LLM-Based Recommendation

SelfDR distills an LLM's own reasoning-enhanced predictions to produce recommendations directly, improving recommendation effectiveness while maintaining inference efficiency, and validate the effectiveness, rationality, and efficiency of SelfDR.

Chumeng Jiang, Jiayin Wang, Xin-Jie Lin et al. · 0 citations

Jul 2026

WhisperRec: Latent Reasoning for Efficient Foundation Recommendation Models

WhisperRec compresses teacher-generated CoT into learnable latent reasoning tokens, enabling a Latent-Reason-then-Answer paradigm that performs reasoning in latent space without producing verbose rationales, and achieves over 10x higher online inference throughput.

Hao Jiang, Pei Du, Pengfei Yao et al. · arXiv.org · 0 citations

Jul 2026

REPREC: Representation Driven Parameter-Efficient Recommendation System

REPREC is a lightweight framework that conditions a frozen LLM using compact user-level representations and consistently improves recommendation performance across different sequential encoders, LLM backbones, and user activity levels.

Harshini Kavuru, Dwipam Katariya, Giri Iyengar et al. · arXiv.org · 0 citations