LLM-based Semantic and ID Representations for Sequential Recommendation

Abstract

ID-based sequential recommendation (SR) methods learn user preferences based on the item-ID interaction sequences, which are often limited by the data sparsity problem. Large language model (LLM)-enhanced SR methods leverage LLM to improve the performance. Although prior works have made significant progress, there are still three key challenges: (i) how to leverage the open-world knowledge and reasoning ability of LLM to enhance item representations; (ii) how to adaptively capture semantic preferences and collaborative preferences from multi-modality sequences; and (iii) how to construct signals to optimize model training and guide the fusion of ID and semantic information. To address these challenges, we propose a novel model, i.e., LLM-based semantic and ID representations for sequential recommendation (SIDSRec). Firstly, we propose an LLM-driven knowledge enhancement module that generates textual features and transfers it to item-level semantic representations. Secondly, we design a dual-channel preference modeling module that captures collaborative preferences and semantic preferences independently, and then aggregates them via a late-fusion layer. Specifically, we introduce modern LLM architectures into recommender systems. The embedding layer introduces a mixture of experts (MoE)-based adapter to improve the discriminative ability of the semantic representations. The sequential encoder introduces a self-attention and a gating mechanism to facilitate user preference learning. Finally, we introduce the next-item prediction task and the user preference alignment task to jointly optimize model training and modality fusion. Experiments conducted on three datasets show that our SIDSRec outperforms the state-of-the-art sequential recommenders by 24.96% and 18.48% on NDCG@5 and NDCG@10 on average. The source codes are available at https://github.com/donglinzhou/SIDSRec.