Preprint

Not Only NTP: Extending Training Signal Coverage for Generative Recommendation

Changhao Li Shuli Wang Junwei Yin Senjie Kou Yinqiu Huang Chiwei Wang Yinhua Zhu Haitao Wang Xingxing Wang

Jul 2026 · 0 citations · 23 references

Computer Science

TL;DR

This work proposes NONTP, extending NTP's signal coverage along both dimensions through two auxiliary objectives, and analyses each component contributes independently, with gradient conflict analyzed as a direction for future work.

Abstract

Next-Token Prediction (NTP) carries two structural training signal limitations. First, NTP optimizes for single-step prediction only, placing no supervised pressure on learning longer-range behavioral structure -- we term this \textbf{temporal locality}. Second, in multi-domain sequences, each target item embedding receives gradient updates exclusively from the immediately preceding hidden state, with no explicit gradient pathway from cross-domain context -- we term this \textbf{spatial locality}. We propose \textbf{NONTP}, extending NTP's signal coverage along both dimensions through two auxiliary objectives. \textbf{TCL (Temporal Contrastive Learning)} uses a BYOL-style EMA teacher with InfoNCE to align hidden states against a $K$-step future trajectory in representation space. \textbf{TDL (Trans-Domain Learning)} mean-pools cross-domain hidden states and predicts through the shared prediction head, opening a second gradient pathway with no additional parameters. Both are discarded at inference: zero overhead. On a four-domain Meituan industrial dataset (full ranking), NONTP achieves HR@10 +34.3\% over NTP and +18.3\% over MBGR. On the public Amazon Movie-Book-CDs benchmark, HR@10 +2.8\% and NDCG@10 +3.7\%. Online A/B tests confirm CTR +1.8\% and GMV +2.1\% (both $p<0.01$). Ablation studies confirm each component contributes independently, with gradient conflict analyzed as a direction for future work.

arXiv Semantic Scholar

View source

Similar papers

Preprint Aug 2026

AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction

AdaMTP is proposed, an adaptive training paradigm that dynamically aligns the prediction horizon with the intrinsic predictability of the sequence, and consistently outperforms standard MTP in both task performance and inference speedup.

Ziqiang Cui, Han Shi, Bowei He et al. · 0 citations

Preprint Aug 2026

Learning What to Remember: Test-Time Training via Context Distillation

This work proposes TTCD, a TTT framework that introduces a self-supervised objective for allocating limited memory capacity for future use, and focuses on an in-place variant: In-Place TTCD, which uses the existing MLP parameters as the fast weights.

Zi-Xuan Wang, Xingyu Dang, Ruiming Zhu et al. · 0 citations

Preprint Aug 2026

Empowering Compact LLMs with Fusion of Layer-wise Exits for Recommendation

The Fusion of Layer-wise Exits for Sequential Recommendation (FLEXRec), a discriminative framework that enhances compact LLMs while retaining scalable full-corpus ranking and achieves state-of-the-art accuracy among competing methods while remaining highly efficient.

Xurong Liang, Tong Chen, Q. Nguyen et al. · 0 citations

Conference Jul 2026

H-RLPOI: A Hybrid LLM and Reinforcement Learning Framework for Next POI Recommendation

Large Language Models (LLMs) have recently been explored for next Point-of-Interest (POI) recommendation. Despite progress, existing approaches face three fundamental challenges: (i) POIs are often represented by simple identifiers or categorical labels, overlooking rich textual semantics; (ii) The task of predicting t...

Zahra Hamdani, Saloua Zammali, S. Yahia · Annual International Compute... · 0 citations

Preprint Aug 2026

Hierarchical Latent Prediction for Language Models

Hierarchical Latent Prediction (HiLP) is introduced, which introduces an auxiliary higher-level abstract latent to help reduce the error accumulation effect in latent-space rollouts and can lead to longer-horizon coherent belief state representation.

Changyan Shi, Tim Pearce, Manan Tomar et al. · 0 citations

Jul 2026

LLM-Based Generative Retrieval for Snapchat Content Recommendation

The design and launch of SnapLGR, an LLM-based generative retrieval system for short-video recommendation at Snapchat shows that successful production SnapLGR requires joint design across representation learning, vocabulary grounding, and efficient training and serving.

Liam Collins, Jiwen Ren, Donald Loveland et al. · arXiv.org · 0 citations