Skip to content

APeB: Benchmarking Personalization Ability of Large Language Model Agents

Garry Yang Zizhe Chen Xinru Chen Yongqiang Chen Jianxiang Wang Deyu Zou Linyi Ding Jialiang Wu Yunzhong He Yu Gong James Cheng Huaixiao Tou
Aug 2026
Artificial Intelligence Human-computer Interaction

Abstract

LLM-powered agents struggle with personalization when users issue raw, underspecified queries. In this setting, agents must infer latent intent, extract preferences from noisy interaction histories, and select among competing alternatives. Existing benchmarks rarely test this capability, as they often rely on user-refined queries or simplified histories. We introduce personalized product search (PPS), a testbed for agentic personalization under raw queries and diverse histories. We construct Agent Personalized Benchmark (APeB) from action logs, pairing underspecified intents with rich histories and user-viewed candidate items. Evaluating state-of-the-art LLMs with multi-step agent workflows, we find that models handle explicit queries well but struggle with early-stage queries requiring intent and preference discovery. Rubric analysis attributes this gap mainly to ineffective history use. A simple history-aware query-refinement pipeline, VQRA, yields consistent gains, highlighting the need for dedicated history-utilization modules in personalized agents.

View source

Similar papers

#artificial intelligence Review Dec 2025

Professional Software Developers Don't Vibe, They Control: AI Agent Use for Coding in 2025

Investigating how experienced developers use agents in building software, including their motivations, strategies, task suitability, and sentiments finds that while experienced developers value agents as a productivity boost, they retain their agency in software design and implementation out of insistence on fundamental software quality attributes.

Ruanqianqian Huang, Avery Reyna, Sorin Lerner et al. · 19 citations · ⚡1
#artificial intelligence Open access Oct 2022

Adaptive surrogate modeling for high-dimensional spatio-temporal output

An adaptive surrogate modeling method for problems with very high-dimensional spatio-temporal outputs is developed that combines exploration and exploitation to improve the surrogate model accuracy with the fewest possible runs of the expensive physics-based model.

B. Kapusuzoglu, S. Mahadevan, Shunsaku Matsumoto et al. · 17 citations
#artificial intelligence Preprint Feb 2025

`From Prompt to Perturbation': An Adaptive Framework for Voice-Based Jailbreaks on Audio LLMs

An adaptive jailbreak attack framework for systematic evaluation of both cascaded pipelines and end-to-end large audio-language models under a unified experimental setting that achieves consistently higher attack success rates across diverse audio-based LLM systems.

Linghan Huang, Bo Li, Huaming Chen et al. · 12 citations · ⚡2
#artificial intelligence Review Open access Oct 2025

Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead

This review provides a systematic literature review of LLM-based Verilog code generation, analyzing 102 papers (70 published and 32 high-quality preprints) from SE, AI, and EDA venues and outlines a roadmap highlighting potential opportunities in LLM-assisted hardware design.

Guang Yang, Wei Zheng, Xiang Chen et al. · 11 citations · ⚡1

From Multi-Agent to Single-Agent: When Is Skill Distillation Beneficial?

This work introduces Behavior-Outcome Freedom (F), a pre-synthesis diagnostic of signed behavior-outcome rank mismatch, and formalizes its candidate-conditional role through Signed Anchor-Rank Transfer, which preserves validated capability resources, removes runtime orchestration, and conditionally inherits pipeline guidance using a calibrated rule over F.

Binyan Xu, Dong Fang, Haitao Li et al. · 10 citations

Diffusion Models for Smarter UAVs: Decision-Making and Modeling

Simulation results confirm the effectiveness and benefits of DMs in generating neighbor velocity estimates in a four-UAV swarm coordination task using Deep Reinforcement Learning (DRL), and explore the integration of DMs with RL and DT.

Yousef Emami, Hao Zhou, Luís Almeida et al. · 9 citations

Related blog posts