On Evaluating and Improving Conversational Agents in Production
We present a framework for evaluating and improving a large-scale, multi-agent shopping assistant in production, and report lessons from its use. Offline evaluation of such a system faces three obstacles. (i) A logged conversation cannot be replayed against a modified system, because a different response changes every...
Kasra Hosseini, Wen-Sen Cheng, Marco-Andrea Buchmann et al.
· 0 citations