Comparing UX of Response Format Personalization in ChatGPT and Google Gemini: A UEQ and ISO 9241-11 Evaluation

Abstract

The rapid development of Generative Artificial Intelligence (GenAI) has prompted growing interest in how personalization features influence user experience across different platforms. However, most previous studies have focused on the general usability of GenAI platforms and have not specifically examined how response format personalization affects user experience. This study analyzes and compares the user experience of response personalization features in two widely used GenAI platforms, ChatGPT and Google Gemini, with a focus on response format customization. A mixed-method approach was employed, combining the User Experience Questionnaire (UEQ) distributed to 100 respondents and a task-based assessment based on the ISO 9241-11 framework involving 10 respondents across three predefined task scenarios. The results show that both platforms provided a positive user experience and achieved 100% task effectiveness. ChatGPT was perceived as clearer, scoring 1.79 against Gemini's 1.41 in Perspicuity, and obtained higher satisfaction at 4.07 against Gemini's 3.85, whereas Gemini completed tasks more efficiently, averaging 34.11 seconds against ChatGPT's 39.46 seconds. In terms of weaknesses, ChatGPT scored lowest in Dependability at 0.72, the dimension in which Gemini performed strongest at 1.40, while Gemini scored lowest in Novelty at 1.03. Based on these findings, the study proposes targeted improvements aimed at improving dependability in ChatGPT and enhancing novelty in Gemini. By addressing the specific weaknesses identified in each platform, this research provides practical and evidence-based insights for developing more user-centered GenAI personalization features in the future.