TopicTune: A Topic Alignment Approach to Improve LLMs Reasoning on Social Questions

Abstract

Large Language Models (LLMs) have demonstrated widespread utility in various tasks, including reasoning over social questions. While LLMs are primarily designed for open-domain applications, recent efforts have adapted them for closed-domain tasks in fields like finance and healthcare to enhance relevance and precision. However, closed-domain LLMs face challenges, including data scarcity, high costs, and limited capacity in addressing intersectional domains that require knowledge across multiple areas. To address these limitations, this study introduces TopicTune, a topic-based prompt-tuning framework that improves the reasoning abilities of open-domain LLMs on social questions by aligning their responses with the specific topics of each query. TopicTune employs a reinforcement learning (RL) model to refine the selection of the top k relevant topics for each query, ensuring the responses align with the query's context. The framework then tailors prompts based on topic-specific nuances, enabling LLMs to generate well-reasoned and contextually relevant responses. We evaluate TopicTune using real-world, open-ended questions from Reddit and Lemmy, demonstrating an improvement in reasoning over social questions across three LLMs. The motivation of this study is to enhance LLM reasoning performance by enabling AI to generate more contextually relevant responses that better meet human needs.