CounselingQA: scaling professional mental-health guidance via dense retrieval and large language models

Abstract

In the digital era, the internet and social media have emerged as essential platforms for individuals facing mental health issues, often used for seeking information and community support. Despite the resources of informal advice available on social media, the complexity of these issues frequently exceeds non-expert knowledge. Specialized sites such as CounselChat and 7Cups offer professional guidance, yet many at-risk individuals still rely on unmoderated sources and general web search. We address this gap by investigating ranking strategies that match pre-existing expert advice to incoming mental-health questions. We introduce , a collection built from two specialized websites, pairing user questions with verified expert responses. We address the task as answer retrieval (AR): given a question, rank expert answers by relevance. We evaluate dense retrieval with SentenceBERT and MentalBERT, and propose a second stage that improves the initial ranking via transformer-based models and large language models (LLMs), used for filtering non-relevant candidates and for reordering. Beyond retrieval, we analyze linguistic style and affective attributes across topics, questions, and responses. Results show that dense retrieval provides strong candidates and that transformer/LLM-driven reranking further elevates relevant, on-topic advice to the top positions. We further conduct qualitative error analyses, including human evaluation to study the benefits and limitations of our approaches. Taken together, these findings indicate that retrieval-first pipelines can help scale access to professional guidance.