Empathy-Enhanced Small Language Models for Digital Mental Health Counseling

Abstract

Access to quality mental health care remains a global challenge, especially in underserved regions. While large language models (LLMs) show promise in providing AI-driven therapeutic support, their massive size often limits practical deployment due to high computational and memory  requirements. This study presents a fine-tuned version of Phi-3, a compact yet powerful language model, tailored specifically for empathetic mental health counseling. By leveraging Low-Rank Adaptation (LoRA) and the Unsloth framework, we significantly improved response relevance and emotional sensitivity, while achieving a 2x increase in training speed and reducing memory consumption by 70%. The model was fine-tuned on three diverse dataset of publicly available mental health dialogues, enabling it to generate supportive and context-aware responses. Evaluation using BERTScore metrics demonstrated strong performance, with an F1-score of 0.8429 during training and 0.8385 on unseen test data. Our results suggest that small-scale, efficiently fine tuned LLMs like Phi-3 can offer accessible, accurate, and scalable mental health support bridging the gap between technological capability and real-world usability for the average user.