DOC-To-ISL: Indian Sign Language Translation System with Automatic Text Summarisation

Abstract

Communication barriers persistently restrict access for Deaf and Hard-of-Hearing individuals in digital environments, especially in systems that rely primarily on text-based interaction. DOCtoISL is introduced as an intelligent web-based Indian Sign Language (ISL) translation system that converts English text and uploaded documents into ISL gesture-based video output. The system integrates Natural Language Processing (NLP), transformer-based text summarisation, sentence simplification, and gesture mapping techniques to enhance readability and accessibility for ISL users. The application utilises a FastAPI backend, React.js frontend, SQLite database, and an OpenCV-based video generation pipeline. Uploaded PDF and DOCX documents are processed to extract textual content, which is subsequently summarised using the Hugging Face DistilBART transformer model. The simplified text is mapped to sign gesture images sourced from a Kaggle-based ISL alphabet and number dataset comprising approximately 24,000 images. The resulting gesture sequence is converted into MP4 video format for visual interpretation. Experimental evaluation indicates enhanced processing speed, more efficient text simplification, and more reliable gesture-mapping performance compared with traditional manual interpretation methods. The modular architecture supports scalability, multilingual extension, and future integration with AI-driven avatar-based ISL generation systems.