Skip to content
Conference Open access

A Gloss-driven Indian Sign Language Production System Using Learned Pose Representations

Sep 2026 · Proceedings of the Thirty-Fifth International Joint Conference on Artificial Intelligence · 0 citations · 45 references

TL;DR

A scalable and modular SLP framework based on Sign-Pose-VQ-VAE model, designed for low-resource settings, achieves state-of-the-art performance among keypoint-based methods on the PHOENIX14T benchmark, attaining a BLEU-4 score of 10.03 and surpassing the previous best method by 0.67 points.

Abstract

Sign Language Production (SLP) system translates spoken or written language into sign language, enabling accessible communication between the deaf/hard-of-hearing and the hearing population. Being one of the most widely used sign languages globally, Indian Sign Language (ISL) is a very low-resource language and lacks such SLP systems. This paper presents a scalable and modular SLP framework based on Sign-Pose-VQ-VAE model, designed for low-resource settings. The model learns discrete pose representations (codes) by disentangling body, left-hand, and right-hand keypoints, enabling efficient pose modeling and co-articulated sign generation. The proposed system is evaluated using a Hindi movie subtitle corpus coupled with an off-the-shelf back-translation model and achieves a gloss BLEU-4 score of 47.20. The system-generated signs are evaluated by certified ISL interpreters with an average rating of 4.33/5, and a BERT precision of 0.7683 on glosses. In addition, the proposed system achieves state-of-the-art performance among keypoint-based methods on the PHOENIX14T benchmark, attaining a BLEU-4 score of 10.03 and surpassing the previous best method by 0.67 points. The system with code is available at https://cs.rkmvu.ac.in/~isl/sl_gen_vqvae.

Read PDF

Similar papers

Conference Aug 2026

A Greedy Skeleton Retrieval Framework for Vietnamese Text-to-Sign Generation

Sign Language Production (SLP) plays a crucial role in bridging the communication gap between the Deaf community and broader society, functioning alongside Sign Language Translation (SLT) and Recognition (SLR). In addition to the limited scale of available data, research on Vietnamese Sign Language (VSL) is further hin...

D. Thanh, Thang Cap · 0 citations
#artificial intelligence Preprint Sep 2026

Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

This work introduces an alternative inspired by language-learning assessment, using an open-weight-LLM QA protocol that measures salient content preservation that aligns more closely with human rankings and is six to seven times more paraphrase-invariant than BLEU-4.

Oline Ranum, Edward Fish, Simon Hadfield et al. · 0 citations
#large language models Review Open access Sep 2026

Leveraging large and vision language models for gloss-free sign language translation in deaf communication: a survey

Sign Language Translation (SLT) is an essential tool for overcoming communication barriers faced by the Deaf and Hard-of-Hearing (DHH) community, enabling equal access to vital information and interactions. This need is especially critical in large-scale, multilingual settings such as Hajj and Umrah, where millions of...

Sufyan Danish, S. Khan, Rama Abdullah Alghamdi et al. · 0 citations
#computer vision Open access Sep 2026

Zero-Shot Cross-Lingual Recognition of Sign Language Handshapes

This work presents the first zero-shot cross-lingual framework for handshape recognition, transferring from ASL to Catalan Sign Language (LSC), and leverages the decomposition of handshapes into five phonological features shared across both languages, to decode LSC handshapes from predicted features via a composite pho...

Marcel Granero Moya, Carolina Del Corral Farrarós, G. Haro et al. · 1 citation
Open access 2026

Bridging Text-to-Sign Translation via Codebook-Oriented Pretraining

This work proposes a novel text-to-sign translation based on model pretraining, which enhances semantic alignment by inheriting codebook-oriented prior knowledge from masked self-supervised models.

Ninlawat Phuangchoke, C. Polprasert · 0 citations
Sep 2026

SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation

SignGPT is presented, a unified, pose-based framework for gloss-free SLT and SLG that integrates part-aware hierarchical representations of body, hand, and facial motion into a shared language model and employs asymmetric multi-token prediction and progressive training for bidirectional modeling.

Rong-Hui Li, Jun Dong, Zhong-Yuan Hu et al. · 0 citations

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.