VSL-HV8: A Vietnamese Sign Language Dataset for Sign Recognition and Translation Research

Abstract

Vietnamese Sign Language (VSL) remains under-resourced for automatic sign language recognition and translation research. Existing VSL datasets primarily focus on isolated gloss-level recordings, while a large-scale dataset containing continuous sentence-level signing sequences is still unavailable. This limitation restricts the development of models that can exploit temporal context across sign sequences and prevents standardized evaluation for continuous sign language recognition and translation in VSL. In this paper, we introduce VSL-HV8, a Vietnamese Sign Language dataset constructed from continuous sentence recordings with gloss-level segmentation annotations. The dataset consists of 4,160 continuous sentence videos collected from 8 signers and is further segmented into 15,792 isolated gloss clips covering 2,062 unique glosses. Unlike previous VSL datasets that only provide isolated sign samples, the sentence-level structure of VSL-HV8 preserves contextual relationships between consecutive signs, enabling isolated sign recognition while establishing a foundation for future continuous recognition and sign language translation research. To establish initial benchmarks, we construct three experimental subsets with vocabulary sizes of 50, 100, and 200 glosses and evaluate an I3D model pretrained on AUTSL. The proposed baseline achieves Top-1 accuracies of 75.99% on the 50-gloss subset and 61.48% on the 200gloss subset. We further provide a preliminary signer-independent (cross-signer) evaluation and discuss remaining limitations, including signer imbalance, long-tailed class distributions, and the lack of Vietnamese text translation annotations. VSL-HV8 is intended as an initial resource for advancing Vietnamese sign language recognition and translation research rather than a complete benchmark for robust cross-signer or end-to-end translation performance.