Skip to content

GLaS-JEPA: Gaussian-Regularized Speech SSL without Engineered Prediction Targets

Sep 2026 · arXiv.org · Vol abs/2609.37798 · 0 citations · 30 references
Engineering Computer Science

TL;DR

GLaS-JEPA is presented, a framework that directly predicts the current encoder's continuous representations at masked positions, without contrastive learning, discrete targets, or separate EMA target encoders, and prevents representation collapse using SIGReg representation-space regularization.

Abstract

Speech self-supervised learning aims to learn general-purpose representations for downstream speech tasks. However, current approaches rely on complex, carefully designed prediction targets. We challenge this necessity with GLaS-JEPA, a framework that directly predicts the current encoder's continuous representations at masked positions, without contrastive learning, discrete targets, or separate EMA target encoders. We prevent representation collapse using SIGReg representation-space regularization, eliminating the need for engineered target-generation mechanisms. Pretrained on 960 hours of LibriSpeech, our 57M-parameter model achieves a 6.89% WER on frozen-encoder SUPERB ASR and a 25.87% CER on slot filling, outperforming the best non-distilled sub-90M baselines by 43.1% and 22.0%, respectively. These results demonstrate that highly competitive speech representations can emerge from a radically simplified training recipe.

View source

Similar papers

#natural language process... Preprint Sep 2026

Does a prosody-trained representation help beyond trainable fusion? A parameter-matched study with frozen HuBERT

Explicit prosodic cues may help automatic speech recognition (ASR) of spontaneous speech, but auxiliary representations typically require additional trainable components, making it unclear whether gains come from the auxiliary information or the fusion mechanism. We address this using a frozen HuBERT backbone and a 64-...

Ki Woong Moon, Dan Brenner · 0 citations

What Survives the Codec Shift: Pooled No-Vocals Residuals for Speech Deepfake Detection

The transition from vocoder-based to neural-codec speech synthesis makes generalization more difficult for speech deepfake detectors, particularly those relying on speech-oriented representations. It remains unclear which acoustic representations retain discriminative information when the generation mechanism changes....

Jia-Jun Xu, Meng-Lu Li, Xiao-Ping Zhang · 0 citations
Sep 2026

SPEAR-Gen: Generation-Aware Pre-training for Unified Speech Representations

Speech understanding and generation place different demands on speech representations, and existing models are typically optimised towards one capability or the other. To reduce this gap, we introduce SPEAR-Gen, a speech representation model that learns a single representation for both capabilities. Task-aligned featur...

Xiao-Yu Yang, Arthur Hinsvark, Antonios Alexos et al. · 0 citations
#artificial intelligence Preprint Sep 2026

GrainSpeech: Less Context, More Detail for Compact Speech Synthesis

A receptive-field-scaling study shows that expanding self-attention beyond 15 phonemes provides no consistent gains in pitch, energy, or duration prediction, and introduces a fixed-receptive-field convolutional encoder that reduces the respective prediction errors.

Zi-Tao Liang, Chang Gao · 0 citations
Preprint Oct 2026

What Do Deepfake Audio Detectors Hear? From Self-Supervised Models to Audio LLMs

Real-world audio recordings for deepfake detection rarely consist solely of speech; they often are mixtures of speech and background events that themselves may also be synthetic. When processing these mixtures, it remains unclear whether detectors attend to the speech or the background event, and where their spectral e...

Wan-Ying Ge, Xin Wang, J. Yamagishi · 0 citations
Preprint Sep 2026

Is Broader Better? A Controlled Study of Multilingual Coverage and Pretraining Objective in Frozen SSL Encoders for Speech Deepfake Detection

It is found that a mid-coverage encoder is strongest on farther out-of-domain sets, matching or surpassing a 577M-parameter model at 315M, and within this fixed frozen-encoder recipe, broader and larger models are not reliably better.

Benjamin Hurt, Oscar O'Donnell · 0 citations

Related blog posts

MIT News · Artificial Intelligence Sep 29, 2026

Who we become when we talk to machines

Professor Sherry Turkle’s new book, “Artificial Intimacy,” offers a withering critique of chatbots and the antisocial dynamics she believes they encourage.

We use cookies to run the site and, with your consent, for analytics and to show ads. See our Cookie Policy.