Jul 2026· Italian National Conference on Sensors· Vol 26, pp. 4583· 0 citations· 41 references
Medicine
TL;DR
A lightweight and projection-free autoencoder that directly outputs compact, Contrastive Language–Image Pre-training (CLIP)-scale latent vectors trained toward the CLIP embedding space, achieving superior signal reconstruction fidelity across time and frequency domains using significantly fewer parameters than traditional masked autoencoder baselines.
Abstract
Electroencephalography is emerging as a promising conditioning modality for generative visual models. However, existing representation learning approaches often rely on high-capacity masked autoencoders and complex projection networks. When constrained to compact embedding dimensions to match vision-language models, these heavy transformer-based bottlenecks frequently suffer from representation collapse and lose critical signal dynamics. To address this, we propose a lightweight and projection-free autoencoder that directly outputs compact, Contrastive Language–Image Pre-training (CLIP)-scale latent vectors trained toward the CLIP embedding space. Our model adopts a U-Net-style architecture combining one-dimensional convolutional residual blocks for temporal dynamics and inter-channel attention modules for spatial dependencies, alongside skip connections to ensure stable reconstruction. Extensive experiments on visual perception datasets demonstrate that our approach successfully tracks complex signal amplitudes without collapsing. Under strict dimensional constraints, the proposed model achieves superior signal reconstruction fidelity across time and frequency domains using significantly fewer parameters than traditional masked autoencoder baselines. Furthermore, latent space visualizations and zero-shot retrieval tasks reveal that while the baseline collapses toward unstructured, near-chance representations, our architecture preserves emerging, partial semantic organization and retrieves several times above chance. This indicates that the proposed design preserves signal structure while exhibiting preliminary, above-chance semantic alignment, enabling integration into brain-driven generative pipelines.
Sparse autoencoders (SAEs) have emerged as a promising technique for mechanistic interpretability by learning a set of sparse latent features in large models, each of which encodes a distinct concept. However, in vision-language models (VLMs), vanilla SAEs struggle to learn modality-consistent concepts, with concepts often exhibiting fragmented coverage (i.e., disjoint regions) in the visual modality. To address this challenge, we propose a Structured Sparse AutoEncoder ($S^2AE$) that enforces concept consistency from both semantic and spatial perspectives in the visual modality. Specifically, we group image patches based on Transformer attention similarity and spatial proximity, and introduce a structured sparsity regularization when training the vanilla SAE. The regularization consists of exclusive sparsity for inter-group concept disentanglement and group sparsity for intra-group concept consistency, which drives the latent neurons by SAEs to specialize in distinct, semantically grounded concepts. Evaluated on the \texttt{Qwen2.5-VL-7B-Instruct} model, the method achieves 6.06% average improvement in semantic alignment (mIoU) and 60.81 in representational efficiency (lower l0 norm) while maintaining near-perfect reconstruction fidelity with an Explained Variance above 99%. Cross-modal analysis further demonstrates that $S^2AE$ enhances neuronal monosemanticity by this visual structural prior, achieving a 3.08% average gain in semantic consistency and a 2.37% average gain in monosemanticity scores for both modalities of multimodal features, thereby fostering more coherent and disentangled representations.
Latent Attention Denoising (LAD) is introduced, a principled and training-free framework that recasts attention calibration as a one-step score-based denoising process that achieves superior performance on both generative and discriminative tasks.
Zhiwen Luo, Siyu Jiang, Weilong Jiang et al.· Annual Meeting of the Associ...· 0 citations
A Fourier-enhanced operator autoencoder for decoder-free reconstruction and latent learning of dynamical systems and achieves accuracy comparable to or better than classical AE-based reduced-order models while providing a more efficient latent-to-field reconstruction path.
Xuandong Lu, Yongming Liu· Machine Learning for Computa...· 0 citations
Modern video transformers typically ignore principles from primate vision and are rarely evaluated against neural data, limiting their biological interpretability. We introduce a sparse winner-takes-all token selection module that replaces dense self-attention to improve efficiency and approximate competitive routing observed in biological visual circuits. We further propose a neuro-inspired split-and-fuse video transformer which uses two complementary pathways: a high-resolution, low-frame-rate"what"stream and a low-resolution, high-frame-rate"where"stream, fused before classification. On Kinetics-400 and Something-Something V2, our best variant operates on the Pareto frontier of accuracy versus inference time among models of comparable scale and pretraining, and showing improved robustness to spatial perturbations. Using representational similarity analysis between model embeddings and time-resolved EEG recordings for the same video stimuli, our model attains a peak brain-model correlation of 0.18 (about 78% of the noise ceiling) and consistently outperforms strong video transformer baselines, suggesting that pathway specialization and sparse competition are useful inductive biases for efficient, brain-aligned video understanding.
AmirHosein Fadaei, Mahyar Maleki, M. Dehaqani· 0 citations
Latents from vision foundation models (VFMs) are semantically rich and well suited for visual understanding. Recent representation autoencoder methods such as RAE have shown that they can provide promising latent spaces for image generation. However, VFM latents remain difficult to model directly: DiT-generated latents exhibit spectral mismatch with encoder latents, especially in high-frequency components. Our channel-wise spectral analysis further reveals that these high-frequency components are diffusely distributed across latent channels and entangled with semantic information, making the latent space difficult for DiT to model. To address these challenges, we propose SPAE, latent adaptation framework for generation. Specifically, SPAE employs a compact bottleneck to distill stable semantic information while suppressing high-frequency components, thereby improving the alignment between DiT-generated latents and encoder latents. In addition, we apply a channel-wise masking strategy to promote the decoupling of semantic information and high-frequency details across bottleneck channels. Experiments show that SPAE achieves a favorable balance among visual understanding, generation quality, and reconstruction fidelity.
Yibin Huang, Jixiang Hong, Zongzhao Li et al.· 0 citations
Latent Acoustic Mapping (LAM) is a self-supervised learning method that generates high-resolution spherical acoustic maps from multichannel recordings without labelled data, matching supervised baselines on direction-of-arrival benchmarks. However, LAM degrades significantly with sparse 4-channel arrays, as the low-resolution cross-spectral matrix captures far less spatial information than the 32-channel inputs LAM was designed for. We benchmark a diverse set of upsampling architectures, spanning lightweight convolutional networks, iterative back-projection models, physics-informed networks, and generative adversarial approaches. We also study whether aligning these upsamplers with LAM by training them jointly or in different stages helps preserve the spatial structure that LAM depends on. Results show that the original full-resolution LAM is the strongest, that separately trained lightweight models are the most competitive learned approaches, and that representation alignment between the upsampler and LAM matters more than model complexity.
Philipp Schmidt, Huw Cheston, Juan Azcarreta et al.· 0 citations