Preprint

GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models

Yiqun Sun Junyu Chen Pengfei Wei Lawrence B. Hsieh

Aug 2026 · 0 citations · 52 references

Computer Science

TL;DR

GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal.

Abstract

Generative vision-language models (VLMs) are increasingly used in human-centered settings, yet they can produce demographically biased outputs even when images differ only in controlled attributes such as perceived race or gender. However, existing inference-time debiasers were largely designed for static embeddings or CLIP-like models rather than generative VLMs. We propose GGSS---Geodesic-Gated Spherical Steering---a norm-preserving intervention that discovers a counterfactual bias subspace on the unit hypersphere, steers visual tokens along geodesic arcs, and uses an adaptive gate to focus correction on tokens that carry stronger demographic signal. We evaluate four generative VLMs against ten adapted inference-time debiasing baselines and prompt-based mitigation under a single operating-point protocol across categorical, pairwise, and occupation-gender bias tests, while also measuring general visual-language capability. GGSS achieves the lowest average bias on all four models, significant on three of four backbones under paired permutation tests, while preserving MMStar accuracy within +/- 0.6 p.p. of the unsteered baseline. Code is available at https://github.com/dukesun99/GGSS.

arXiv Semantic Scholar

View source

Similar papers

#artificial intelligence Preprint Aug 2026

From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model

This work proposes a principled VLM TTA method called \algname, and theoretically reveals that the InfoNCE loss can be neatly reformulated as a Wasserstein OT formulation, thereby unifying the objectives of the inference and adaptation of VLMs to achieve their mutual benefits.

Qi Yu, Zhichen Zeng, Katherine Tieu et al. · 0 citations

#artificial intelligence Preprint Sep 2026

Steering Geometry: Validating Human Value Geometry in LLM Steering Space

As large language models (LLMs) are increasingly deployed in alignment-sensitive contexts, activation steering has emerged as a lightweight, inference-time alternative to fine-tuning methods (e.g., RLHF, DPO) for behavioral control. However, existing work typically validates steering on isolated behaviors, leaving it u...

Mohammad Mahdi Abootorabi, Armin Saghafian, Ali Bazshoushtari et al. · 0 citations

Preprint Aug 2026

Interpreting Language Model Hidden States at Scale

OmniLens is presented, which applies a single lens family to any model-width activation, whether residual stream, attention, or MLP, and combines two independent scaling techniques, which reproduces key published results at substantially lower cost.

Jordan Pettyjohn, Mansi Sakarvadia, Nathaniel Hudson et al. · 0 citations

#artificial intelligence Preprint Sep 2026

RiLM: Parameter-Efficient Language Modeling via Geodesic Decoding

Language models under one million parameters matter for edge deployment, domain adaptation, and reproducible research, yet a two-layer LSTM or Transformer at embedding width d = 128 still spends roughly one third of its capacity on the output matrix W_out in R^(d x |V|). We propose Riemannian Language Models (RiLM), wh...

Fang Li · 0 citations

Preprint Aug 2026

StateSight: Benchmarking Latent Spatial-State Reconstruction in Vision-Language Models

The results show that format-valid responses can mask failures to recover the spatial structure required for verifiable visual inference, and that format-valid responses can mask failures to recover the spatial structure required for verifiable visual inference.

Michelle Lin · 0 citations

Preprint Sep 2026

DIVA: Exploiting Cross-Step Conditional Propagation for Visual Jailbreaks in Discrete Diffusion Vision-Language Models

Large vision-language models (VLMs) are increasingly deployed in safety-critical settings, yet existing visual jailbreak research has focused almost exclusively on autoregressive architectures, leaving an important emerging family unstudied: multimodal discrete diffusion vision-language models (dVLMs). We identify a vu...

Guo-Rui Song, Run-Qing Tang, Jing-Ye Zhang et al. · 0 citations