How Far Does a Shared Linear Map Go? Probing Feature-Space Manipulability for Image Editing

Abstract

Understanding how image-space transformations manifest in a model's internal representations is a longstanding goal in representation analysis. Prior work has shown that geometric transformations can often be captured by learned linear operators between feature maps, but it remains unclear whether this extends to photometric, local, and semantically defined edits. We train probes of increasing capacity from a spatially shared linear map to nonlinear per-vector, receptive-field, and global transformer models to predict feature-space changes induced by geometric transforms, photometric edits, occlusions, and diffusion-generated semantic edits. Across ConvNeXt, SwinV2, and DINOv3, a single shared linear map often predicts held-out manipulation outcomes nearly as well as substantially more expressive probes for the supervised backbones, with sufficiency generally increasing with depth; this pattern is less consistent for DINOv3. These results suggest that a simple spatially shared linear operator is often sufficient to represent diverse image manipulations, while its leading singular components capture semantic content and higher-rank components primarily refine image details. We frame these findings as predictive representational sufficiency rather than evidence of intrinsic linear feature-space geometry.