Hamza Omar Alkaf

Segment Where You Look: Attention as the Interface Between Language Models and Segmenters

Vision-language models (VLMs) that segment from language must pass the location of the referent to a pixel decoder. The dominant interface compresses it into the hidden state of a single [SEG] token, a d-dimensional vector with no spatial index, and leaves the decoder, which never sees the expression, to recover the re...