Attention is known to be sparse in vision transformers. In particular, we analyze the unequal relevance of attention-matrix columns to the final computation and show that this phenomenon becomes increasingly pronounced with network depth. This can be interpreted as saying that tokens have an absolute importance for the...
Charles Herr, Gabriele Spadaro, Enzo Tartaglione· 0 citations
This work proposes an efficient strategy that enables merging of layers without an existing analytical solution, and also without increasing kernel size, and validate the approach across multiple architectures and datasets, and measure inference speed-up gains on real embedded platforms.
P. Shul'zhenko, Gabriele Spadaro, Enzo Tartaglione· 0 citations
This work introduces TORINO (TOken Reduction via Interpretable coNcept Overlap), a plug-and-play framework for adaptive visual token reduction in VLMs that requires no fine-tuning of the underlying model.