Bigger Text Encoders Can Hurt CLIP Zero-Shot Performance
Contrastive Language-Image Pretraining (CLIP) is a building block of many machine learning applications. Scaling laws have guided resource allocation for large-scale training, yet prior work treats total CLIP model size as a single variable, without exploring how the capacity split between encoders impacts downstream p...
Samir Char, Carles Domingo-Enrich, Randall Balestriero
· 0 citations