Theoretical Refinement of CLIP by Utilizing Linear Structure of Optimal Similarity
KME-CLIP is proposed, which leverages this structure through the inner product in a reproducing kernel Hilbert space (RKHS) and theoretically proves that the KME-CLIP similarity can bring the contrastive loss arbitrarily close to its optimal value, which is attained by PMI, as the size of the point set grows.