year: 2020
paper: https://arxiv.org/pdf/2002.05709.pdf
website:
code:
connections: contrastive learning, SSL
Contrastive learning between different representations of the same image (created via noise, cut-outs / ….) which should maximize similarity, but also using contrastive / negative examples, in order to avoid collapse, where both encoders just output the same constant:
The part in the numerator should be high (maximum simmilarity from the same images) and the one in the denominator should be low (minimum simmilarity from different images).
Obious problem: You might have an image for a labrador, but also a different image of a labrador: Dubious to use that as a contrastive sample.
Making non-same things equally dissimilar, even though that’s not the case (Black & White approach).