learning-ai.cafe learning-ai.cafe / Learning by Comparison / 10

10 · Learning by Comparison

References

Learning a Similarity Metric Discriminatively, with Application to Face Verification — Chopra, Hadsell & LeCun. CVPR 2005.

@inproceedings{chopra2005learning,
  title     = {Learning a Similarity Metric Discriminatively, with Application to Face Verification},
  author    = {Chopra, Sumit and Hadsell, Raia and LeCun, Yann},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2005},
  doi       = {10.1109/CVPR.2005.202}
}

Contrastive Representation Learning — Lilian Weng. Lil'Log, 2021.

@misc{weng2021contrastive,
  title   = {Contrastive Representation Learning},
  author  = {Weng, Lilian},
  journal = {Lil'Log},
  year    = {2021},
  url     = {https://lilianweng.github.io/posts/2021-05-31-contrastive/}
}

Noise-contrastive estimation: A new estimation principle for unnormalized statistical models — Gutmann & Hyvärinen. AISTATS 2010.

@inproceedings{gutmann2010noise,
  title     = {Noise-contrastive estimation: A new estimation principle for unnormalized statistical models},
  author    = {Gutmann, Michael and Hyv{\"a}rinen, Aapo},
  booktitle = {International Conference on Artificial Intelligence and Statistics (AISTATS)},
  series    = {JMLR Workshop and Conference Proceedings},
  volume    = {9},
  year      = {2010},
  url       = {https://proceedings.mlr.press/v9/gutmann10a.html}
}

Training Products of Experts by Minimizing Contrastive Divergence — Geoffrey Hinton. Neural Computation 14(8), 2002.

@article{hinton2002training,
  title   = {Training Products of Experts by Minimizing Contrastive Divergence},
  author  = {Hinton, Geoffrey E.},
  journal = {Neural Computation},
  volume  = {14},
  number  = {8},
  pages   = {1771--1800},
  year    = {2002}
}

Estimation of Non-Normalized Statistical Models by Score Matching — Aapo Hyvärinen. JMLR 6, 2005.

@article{hyvarinen2005estimation,
  title   = {Estimation of Non-Normalized Statistical Models by Score Matching},
  author  = {Hyv{\"a}rinen, Aapo},
  journal = {Journal of Machine Learning Research},
  volume  = {6},
  pages   = {695--709},
  year    = {2005}
}

A Cookbook of Self-Supervised Learning — Balestriero et al. 2023.

@article{balestriero2023cookbook,
  title   = {A Cookbook of Self-Supervised Learning},
  author  = {Balestriero, Randall and Ibrahim, Mark and Sobal, Vlad and Morcos, Ari and Shekhar, Shashank
             and Goldstein, Tom and Bordes, Florian and Bardes, Adrien and Mialon, Gregoire and Tian, Yuandong and others},
  journal = {arXiv preprint arXiv:2304.12210},
  year    = {2023},
  eprint  = {2304.12210},
  archivePrefix = {arXiv}
}

FaceNet: A Unified Embedding for Face Recognition and Clustering — Schroff, Kalenichenko & Philbin. CVPR 2015.

@inproceedings{schroff2015facenet,
  title     = {FaceNet: A Unified Embedding for Face Recognition and Clustering},
  author    = {Schroff, Florian and Kalenichenko, Dmitry and Philbin, James},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages     = {815--823},
  year      = {2015},
  eprint    = {1503.03832},
  archivePrefix = {arXiv}
}

Deep Metric Learning via Lifted Structured Feature Embedding — Song, Xiang, Jegelka & Savarese. CVPR 2016.

@inproceedings{song2016deep,
  title     = {Deep Metric Learning via Lifted Structured Feature Embedding},
  author    = {Song, Hyun Oh and Xiang, Yu and Jegelka, Stefanie and Savarese, Silvio},
  booktitle = {IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  pages     = {4004--4012},
  year      = {2016},
  eprint    = {1511.06452},
  archivePrefix = {arXiv}
}

Improved Deep Metric Learning with Multi-class N-pair Loss Objective — Kihyuk Sohn. NeurIPS 2016.

@inproceedings{sohn2016improved,
  title     = {Improved Deep Metric Learning with Multi-class N-pair Loss Objective},
  author    = {Sohn, Kihyuk},
  booktitle = {Advances in Neural Information Processing Systems (NeurIPS)},
  volume    = {29},
  year      = {2016}
}

Representation Learning with Contrastive Predictive Coding — van den Oord, Li & Vinyals. 2018.

@article{oord2018representation,
  title   = {Representation Learning with Contrastive Predictive Coding},
  author  = {van den Oord, Aaron and Li, Yazhe and Vinyals, Oriol},
  journal = {arXiv preprint arXiv:1807.03748},
  year    = {2018},
  eprint  = {1807.03748},
  archivePrefix = {arXiv}
}

A Simple Framework for Contrastive Learning of Visual Representations — Chen, Kornblith, Norouzi & Hinton. ICML 2020.

@inproceedings{chen2020simple,
  title     = {A Simple Framework for Contrastive Learning of Visual Representations},
  author    = {Chen, Ting and Kornblith, Simon and Norouzi, Mohammad and Hinton, Geoffrey},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2020},
  eprint    = {2002.05709},
  archivePrefix = {arXiv}
}

Momentum Contrast for Unsupervised Visual Representation Learning — He, Fan, Wu, Xie & Girshick. CVPR 2020.

@inproceedings{he2020momentum,
  title     = {Momentum Contrast for Unsupervised Visual Representation Learning},
  author    = {He, Kaiming and Fan, Haoqi and Wu, Yuxin and Xie, Saining and Girshick, Ross},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2020},
  eprint    = {1911.05722},
  archivePrefix = {arXiv}
}

learning-ai.cafe

Learning by Comparison

Teach the model what makes two examples similar (and what makes them different) without ever looking at the labels.

  1. 01 Introduction
  2. 02 Noise-Contrastive Estimation
  3. 03 Triplet Loss
  4. 04 Lifted Structured Loss
  5. 05 N-pair Loss
  6. 06 InfoNCE Loss
  7. 07 SimCLR
  8. 08 MoCo
  9. 09 BYOL & Beyond
  10. 10 References

Who built this page

Every merged contribution is typed and scored at review. Chips next to a heading show who touched that section. Scores feed the contributor share of project support.

History

    Propose a change

    Settings

    Language
    Appearance
    Accent
    Font
    Text size
    Column width
    Book