Command Palette
Search for a command to run...
Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross Girshick
Abstract
We present Momentum Contrast (MoCo) for unsupervised visual representation learning. From a perspective on contrastive learning as dictionary look-up, we build a dynamic dictionary with a queue and a moving-averaged encoder. This enables building a large and consistent dictionary on-the-fly that facilitates contrastive unsupervised learning. MoCo provides competitive results under the common linear protocol on ImageNet classification. More importantly, the representations learned by MoCo transfer well to downstream tasks. MoCo can outperform its supervised pre-training counterpart in 7 detection/segmentation tasks on PASCAL VOC, COCO, and other datasets, sometimes surpassing it by large margins. This suggests that the gap between unsupervised and supervised representation learning has been largely closed in many vision tasks.
Code Repositories
Benchmarks
| Benchmark | Methodology | Metrics |
|---|---|---|
| contrastive-learning-on-imagenet-1k | ResNet50 | ImageNet Top-1 Accuracy: 60.6 |
| image-classification-on-omnibenchmark | MoCoV2 | Average Top-1 Accuracy: 34.8 |
| self-supervised-image-classification-on | MoCo (ResNet-50 4x) | Number of Params: 375M Top 1 Accuracy: 68.6% |
| self-supervised-image-classification-on | MoCo (ResNet-50) | Number of Params: 24M Top 1 Accuracy: 60.6% |
| self-supervised-image-classification-on | MoCo (ResNet-50 2x) | Number of Params: 94M Top 1 Accuracy: 65.4% |
| self-supervised-image-classification-on-1 | MoCo (Resnet-50) | Top 1 Accuracy: 77.0% |
| self-supervised-image-classification-on-1 | MoCo (Resnet-50) | Top 1 Accuracy: 77.3% |
Build AI with AI
From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.