HyperAIHyperAI
HomeConsoleDocsNewsPapersTutorialsDatasetsWikiSOTALLM ModelsGPU LeaderboardEvents
Search
About
HyperAIHyperAI

Command Palette

Search for a command to run...

5 months ago

Label Propagation for Zero-shot Classification with Vision-Language Models

View Paper Details

Stojnić Vladan ; Kalantidis Yannis ; Tolias Giorgos

Label Propagation for Zero-shot Classification with Vision-Language Models

Abstract

Vision-Language Models (VLMs) have demonstrated impressive performance onzero-shot classification, i.e. classification when provided merely with a listof class names. In this paper, we tackle the case of zero-shot classificationin the presence of unlabeled data. We leverage the graph structure of theunlabeled data and introduce ZLaP, a method based on label propagation (LP)that utilizes geodesic distances for classification. We tailor LP to graphscontaining both text and image features and further propose an efficient methodfor performing inductive inference based on a dual solution and asparsification step. We perform extensive experiments to evaluate theeffectiveness of our method on 14 common datasets and show that ZLaPoutperforms the latest related works. Code:https://github.com/vladan-stojnic/ZLaP

Code Repositories

vladan-stojnic/zlap
Official
pytorch
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
transductive-zero-shot-classification-onZLaP
Top 1 Accuracy: 72.7
transductive-zero-shot-classification-on-1ZLaP
Accuracy: 60.9
transductive-zero-shot-classification-on-2ZLaP
Accuracy: 73.4
transductive-zero-shot-classification-on-3ZLaP
Accuracy: 92.8
transductive-zero-shot-classification-on-4ZLaP
Accuracy: 71.9
transductive-zero-shot-classification-on-5ZLaP
Accuracy: 72.1
transductive-zero-shot-classification-on-6ZLaP
Accuracy: 83.7
transductive-zero-shot-classification-on-7ZLaP
Accuracy: 93.6
transductive-zero-shot-classification-on-8ZLaP
Accuarcy: 73.3
transductive-zero-shot-classification-on-9ZLaP
Accuracy: 77.7
transductive-zero-shot-classification-on-cubZLaP
Accuracy: 64.1
transductive-zero-shot-classification-on-dtdZLaP
Accuracy: 51.8
transductive-zero-shot-classification-on-fgvcZLaP
Accuracy: 28.4
transductive-zero-shot-classification-on-foodZLaP
Accuracy: 87.9
zero-shot-learning-on-caltech-101ZLaP
Accuracy: 84
zero-shot-learning-on-caltech-101ZLaP*
Accuracy: 83.1
zero-shot-learning-on-cifar-10ZLaP*
Accuracy: 93.6
zero-shot-learning-on-cifar-10ZLaP
Accuracy: 93.4
zero-shot-learning-on-cifar-100ZLaP*
Accuracy: 74.2
zero-shot-learning-on-cifar-100ZLaP
Accuracy: 74
zero-shot-learning-on-cub-200-2011ZLaP*
Accuracy: 64.2
zero-shot-learning-on-cub-200-2011ZLaP
Accuracy: 64.3
zero-shot-learning-on-dtdZLaP*
Accuracy: 51
zero-shot-learning-on-dtdZLaP
Accuracy: 51.2
zero-shot-learning-on-eurosatZLaP*
Accuracy: 63.2
zero-shot-learning-on-fgvc-aircraftZLaP
Accuracy: 29.1
zero-shot-learning-on-fgvc-aircraftZLaP*
Accuracy: 29
zero-shot-learning-on-flowers-102ZLaP*
Accuracy: 75.5
zero-shot-learning-on-flowers-102ZLaP
Accuracy: 75.9
zero-shot-learning-on-food-101ZLaP*
Accuracy: 87.9
zero-shot-learning-on-food-101ZLaP
Accuracy: 87.8
zero-shot-learning-on-imagenetZLaP
Top 1 Accuracy: 72.1
zero-shot-learning-on-imagenetZLaP*
Top 1 Accuracy: 72.1
zero-shot-learning-on-oxford-iiit-petsZLaP*
Accuracy: 89
zero-shot-learning-on-oxford-iiit-petsZLaP
Accuracy: 90
zero-shot-learning-on-stanford-carsZLaP*
Accuracy: 71.8
zero-shot-learning-on-stanford-carsZLaP
Accuracy: 71.2
zero-shot-learning-on-sun397ZLaP*
Accuracy: 71.4
zero-shot-learning-on-sun397ZLaP
Accuracy: 71
zero-shot-learning-on-ucf101ZLaP*
Accuracy: 76.3
zero-shot-learning-on-ucf101ZLaP
Accuracy: 76.3

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp