HyperAIHyperAI

Command Palette

Search for a command to run...

3 months ago

Nearest Neighbor Guidance for Out-of-Distribution Detection

Jaewoo Park Yoon Gyo Jung Andrew Beng Jin Teoh

Nearest Neighbor Guidance for Out-of-Distribution Detection

Abstract

Detecting out-of-distribution (OOD) samples are crucial for machine learning models deployed in open-world environments. Classifier-based scores are a standard approach for OOD detection due to their fine-grained detection capability. However, these scores often suffer from overconfidence issues, misclassifying OOD samples distant from the in-distribution region. To address this challenge, we propose a method called Nearest Neighbor Guidance (NNGuide) that guides the classifier-based score to respect the boundary geometry of the data manifold. NNGuide reduces the overconfidence of OOD samples while preserving the fine-grained capability of the classifier-based score. We conduct extensive experiments on ImageNet OOD detection benchmarks under diverse settings, including a scenario where the ID data undergoes natural distribution shift. Our results demonstrate that NNGuide provides a significant performance improvement on the base detection scores, achieving state-of-the-art results on both AUROC, FPR95, and AUPR metrics. The code is given at \url{https://github.com/roomo7time/nnguide}.

Code Repositories

jingkang50/openood
Official
pytorch
Mentioned in GitHub
roomo7time/nnguide
Official
pytorch
Mentioned in GitHub

Benchmarks

BenchmarkMethodologyMetrics
out-of-distribution-detection-on-imagenet-1k-1NNGuide-ViM (ViT-B/16)
AUROC: 92.96
FPR95: 33.10
out-of-distribution-detection-on-imagenet-1k-10NNGuide (RegNet)
AUROC: 95.82
FPR95: 17.00
Latency, ms: 31.00
out-of-distribution-detection-on-imagenet-1k-10NNGuide (ResNet50 w/ ReAct)
AUROC: 96.11
FPR95: 17.27
Latency, ms: 11.10
out-of-distribution-detection-on-imagenet-1k-11NNGuide (ResNet50 w/ ReAct)
AUROC: 92.49
FPR95: 35.1
Latency, ms: 11.10
out-of-distribution-detection-on-imagenet-1k-11NNGuide (RegNet)
AUROC: 97.73
FPR95: 10.79
Latency, ms: 31.00
out-of-distribution-detection-on-imagenet-1k-12NNGuide (RegNet)
AUROC: 95.42
FPR95: 17.97
out-of-distribution-detection-on-imagenet-1k-12NNGuide (ResNet50 w/ ReAct)
AUROC: 95.45
FPR95: 19.72
out-of-distribution-detection-on-imagenet-1k-3NNGuide (ResNet50 w/ ReAct)
AUROC: 97.7
FPR95: 11.12
Latency, ms: 11.10
out-of-distribution-detection-on-imagenet-1k-3NNGuide (RegNet)
AUROC: 99.57
FPR95: 1.83
Latency, ms: 31.00
out-of-distribution-detection-on-imagenet-1k-8NNGuide (RegNet)
AUROC: 94.43
FPR95: 21.58
out-of-distribution-detection-on-imagenet-1k-9NNGuide (RegNet)
AUROC: 91.87
FPR95: 31.47

Build AI with AI

From idea to launch — accelerate your AI development with free AI co-coding, out-of-the-box environment and best price of GPUs.

AI Co-coding
Ready-to-use GPUs
Best Pricing
Get Started

Hyper Newsletters

Subscribe to our latest updates
We will deliver the latest updates of the week to your inbox at nine o'clock every Monday morning
Powered by MailChimp