Tian Zhuotao ; Lai Xin ; Jiang Li ; Liu Shu ; Shu Michelle ; Zhao Hengshuang ; Jia Jiaya

Abstract

Training semantic segmentation models requires a large amount of finelyannotated data, making it hard to quickly adapt to novel classes not satisfyingthis condition. Few-Shot Segmentation (FS-Seg) tackles this problem with manyconstraints. In this paper, we introduce a new benchmark, called GeneralizedFew-Shot Semantic Segmentation (GFS-Seg), to analyze the generalization abilityof simultaneously segmenting the novel categories with very few examples andthe base categories with sufficient examples. It is the first study showingthat previous representative state-of-the-art FS-Seg methods fall short inGFS-Seg and the performance discrepancy mainly comes from the constrainedsetting of FS-Seg. To make GFS-Seg tractable, we set up a GFS-Seg baseline thatachieves decent performance without structural change on the original model.Then, since context is essential for semantic segmentation, we propose theContext-Aware Prototype Learning (CAPL) that significantly improves performanceby 1) leveraging the co-occurrence prior knowledge from support samples, and 2)dynamically enriching contextual information to the classifier, conditioned onthe content of each query image. Both two contributions are experimentallyshown to have substantial practical merit. Extensive experiments on Pascal-VOCand COCO manifest the effectiveness of CAPL, and CAPL generalizes well toFS-Seg by achieving competitive performance. Code is available athttps://github.com/dvlab-research/GFS-Seg.