Zhao Jiawei ; Yan Ke ; Zhao Yifan ; Guo Xiaowei ; Huang Feiyue ; Li Jia

Abstract

The simultaneous recognition of multiple objects in one image remains achallenging task, spanning multiple events in the recognition field such asvarious object scales, inconsistent appearances, and confused inter-classrelationships. Recent research efforts mainly resort to the statistic labelco-occurrences and linguistic word embedding to enhance the unclear semantics.Different from these researches, in this paper, we propose a novelTransformer-based Dual Relation learning framework, constructing complementaryrelationships by exploring two aspects of correlation, i.e., structuralrelation graph and semantic relation graph. The structural relation graph aimsto capture long-range correlations from object context, by developing across-scale transformer-based architecture. The semantic graph dynamicallymodels the semantic meanings of image objects with explicit semantic-awareconstraints. In addition, we also incorporate the learnt structuralrelationship into the semantic graph, constructing a joint relation graph forrobust representations. With the collaborative learning of these two effectiverelation graphs, our approach achieves new state-of-the-art on two popularmulti-label recognition benchmarks, i.e., MS-COCO and VOC 2007 dataset.