Kao Yueying He Ran Huang Kaiqi

Abstract

Human beings often assess the aesthetic quality of an image coupled with theidentification of the image's semantic content. This paper addresses thecorrelation issue between automatic aesthetic quality assessment and semanticrecognition. We cast the assessment problem as the main task among a multi-taskdeep model, and argue that semantic recognition task offers the key to addressthis problem. Based on convolutional neural networks, we employ a single andsimple multi-task framework to efficiently utilize the supervision of aestheticand semantic labels. A correlation item between these two tasks is furtherintroduced to the framework by incorporating the inter-task relationshiplearning. This item not only provides some useful insight about the correlationbut also improves assessment accuracy of the aesthetic task. Particularly, aneffective strategy is developed to keep a balance between the two tasks, whichfacilitates to optimize the parameters of the framework. Extensive experimentson the challenging AVA dataset and Photo.net dataset validate the importance ofsemantic recognition in aesthetic quality assessment, and demonstrate thatmulti-task deep models can discover an effective aesthetic representation toachieve state-of-the-art results.