Yan Tianyu ; Wan Zifu ; Deng Xinhao ; Zhang Pingping ; Liu Yang ; Lu Huchuan

Abstract

Recently, Segment Anything Model (SAM) shows exceptional performance ingenerating high-quality object masks and achieving zero-shot imagesegmentation. However, as a versatile vision model, SAM is primarily trainedwith large-scale natural light images. In underwater scenes, it exhibitssubstantial performance degradation due to the light scattering and absorption.Meanwhile, the simplicity of the SAM's decoder might lead to the loss offine-grained object details. To address the above issues, we propose a novelfeature learning framework named MAS-SAM for marine animal segmentation, whichinvolves integrating effective adapters into the SAM's encoder and constructinga pyramidal decoder. More specifically, we first build a new SAM's encoder witheffective adapters for underwater scenes. Then, we introduce a HypermapExtraction Module (HEM) to generate multi-scale features for a comprehensiveguidance. Finally, we propose a Progressive Prediction Decoder (PPD) toaggregate the multi-scale features and predict the final segmentation results.When grafting with the Fusion Attention Module (FAM), our method enables toextract richer marine information from global contextual cues to fine-grainedlocal details. Extensive experiments on four public MAS datasets demonstratethat our MAS-SAM can obtain better results than other typical segmentationmethods. The source code is available at https://github.com/Drchip61/MAS-SAM.