WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection

Dongyu, She, Jufeng, Yang, Ming-Ming, Cheng, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Rosin, Paul ORCID: https://orcid.org/0000-0002-4965-3884 and Liang, Wang 2020. WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection. IEEE Transactions on Multimedia 22 (5) , pp. 1358-1371. 10.1109/TMM.2019.2939744

Preview

PDF - Accepted Post-Print Version Download (4MB) | Preview

Abstract

Automatic assessment of sentiment from visual content has gained considerable attention with the increasing tendency of expressing opinions online. In this paper, we solve the problem of visual sentiment analysis, which is challenging due to the high-level abstraction in the recognition process. Existing methods based on convolutional neural networks learn sentiment representations from the holistic image, despite the fact that different image regions can have different influence on the evoked sentiment. In this paper, we introduce a weakly supervised coupled convolutional network (WSCNet). Our method is dedicated to automatically selecting relevant soft proposals from weak annotations (e.g., global image labels), thereby significantly reducing the annotation burden, and encompasses the following contributions. First, WSCNet detects a sentiment-specific soft map by training a fully convolutional network with the cross spatial pooling strategy in the detection branch. Second, both the holistic and localized information are utilized by coupling the sentiment map with deep features for robust representation in the classification branch. We integrate the sentiment detection and classification branches into a unified deep framework, and optimize the network in an end-to-end way. Through this joint learning strategy, weakly supervised sentiment classification and detection benefit each other. Extensive experiments demonstrate that the proposed WSCNet outperforms the state-of-the-art results on seven benchmark datasets.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Institute of Electrical and Electronics Engineers (IEEE)
ISSN:	1520-9210
Date of First Compliant Deposit:	1 October 2019
Date of Acceptance:	18 August 2019
Last Modified:	06 Nov 2023 18:31
URI:	https://orca.cardiff.ac.uk/id/eprint/125791

Citation Data

Cited 31 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item