Dongyu, She, Jufeng, Yang, Ming-Ming, Cheng, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Rosin, Paul ORCID: https://orcid.org/0000-0002-4965-3884 and Liang, Wang 2020. WSCNet: Weakly Supervised Coupled Networks for Visual Sentiment Classification and Detection. IEEE Transactions on Multimedia 22 (5) , pp. 1358-1371. 10.1109/TMM.2019.2939744 |
Preview |
PDF
- Accepted Post-Print Version
Download (4MB) | Preview |
Abstract
Automatic assessment of sentiment from visual content has gained considerable attention with the increasing tendency of expressing opinions online. In this paper, we solve the problem of visual sentiment analysis, which is challenging due to the high-level abstraction in the recognition process. Existing methods based on convolutional neural networks learn sentiment representations from the holistic image, despite the fact that different image regions can have different influence on the evoked sentiment. In this paper, we introduce a weakly supervised coupled convolutional network (WSCNet). Our method is dedicated to automatically selecting relevant soft proposals from weak annotations (e.g., global image labels), thereby significantly reducing the annotation burden, and encompasses the following contributions. First, WSCNet detects a sentiment-specific soft map by training a fully convolutional network with the cross spatial pooling strategy in the detection branch. Second, both the holistic and localized information are utilized by coupling the sentiment map with deep features for robust representation in the classification branch. We integrate the sentiment detection and classification branches into a unified deep framework, and optimize the network in an end-to-end way. Through this joint learning strategy, weakly supervised sentiment classification and detection benefit each other. Extensive experiments demonstrate that the proposed WSCNet outperforms the state-of-the-art results on seven benchmark datasets.
Item Type: | Article |
---|---|
Date Type: | Publication |
Status: | Published |
Schools: | Computer Science & Informatics |
Publisher: | Institute of Electrical and Electronics Engineers (IEEE) |
ISSN: | 1520-9210 |
Date of First Compliant Deposit: | 1 October 2019 |
Date of Acceptance: | 18 August 2019 |
Last Modified: | 04 Dec 2024 11:00 |
URI: | https://orca.cardiff.ac.uk/id/eprint/125791 |
Citation Data
Cited 31 times in Scopus. View in Scopus. Powered By Scopus® Data
Actions (repository staff only)
Edit Item |