Weakly supervised coupled networks for visual sentiment analysis

Jufeng, Yang, Dongyu, She, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680, Rosin, Paul ORCID: https://orcid.org/0000-0002-4965-3884 and Ming-Hsuan, Yang 2018. Weakly supervised coupled networks for visual sentiment analysis. Presented at: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Lake Salt City, UT, USA, 18-22 Jun 2018. IEEE Computer Society Conference on Computer Vision and Pattern Recognition workshops.

Preview

PDF - Accepted Post-Print Version Download (2MB) | Preview

Abstract

Automatic assessment of sentiment from visual content has gained considerable attention with the increasing tendency of expressing opinions on-line. In this paper, we solve the problem of visual sentiment analysis using the high-level abstraction in the recognition process. Existing methods based on convolutional neural networks learn sentiment representations from the holistic image appearance. However, different image regions can have a different influence on the intended expression. This paper presents a weakly supervised coupled convolutional network with two branches to leverage the localized information. The first branch detects a sentiment specific soft map by training a fully convolutional network with the cross spatial pooling strategy, which only requires image-level labels, thereby significantly reducing the annotation burden. The second branch utilizes both the holistic and localized information by coupling the sentiment map with deep features for robust classification. We integrate the sentiment detection and classification branches into a unified deep framework and optimize the network in an end-to-end manner. Extensive experiments on six benchmark datasets demonstrate that the proposed method performs favorably against the state-ofthe- art methods for visual sentiment analysis.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Computer Science & Informatics
ISBN:	9781538664216
ISSN:	2160-7508
Date of First Compliant Deposit:	29 March 2018
Date of Acceptance:	19 February 2018
Last Modified:	23 Oct 2022 13:19
URI:	https://orca.cardiff.ac.uk/id/eprint/110340

Actions (repository staff only)

Edit Item