SWAG-V: explanations for video using superpixels weighted by average gradients

Hartley, Thomas, Sidorov, Kirill, Willis, Christopher and Marshall, David ORCID: https://orcid.org/0000-0003-2789-1395 2022. SWAG-V: explanations for video using superpixels weighted by average gradients. Presented at: 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, 3-8 January 2022. 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). IEEE, pp. 1576-1585. 10.1109/WACV51458.2022.00164

Preview

PDF - Accepted Post-Print Version Download (2MB) | Preview

Abstract

CNN architectures that take videos as an input are often overlooked when it comes to the development of explanation techniques. This is despite their use in often critical domains such as surveillance and healthcare. Explanation techniques developed for these networks must take into account the additional temporal domain if they are to be successful. In this paper we introduce SWAG-V, an extension of SWAG for use with networks that take video as an input. In addition we show how these explanations can be created in such a way that they are balanced between fine and coarse explanations. By creating superpixels that incorporate the frames of the input video we are able to create explanations that better locate regions of the input that are important to the networks prediction. We compare SWAG-V against a number of similar techniques using metrics such as insertion and deletion, and weak localisation. We compute these using Kinetics-400 with both the C3D and R(2+1)D network architectures and find that SWAG-V is able to outperform multiple techniques.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Advanced Research Computing @ Cardiff (ARCCA) Computer Science & Informatics
Publisher:	IEEE
ISBN:	9781665409155
Funders:	EPSRC
Date of First Compliant Deposit:	14 March 2022
Date of Acceptance:	5 October 2021
Last Modified:	14 Jun 2024 15:19
URI:	https://orca.cardiff.ac.uk/id/eprint/148351

Actions (repository staff only)

Edit Item