CCDb+: enhanced annotations and multi-modal benchmark for natural dyadic conversations

Deng, Yang, Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and Rosin, Paul ORCID: https://orcid.org/0000-0002-4965-3884 2025. CCDb+: enhanced annotations and multi-modal benchmark for natural dyadic conversations. Presented at: ACM Multimedia, Dublin, Ireland, 27-31 Oct. 2025.

Preview

PDF - Accepted Post-Print Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (796kB) | Preview

Abstract

Backchannel signals play a critical role in social interaction, expressing attentiveness, agreement, and emotion in both human and human-agent conversations. However, few multi-modal databases exist in this area due to the complexity of categorisation and the high cost of precise timing, especially in naturalistic dyadic conversations. To address these challenges, we introduce CCDb+ (Cardiff Conversation Database +), an enhanced version of CCDb, with 25 newly annotated conversations and corrections to 14 previously annotated conversations, along with thorough consistency checks to ensure annotation reliability. Additionally, we propose a multi-modal process for backchannel detection as a baseline, showing that both visual and acoustic cues contribute significantly to understanding backchannel behaviour. Recognising that backchannel signals often intersect with other social cues, we introduce several detection sub-tasks — such as smile, nodding, and agreement — with baseline results for each. Finally, we demonstrate multi-modal paradigms for nuanced signals like nodding and thinking. The database and associated annotations are publicly available at https://huggingface.co/datasets/CardiffVisualComputing/CCDb.

Item Type:	Conference or Workshop Item (Paper)
Status:	Unpublished
Schools:	Schools > Computer Science & Informatics
Funders:	EPSRC
Related URLs:	Project
Date of First Compliant Deposit:	24 September 2025
Date of Acceptance:	5 July 2025
Last Modified:	25 Sep 2025 13:15
URI:	https://orca.cardiff.ac.uk/id/eprint/181328

Actions (repository staff only)

Edit Item