Improving speech emotion recognition through hierarchical classification and text integration for enhanced emotional analysis and contextual understanding

Alqurashi, Nawal, Li, Yuhua ORCID: https://orcid.org/0000-0003-2913-4478 and Sidorov, Kirill ORCID: https://orcid.org/0000-0001-7935-4132 2024. Improving speech emotion recognition through hierarchical classification and text integration for enhanced emotional analysis and contextual understanding. Presented at: International Joint Conference on Neural Networks, Yokohama, Japan, 30 June – 5 July 2024. Proceedings of IJCNN. IEEE, pp. 1-8. 10.1109/IJCNN60899.2024.10650087

Preview

PDF - Accepted Post-Print Version Download (884kB) | Preview

Abstract

Speech emotion recognition (SER) systems are designed to classify spoken audio samples into different emotion categories. However, misclassifying emotional samples and predicting them as neutral remains a challenging problem in these systems. One primary contributing factor to this issue is the limitation of speech features to recognize emotions from neutral spoken samples that convey emotional context, as these features do not account for contextual or meaning-based features. To address this issue and improve the recognition performance in SER, we propose a hierarchically structured classification model and integrate text features as a supportive modality to address the misclassification of emotional samples. Text-based features provide valuable contextual information that can aid in identifying emotional content in otherwise neutral speech. This work could be potentially applied in various fields, such as healthcare, education, and entertainment, where recognizing emotions from speech can be crucial for effective communication and decision-making.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Computer Science & Informatics
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Publisher:	IEEE
ISBN:	9798350359329
ISSN:	2161-4407
Date of First Compliant Deposit:	10 June 2024
Date of Acceptance:	15 March 2024
Last Modified:	30 Sep 2024 18:39
URI:	https://orca.cardiff.ac.uk/id/eprint/168675

Actions (repository staff only)

Edit Item