Furby, Jack, Cunnington, Dan, Braines, Dave and Preece, Alun  ORCID: https://orcid.org/0000-0003-0349-9057
2025.
The impact of concept explanations and interventions on human-machine collaboration.
Presented at: The 3rd World Conference on Explainable Artificial Intelligence,
Istanbul, Turkey,
9 -11 July 2025.
Proceedings of the 3rd World Conference on Explainable Artificial Intelligence.
Communications in Computer and Information Science
, vol.2576
Springer Nature,
pp. 255-280.
10.1007/978-3-032-08317-3_12
|
Preview |
PDF
- Published Version
Available under License Creative Commons Attribution. Download (3MB) | Preview |
Abstract
Deep Neural Networks (DNNs) are often considered black boxes due to their opaque decision-making processes. To reduce their opacity Concept Models (CMs), such as Concept Bottleneck Models (CBMs), were introduced to predict human-defined concepts as an intermediate step before predicting task labels. This enhances the interpretability of DNNs. In a human-machine setting greater interpretability enables humans to improve their understanding and build trust in a DNN. In the introduction of CBMs, the models demonstrated increased task accuracy as incorrect concept predictions were replaced with their ground truth values, known as intervening on the concept predictions. In a collaborative setting, if the model task accuracy improves from interventions, trust in a model and the human-machine task accuracy may increase. However, the result showing an increase in model task accuracy was produced without human evaluation and thus it remains unknown if the findings can be applied in a collaborative setting. In this paper, we ran the first human studies using CBMs to evaluate their human interaction in collaborative task settings. Our findings show that CBMs improve interpretability compared to standard DNNs, leading to increased human-machine alignment. However, this increased alignment did not translate to a significant increase in task accuracy. Understanding the model’s decision-making process required multiple interactions, and misalignment between the model’s and human decision-making processes could undermine interpretability and model effectiveness.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Date Type: | Publication |
| Status: | Published |
| Schools: | Schools > Computer Science & Informatics |
| Publisher: | Springer Nature |
| ISBN: | 9783032083166 |
| Date of First Compliant Deposit: | 6 April 2025 |
| Date of Acceptance: | 24 March 2025 |
| Last Modified: | 30 Oct 2025 14:45 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/177429 |
Actions (repository staff only)
 |
Edit Item |
Altmetric
Altmetric