Human-Agent Knowledge Fusion: Collaborative sensemaking with explainable and tellable AI

Braines, David 2024. Human-Agent Knowledge Fusion: Collaborative sensemaking with explainable and tellable AI. PhD Thesis, Cardiff University. Item availability restricted.

Preview	PDF - Accepted Post-Print Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (41MB) | Preview
	PDF (Cardiff University Electronic Publication Form) - Supplemental Material Restricted to Repository staff only Download (192kB) | Request a copy

Abstract

Augmenting human cognitive activities with Artificial Intelligence (AI) powered machine agents shows promising potential, with new cloud services released regularly. However, rapidly using these in traditional applications requires technical skills beyond typical users. Developers build or extend applications to harness these services, often delaying availability to these users. Chatbot-style conversational interfaces attempt to address this but favour simple interactions. To support richer solutions, I propose knowledge sharing through co-construction of task-relevant information between humans and machine agents. Specifically, shared knowledge supporting multiple modalities and a range of specificity, from rapidly foraged and fluid information to more formally defined knowledge. Moreover, users should be able to invoke relevant cloud services, quickly establishing a level of trust appropriate to those services. By fusing knowledge through co-construction, we can move beyond simple conversational interactions or bespoke applications common for machine agent integration today, enabling faster and richer collaboration mechanisms. This thesis introduces Human-Agent Knowledge Fusion (HAKF) as a conceptual framework to support co-construction of multi-modal knowledge, and support human-agent teams in task-specific and time-constrained problem-solving activities. Specifically, HAKF highlights the need for explainable AI to establish trust rapidly, and tellable AI for fluid knowledge exchange. An open-source instantiation of HAKF, Cogni-sketch, is defined, enabling experimentation for: (1) human-led information foraging, sensemaking and storytelling for open source intelligence analysis, and (2) information fusion from machine agents and data feeds, alongside human analysts. Results from (1) show that users successfully completed the task, concurrently progressing multiple sensemaking activities. Results from (2), featuring fusion of machine vision and object identification, demonstrate co-construction of knowledge from machine agents for consumption by human users. Through HAKF and Cogni-sketch I show the potential for powerful but flexible solutions, enabling task-relevant problem-solving activities between human and machine agents, ranging from information gathering and organisation through sensemaking and storytelling.

Item Type:	Thesis (PhD)
Date Type:	Completion
Status:	Unpublished
Schools:	Computer Science & Informatics
Subjects:	Q Science > Q Science (General) Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Date of First Compliant Deposit:	18 November 2024
Date of Acceptance:	14 November 2024
Last Modified:	19 Nov 2024 10:57
URI:	https://orca.cardiff.ac.uk/id/eprint/174061

Actions (repository staff only)

Edit Item