Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

The conversational action test: detecting the artificial sociality of artificial intelligence

Albert, Saul, Housley, William ORCID: https://orcid.org/0000-0003-1568-9093, Sikveland, Rein Ove and Stokoe, Elizabeth 2025. The conversational action test: detecting the artificial sociality of artificial intelligence. New Media and Society 27 (10) , pp. 5592-5621. 10.1177/14614448251338277

[thumbnail of 10.1177_14614448251338277.pdf] PDF - Published Version
Available under License Creative Commons Attribution.

Download (727kB)
License URL: https://creativecommons.org/licenses/by/4.0/
License Start date: 4 October 2025

Abstract

Drawing on the “Voigt-Kampff Empathy Test”—a science fiction version of Turing’s famous thought experiment—we propose the Conversational Action Test (CAT): a new approach to evaluating conversational artificial intelligence (AI) voice agents. We compare social actions in a range of telephone service encounters where one party is an artificial conversational agent to a range of similar human-human calls. The CAT demonstrates a novel paradigm that addresses long-standing theoretical and methodological problems for ostensible “tests” of conversational AI by (a) revealing the conceptual confusion of attempting to “detect” an AI in routine service interactions and (b) focusing, instead, on the situated interactional practices through which an AI “passes” for human. We discuss the implications of the CAT for the design and evaluation of conversational AI, and for the notion of “humanness” as a goal or benchmark for such systems. Data include publicly available human/AI service calls and comparable human-human calls in British and American English.

Item Type: Article
Date Type: Publication
Status: Published
Schools: ?? AHS1 ??
Additional Information: License information from Publisher: LICENSE 1: URL: https://creativecommons.org/licenses/by/4.0/, Start Date: 2025-10-04
Publisher: SAGE Publications
ISSN: 1461-4448
Date of First Compliant Deposit: 10 October 2025
Date of Acceptance: 28 October 2024
Last Modified: 10 Oct 2025 14:52
URI: https://orca.cardiff.ac.uk/id/eprint/181596

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics