Automation on Twitter: Measuring the effectiveness of approaches to Bot detection

Beatson, Oliver, Gibson, Rachel, Cunill, Marta Cantijoch and Elliot, Mark 2023. Automation on Twitter: Measuring the effectiveness of approaches to Bot detection. Social Science Computer Review 41 (1) , pp. 181-200. 10.1177/08944393211034991

PDF - Published Version Download (640kB)

Abstract

The effectiveness of approaches to bot detection varies, with real-time detection being almost impossible. As a result, this article argues that the general Twitter using public cannot be expected to judge which accounts are bots with certainty and therefore do not know to what extent they are being manipulated online. In this article, the challenge of detecting bots and fake accounts is demonstrated by constructing two distinct methods to bot detection. The first method takes a fixed criteria-based approach, by building on commonly cited identifiers for bots. The second method takes a more flexible, investigative approach in order to uncover bots involved in coordinated efforts to influence online debates. As well as profiling the specific mechanics of how each one operates, we argue that they can be compared against an evaluative framework that specifies a set of key criteria that bot detection methods should meet in order to perform. Here, we identify four key criteria on which these methods can be evaluated and then examine how they perform in terms of the key criteria of accuracy. The results of these methods are then compared and cross-checked against an existing and widely used bot detection service. The findings show that different bot detection methods can present significantly different results and that only confirmation from Twitter, through suspensions or announcements, can truly allow users to know whether an account is a bot or not. We argue that this development could have a significant effect on the level of trust that social media users have both in the information they receive through social media and also in the political process.

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics Crime and Security Research Institute (CSURI) Social Sciences (Includes Criminology and Education)
Publisher:	SAGE Publications
ISSN:	0894-4393
Date of First Compliant Deposit:	10 August 2021
Date of Acceptance:	7 July 2021
Last Modified:	05 May 2023 14:06
URI:	https://orca.cardiff.ac.uk/id/eprint/143279

Citation Data

Cited 2 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item