Krykoniuk, Kateryna, Hopkin-King, Cleo and Roberts, Seán G.  ORCID: https://orcid.org/0000-0001-5990-9161
2026.
Using LLMs to identify discourse derailment as a potential cue for disinformation in social media posts.
Humanities & Social Sciences Communications
10.1057/s41599-026-07900-x
|
Preview |
PDF
- Accepted Post-Print Version
Available under License Creative Commons Attribution. Download (1MB) | Preview |
Abstract
Identifying disinformation in online social media is important for addressing several risks to society. However, methods for automatically detecting possible disinformation still mostly rely on low-level lexical cues, which are increasingly outdated in a world with access to generative language models. We present a new approach to the automatic detection of disinformation based on measuring discourse ‘derailment’ – messages that try to force the topic of discourse away from one topic and onto another. While this may include both malicious and benign derailment, it could serve as an important signal for early-warning systems. In this study, we implement a system for automatic detection of discourse derailment which uses a Large Language Model (LLM) to generate expected replies and compares them to the real replies. We test this system on a set of human-annotated data to show that the system outperforms various baselines and approaches the agreement between human annotators. This suggests that LLMs can be sensitive to discourse-level information. However, we also identify evidence of several limitations, including that the automatic system relies on different cues compared to human annotators, which leads to some amount of bias. Nevertheless, our project represents a considerable step towards understanding how to use LLMs to analyse discourse and a new angle on tackling disinformation.
| Item Type: | Article |
|---|---|
| Date Type: | Published Online |
| Status: | In Press |
| Schools: | Schools > English, Communication and Philosophy |
| Publisher: | Springer Nature [academic journals on nature.com] |
| ISSN: | 2662-9992 |
| Date of First Compliant Deposit: | 16 June 2026 |
| Date of Acceptance: | 31 May 2026 |
| Last Modified: | 31 Jul 2026 13:30 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/187563 |
Actions (repository staff only)
 |
Edit Item |
Dimensions
Dimensions