Pun unintended: LLMs and the illusion of humor understanding

Zangari, Alessandro, Marcuzzo, Matteo, Albarelli, Andrea, Pilehvar, Mohammad Taher and Camacho-Collados, Jose ORCID: https://orcid.org/0000-0003-1618-7239 2025. Pun unintended: LLMs and the illusion of humor understanding. Presented at: 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China, 4-9 November 2025. Published in: Christodoulopoulos, Christos, Chakraborty, Tanmoy, Rose, Carolyn and Peng, Violet eds. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 27924-27959. 10.18653/v1/2025.emnlp-main.1419

PDF - Published Version Available under License Creative Commons Attribution. Download (5MB)

Abstract

Puns are a form of humorous wordplay that exploits polysemy and phonetic similarity. While LLMs have shown promise in detecting puns, we show in this paper that their understanding often remains shallow, lacking the nuanced grasp typical of human interpretation. By systematically analyzing and reformulating existing pun benchmarks, we demonstrate how subtle changes in puns are sufficient to mislead LLMs. Our contributions include comprehensive and nuanced pun detection benchmarks, human evaluation across recent LLMs, and an analysis of the robustness challenges these models face in processing puns.

Item Type:	Conference or Workshop Item - published (Paper)
Date Type:	Publication
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	Association for Computational Linguistics
ISBN:	979-8-89176-332-6
Date of First Compliant Deposit:	16 June 2026
Last Modified:	02 Aug 2026 01:20
URI:	https://orca.cardiff.ac.uk/id/eprint/187583

Actions (repository staff only)

Edit Item