Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Leaving no stone unturned: flexible retrieval of idiomatic expressions from a large text corpus

Hughes, Callum, Filimonov, Maxim, Wray, Alison ORCID: and Spasic, Irena ORCID: 2021. Leaving no stone unturned: flexible retrieval of idiomatic expressions from a large text corpus. Machine Learning and Knowledge Extraction 3 (1) , pp. 263-283. 10.3390/make3010013

[thumbnail of make-03-00013.pdf] PDF - Published Version
Available under License Creative Commons Attribution.

Download (2MB)


Idioms are multi-word expressions whose meaning cannot always be deduced from the literal meaning of constituent words. A key feature of idioms that is central to this paper is their peculiar mixture of fixedness and variability, which poses challenges for their retrieval from large corpora using traditional search approaches. These challenges hinder insights into idiom usage affecting users who are conducting linguistic research as well as those involved in language ed-ucation. To facilitate access to idioms examples taken from real-world contexts, we introduce an information retrieval system designed specifically for idioms. Given a search query that represents an idiom, typically in its canonical form, the system expands it automatically to account for the most common types of idiom variation including inflection, open slots, adjectival or adverbial modification, and passivisation. As a by-product of query expansion, other types of idiom varia-tion captured include derivation, compounding, negation, distribution across multiple clauses as well as other unforeseen types of variation. The system was implemented on top of Elasticsearch, an open-source, distributed, scalable, real-time search engine. Flexible retrieval of idioms is supported by a combination of linguistic pre-processing of the search queries, their translation into a set of query clauses written in a query language called Query DSL, and analysis, an indexing process that involves tokenisation and normalisation. Our system outperformed the phrase search in terms of recall and outperformed the keyword search in terms of precision. Out of the three, our approach was found to provide the best balance between precision and recall. By providing a fast and easy way of finding idioms in large corpora, our approach can facilitate further developments in fields such as linguistics, language education and natural language processing. Keywords: information retrieval; natural language processing; corpus linguistics; multi-word expressions; idioms

Item Type: Article
Date Type: Publication
Status: Published
Schools: English, Communication and Philosophy
Computer Science & Informatics
Data Innovation Research Institute (DIURI)
Subjects: P Language and Literature > P Philology. Linguistics
Q Science > QA Mathematics > QA76 Computer software
Publisher: MDPI
ISSN: 2504-4990
Related URLs:
Date of First Compliant Deposit: 3 March 2021
Date of Acceptance: 25 February 2021
Last Modified: 05 May 2023 18:56

Actions (repository staff only)

Edit Item Edit Item


Downloads per month over past year

View more statistics