Leaving no stone unturned: flexible retrieval of idiomatic expressions from a large text corpus

Hughes, Callum, Filimonov, Maxim, Wray, Alison ORCID: https://orcid.org/0000-0002-2144-4458 and Spasic, Irena ORCID: https://orcid.org/0000-0002-8132-3885 2021. Leaving no stone unturned: flexible retrieval of idiomatic expressions from a large text corpus. Machine Learning and Knowledge Extraction 3 (1) , pp. 263-283. 10.3390/make3010013

PDF - Published Version Available under License Creative Commons Attribution. Download (2MB)

Abstract

Idioms are multi-word expressions whose meaning cannot always be deduced from the literal meaning of constituent words. A key feature of idioms that is central to this paper is their peculiar mixture of fixedness and variability, which poses challenges for their retrieval from large corpora using traditional search approaches. These challenges hinder insights into idiom usage affecting users who are conducting linguistic research as well as those involved in language ed-ucation. To facilitate access to idioms examples taken from real-world contexts, we introduce an information retrieval system designed specifically for idioms. Given a search query that represents an idiom, typically in its canonical form, the system expands it automatically to account for the most common types of idiom variation including inflection, open slots, adjectival or adverbial modification, and passivisation. As a by-product of query expansion, other types of idiom varia-tion captured include derivation, compounding, negation, distribution across multiple clauses as well as other unforeseen types of variation. The system was implemented on top of Elasticsearch, an open-source, distributed, scalable, real-time search engine. Flexible retrieval of idioms is supported by a combination of linguistic pre-processing of the search queries, their translation into a set of query clauses written in a query language called Query DSL, and analysis, an indexing process that involves tokenisation and normalisation. Our system outperformed the phrase search in terms of recall and outperformed the keyword search in terms of precision. Out of the three, our approach was found to provide the best balance between precision and recall. By providing a fast and easy way of finding idioms in large corpora, our approach can facilitate further developments in fields such as linguistics, language education and natural language processing. Keywords: information retrieval; natural language processing; corpus linguistics; multi-word expressions; idioms

Item Type:	Article
Date Type:	Publication
Status:	Published
Schools:	English, Communication and Philosophy Computer Science & Informatics Data Innovation Research Institute (DIURI)
Subjects:	P Language and Literature > P Philology. Linguistics Q Science > QA Mathematics > QA76 Computer software
Publisher:	MDPI
ISSN:	2504-4990
Related URLs:	https://datainnovation.cardiff.ac.uk/idi...
Date of First Compliant Deposit:	3 March 2021
Date of Acceptance:	25 February 2021
Last Modified:	05 May 2023 18:56
URI:	https://orca.cardiff.ac.uk/id/eprint/138986

Actions (repository staff only)

Edit Item