Understanding the source of semantic regularities in word embeddings

Chiang, Hsiao-Yu, Camacho-Collados, Jose ORCID: https://orcid.org/0000-0003-1618-7239 and Pardos, Zachary 2020. Understanding the source of semantic regularities in word embeddings. Presented at: SIGNLL Conference Computational Natural Language Learning (CoNLL 2020), Virtual, 19-20 November 2020. Proceedings of the 24th Conference on Computational Natural Language Learning. Association for Computational Linguistics, pp. 119-131.

PDF - Published Version Available under License Creative Commons Attribution. Download (422kB)

Abstract

Semantic relations are core to how humans understand and express concepts in the real world using language. Recently, there has been a thread of research aimed at modeling these relations by learning vector representations from text corpora. Most of these approaches focus strictly on leveraging the co-occurrences of relationship word pairs within sentences. In this paper, we investigate the hypothesis that examples of a lexical relation in a corpus are fundamental to a neural word embedding’s ability to complete analogies involving the relation. Our experiments, in which we remove all known examples of a relation from training corpora, show only marginal degradation in analogy completion performance involving the removed relation. This finding enhances our understanding of neural word embeddings, showing that co-occurrence information of a particular semantic relation is the not the main source of their structural regularity.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Additional Information:	Creative Commons Attribution 4.0 International License
Publisher:	Association for Computational Linguistics
Date of First Compliant Deposit:	15 December 2020
Date of Acceptance:	18 September 2020
Last Modified:	09 Nov 2022 09:47
URI:	https://orca.cardiff.ac.uk/id/eprint/137047

Citation Data

Cited 6 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item