Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Understanding the source of semantic regularities in word embeddings

Chiang, Hsiao-Yu, Camacho-Collados, Jose and Pardos, Zachary 2020. Understanding the source of semantic regularities in word embeddings. Presented at: SIGNLL Conference Computational Natural Language Learning (CoNLL 2020), Virtual, 19-20 November 2020. Proceedings of the 24th Conference on Computational Natural Language Learning. Association for Computational Linguistics, pp. 119-131.

[img] PDF - Published Version
Available under License Creative Commons Attribution.

Download (422kB)

Abstract

Semantic relations are core to how humans understand and express concepts in the real world using language. Recently, there has been a thread of research aimed at modeling these relations by learning vector representations from text corpora. Most of these approaches focus strictly on leveraging the co-occurrences of relationship word pairs within sentences. In this paper, we investigate the hypothesis that examples of a lexical relation in a corpus are fundamental to a neural word embedding’s ability to complete analogies involving the relation. Our experiments, in which we remove all known examples of a relation from training corpora, show only marginal degradation in analogy completion performance involving the removed relation. This finding enhances our understanding of neural word embeddings, showing that co-occurrence information of a particular semantic relation is the not the main source of their structural regularity.

Item Type: Conference or Workshop Item (Paper)
Date Type: Publication
Status: Published
Schools: Computer Science & Informatics
Additional Information: Creative Commons Attribution 4.0 International License
Publisher: Association for Computational Linguistics
Date of First Compliant Deposit: 15 December 2020
Date of Acceptance: 18 September 2020
Last Modified: 15 Dec 2020 16:30
URI: http://orca.cardiff.ac.uk/id/eprint/137047

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics