Modeling context words as regions: an ordinal regression approach to word embedding

Jameel, Shoaib and Schockaert, Steven ORCID: https://orcid.org/0000-0002-9256-2881 2017. Modeling context words as regions: an ordinal regression approach to word embedding. Presented at: SIGNLL Conference on Computational Natural Language Learning, Vancouver, Canada, 3-4 August 2017. Published in: Levy, Roger and Specia, Lucia eds. Proceedings of the 21st Conference on Computational Natural Language Learning (CoNLL 2017). Association of Computing Language, pp. 123-133. 10.18653/v1/K17-1014

Preview

PDF - Accepted Post-Print Version Download (309kB) | Preview

Abstract

Vector representations of word meaning have found many applications in the field of natural language processing. Word vectors intuitively represent the average context in which a given word tends to occur, but they cannot explicitly model the diversity of these contexts. Although region representations of word meaning offer a natural alternative to word vectors, only few methods have been proposed that can effectively learn word regions. In this paper, we propose a new word embedding model which is based on SVM regression. We show that the underlying ranking interpretation of word contexts is sufficient to match, and sometimes outperform, the performance of popular methods such as Skip-gram. Furthermore, we show that by using a quadratic kernel, we can effectively learn word regions, which outperform existing unsupervised models for the task of hypernym detection.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Computer Science & Informatics
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Publisher:	Association of Computing Language
Funders:	ERC
Date of First Compliant Deposit:	12 July 2017
Date of Acceptance:	30 May 2017
Last Modified:	02 Nov 2022 11:32
URI:	https://orca.cardiff.ac.uk/id/eprint/102352

Citation Data

Cited 4 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item