Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Meemi: a simple method for post-processing and integrating cross-lingual word embeddings

Doval, Yerai, Camacho Collados, Jose ORCID: https://orcid.org/0000-0003-1618-7239, Espinosa-Anke, Luis ORCID: https://orcid.org/0000-0001-6830-9176 and Schockaert, Steven ORCID: https://orcid.org/0000-0002-9256-2881 2021. Meemi: a simple method for post-processing and integrating cross-lingual word embeddings. Natural Language Engineering 10.1017/S1351324921000280

[thumbnail of NLE_Multilingual_embeddings.pdf] PDF - Accepted Post-Print Version
Download (401kB)

Abstract

Word embeddings have become a standard resource in the toolset of any Natural Language Processing practitioner. While monolingual word embeddings encode information about words in the context of a particular language, cross-lingual embeddings define a multilingual space where word embeddings from two or more languages are integrated together. Current state-of-the-art approaches learn these embeddings by aligning two disjoint monolingual vector spaces through an orthogonal transformation which preserves the structure of the monolingual counterparts. In this work, we propose to apply an additional transformation after this initial alignment step, which aims to bring the vector representations of a given word and its translations closer to their average. Since this additional transformation is non-orthogonal, it also affects the structure of the monolingual spaces. We show that our approach both improves the integration of the monolingual spaces as well as the quality of the monolingual spaces themselves. Furthermore, because our transformation can be applied to an arbitrary number of languages, we are able to effectively obtain a truly multilingual space. The resulting (monolingual and multilingual) spaces show consistent gains over the current state-of-the-art in standard intrinsic tasks, namely dictionary induction and word similarity, as well as in extrinsic tasks such as cross-lingual hypernym discovery and cross-lingual natural language inference.

Item Type: Article
Date Type: Published Online
Status: In Press
Schools: Computer Science & Informatics
Publisher: Cambridge University Press
ISSN: 1351-3249
Date of First Compliant Deposit: 1 September 2021
Date of Acceptance: 28 June 2021
Last Modified: 09 Nov 2022 11:30
URI: https://orca.cardiff.ac.uk/id/eprint/143615

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics