On the curious case of l2 norm of sense embeddings

Zhou, Yi ORCID: https://orcid.org/0000-0001-7009-8515 and Bollegala, Danushka 2022. On the curious case of l2 norm of sense embeddings. Presented at: The 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), Abu Dhabi, United Arab Emirates, 7 - 11 December 2022. Published in: Goldberg, Yoav, Kozareva, Zornitsa and Zhang, Yue eds. Findings of the Association for Computation Linguistics: EMNLP 2022. Association for Computational Linguistics, 10.18653/v1/2022.findings-emnlp.190

Full text not available from this repository.

Abstract

We show that the l2 norm of a static sense embedding encodes information related to the frequency of that sense in the training corpus used to learn the sense embeddings. This finding can be seen as an extension of a previously known relationship for word embeddings to sense embeddings. Our experimental results show that in spite of its simplicity, the l2 norm of sense embeddings is a surprisingly effective feature for several word sense related tasks such as (a) most frequent sense prediction, (b) word-in-context (WiC), and (c) word sense disambiguation (WSD). In particular, by simply including the l2 norm of a sense embedding as a feature in a classifier, we show that we can improve WiC and WSD methods that use static sense embeddings.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Unpublished
Schools:	Computer Science & Informatics
Publisher:	Association for Computational Linguistics
Last Modified:	01 Aug 2024 14:45
URI:	https://orca.cardiff.ac.uk/id/eprint/170398

Actions (repository staff only)

Edit Item