Mtumbuka, Frank and Schockaert, Steven  ORCID: https://orcid.org/0000-0002-9256-2881
2024.
Entity or relation embeddings? An analysis of encoding strategies for relation extraction.
Presented at: Findings of EMNLP,
Miami, Florida,
12-16 November 2024.
Published in: Al-Onaizan, Yaser, Bansal, Mohit and Chen, Yun-Nung eds.
Findings of the Association for Computational Linguistics: EMNLP 2024.
Association for Computational Linguistics,
pp. 6003-6022.
10.18653/v1/2024.findings-emnlp.348
|
Preview |
PDF
- Accepted Post-Print Version
Download (1MB) | Preview |
Abstract
Existing approaches to relation extraction obtain relation embeddings by concatenating embeddings of the head and tail entities. Despite the popularity of this approach, we find that such representations mostly capture the types of the entities involved, leading to false positives and confusion between relations that involve entities of the same type. Another possibility is to use a prompt with a [MASK] token to directly learn relation embeddings, but this approach tends to perform poorly. We show that this underperformance comes from the fact that information about entity types is insufficiently captured by the [MASK] embeddings. We therefore propose a simple model, which combines such [MASK] embeddings with entity embeddings. Despite its simplicity, our model consistently outperforms the state-of-the-art across several benchmarks, even when the entity embeddings are obtained from a pre-trained entity typing model. We also experiment with a self-supervised pre-training strategy which further improves the results.
| Item Type: | Conference or Workshop Item - published (Paper) |
|---|---|
| Date Type: | Published Online |
| Status: | Published |
| Schools: | Professional Services > Advanced Research Computing @ Cardiff (ARCCA) Schools > Computer Science & Informatics |
| Publisher: | Association for Computational Linguistics |
| Date of First Compliant Deposit: | 25 October 2024 |
| Date of Acceptance: | 20 September 2024 |
| Last Modified: | 02 Apr 2026 13:59 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/173142 |
Actions (repository staff only)
 |
Edit Item |
Altmetric
Altmetric