Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

Cardiff University at SemEval-2020 Task 6: fine-tuning BERT for domain-specific definition classification

Jeawak, Shelan S., Espinosa-Anke, Luis ORCID: https://orcid.org/0000-0001-6830-9176 and Schockaert, Steven ORCID: https://orcid.org/0000-0002-9256-2881 2020. Cardiff University at SemEval-2020 Task 6: fine-tuning BERT for domain-specific definition classification. Presented at: International Workshop on Semantic Evaluation (SemEval 2020), Barcelona, Spain, 12-13 December 2020. Published in: Herbelot, A., Zhu, X., Palmer, A., Schneider, N., May, J. and Shutova, E. eds. Proceedings of the Fourteenth International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics. International Committee for Computational Linguistics, pp. 361-366.

[thumbnail of SemEval_2020_definition_extraction.pdf]
Preview
PDF - Accepted Post-Print Version
Download (121kB) | Preview

Abstract

We describe the system submitted to SemEval-2020 Task 6, Subtask 1. The aim of this subtask is to predict whether a given sentence contains a definition or not. Unsurprisingly, we found that strong results can be achieved by fine-tuning a pre-trained BERT language model. In this paper,we analyze the performance of this strategy. Among others, we show that results can be improved by using a two-step fine-tuning process, in which the BERT model is first fine-tuned on the full training set, and then further specialized towards a target domain.

Item Type: Conference or Workshop Item (Paper)
Date Type: Publication
Status: Published
Schools: Professional Services > Advanced Research Computing @ Cardiff (ARCCA)
Schools > Computer Science & Informatics
Publisher: International Committee for Computational Linguistics
ISBN: 9781952148316
Date of First Compliant Deposit: 17 August 2020
Date of Acceptance: 26 June 2020
Last Modified: 04 Sep 2025 10:45
URI: https://orca.cardiff.ac.uk/id/eprint/134231

Citation Data

Cited 5 times in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item Edit Item

Downloads

Downloads per month over past year

View more statistics