Jeawak, Shelan, Espinosa-Anke, Luis ORCID: https://orcid.org/0000-0001-6830-9176 and Schockaert, Steven ORCID: https://orcid.org/0000-0002-9256-2881 2020. Cardiff University at SemEval-2020 Task 6: fine-tuning BERT for domain-specific definition classification. Presented at: International Workshop on Semantic Evaluation (SemEval 2020), Barcelona, Spain, 12-13 December 2020. |
Preview |
PDF
- Accepted Post-Print Version
Download (121kB) | Preview |
Abstract
We describe the system submitted to SemEval-2020 Task 6, Subtask 1. The aim of this subtask is to predict whether a given sentence contains a definition or not. Unsurprisingly, we found that strong results can be achieved by fine-tuning a pre-trained BERT language model. In this paper,we analyze the performance of this strategy. Among others, we show that results can be improved by using a two-step fine-tuning process, in which the BERT model is first fine-tuned on the full training set, and then further specialized towards a target domain.
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Status: | In Press |
Schools: | Advanced Research Computing @ Cardiff (ARCCA) Computer Science & Informatics |
Date of First Compliant Deposit: | 17 August 2020 |
Date of Acceptance: | 26 June 2020 |
Last Modified: | 14 Jun 2024 15:29 |
URI: | https://orca.cardiff.ac.uk/id/eprint/134231 |
Citation Data
Cited 2 times in Scopus. View in Scopus. Powered By Scopus® Data
Actions (repository staff only)
Edit Item |