Waqar, Eeshan, Davies, Jonathan  ORCID: https://orcid.org/0000-0002-6920-4031, Knight, Dawn ORCID: https://orcid.org/0000-0002-4745-6502 and Alva Manchego, Fernando ORCID: https://orcid.org/0000-0001-6218-8377
2026.
CEFR-Cymraeg: A dataset and baseline models for language proficiency assessment in Welsh.
Presented at: The Fifteenth Language Resources and Evaluation Conference (LREC 2026),
Palma de Mallorca, Spain,
13-15 May 2026.
Proceedings of Learning Resources Evaluation Conference 2026 (LREC).
ELRA,
pp. 3496-3505.
10.63317/2dvuy5ucr9g2
|
Preview |
PDF
- Accepted Post-Print Version
Available under License Creative Commons Attribution. Download (384kB) | Preview |
Abstract
We introduce CEFR-Cymraeg, the first dataset annotated with Common European Framework of Reference (CEFR) levels for Welsh. The dataset is built from learning materials for adult learners, carefully extracted from widely used coursebooks and verified by teachers of Welsh as a second language. It spans levels A1 to B2 and includes multiple units of analysis: sentences, dialogues, paragraphs, and documents. In total, 2,658 entries are provided with gold-standard CEFR annotations, making CEFR-Cymraeg a valuable resource for research on language learning and low-resourced Celtic languages. To illustrate its potential applications, we define language proficiency assessment as a multi-class classification task and fine-tune multilingual pre-trained language models. Given the limited size of the dataset, we also experiment with data augmentation. Results show that these models successfully capture proficiency distinctions and generalise well to Welsh, with the best-performing model reaching a weighted F1-score of 0.83. Qualitative analysis confirmed that most apparent errors reflected valid pedagogical variation rather than model inconsistencies. CEFR-Cymraeg establishes a benchmark resource for Welsh and opens new opportunities for educational NLP, corpus linguistics, and multilingual proficiency research.
| Item Type: | Conference or Workshop Item - published (Paper) |
|---|---|
| Date Type: | Publication |
| Status: | Published |
| Schools: | Schools > English, Communication and Philosophy Schools > Computer Science & Informatics |
| Additional Information: | RRS policy applied |
| Publisher: | ELRA |
| ISBN: | 9782493814494 |
| ISSN: | 2522-2686 |
| Date of First Compliant Deposit: | 5 May 2026 |
| Last Modified: | 05 Aug 2026 02:00 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/184885 |
Actions (repository staff only)
 |
Edit Item |
Dimensions
Dimensions