Language models for text classification: is in-context learning enough?

Edwards, Aleksandra and Camacho-Collados, Jose ORCID: https://orcid.org/0000-0003-1618-7239 2025. Language models for text classification: is in-context learning enough? Presented at: Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), Torino, Italy, 20 - 25 May, 2024. Published in: Calzolari, Nicoletta, Kan, Min-Yen, Hoste, Veronique, Lenci, Alessandro, Sakti, Sakriani and Xue, Nianwen eds. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). Proceedings of the 29th International Conference on Computational Linguistics. Torino, Italy: ELRA and ICCL, 10058–10072.

Preview

PDF - Published Version Available under License Creative Commons Attribution Non-commercial. Download (959kB) | Preview

Abstract

Recent foundational language models have shown state-of-the-art performance in many NLP tasks in zero- and few-shot settings. An advantage of these models over more standard approaches based on fine-tuning is the ability to understand instructions written in natural language (prompts), which helps them generalise better to different tasks and domains without the need for specific training data. This makes them suitable for addressing text classification problems for domains with limited amounts of annotated instances. However, existing research is limited in scale and lacks understanding of how text generation models combined with prompting techniques compare to more established methods for text classification such as fine-tuning masked language models. In this paper, we address this research gap by performing a large-scale evaluation study for 16 text classification datasets covering binary, multiclass, and multilabel problems. In particular, we compare zero- and few-shot approaches of large language models to fine-tuning smaller language models. We also analyse the results by prompt, classification type, domain, and number of labels. In general, the results show how fine-tuning smaller and more efficient language models can still outperform few-shot approaches of larger language models, which have room for improvement when it comes to text classification.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Publication
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	ELRA and ICCL
ISBN:	9782493814104
ISSN:	2951-2093
Related URLs:	Organisation
Date of First Compliant Deposit:	24 September 2025
Date of Acceptance:	1 May 2024
Last Modified:	25 Sep 2025 15:00
URI:	https://orca.cardiff.ac.uk/id/eprint/181324

Actions (repository staff only)

Edit Item