| Maddela, Mounica and Alva Manchego, Fernando 2025. Adapting sentence-level automatic metrics for document-level simplification evaluation. Presented at: NAACL 2025, Albuquerque, New Mexico, 29 April - 04 May 2025. Published in: Chiruzzo, Luis, Ritter, Alan and Wang, Lu eds. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies. , vol.1 Association for Computational Linguistics, 6444–6459. 10.18653/v1/2025.naacl-long.327 |
Abstract
Text simplification aims to enhance the clarity and comprehensibility of a complex text while preserving its original meaning. Previous research on the automatic evaluation of text simplification has primarily focused on sentence simplification, with commonly used metrics such as SARI and advanced metrics such as LENS being trained and evaluated at the sentence level. However, these metrics often underperform on longer texts. In our study, we propose a novel approach to adapt existing sentence-level metrics for paragraph- or document-level simplification. We benchmark our approach against a wide variety of existing reference-based and reference-less metrics across multiple domains. Empirical results demonstrate that our approach outperforms traditional sentence-level metrics in terms of correlation with human judgment. Furthermore, we evaluate the sensitivity and robustness of various metrics to different types of errors produced by existing text simplification systems.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Date Type: | Publication |
| Status: | Published |
| Schools: | Schools > Computer Science & Informatics |
| Publisher: | Association for Computational Linguistics |
| ISBN: | 979-8-89176-189-6 |
| Last Modified: | 12 Aug 2025 11:30 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/180321 |
Actions (repository staff only)
 |
Edit Item |
Altmetric
Altmetric