CASE – Condition-aware sentence embeddings for conditional semantic textual similarity measurement

Zhang, Gaifan, Zhou, Yi ORCID: https://orcid.org/0000-0001-7009-8515 and Bollegala, Danushka 2026. CASE – Condition-aware sentence embeddings for conditional semantic textual similarity measurement. Presented at: 19th Conference of the European Chapter of the Association for Computational Linguistics, Rabat, Morocco, 24-29 March 2026. Published in: Demberg, Vera, Inui, Kentaro and Marquez, Lluis eds. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics. , vol.1 Association for Computational Linguistics, pp. 4954-4968. 10.18653/v1/2026.eacl-long.231

PDF - Published Version Available under License Creative Commons Attribution. Download (1MB)

Abstract

The meaning conveyed by a sentence often depends on the context in which it appears. Despite the progress of sentence embedding methods, it remains unclear as how to best modify a sentence embedding conditioned on its context. To address this problem, we propose Condition-Aware Sentence Embeddings (CASE), an efficient and accurate method to create an embedding for a sentence under a given condition. First, CASE creates an embedding for the condition using an Large Language Model (LLM) encoder, where the sentence influences the attention scores computed for the tokens in the condition during pooling. Next, a supervised method is learnt to align the LLM-based text embeddings with the Conditional Semantic Textual Similarity (C-STS) task. We find that subtracting the condition embedding will consistently improve the C-STS performance of LLM-based text embeddings and improve the isotropy of the embedding space. Moreover, our supervised projection method significantly improves the performance of LLM-based embeddings despite requiring a small number of embedding dimensions.

Item Type:	Conference or Workshop Item - published (Paper)
Date Type:	Publication
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	Association for Computational Linguistics
ISBN:	9798891763807
Date of First Compliant Deposit:	16 June 2026
Last Modified:	16 Jun 2026 10:15
URI:	https://orca.cardiff.ac.uk/id/eprint/187579

Actions (repository staff only)

Edit Item