Evaluating the effect of retrieval augmentation on social biases

Zhang, Tianhui, Zhou, Yi ORCID: https://orcid.org/0000-0001-7009-8515 and Bollegala, Danushka 2026. Evaluating the effect of retrieval augmentation on social biases. Presented at: 19th Conference of the European Chapter of the Association for Computational Linguistics, Rabat, Morocco, 24-29 March 2026. Published in: Demberg, Vera, Inui, Kentaro and Marquez, Lluis eds. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, pp. 5004-5026. 10.18653/v1/2026.eacl-long.233

PDF - Published Version Available under License Creative Commons Attribution. Download (874kB)

Abstract

Retrieval Augmented Generation (RAG) is a popular method for injecting up-to-date information into Large Language Model (LLM)-based Natural Language Generation (NLG) systems. While RAG can enhance factual accuracy, its effect on the social biases inherent in LLMs is not well understood. This paper systematically investigates how RAG modulates social biases across three languages (English, Japanese, and Chinese) and four categories (gender, race, age, and religion). By evaluating various generator LLMs on the BBQ benchmark, we analyse how document collections with controlled stereotypical content affect RAG outputs. We find that biases present in the retrieved documents are often significantly amplified in the generated texts, even when the base LLM itself has a low-level of intrinsic bias. These findings raise concerns about the social fairness of RAG systems, underscoring the urgent need for careful bias evaluation before real-world deployment.

Item Type:	Conference or Workshop Item - published (Paper)
Date Type:	Publication
Status:	Published
Schools:	Schools > Computer Science & Informatics
Publisher:	Association for Computational Linguistics
ISBN:	9798891763807
Date of First Compliant Deposit:	16 June 2026
Last Modified:	16 Jun 2026 10:00
URI:	https://orca.cardiff.ac.uk/id/eprint/187580

Actions (repository staff only)

Edit Item