Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language Models

Siddique, Zara ORCID: https://orcid.org/0009-0000-2245-5338, Turner, Liam ORCID: https://orcid.org/0000-0003-4877-5289 and Espinosa-Anke, Luis ORCID: https://orcid.org/0000-0001-6830-9176 2024. Who is better at math, Jenny or Jingzhen? Uncovering Stereotypes in Large Language Models. Presented at: The 2024 Conference on Empirical Methods in Natural Language Processing, Miami, FL, USA, 12-16 November 2024. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pp. 18601-18619. 10.18653/v1/2024.emnlp-main.1035

Preview

PDF - Published Version Available under License Creative Commons Attribution. Download (795kB) | Preview

Abstract

Large language models (LLMs) have been shown to propagate and amplify harmful stereotypes, particularly those that disproportionately affect marginalised communities. To understand the effect of these stereotypes more comprehensively, we introduce GlobalBias, a dataset of 876k sentences incorporating 40 distinct gender-by-ethnicity groups alongside descriptors typically used in bias literature, which enables us to study a broad set of stereotypes from around the world. We use GlobalBias to directly probe a suite of LMs via perplexity, which we use as a proxy to determine how certain stereotypes are represented in the model’s internal representations. Following this, we generate character profiles based on given names and evaluate the prevalence of stereotypes in model outputs. We find that the demographic groups associated with various stereotypes remain consistent across model likelihoods and model outputs. Furthermore, larger models consistently display higher levels of stereotypical outputs, even when explicitly instructed not to.

Item Type:	Conference or Workshop Item (Paper)
Status:	Published
Schools:	Computer Science & Informatics
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Publisher:	Association for Computational Linguistics
Date of First Compliant Deposit:	26 November 2024
Last Modified:	03 Dec 2024 11:30
URI:	https://orca.cardiff.ac.uk/id/eprint/170671

Actions (repository staff only)

Edit Item