Evaluating short-term temporal fluctuations of social biases in social media data and masked language models

Zhou, Yi ORCID: https://orcid.org/0000-0001-7009-8515 2024. Evaluating short-term temporal fluctuations of social biases in social media data and masked language models. Presented at: The 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), Miami, Florida, 12-16 November 2024. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 19693–19708.

Preview

PDF - Accepted Post-Print Version Download (1MB) | Preview

Abstract

Social biases such as gender or racial biases have been reported in language models (LMs), including Masked Language Models (MLMs). Given that MLMs are continuously trained with increasing amounts of additional data collected over time, an important yet unanswered question is how the social biases encoded with MLMs vary over time. In particular, the number of social media users continues to grow at an exponential rate, and it is a valid concern for the MLMs trained specifically on social media data whether their social biases (if any) would also amplify over time. To empirically analyse this problem, we use a series of MLMs pretrained on chronologically ordered temporal snapshots of corpora. Our analysis reveals that, although social biases are present in all MLMs, most types of social bias remain relatively stable over time (with a few exceptions). To further understand the mechanisms that influence social biases in MLMs, we analyse the temporal corpora used to train the MLMs. Our findings show that some demographic groups, such as male, obtain higher preference over the other, such as female on the training corpora constantly.

Item Type:	Conference or Workshop Item (Paper)
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	Association for Computational Linguistics
Date of First Compliant Deposit:	20 November 2024
Date of Acceptance:	2024
Last Modified:	20 Nov 2024 17:00
URI:	https://orca.cardiff.ac.uk/id/eprint/173669

Actions (repository staff only)

Edit Item