Gunawardana, Liyanaarachchige Dona Rashmi Nirasha, Randika, Ayantha, Perera, Dushani and Karunanayaka, Kasun
2026.
Towards Responsible AI in Low-Resource Languages: Evaluating LLM Responses with a Culturally Aware Sinhala Dataset.
Presented at: ICMI '26: International Conference on Multimodal Interaction,
Naples, Italy,
5 - 9 October 2026.
Published in: Rossi, Silvia, Esposito, Anna, Tortora,, Genoveffa, Conati, Cristina, Cutugno, Francesco, Siegert, Ingo Siegert, Rossi, Alessandra Rossi and Barmaki, Leila Barmaki eds.
Proceedings of the 28th International Conference On Multimodal Interaction.
New York, USA:
ACM,
p. 437.
10.1145/3776574.3831155
|
Preview |
PDF
- Published Version
Available under License Creative Commons Attribution. Download (911kB) | Preview |
Abstract
Large Language Models (LLMs) are increasingly used as primary information sources, raising critical concerns about their safety and reliability across languages. While prior work has established the importance of harmless yet non-evasive responses, these guarantees are largely evaluated in high-resource languages. In this paper, we investigate the safety of responses in LLMs in Sinhala, a low-resource language with strong cultural grounding. We introduce a framework that extends existing definitions of harmlessness by incorporating responsible behaviour, requiring models to actively discourage harmful actions and provide safety-oriented guidance for culturally grounded question prompts. Using a curated set of culturally grounded and safety-critical queries, we evaluate multiple publicly available LLMs across categories, including self-harm, medical advice, culturally sensitive topics, and illegal activities. Our findings reveal significant safety gaps, including the generation of dangerous misinformation and the absence of appropriate intervention in high-risk scenarios. We further propose a Sinhala-specific diagnostic dataset to evaluate the safety of responses generated by LLMs in the Sinhala language.
| Item Type: | Conference or Workshop Item - published (Paper) |
|---|---|
| Date Type: | Publication |
| Status: | Published |
| Schools: | Schools > Computational & Mathematical Sciences Schools > Computer Science & Informatics |
| Publisher: | ACM |
| ISBN: | 9798400723186 |
| Date of First Compliant Deposit: | 12 October 2026 |
| Last Modified: | 12 Oct 2026 10:13 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/190126 |
Actions (repository staff only)
 |
Edit Item |
Dimensions
Dimensions