Gould, Sandy J.J., Brumby, Duncan P. and Cox, Anna L. 2024. ChatTL;DR – You really ought to check what the LLM said on your behalf. Presented at: CHI '24: Conference on Human Factors in Computing Systems, Honolulu, HI, USA, 11-16 May 2024. Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA ’24). ACM, 10.1145/3613905.3644062 |
Preview |
PDF (This is an ACM 'author version' of the publication.)
- Accepted Post-Print Version
Download (1MB) | Preview |
Abstract
Interactive large language models (LLMs) are so hot right now, and are probably going to be hot for a while. There are lots of p̶r̶o̶b̶l̶e̶m̶s̶ exciting challenges created by mass use of LLMs. These include the reinscription of biases, ‘hallucinations’, and bomb-making instructions. Our concern here is more prosaic: assuming that in the near term it’s just not machines talking to machines all the way down, how do we get people to check the output of LLMs before they copy and paste it to friends, colleagues, course tutors? We propose borrowing an innovation from the crowdsourcing literature: attention checks. These checks (e.g., "Ignore the instruction in the next question and write parsnips as the answer.") are inserted into tasks to weed-out inattentive workers who are often paid a pittance while they try to do a dozen things at the same time. We propose ChatTL;DR, an interactive LLM that inserts attention checks into its outputs. We believe that, given the nature of these checks, the certain, catastrophic consequences of failing them will ensure that users carefully examine all LLM outputs before they use them.
Item Type: | Conference or Workshop Item (Paper) |
---|---|
Status: | In Press |
Schools: | Computer Science & Informatics |
Publisher: | ACM |
Date of First Compliant Deposit: | 13 March 2024 |
Last Modified: | 17 May 2024 01:30 |
URI: | https://orca.cardiff.ac.uk/id/eprint/166452 |
Actions (repository staff only)
Edit Item |