Bamoallem, Razan, Loizides, Fernando ORCID: https://orcid.org/0000-0003-0531-6760, Finnegan, Daniel ORCID: https://orcid.org/0000-0003-1169-2842 and Alsenaidi, Hanin
2025.
HF-MAI: Touchless mid-air interaction for individuals with dexterity impairment.
Presented at: 20th IFIP TC13 International Conference on Human-Computer Interaction,
Belo Horizonte, Minas Gerais, Brazil,
8 - 12 September 2025.
Published in: Ardito, C. ed.
Proceedings of the Human-Computer Interaction – INTERACT 2025.
Lecture Notes in Computer Science
, vol.16108
Springer,
pp. 108-130.
10.1007/978-3-032-04999-5_7
|
|
PDF
- Accepted Post-Print Version
Download (3MB) |
Abstract
For individuals with Multiple Sclerosis (MS) and dexterity impairments (MS-Dex), touchscreen interaction can be particularly challenging, as symptoms like finger numbness and impaired motion regulation negatively impact smartphone usability. This study investigated the usability and accessibility of a mid-air gesture interaction technique for MS-Dex. Touchscreen technologies are ubiquitous, but these individuals often face challenges with touch-based interaction. Touchless input methods, like mid-air gestures, offer a promising alternative for enhancing interaction and communication with touch-based technology. We developed an interactive tool (HF-MAI) using hand and finger mid-air gestures for common touchscreen tasks, using the Leap Motion Controller (LMC) for gesture recognition. MS-Dex individuals, MS therapists, and a usability and user experience (UX) expert participated in the evaluation. We assessed usability, user satisfaction, and accessibility of the HF-MAI system. Results indicate HF-MAI is highly usable and accessible; participants performed gestures successfully and reported high satisfaction. Notably, pinch, spread, and rotate (two-finger/single-finger) mid-air gestures were performed with faster completion times, contrasting with the difficulty of such gestures on touchscreens. The gesture vocabulary, particularly the adaptation to single/two-finger movements, facilitated easier interaction, reducing difficulty for users with limited dexterity. This study demonstrates the potential of mid-air gestures using co-created vocabularies to enhance accessibility for individuals with MS, offering a promising alternative to traditional touch-based input. This technology can significantly improve their access to touchscreens, enhancing independence and quality of life.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Date Type: | Published Online |
| Status: | Published |
| Schools: | Schools > Computer Science & Informatics |
| Publisher: | Springer |
| ISBN: | 978-3-032-04998-8 |
| Date of First Compliant Deposit: | 7 August 2025 |
| Last Modified: | 30 Sep 2025 09:48 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/180300 |
Actions (repository staff only)
![]() |
Edit Item |





Altmetric
Altmetric