Handwriting velcro: Endowing AR glasses with personalized and posture-adaptive text input using flexible touch sensor

Fang, Fengyi, Zhang, Hongwei, Zhan, Lishuang, Guo, Shihui, Zhang, Minying, Lin, Juncong, Qin, Yipeng ORCID: https://orcid.org/0000-0002-1551-9126 and Fu, Hongbo 2023. Handwriting velcro: Endowing AR glasses with personalized and posture-adaptive text input using flexible touch sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) 6 (4) , pp. 1-31. 10.1145/3569461

Preview

PDF - Accepted Post-Print Version Download (10MB) | Preview

Abstract

Text input is a desired feature for AR glasses. While there already exist various input modalities (e.g., voice, mid-air gesture), the diverse demands required by different input scenarios can hardly be met by the small number of fixed input postures offered by existing solutions. In this paper, we present Handwriting Velcro, a novel text input solution for AR glasses based on flexible touch sensors. The distinct advantage of our system is that it can easily stick to different body parts, thus endowing AR glasses with posture-adaptive handwriting input. We explored the design space of on-body device positions and identified the best interaction positions for various user postures. To flatten users' learning curves, we adapt our device to the established writing habits of different users by training a 36-character (i.e., A-Z, 0-9) recognition neural network in a human-in-the-loop manner. Such a personalization attempt ultimately achieves a low error rate of 0.005 on average for users with different writing styles. Subjective feedback shows that our solution has a good performance in system practicability and social acceptance. Empirically, we conducted a heuristic study to explore and identify the best interaction Position-Posture Correlation. Experimental results show that our Handwriting Velcro excels similar work [6] and commercial product in both practicality (12.3 WPM) and user-friendliness in different contexts.

Item Type:	Article
Date Type:	Published Online
Status:	Published
Schools:	Computer Science & Informatics
Publisher:	ACM
ISSN:	2474-9567
Date of First Compliant Deposit:	7 March 2023
Date of Acceptance:	7 October 2022
Last Modified:	07 Nov 2023 01:16
URI:	https://orca.cardiff.ac.uk/id/eprint/155797

Actions (repository staff only)

Edit Item