Detecting hand-object interactions in the wild: Automated analysis of headcam videos from young children with and without Down syndrome

Thompson, Craig D. J., Lai, Yukun ORCID: https://orcid.org/0000-0002-2094-5680 and D'Souza, Hana 2026. Detecting hand-object interactions in the wild: Automated analysis of headcam videos from young children with and without Down syndrome. Presented at: IEEE ICDL 2026, Kyoto, Japan, 15-18 September 2026. Proceedings of the IEEE International Conference on Development and Learning 2026. IEEE, Item availability restricted.

	PDF - Accepted Post-Print Version Restricted to Repository staff only Download (1MB)
	PDF (Provisional file) - Accepted Post-Print Version Download (17kB)

Abstract

Observing children’s manual interactions within their naturalistic environment offers an important window into early motor development, yet measuring these behaviours in naturalistic settings remains a major methodological challenge. Until recently, characterising such behaviour relied on labourintensive manual annotation, limiting scalability. In this study, we leverage recent advances in machine learning to (1) improve automated quantification of naturalistic hand-object interactions, and (2) examine how these behaviours differ between young children with and without Down syndrome (DS). Utilising our developmental head-mounted camera (headcam) dataset (TinyExplorer; 74 children aged 2–60 months), we finetuned the 100 Days of Hands (100DOH) model on child-centred egocentric footage, raising its hand-detection F1 score from .79 to .87, and extended it with a hand-ownership classifier (own vs other hands), essential for developmental analysis of children’s egocentric views. Applied to over 109,000 frames from 30 participants, the resulting TinyExplorer-Tuned (100DOH-TET) framework revealed broadly similar overall hand and object presence across groups. Exploratory, non-significant trends pointed in opposite directions for the two groups: typically developing children tended to handle objects themselves more often, whereas children with DS tended to encounter more objects held by others. These patterns require confirmation in larger samples, but they demonstrate the potential of automated methods for quantifying early manual behaviour in real-world environments.

Item Type:	Conference or Workshop Item - published (Paper)
Status:	In Press
Schools:	Schools > Computational & Mathematical Sciences Schools > Psychology Schools > Computer Science & Informatics
Publisher:	IEEE
Date of First Compliant Deposit:	30 September 2026
Last Modified:	30 Sep 2026 09:05
URI:	https://orca.cardiff.ac.uk/id/eprint/189901

Actions (repository staff only)

Edit Item