Cardiff University | Prifysgol Caerdydd ORCA
Online Research @ Cardiff 
WelshClear Cookie - decide language by browser settings

GAM: General Affordance-based Manipulation for contact-rich object disentangling tasks

Yang, Xintong ORCID:, Wu, Jing ORCID:, Lai, Yu-Kun ORCID: and Ji, Ze ORCID: 2024. GAM: General Affordance-based Manipulation for contact-rich object disentangling tasks. Neurocomputing 578 , 127386. 10.1016/j.neucom.2024.127386

[thumbnail of 1-s2.0-S0925231224001577-main.pdf]
PDF - Published Version
Available under License Creative Commons Attribution.

Download (2MB) | Preview
License URL:
License Start date: 7 February 2024


Picking up an entangled object is a difficult manipulation task due to its rich contact dynamics. Most existing solutions fail to produce grasp poses to enable reliable manipulation due to the dependence on simplified assumptions for the motion policies. Grasps generated by these methods tend to drop objects or cause undesired movements of non-grasped objects. To improve such object-disentangling tasks, we propose to extend the concept of reinforcement learning (RL)-based affordance to include arbitrary action consequences and implement a general affordance-based manipulation (GAM) framework. In the GAM, we train an RL agent that uses more fine-grained actions and outperforms previous methods with a smaller chance of dropping objects and making contact with non-grasped hooks. Then, a manipulation affordance prediction (MAP) model is trained to estimate the performances of the RL agent. Finally, the manipulation affordance-based grasp filter (MAGF) selects grasp poses that afford the desired manipulation performances, showing substantial improvements in five challenging hook disentangling tasks in simulation. The experiments show (1) the limitation of TAG generators, (2) the effectiveness of filtering TAGs with predicted manipulation performances based on the general affordance theory, and (3) the importance of avoiding contact with non-grasped objects in contact-rich manipulation.

Item Type: Article
Date Type: Publication
Status: Published
Schools: Engineering
Publisher: Elsevier
ISSN: 0925-2312
Funders: EPSRC
Date of First Compliant Deposit: 6 February 2024
Date of Acceptance: 3 February 2024
Last Modified: 16 Apr 2024 09:19

Actions (repository staff only)

Edit Item Edit Item


Downloads per month over past year

View more statistics