An open-source multi-goal reinforcement learning environment for robotic manipulation with Pybullet

Yang, Xintong ORCID: https://orcid.org/0000-0002-7612-614X, Ji, Ze ORCID: https://orcid.org/0000-0002-8968-9902, Wu, Jing ORCID: https://orcid.org/0000-0001-5123-9861 and Lai, Yu-Kun ORCID: https://orcid.org/0000-0002-2094-5680 2021. An open-source multi-goal reinforcement learning environment for robotic manipulation with Pybullet. Presented at: 21st Towards Autonomous Robotic Systems Conference (TAROS 2021), Virtual, 8-10 September 2021. Published in: Fox, C., Gao, J., Ghalamzan Esfahani, A., Saaj, M., Hanheide, M. and Parsons, S. eds. Towards Autonomous Robotic Systems. TAROS 2021. Lecture Notes in Computer Science Springer, pp. 14-24. 10.1007/978-3-030-89177-0_2

	PDF - Accepted Post-Print Version Download (1MB)
	PDF - Supplemental Material Download (3MB)

Abstract

This work re-implements the OpenAI Gym multi-goal robotic manipulation environment, originally based on the commercial Mujoco engine, onto the open-source Pybullet engine. By comparing the performances of the Hindsight Experience Replay-aided Deep Deterministic Policy Gradient agent on both environments, we demonstrate our successful re-implementation of the original environment. Besides, we provide users with new APIs to access a joint control mode, image observations and goals with customisable camera and a built-in on-hand camera. We further design a set of multi-step, multi-goal, long-horizon and sparse reward robotic manipulation tasks, aiming to inspire new goal-conditioned reinforcement learning algorithms for such challenges. We use a simple, human-prior-based curriculum learning method to benchmark the multi-step manipulation tasks. Discussions about future research opportunities regarding this kind of tasks are also provided.

Item Type:	Conference or Workshop Item (Paper)
Date Type:	Published Online
Status:	Published
Schools:	Schools > Engineering
Publisher:	Springer
ISBN:	9783030891763
Date of First Compliant Deposit:	20 July 2021
Date of Acceptance:	11 February 2021
Last Modified:	14 May 2025 14:49
URI:	https://orca.cardiff.ac.uk/id/eprint/142729

Citation Data

Cited 1 time in Scopus. View in Scopus. Powered By Scopus® Data

Actions (repository staff only)

Edit Item