Gong, Meiyuan, Gao, Yan and Ji, Ze  ORCID: https://orcid.org/0000-0002-8968-9902
2026.
Prompted to explore: training-time-only LLM proposals for sample-efficient pushing grasping policies.
Presented at: International Conference on Mechatronics and Automation,
Changchun, China,
2-5 August 2026.
Proceedings of ICMA.
IEEE,
pp. 2018-2024.
10.1109/icma69663.2026.11647618
|
Preview |
PDF
- Accepted Post-Print Version
Available under License Creative Commons Attribution. Download (1MB) | Preview |
Abstract
Learning long-horizon robot manipulation remains difficult and time-consuming, especially under sparse rewards due to inefficient exploration and reward assignment. We present a minimal integration of large language models (LLMs) with reinforcement learning (RL) in which the LLM is used strictly as an online action proposer during early training to help with the RL agent. Given a task description and a compact scene abstraction, the LLM outputs a single parameterised primitive (pushing or grasping) that is validated by lightweight safety checks and mixed with policy actions via an epsilon-greedy scheduler. Besides, we study two simple policies that preserve the same low-level learner (grasping-biased model): (i) pushing-first reward shaping that temporarily increases the relative reward of pushing early in an episode, and (ii) a Proximal Policy Optimization (PPO) high-level switch that explicitly selects between push and grasp while Soft Actor-Critic (SAC) handles low-level parameterisation. In simulation, LLM-guided exploration accelerates early learning, and the proposed methods help further mitigate grasping overuse in the cluttered environment. Besides, the LLM-guided RL agent could achieve a higher success rate at 89.5% compared with the baseline method at 68%.
| Item Type: | Conference or Workshop Item - published (Paper) |
|---|---|
| Date Type: | Published Online |
| Status: | Published |
| Schools: | Schools > Engineering |
| Additional Information: | RRS policy applied |
| Publisher: | IEEE |
| ISBN: | 9798331562847 |
| ISSN: | 2152-7431 |
| Date of First Compliant Deposit: | 2 September 2026 |
| Last Modified: | 02 Sep 2026 13:45 |
| URI: | https://orca.cardiff.ac.uk/id/eprint/189143 |
Actions (repository staff only)
 |
Edit Item |
Dimensions
Dimensions