The field of robotics and autonomous systems has witnessed significant advancements in recent years, with an increasing focus on enhancing the capabilities of robotic agents through RL. This project centres around applying Reinforcement Learning (RL) techniques to do object manipulation tasks with a specific focus on making a robot arm reach a target object. Using a combination of Gazebo and Robot Operating System (ROS) environments, the robot arm is trained using custom OpenAI Gym environments to simulate the task. The primary objective involves positioning the end-effector of the robot arm close to a designated object and overcoming challenges such as self-collisions during movements. Various iterations of RL training, including different reward logics, curriculum learning approaches, and fine-tuning parameters, are explored to refine the decision-making capabilities of the agent. The training process of Curriculum Learning involves a phased approach, starting with basic movements and progressing to more complex tasks, demonstrating improved performance. However, challenges such as prolonged training times and uncertainties in arm behaviour persist. The project highlights the complexities inherent in designing effective RL strategies for robotic systems and stresses the need for further research to enhance computational efficiency and reliability in real-world applications.
Keywords
Reinforcement LearningRobot ArmObject ManipulationOpenAI GymStable BaselinesRobot Operating System
References
R.S. Sutton and A.G. Barto, "Reinforcement Learning: An Introduction," IEEE Transactions on Neural Networks, vol. 9, no. 5, pp. 1054-1054, 1998, doi: 10.1109/TNN.1998.712192.
Z. Zhu et al., "High precision and efficiency robotic milling of complex parts: Challenges, approaches and trends," Chinese Journal of Aeronautics, vol. 35, no. 2, pp. 22-46, 2022, doi: 10.1016/j.cja.2020.12.030.
H. Ju et al., "Transferring policy of deep reinforcement learning from simulation to reality for robotics," Nat Mach Intell, vol. 4, pp. 1077-1087, 2022, doi: 10.1038/s42256-022-00573-6.
A. K. Shakya et al., "Reinforcement learning algorithms: A brief survey," Expert Systems with Applications, vol. 231, p. 120495, 2023, doi: 10.1016/j.eswa.2023.120495.
A. Ray et al., "Spinning Up OpenAI," 2018, [Online]. Available: https://spinningup.openai.com/en/latest/spinningup/rl_intro.html#.
V. Mnih et al., "Human-level control through deep reinforcement learning," 2015.
J. Schulman et al., "Proximal Policy Optimization Algorithms," 2017.
A. Plaat, "Model-Based Reinforcement Learning," 2022.
A. Stone et al., "Open-World Object Manipulation using Pre-trained Vision-Language Models," ArXiv, /abs/2303.00905, 2023.
D. Han et al., "A Survey on Deep Reinforcement Learning Algorithms for Robotic Manipulation," Sensors, 2023, doi: 10.3390/s23073762.
S. Chen and Y. Li, "Active vision in robotic systems: a survey of recent developments," The International Journal of Robotics Research, vol. 30, no. 11, pp. 1343-1377, 2011, doi: 10.1177/0278364911410755.
R. S. Pol and M. Murugan, "A review on indoor human aware autonomous mobile robot navigation through a dynamic environment: survey of different path planning algorithm and methods," 2015 International Conference on Industrial Instrumentation and Control (ICIC), IEEE, 2015.
M. Foukarakis et al., "Combining finite state machine and decision-making tools for adaptable robot behavior," International conference on universal access in human-computer interaction, Heraklion, Crete, Greece, pp. 625–635, Springer, 2-27 June 2014.
R.-E. Precup and H. Hellendoorn, "A survey on industrial applications of fuzzy control," Computers in Industry, vol. 62, no. 3, pp. 213-226, 2011.
O. Boubaker, "The inverted pendulum benchmark in nonlinear control theory," International Journal of Advanced Robot Systems, vol. 10, no. 233, pp. 1-9, 2013.
L. Sciavicco and B. Siciliano, "Modelling and control of robot manipulators," Springer Science & Business Media, Berlin/Heidelberg, 2012.
A. Sharma et al., "Autonomous Reinforcement Learning: Formalism and Benchmarking," 2021.
H. Nguyen and H. La, "Review of Deep Reinforcement Learning for Robot Manipulation," 2019.