It is a challenging task for an autonomous robot to work safely in dynamic and unstructured environments, such as vast space, open land and deep sea. Compared with the traditional industrial robot in manufacturing, robot autonomy in uncontrolled scenarios requires significantly extra capabilities to support, e.g., perception, navigation, decision-making, and manipulation.

To solve this problem, researchers at Shenyang Institute of Automation (SIA), Chinese Academy of Sciences (CAS) have developed a novel deep reinforcement learning (DRL) based control system with partners at Edinburgh Centre for Robotics (ECR) in the United Kingdom, and published their study in Sensors.

The novel method is set to achieve autonomous mobile manipulation in dynamic and unstructured environments. The cutting-edge AI techniques have been successfully applied to a complex real-world mobile manipulator robot.

The joint research team designs new neural network architecture to build a RL-based whole-body robot control model, which leverages the state-of-the-art deep learning method to perceive and understand the environment and targets through an on-board camera.

Then, they utilize the acquired information and the robot state to autonomously control the robot. Through the interactive learning and training in the simulation and actual environment, the autonomous operation of the mobile manipulator in the real environment is finally achieved.

This research lays the foundation of deploying the DRL for autonomous operation research using a more complex floating-based underwater robot system.

Under a strategic cooperation framework (MoU) between the State Key Laboratory of Robotics of SIA, the supporting organization of the State Key Lab, and ECR, SIA and ECR have been actively carrying out personnel exchange, academic exchange and joint training of graduate students, and have been closely collaborating in several research fields, including DRL-based robot control, vision-based underwater 3D scene reconstruction and target recognition, autonomous operation of underwater robot.

With the cooperation being further strengthened and extended, more scientific research results will be delivered in the future to advance the relevant scientific research of both sides.

Deep-RL based mobile manipulation (Image by WANG Cong)