Amid the rapid advancement of intelligent manufacturing, the Wireless Cloud Robotic System (WCRS) is envisioned as a cornerstone of future factories. However, achieving high-precision, low-latency control of robots in complex and dynamic wireless environments remains a critical challenge demanding a solution.

Recently, a research team from Industrial Control Network and System Department, the Shenyang Institute of Automation (SIA) of the Chinese Academy of Sciences, proposed an innovative solution. By integrating Reconfigurable Intelligent Surface (RIS) with an advanced algorithm termed "multi-agent transfer reinforcement learning," they successfully achieved co-optimization of robotic control and wireless communication, offering new possibilities for overcoming this bottleneck. The relevant research findings have been published in the leading journal in the field, the IEEE/CAA Journal of Automatica Sinica.

The research team constructed a novel wireless cloud robotic system architecture using Reconfigurable Intelligence Surface (RIS) technology. This architecture supports on-demand adjustment of the RIS phase shifts and beams, effectively countering environmental interference, thereby "reshaping" a deterministic communication link with low-latency, and low-jitter for high-precision closed-loop motion control.

RIS-assisted Wireless Cloud Robotic System Architecture (Image by SIA)

A Joint Optimization Method for Control and Communication Based on Multi-Agent Transfer Reinforcement Learning (Image by SIA)

Under this new RIS-assisted architecture, the robot's control decisions become strongly coupled with the configuration of the wireless channel, making it difficult for single-dimension optimization to improve overall performance, thus necessitating co-optimization. After fully considering constraints related to control input thresholds, control delay deadlines, beam phase, antenna power, and information distortion, the research team established a system stability maximization problem oriented towards control error and communication jitter. They defined a novel jitter-oriented system stability objective and derived a closed-form expression for the control delay deadline based on Jensen's inequality and the Lyapunov-Krasovskii functional.

Furthermore, considering the time-varying characteristics and partial observability of robot and communication channel states, the researchers modeled the problem as a Partially Observable Markov Decision Process (POMDP). They proposed a multi-agent transfer reinforcement learning method based on LSTM-PPO-MATRL, achieving joint optimization of control and communication parameters including control input compensation, RIS phase shift, and beamforming, thereby enhancing system stability.

Comparison of Cloud Robot Control Performance Under Different Scenarios (Image by SIA)

To validate the effectiveness of the algorithm, the research team conducted verification across four control tasks on a simulation platform: low-dynamic tasks such as Inverted Pendulum (INV) and Hopper (HOP), and high-dynamic tasks such as HalfCheetah (HAL) and Ant (ANT), corresponding to a 2-DOF robotic arm, a monopod robot, a bipedal robot, and a quadruped robot, respectively. Experimental results demonstrate that, within the new architecture, the control-communication joint optimization method exhibits advantages such as fast algorithm convergence and high rewards, meeting the requirements for cloud robot control and outperforming single-dimensional optimization of either control or communication alone.

This research not only theoretically solves the challenging problem of control and communication co-optimization in wireless cloud robotic systems but also, through the combination of RIS and intelligent algorithms, provides new insights for achieving large-scale, high-precision, and highly reliable robot collaboration in future factories. The outcomes hold promise for applications demanding extremely low latency and high-precision control, such as remote surgery, collaborative assembly, and autonomous driving, and are significant for advancing the development of industrial automation and intelligent systems.