Wireless networked control systems, with their significant advantages in flexibility, scalability, and cost-effectiveness, have become a key enabling technology in modern industrial control. However, the complex radio frequency environment and limited spectrum resources in industrial settings can easily induce random packet loss and non-deterministic communication delays, severely degrading the closed-loop control performance of the system.

Addressing the challenges posed by limited communication resources and highly dynamic environmental conditions in industrial scenarios, the research team led by Professor LIANG Wei from the Industrial Control Network and System Department, the Shenyang Institute of Automation (SIA) of the Chinese Academy of Sciences, has proposed a Joint Estimation-Control-Scheduling (JECS) method based on deep reinforcement learning.

The research findings have been published in the international journal IEEE Transactions on Cognitive Communications and Networking, under the title Joint Estimation-Control-Scheduling for Wireless Networked Control Systems via Deep Reinforcement Learning. ZHANG Lei, a doctoral student at the SIA, is the first author, and his supervisor is Professor LIANG Wei.

Traditional design approaches often treat communication scheduling and control algorithms separately or rely on accurate system models, making it difficult to achieve ideal results in practical applications. Furthermore, many existing studies face scalability issues, as their trained models often fail when the number of subsystems changes or network resources fluctuate.

To tackle the problem of packet loss in industrial environments, the research team designed a state estimation model based on deep learning and a control strategy based on deep reinforcement learning for individual subsystems. This significantly enhances the system's operational stability under the constraint of unreliable communication. Building on this, for complex scenarios involving the concurrent operation of multiple subsystems, the team further developed a centralized scheduling decision model based on the Transformer architecture. By innovatively introducing a priority scoring mechanism, this model significantly improves the system's generalization ability across different scales and resource constraints.

The research team comprehensively evaluated the JECS method on two benchmark tasks: the classic inverted pendulum (using the OpenAI Gymnasium environment) and a ball balancing platform (using a custom environment). They compared its performance against methods minimizing the Age of Information (AoI), Round-robin scheduling, and another DRL-based method called DeepCAS.

The results indicate that their proposed JECS method significantly reduces the LQG control cost and supports the stable operation of a greater number of subsystems simultaneously, demonstrating a marked increase in system capacity. Even when compared to the model-dependent CARS method, JECS achieves comparable control performance without requiring any model information.

This research was supported by programs under the National Natural Science Foundation of China, the Liaoning Provincial Natural Science Foundation,etc.