|
| Title: | Safe adaptive policy transfer reinforcement learning for distributed multi-agent control | | | Authors: | B. Du, W. Xie, Y. Li, Q. Yang, W. Zhang, R.R. Negenborn, Y. Pang, H. Chen |
| | Journal: | IEEE Transactions on Neural Networks and Learning Systems | | |
| | Abstract: | Multi-agent reinforcement learning training is usually difficult and time-consuming due to mutual interference among agents. Safety concerns make an already difficult training process even harder. This study proposes a safe adaptive policy transfer reinforcement learning approach for multi-agent cooperative control. Specifically, a pioneer and follower off-policy policy transfer learning method is presented to help follower agents to acquire knowledge and experience from a single well-trained pioneer agent. Notably, the designed approach can transfer both the policy representation and sample experience provided by the pioneer policy in the off-policy learning. More importantly, the proposed method can adaptively adjust the learning weight of prior experience and exploration according to the Wasserstein distance between policy probability distributions of pioneer and follower. Case studies show that the distributed agents trained by the proposed method can complete a collaborative task and acquire the maximum rewards while minimizing the violation of constraints. Moreover, the proposed method can also achieve satisfactory performance in terms of learning speed and success rate. |
| | Reference: | Safe adaptive policy transfer reinforcement learning for distributed multi-agent control. B. Du, W. Xie, Y. Li, Q. Yang, W. Zhang, R.R. Negenborn, Y. Pang, H. Chen. Accepted for publication in IEEE Transactions on Neural Networks and Learning Systems, 2023. | | | Request: | A
copy of this publication. |
|
|
