Event-triggered control provides a mechanism for avoiding excessive use of constrained communication bandwidth in networked multi-agent systems. However, most existing methods rely on accurate system models, which may be unavailable in practice. In this work, we propose a model-free, priority-driven reinforcement learning algorithm that learns communication priorities and control policies jointly from data in decentralized multi-agent systems. By learning communication priorities, we circumvent the hybrid action space typical in event-triggered control with binary communication decisions. We evaluate our algorithm on benchmark tasks and demonstrate that it outperforms the baseline method.
Overview
Our method jointly learns control policies and communication priorities using the
independent proximal policy optimization algorithm. A decentralized multi-agent system where each agent \(i\) learns the policy \(\pi(a_i | o_i, \hat{o})\)
based on local observations \(o_i\) and communicated observations \(\hat{o}\).
Agent \(i\) exchanges the communication priority \(a_{i,\mathrm{p}}\) through a wireless network.
Moreover, \(L\) agents with the highest priorities are allocated bandwidth and send crucial information \(\hat{o}\) to other agents.
Agent \(i\) executes the control action \(a_{i,c}\) to interact with the environment, getting corresponding rewards and local observations,
which feed back into the policy learning.
Coverage Control (Three Agents)
(a) Ours
(b) Round-Robin Communication
In coverage control, our algorithm enables agents to quickly coordinate and
cover the corresponding landmarks. In the final performance, two agents are nearly
aligned with their respective landmarks, while one
agent has a short distance away from its landmark.
The policy learned by the baseline lets the agents go to different landmarks at the initial stage,
but in the middle, two of the agents show a lack of awareness regarding which landmark
they are supposed to cover. In the end, none of the three agents can be fully
aligned with landmarks.
Formation Control (Eight Agents)
(a) Ours
(b) Round-Robin Communication
In formation control, following the policy from our algorithm, the agents coordinate
and eventually surround the landmark, forming an approximate regular polygon.
The policy from the baseline fails to allow the agents to be distributed in a
desired manner around the landmark. Some agents place themselves in appropriate
locations, while the remaining agents do not reach the corresponding locations.
Multiwalker
Three Agents
(a) Ours
(b) Round-Robin Communication
(c) MAPPOC
Under limited bandwidth conditions, the agents trained with our algorithm initially adopt a simplified policy,
passing the package to the rightmost agent to minimize complex coordination, which would
otherwise require more communication. The rightmost agent stabilizes the package by
positioning itself at its center of mass, balancing it effectively—even performing
jumps to maintain stability. Eventually, however, the package falls to the ground due
to risky balancing and an unexpected terrain configuration not encountered
in training. Overall, our method enables agents to transport packages over considerable
distances, despite communication constraints.
With the baseline, the agents adopt a similar policy, that is, let as few agents as possible transport the package. However,
around the midpoint, the middle agent approaches the rightmost one, aiming to
transfer the package. Here, a slight misalignment occurs due to the lack of the important information, with the middle agent moving out
of sync with the rightmost agent's reach. This desynchronization leads to the package
slipping, resulting in it falling to the ground.
Agents trained with MAPPOC rely mainly on the middle and rightmost agents for package transport. However, their transport speed is much lower than that achieved by our method, and the rightmost agent eventually loses balance and falls while attempting to stabilize the package. Notably, MAPPOC achieves only about 55% communication savings, whereas our method reduces communication usage by approximately 67%.
Six Agents
(a) Ours
(b) Round-Robin Communication
When the number of agents increases to six, the policy learned by our algorithm is
to pass the package to the four agents on the right, who then transport it over a
considerable distance. However, because the agents are so close together and lack
sufficient communication, the leftmost agent eventually steps on the second agent
from the left.
With six agents, the policy obtained from the baseline is still to let as few agents as
possible transport the package. However, this approach introduces certain risks:
some middle agents exhibit less effort during transfers and fail to recognize how
their forward movement impacts the overall task. As a result, they end up tripping
other agents, and the package does not travel very far.
