Spatial Intention Maps for Multi-Agent Mobile Manipulation

The ability to communicate intention enables decentralized multi-agent robots to collaborate while performing physical tasks. In this work, we present spatial intention maps, a new intention representation for multi-agent vision-based deep reinforcement learning that improves coordination between decentralized mobile manipulators. In this representation, each agent's intention is provided to other agents, and rendered into an overhead 2D map aligned with visual observations. This synergizes with the recently proposed spatial action maps framework, in which state and action representations are spatially aligned, providing inductive biases that encourage emergent cooperative behaviors requiring spatial coordination, such as passing objects to each other or avoiding collisions. Experiments across a variety of multi-agent environments, including heterogeneous robot teams with different abilities (lifting, pushing, or throwing), show that incorporating spatial intention maps improves performance for different mobile manipulation tasks while significantly enhancing cooperative behaviors.

Foraging with lifting robots

Foraging with pushing robots

Search and rescue

Lifting robots with a divider obstacle

Pushing robots with a divider obstacle

Throwing and lifting teamwork

Paper

To appear at IEEE International Conference on Robotics and Automation (ICRA), 2021.
Latest version (March 23, 2021): arXiv:2103.12710 [cs.RO] or here.

Method

Team

Jimmy Wu ^1,2

Xingyuan Sun ¹

Andy Zeng ²

Shuran Song ³

Szymon Rusinkiewicz ¹

Thomas Funkhouser ^1,2

@inproceedings{wu2021spatial,
  title = {Spatial Intention Maps for Multi-Agent Mobile Manipulation},
  author = {Wu, Jimmy and Sun, Xingyuan and Zeng, Andy and Song, Shuran and Rusinkiewicz, Szymon and Funkhouser, Thomas},
  booktitle = {IEEE International Conference on Robotics and Automation (ICRA)},
  year = {2021}
}

Qualitative Results

We ran experiments on a variety of different environments with and without spatial intention maps. In this section, we show some of our main qualitative results.

Emergent Foraging Strategy

In the SmallDivider environment, we find that using spatial intention maps enables teams of robots to learn the strategy shown in the pair of images below. The strategy is highly efficient, and consistently emerges from training for both lifting robots and pushing robots.

Team of lifting robots

Team of pushing robots

Emergent Division of Labor

When we train heterogeneous teams of robots with spatial intention maps, we observe that a natural division of labor emerges, as shown in the images below. The movement trajectories are colored by robot type (lifting is blue, pushing/throwing is green). Notice that there is almost no overlap between blue and green trajectories in either image. The pushing robots focus on objects along the wall, the throwing robots focus on faraway objects, and the lifting robots take care of the rest.

Lifting and pushing team

Lifting and throwing team

Search and Rescue Efficiency

We find that teams trained with spatial intention maps are able to execute the search and rescue task more efficiently, as shown by movement trajectories in the pair of images below. With intention maps (left), robots are able to choose actions that avoid overlapping with other robots and distribute themselves more effectively. Without intention maps (right), they are not able to coordinate as well since they do not know each other's intentions.

Robots trained with spatial intention maps

Robots trained without spatial intention maps (baseline)

Baseline Policies

Here we highlight common undesirable behaviors that we see in baseline policies trained without spatial intention maps. For lifting robots, these behaviors include collisions with other robots (left), or indecisive back-and-forth movement in an effort to avoid collisions (right).

Lifting robots colliding

Lifting robots moving indecisively

Similarly, pushing and rescue robots that are trained without spatial intention maps have a hard time avoiding overlap in their efforts. Below are examples of pushing robots (left) and rescue robots (right) going after the same object.

Pushing robots trying to push the same object

Rescue robots trying to rescue the same object

Real Robot Videos

In this section, we show full episodes (8x speed) of policies trained with spatial intention maps and running on the real robots. These videos are grouped by robot team composition, and each video within a group shows a different test environment.

4 Lifting Robots

4 Pushing Robots

4 Rescue Robots

Simulation Videos

In this section, we show full episodes in the simulation (4x speed) of policies trained with spatial intention maps. Each video within a group shows a different test environment.

4 Lifting Robots

4 Pushing Robots

2 Lifting + 2 Pushing

2 Lifting + 2 Throwing

4 Rescue Robots

Acknowledgments

We would like to thank Naomi Leonard, Anirudha Majumdar, Naveen Verma, and Anish Singhani for fruitful technical discussions. This work was supported in part by the Princeton School of Engineering, as well as the National Science Foundation under IIS-1815070 and DGE-1656466.

Contact

If you have any questions, please feel free to contact Jimmy Wu.