Abstract

Data-driven simulation has become a favorable way to train and test autonomous driving algorithms. The idea of replacing the actual environment with a learned simulator has also been explored in model-based reinforcement learning in the context of world models. In this work, we show data-driven traffic simulation can be formulated as a world model. We present TrafficBots, a multi-agent policy built upon motion prediction and end-to-end driving, and based on TrafficBots we obtain a world model tailored for the planning module of autonomous vehicles. Existing data-driven traffic simulators are lacking configurability and scalability. To generate configurable behaviors, for each agent we introduce a destination as navigational information, and a time-invariant latent personality that specifies the behavioral style. To improve the scalability, we present a new scheme of positional encoding for angles, allowing all agents to share the same vectorized context and the use of an architecture based on dot-product attention. As a result, we can simulate all traffic participants seen in dense urban scenarios. Experiments on the Waymo open motion dataset show TrafficBots can simulate realistic multi-agent behaviors and achieve good performance on the motion prediction task.

Video

Paper

Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu and Luc van Gool TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction ICRA 2023

Code

github.com/SysCV/TrafficBots

Citation

@inproceedings{zhang2023trafficbots,
  title = {{TrafficBots}: Towards World Models for Autonomous Driving Simulation and Motion Prediction},
  booktitle = {International Conference on Robotics and Automation (ICRA)},
  author = {Zhang, Zhejun and Liniger, Alexander and Dai, Dengxin and Yu, Fisher and Van Gool, Luc},
  year = {2023},
}