End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

Zhejun Zhang, Alexander Liniger, Dengxin Dai, Fisher Yu, Luc Van Gool
ICCV 2021

End-to-End Urban Driving by Imitating a Reinforcement Learning Coach

Abstract

End-to-end approaches to autonomous driving commonly rely on expert demonstrations. Although humans are good drivers, they are not good coaches for end-to-end algorithms that demand dense on-policy supervision. On the contrary, automated experts that leverage privileged information can efficiently generate large scale on-policy and off-policy demonstrations. However, existing automated experts for urban driving make heavy use of hand-crafted rules and perform suboptimally even on driving simulators, where ground-truth information is available. To address these issues, we train a reinforcement learning expert that maps bird’s-eye view images to continuous low-level actions. While setting a new performance upper-bound on CARLA, our expert is also a better coach that provides informative supervision signals for imitation learning agents to learn from. Supervised by our reinforcement learning coach, a baseline end-to-end agent with monocular camera-input achieves expert-level performance. Our end-to-end agent achieves a 78% success rate while generalizing to a new town and new weather on the NoCrash-dense benchmark and state-of-the-art performance on the more challenging CARLA LeaderBoard.

ICCV 2021 Presentation

Result Videos

Highlights: IL Agent Supervised by Roach Diving with single camera image as input.

Unedited: Roach Driving with bird’s-eye-view image as input.

Unedited: Autopilot Driving with ground-truth information as input.

Paper

Code

paper
github.com/zhejz/carla-roach

Citation

@inproceedings{zhang2021roach,
  title = {End-to-End Urban Driving by Imitating a Reinforcement Learning Coach},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  author = {Zhang, Zhejun and Liniger, Alexander and Dai, Dengxin and Yu, Fisher and Van Gool, Luc},
  year = {2021},
}

Related


Instance-Aware Predictive Navigation in Multi-Agent Environments

Instance-Aware Predictive Navigation in Multi-Agent Environments

ICRA 2021 A new visual model-based RL method with consideration of multiple hypotheses for future object movement.


Semantic Predictive Control for Explainable and Efficient Policy Learning

Semantic Predictive Control for Explainable and Efficient Policy Learning

ICRA 2019 We propose a driving policy learning framework that predicts feature representations of future visual inputs.


Deep Object-Centric Policies for Autonomous Driving

Deep Object-Centric Policies for Autonomous Driving

ICRA 2019 We show that object-centric models outperform object-agnostic methods in scenes with other vehicles and pedestrians.


End-to-end Learning of Driving Models from Large-scale Video Datasets

End-to-end Learning of Driving Models from Large-scale Video Datasets

CVPR 2017 Oral We develop an end-to-end trainable architecture for learning to predict a distribution over future vehicle egomotion from instantaneous monocular camera observations and previous vehicle state.