Warp Consistency for Unsupervised Learning of Dense Correspondences

Prune Truong, Martin Danelljan, Fisher Yu, Luc Van Gool
ICCV 2021 Oral

Warp Consistency for Unsupervised Learning of Dense Correspondences

Abstract

The key challenge in learning dense correspondences lies in the lack of ground-truth matches for real image pairs. While photometric consistency losses provide unsupervised alternatives, they struggle with large appearance changes, which are ubiquitous in geometric and semantic matching tasks. Moreover, methods relying on synthetic training pairs often suffer from poor generalisation to real data. We propose Warp Consistency, an unsupervised learning objective for dense correspondence regression. Our objective is effective even in settings with large appearance and view-point changes. Given a pair of real images, we first construct an image triplet by applying a randomly sampled warp to one of the original images. We derive and analyze all flow-consistency constraints arising between the triplet. From our observations and empirical results, we design a general unsupervised objective employing two of the derived constraints. We validate our warp consistency loss by training three recent dense correspondence networks for the geometric and semantic matching tasks. Our approach sets a new state-of-the-art on several challenging benchmarks, including MegaDepth, RobotCar and TSS.

ICCV 2021 Presentation

Download the slides.

Poster

Click here to open high-res pdf poster.

Paper

Code

paper
github.com/PruneTruong/DenseMatching

Citation

@inproceedings{warpc,
  author    = {Prune Truong and Martin Danelljan and Fisher Yu and Luc Van Gool},
  title     = {Warp Consistency for Unsupervised Learning of Dense Correspondences},
  booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
  year      = {2021}
}

Related


Robust Object Detection via Instance-Level Temporal Cycle Confusion

Robust Object Detection via Instance-Level Temporal Cycle Confusion

ICCV 2021 We study the effectiveness of auxiliary self-supervised tasks to improve the out-of-distribution generalization of object detectors.


Hierarchical Discrete Distribution Decomposition for Match Density Estimation

Hierarchical Discrete Distribution Decomposition for Match Density Estimation

CVPR 2019 We propose Hierarchical Discrete Distribution Decomposition (HD^3), a framework suitable for learning probabilistic pixel correspondences in both optical flow and stereo matching.