Map-free Visual Relocalization: Metric Pose Relative to a Single Image

Abstract

Can we relocalize in a scene represented by a single reference image? Standard visual relocalization requires hundreds of images and scale calibration to build a scene-specific 3D map. In contrast, we propose Map-free Relocalization, i.e., using only one photo of a scene to enable instant, metric scaled relocalization. Existing datasets are not suitable to benchmark map-free relocalization, due to their focus on large scenes or their limited variability. Thus, we have constructed a new dataset of 655 small places of interest, such as sculptures, murals and fountains, collected worldwide. Each place comes with a reference image to serve as a relocalization anchor, and dozens of query images with known, metric camera poses. The dataset features changing conditions, stark viewpoint changes, high variability across places, and queries with low to no visual overlap with the reference image. We identify two viable families of existing methods to provide baseline results: relative pose regression, and feature matching combined with single-image depth prediction. While these methods show reasonable performance on some favorable scenes in our dataset, map-free relocalization proves to be a challenge that requires new, innovative solutions.
PaperSupplementarXivDatasetCodeBibTeX
Map-free Visual Relocation Paper Thumbnail

Dataset

We make available a dataset of 655 places, collected by non-expert users world-wide. The full dataset is publicly available here.

Animated image showing example scenes from the dataset.

Benchmark

We propose new metrics based on the reprojection error of virtual objects. To successfully compete in our benchmark, methods need to be precise but also provide a robust confidence estimate of their predictions. You can submit to our public leaderboard here.

BibTeX

Please cite our paper if you use our dataset or code.

@inproceedings{arnold2022mapfree, title={Map-free Visual Relocalization: Metric Pose Relative to a Single Image}, author={Arnold, Eduardo and Wynn, Jamie and Vicente, Sara and Garcia-Hernando, Guillermo and Monszpart, {\'{A}}ron and Prisacariu, Victor Adrian and Turmukhambetov, Daniyar and Brachmann, Eric}, booktitle={ECCV}, year={2022}, }