[ECCV '24 Workshop Baseline]: Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences, without using multiple images or device poses.
Only using the last frame in the query sequence, not using device poses.

[Baseline] MicKey (using only a single frame)

Map-Free Visual Relocalization Enhanced by Instance Knowledge and Depth Knowledge use only single frame

Map-Free Visual Relocalization Enhanced by Instance Knowledge and Depth Knowledge

Forward with (ref, query) gives pointmaps and pixel matches. These latter ones are used for Ess. matrix estimation, and the scale is obtained from the predicted pointmaps.
(re-submission of previous results)

MASt3R (Ess.Mat + D.Scale)

Metric3d for 3d scaled depth estimation and loftr for initial patch matching.Then an geometric pnp algorithm is used for pose estimation.

interp_metric3d_loftr_3d2d

[ECCV '24 Workshop Baseline]: RoMa correspondences with MicKey depth maps predictions.

RoMa w/ MicKey depth maps

Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences

MicKey trained w/ Overlap Score

MicKey

[ECCV '24 Workshop Baseline]: SuperPoint + SuperGlue correspondences with MicKey depth maps predictions.

SuperGlue w/ MicKey depth maps

[ECCV '24 Workshop Baseline]: LoFTR correspondences with MicKey depth maps predictions.

LoFTR w/ MicKey depth maps

Using the depth map from the Depth Anything Model as the ground truth, we trained the Mickey Variant network in a supervised manner.

Mickey Variant GT_Depth

Our flexible method produces accurate and robust pose estimates using complementary strengths of Correspondence + Solver and Learning-Based methods.

FAR (LoFTR)

DUSt3R --> pixel correspondences between output 3d points

DPT-KITTI & DUSt3R (Ess.Mat. + D.Scale)

FAR (SuperGlue)

Same as loftr one, 2px opencv MAGSAC etc.

DPT-KITTI & RoMa (Ess.Mat. + D.Scale) Note: maybe buggy

We replace the baseline DPT-KITTI with KBR++ depth predictions. KBR++ is a self-supervised monocular depth estimation system trained on a combination of SlowTV, CribsTV, MannequinChallenge & Kitti. The model is transferred to MapFreeReloc in a zero-shot fashion, without fine-tuning. 

Since KBR++ is trained only on monocular data (ie without a stereo baseline), the predictions are scaleless. Metric-scale predictions are obtained by aligning KBR++ to the provided baseline DPT-KITTI using least-squares.

KBR++ & LoFTR (PnP)

We replace the baseline DPT-KITTI with KBR depth predictions. KBR is a self-supervised monocular depth estimation system trained on a combination of SlowTV, MannequinChallenge & Kitti. The model is transferred to MapFreeReloc in a zero-shot fashion, without fine-tuning. 

Since KBR is trained only on monocular data (ie without a stereo baseline), the predictions are scaleless. Metric-scale predictions are obtained by aligning KBR to the provided baseline DPT-KITTI using least-squares.

KBR & LoFTR (PnP)

[ECCV'24 Workshop Baseline]: ASpanFormer: Detector-Free Image Matching
with Adaptive Span Transformer

DPT-KITTI & ASpanFormer (Ess.Mat. + D.Scale)

DPT-KITTI & LoFTR (PnP)

DPT-KITTI & LoFTR (Ess.Mat. + D.Scale)

DPT-KITTI & SuperGlue (Ess.Mat. + D.Scale)

DPT-KITTI & SuperGlue (PnP)

[ECCV '24 Workshop Baseline]: DISK+LightGlue correspondences with DPT-KITTI depth maps.

DPT-KITTI & LightGlue w/ DISK (Ess.Mat. + D.Scale)

[ECCV '24 Workshop Baseline]: DISK correspondences with DPT-KITTI depth maps.

DPT-KITTI & DISK (Ess.Mat. + D.Scale)

[ECCV'24 Workshop Baseline]: DeDoDe: Detect, Don't Describe -- Describe, Don't Detect for Local Feature Matching

Evaluation Leaderboard: Multi Frame