REVERIE Challenge

Leaderboard

Note: Technical report template is provided. Please refer to the recent update on the Home page

Note: The 3D Coordinates of viewpoints and objects (GPS) in the dataset cannot be used as input data. This is because the REVERIE challenge aims to encourage developing models that work like we humans. You can use rel_distance, rel_heading, and rel_elevation among viewpoints from the simulator. This means you do not have a global map at the beginning. If you prefer the GPS information, you can use algorithms (eg., SLAM) to estimate it, do pre-exploration, etc.

Note: Make sure you are using the new data. Check whether your object features are consistent with objects listed in the BBoxes folder.

Channel 1

Team	Nav-Succ	Nav-OSucc	Nav-SPL	Nav-Length	RGS	RGSPL
Baseline_Ch1	14.44	38.67	9.02	23.6	5.16	3.27
BPT	65.84	71.83	50.24	17.72	22.53	17.09
Navigo^*	63.1	70.43	47.0	19.09	22.17	17.02
TouchFish	61.49	67.48	45.75	20.1	22.36	16.81
ProjectD	61.39	66.6	44.99	21.14	21.19	15.7
TeamKeio1	43.42	58.02	29.67	18.21	14.71	10.33
Lancer	41.46	54.72	28.22	19.64	12.53	9.52
NorthWarrior	41.28	53.72	28.17	19.88	13.03	9.48
GrooBoys	39.83	55.64	27.36	19.26	12.5	9.44
GingerGin	39.03	53.36	27.05	19.75	13.21	9.37
Dragon	38.66	48.27	26.07	16.62	12.98	9.25
Reapers	36.72	44.15	25.43	13.44	11.64	9.11
NDGMa	28.88	35.35	25.64	11.29	12.45	9.09
Kzio	37.44	42.07	23.08	18.42	11.37	9.08
Bevo	36.88	46.65	24.08	17.63	11.75	9.05
FirstFleet	29.62	37.82	17.54	15.23	11.56	9.02
Toath	35.54	44.63	21.06	15.07	11.21	8.95
EBmonk	35.68	41.33	24.22	15.58	12.23	8.93
Phoenix	36.46	42.06	24.76	16.1	12.18	8.89
Aces	35.23	40.38	23.19	16.38	10.64	8.67
Arvehol	34.19	43.52	25.65	14.01	10.44	8.52
Alpha	28.63	42.27	19.22	15.38	10.67	8.48
Scorpion	28.52	44.81	20.12	14.7	10.71	8.47
Saga	28.46	41.35	20.05	15.23	11.03	8.44
Commando	28.37	42.63	21.38	13.17	10.33	8.19
Thunder	28.05	40.82	19.26	14.43	10.14	8.07
ThreeZero	28.23	39.56	20.19	13.38	9.96	7.47
Dun	27.43	30.08	19.83	13.42	9.83	7.45
Zong	27.36	31.52	21.27	12.61	9.68	7.41
Hemster	28.17	34.35	20.64	13.29	9.72	7.4
Xpress	27.53	38.29	19.37	14.55	9.68	7.38
TurfDog	26.92	35.46	19.67	14.81	9.81	7.26
Chiladas	26.64	32.43	21.44	13.75	9.44	6.35
Gafferty	26.33	34.28	20.36	14.11	8.52	6.24
Sannas	25.48	33.63	20.14	12.06	8.48	6.09
Titan	24.33	32.65	18.73	14.02	8.43	6.05
Reds	24.48	31.64	18.22	12.76	8.26	6.03
Beast	25.13	33.87	17.85	14.2	7.92	6.01
Genesis	25.32	34.59	19.02	13.52	7.86	5.56
Vapo	25.76	34.41	19.28	11.63	7.83	5.52
Noel	25.63	32.25	18.37	10.84	7.92	5.46
Elves	25.59	34.46	18.39	11.26	7.71	5.42
Blizzard	24.45	33.53	18.26	11.57	7.46	5.39
DesertEye	25.37	30.39	17.27	12.22	7.24	5.33
Shophy	24.58	34.69	18.36	11.73	7.42	5.28
Chub	26.46	33.38	22.58	10.73	7.13	5.21
Grater	25.47	35.66	19.45	11.81	6.95	5.04
Bullet	25.51	36.26	21.63	12.47	6.88	5.01
Yamball	23.65	30.72	18.14	13.45	6.71	4.85
KaiPai	22.39	27.61	15.23	12.84	6.56	4.74
KMA	20.23	25.29	16.19	11.56	6.35	4.52
Braver	20.21	24.35	16.01	12.46	6.59	4.13
YeongJ	18.54	36.24	10.27	20.65	6.23	4.04

Note: * Navigo uses ground truth depth to exclude objects that are 3 meters away from each viewpoint.

Channel 2

Team	Nav-Succ	Nav-OSucc	Nav-SPL	Nav-Length	RGS	RGSPL
Baseline_Ch2	14.44	38.67	9.02	23.6	4.9	3.1
BPT	65.84	71.83	50.24	17.72	37.21	28.36
Navigo^*	63.1	70.43	47.0	19.09	35.4	26.47
TouchFish	61.49	67.48	45.75	20.1	34.77	25.77
ProjectD	61.39	66.6	44.99	21.14	33.79	24.89
Findwell	51.95	57.19	37.82	17.37	23.16	16.52
TeamKeio2	43.42	58.02	29.67	18.21	15.85	10.96
Bumblebee	42.03	57.36	28.62	18.05	15.14	10.31
FirstFleet	41.12	52.15	28.45	18.26	15.02	10.26
Lancer	41.46	54.72	28.22	19.64	14.86	10.2
NorthWarrior	41.28	53.72	28.17	19.88	14.75	10.16
GrooBoys	39.83	55.64	27.36	19.26	14.72	10.14
GingerGin	39.03	53.36	27.05	19.75	14.61	10.11
Dragon	38.66	48.27	26.07	16.62	14.35	9.92
Reapers	36.72	44.15	25.43	13.44	13.65	9.88
EBmonk	35.68	41.33	24.22	15.58	13.14	9.41
Phoenix	36.46	42.06	24.76	16.1	13.29	9.33
Aces	35.23	40.38	23.19	16.38	14.43	9.26
Alpha	28.63	42.27	19.22	15.38	12.11	8.89
Scorpion	28.52	44.81	20.12	14.7	11.92	8.64
Saga	28.46	41.35	20.05	15.23	11.43	8.51
Commando	28.37	42.63	21.38	13.17	11.75	8.49
Thunder	28.05	40.82	19.26	14.43	11.66	8.13
ThreeZero	28.23	39.56	20.19	13.38	11.63	7.66
Zong	27.36	31.52	21.27	12.61	11.28	7.62
Hemster	28.17	34.35	20.64	13.29	11.43	7.45
Xpress	27.53	38.29	19.37	14.55	11.15	7.22
TurfDog	26.92	35.46	19.67	14.81	10.79	7.18
Chiladas	26.64	32.43	21.44	13.75	10.38	6.81
Gafferty	26.33	34.28	20.36	14.11	10.26	6.74
Titan	24.33	32.65	18.73	14.02	9.89	6.55
Yamball	23.65	30.72	18.14	13.45	9.81	6.43
Reds	24.48	31.64	18.22	12.76	9.77	6.35
Beast	25.13	33.87	17.85	14.2	9.67	6.26
Genesis	25.32	34.59	19.02	13.52	9.41	5.83
Vapo	25.76	34.41	19.28	11.63	9.41	5.74
Noel	25.63	32.25	18.37	10.84	9.32	5.65
Elves	25.59	34.46	18.39	11.26	9.35	5.47
Blizzard	24.45	33.53	18.26	11.57	9.28	5.44
DesertEye	25.37	30.39	17.27	12.22	9.19	5.38
Shophy	24.58	34.69	18.36	11.73	9.31	5.31
Chub	26.46	33.38	22.58	10.73	9.43	5.26
Grater	25.47	35.66	19.45	11.81	9.25	5.13
Bullet	25.51	36.26	21.63	12.47	9.37	5.06
KMA	20.23	25.29	16.19	11.56	8.63	4.99

Note: * Navigo uses ground truth depth to exclude objects that are 3 meters away from each viewpoint.