AI Driving Olympics Home Dashboard Logs Challenges Submissions Evaluators Jobs (TODO) Users Docs

Welcome to the NIPS 2018 AI Driving Olympics (AI-DO) challenges server.

You can find an introduction to AI-DO here.

The reference manual will get you started.

AI-DO is brought to you by:

News and updates

2018-12-11AI-DO 1 at NeurIPS report. Congratulations to our winners!
2018-12-04AI-DO1 Submission Deadline: Thursday Dec 6 at 11:59pm PST
2018-11-07Update to challenge LF, LFV evaluation code
2018-10-31Server capacity increased; leaderboards logic updated.
2018-10-30AI-DO I Interactive Tutorials
2018-10-24Updated duckietown-challenges server fixes speed problems; updated “dts commands evaluate”
2018-10-20Problems with local evaluator (dts challenges evaluate)
2018-10-14AI Driving Olympics Ready for Submissions
2018-07-13Announcing the AI Driving Olympics (AI-DO)

AI-DO 1 - Embodied tasks

In these challenges, the participants must guide a Duckiebot through the streets of Duckietown. The only input available is a camera.

Repositories
Challenge definition challenge-aido1_LF1
Submission templates
Agent template- challenge-aido1_LF1-template-random
Pytorch templatePyTorch challenge-aido1_LF1-template-pytorch
Tensorflow templateTensorflow challenge-aido1_LF1-template-tensorflow
ROS templateROS challenge-aido1_LF1-template-ros
Baseline solutions
Imitation learning from real logsTensorflow challenge-aido1_LF1-baseline-IL-logs-tensorflow
Imitation learning from simulationTensorflow challenge-aido1_LF1-baseline-IL-sim-tensorflow
Reinforcement learning from simulationPyTorch challenge-aido1_LF1-baseline-RL-sim-pytorch
Traditional Duckietown pipelineROS challenge-aido1_LF1-baseline-duckietown

Challenge LF - Lane following (rev. 3)

RankUserSubmissionUser labelTraveled distanceSurvival timeLateral deviationMajor infractions
1WEIGAO1633First trial18.618.0.580.2
2heyt0ny1579SAIC MOSCOW MML18.618.0.870.4
3miksaz1859JetBrains Research18.618.1.340.2
4Jon1684JP pipeline16.7418.1.051.8
5VincentMai1033ROS-based Lane Following14.8818.1.861.8
6Benjamin1063My ROS solution14.2618.0.910.6
7lavoiems819Baby Duke13.0218.0.892.2
8David1139Pytorch IL12.418.1.491.4
9orlando.m.bol@gmail.com436PyTorch DDPG template9.9218.1.381.4
10placaille985Dolores' Awakening8.6810.0.590.8
11hvrigazov1552Visteon perception team7.4418.0.911.
12Allen Ou590Tensorflow template5.5810.0.560.2
13tongqiusui1363Tensorflow template5.5810.0.560.2
14shandonguniversityInspur1482Inspur CNN5.5810.0.560.2
15yangzm111491Tensorflow template5.5810.0.560.2

See also the extended leaderboard.

Challenge LFV - Lane following with vehicles (rev. 1)

RankUserSubmissionUser labelTraveled distanceSurvival timeLateral deviationMajor infractions
1heyt0ny1779SAIC MOSCOW MML1.2418.0.260.
2WEIGAO2051Panasonic R&D Center Singapore & NUS0.6218.0.250.4
3sodabeta1997Copy of #20: sub 903 by sodabeta (PyTorch DDPG template)0.6218.0.458.4
4pravishsainath1998Copy of #21: sub 1085 by pravishsainath (PyTorch DDPG template)0.6218.0.458.4
5anna.tsalapova2027Copy of #50: sub 396 by anna.tsalapova (simple submition)0.6218.0.640.
6Sashkasem2028Copy of #51: sub 428 by Sashkasem (Random execution)0.6218.0.640.
7David1893Pytorch IL0.6218.0.643.2
8orlando.m.bol@gmail.com1987Copy of #9: sub 436 by orlando.m.bol@gmail.com (PyTorch DDPG template)0.6218.1.11.8
9gunshi1999Copy of #22: sub 1268 by gunshi (ROS-based Lane Following)0.6218.2.090.
10Benjamin1984Copy of #6: sub 1063 by Benjamin (My ROS solution)0.6218.2.230.
11krishna2002Copy of #25: sub 1143 by krishna (gym_duckietown + opencv)0.628.0.41.6
12Jon668JP pipeline0.626.0.132.
13hvrigazov1989Copy of #11: sub 1552 by hvrigazov (Visteon perception team)0.626.0.320.8
14trimcao1627Tensorflow template0.626.0.382.
15wenhui2033Copy of #56: sub 1386 by wenhui (ROS-based Lane Following)0.626.0.40.4

See also the extended leaderboard.

AI-DO 1 - Autonomous mobility on demand

In these challenges, the participant must coordinate a fleet of robo-taxis in various cities environments.

Challenge [AIDO 1] AMOD - Efficiency (rev. 1)

RankUserSubmissionUser labelefficiency
1lic1883super fishy-11.88762
2JoelGaechter1696greedy but not so lazy-34.87095

See also the extended leaderboard.

Challenge [AIDO 1] AMOD - Service Quality (rev. 1)

RankUserSubmissionUser labelservice_quality
1lic1885inaccessible code but it scores better than without-20.224397
2JoelGaechter1697greedy but not so lazy-20.554737

See also the extended leaderboard.

Challenge [AIDO 1] AMOD - Fleet Size (rev. 1)

RankUserSubmissionUser labelfleet_size
1rumpelstilzchen1665Simsalabim-1000000000
2lic1879freq no rebalance-1000000000
3JoelGaechter1698greedy but not so lazy-1000000000

See also the extended leaderboard.

Test challenges

Test challenges

Test challenges

Challenge [Test] Try your luck

RankUserSubmissionUser labelscore1
1Edward1712SemSeg_test100
2haliang1719Solution template100
3upkoi1278Solution template95.901604
4brianhw105Solution template95.628742
5vguerra718Solution template92.620089
6tgemicioglu873Solution template91.777224
7yun chen617Solution template89.270452
8miksaz640Solution template87.919199
9Raiszo1629Solution template85.421533
10Maxim Kuzmin107Solution template85.290372
11YIPTSZFUNG1500Solution template83.988095
12hvrigazov404Solution template83.710528
13GWLee1397Solution template83.307442
14hellolai673Solution template79.957535
15stoplight1492Solution template77.201782

See also the extended leaderboard.

Challenge [Test] Multi-step evaluation

No entries to display.

See also the extended leaderboard.

Challenge [Test] Log processing

No entries to display.

See also the extended leaderboard.

Others

Other challenges

Others

Challenge [old] Lane following (rev 1)

RankUserSubmissionUser labelreward
1Andy Ser49Tensorflow template-30.045445
2hchengwang66Tensorflow template-14.072496

See also the extended leaderboard.

Challenge [old] AMOD challenge (rev. 1)

No entries to display.

See also the extended leaderboard.

Challenge [old] Lane following (rev 2)

RankUserSubmissionUser labeltraveled_tiles_mediansurvival_time_meangood_angle_meanvalid_direction_mean
1BrianWang187ROS-based Lane Following47.4266676.2344523.233333
2PolyProgrammist95Random execution316.61333310.12917811.266667
3spetryk222Random execution316.61333311.28566311.546667
4cc2294186Random execution316.61333314.77309112.553333
5ikra248Random execution316.61333319.63598913.373333
6lmandrile249Random execution316.61333319.88039213.566667
7apacheck193Random execution316.61333321.61057913.066667
8emmasung241Random execution316.61333322.02311612.56
9anna.tsalapova94Random execution316.61333323.23038213.473333
10sischaef132Random execution316.61333324.21992413.7
11Jon236Random execution316.61333334.42087612.626667
12jlh457232Random execution312.98666711.3804119.42
13David251Random execution39.6066678.7389328.466667
14MandanaSamiei256Tensorflow template33.10.8540871.093333
15hoiawog137TF-v033.0933330.8407461.053333

See also the extended leaderboard.

Challenge LF - Lane following (rev. 4, more samples)

RankUserSubmissionUser labelTraveled distanceSurvival timeLateral deviationMajor infractions
1heyt0ny2133Copy of #2: sub 1579 by heyt0ny (SAIC MOSCOW MML)19.2218.0.720.
2WEIGAO2132Copy of #1: sub 1633 by WEIGAO (First trial)18.618.0.60.
3Benjamin2137Copy of #6: sub 1063 by Benjamin (My ROS solution)13.6418.1.431.2
4Jon2135Copy of #4: sub 1684 by Jon (JP pipeline)11.1612.0.770.
5VincentMai2136Copy of #5: sub 1033 by VincentMai (ROS-based Lane Following)9.310.0.982.2
6hvrigazov2142Copy of #11: sub 1552 by hvrigazov (Visteon perception team)5.5818.1.020.8
7shandonguniversityInspur2145Copy of #14: sub 1482 by shandonguniversityInspur (Inspur CNN)5.5810.0.380.2
8trimcao2148Copy of #17: sub 1515 by trimcao (Tensorflow template)5.5810.0.380.2
9Allen Ou2143Copy of #12: sub 590 by Allen Ou (Tensorflow template)5.5810.0.430.2
10madelhorius2147Copy of #16: sub 1832 by madelhorius (Tensorflow template)5.5810.0.430.2
11melorian2149Copy of #18: sub 1378 by melorian (Tensorflow template)5.5810.0.430.2
12tongqiusui2144Copy of #13: sub 1363 by tongqiusui (Tensorflow template)5.5810.0.510.
13yangzm112146Copy of #15: sub 1491 by yangzm11 (Tensorflow template)5.5810.0.510.2
14orlando.m.bol@gmail.com2140Copy of #9: sub 436 by orlando.m.bol@gmail.com (PyTorch DDPG template)4.9618.1.051.8
15lavoiems2138Copy of #7: sub 819 by lavoiems (Baby Duke)4.968.0.420.2

See also the extended leaderboard.

Challenge LFV - Lane following with vehicles (rev. 2 - dynamic, 20 samples)

RankUserSubmissionUser labelTraveled distanceSurvival timeLateral deviationMajor infractions
1WEIGAO2073Copy of #1: sub 1633 by WEIGAO (First trial)17.9818.0.310.
2Benjamin2079Copy of #6: sub 1063 by Benjamin (My ROS solution)7.448.1.070.4
3sodabeta2093Copy of #20: sub 903 by sodabeta (PyTorch DDPG template)3.118.0.458.4
4pravishsainath2094Copy of #21: sub 1085 by pravishsainath (PyTorch DDPG template)3.118.0.458.4
5VincentMai2078Copy of #5: sub 1033 by VincentMai (ROS-based Lane Following)3.14.0.250.4
6miksaz2076Copy of #3: sub 1859 by miksaz (JetBrains Research)2.482.0.150.2
7anna.tsalapova2123Copy of #50: sub 396 by anna.tsalapova (simple submition)1.2418.1.710.
8Sashkasem2124Copy of #51: sub 428 by Sashkasem (Random execution)1.2418.1.710.
9haliang2108Copy of #35: sub 784 by haliang (Random execution)1.244.0.120.8
10gunshi2095Copy of #22: sub 1268 by gunshi (ROS-based Lane Following)1.244.0.260.
11Jon2077Copy of #4: sub 1684 by Jon (JP pipeline)1.242.0.060.
12placaille2083Copy of #10: sub 985 by placaille (Dolores Awakening)1.242.0.060.2
13dcharrez2092Copy of #19: sub 1479 by dcharrez (Tensorflow)1.242.0.060.4
14heyt0ny2075Copy of #2: sub 1579 by heyt0ny (SAIC MOSCOW MML)1.242.0.070.
15tongqiusui2086Copy of #13: sub 1363 by tongqiusui (Tensorflow template)1.242.0.120.

See also the extended leaderboard.

AI-DO 1 - outdated versions of challenges

These are previous versions of the AI-DO tasks.

AI-DO 1 - outdated versions of challenges

Challenge [old] Lane following (rev 1)

RankUserSubmissionUser labelreward
1Andy Ser49Tensorflow template-30.045445
2hchengwang66Tensorflow template-14.072496

See also the extended leaderboard.

Challenge [old] AMOD challenge (rev. 1)

No entries to display.

See also the extended leaderboard.

Challenge [old] Lane following (rev 2)

RankUserSubmissionUser labeltraveled_tiles_mediansurvival_time_meangood_angle_meanvalid_direction_mean
1BrianWang187ROS-based Lane Following47.4266676.2344523.233333
2PolyProgrammist95Random execution316.61333310.12917811.266667
3spetryk222Random execution316.61333311.28566311.546667
4cc2294186Random execution316.61333314.77309112.553333
5ikra248Random execution316.61333319.63598913.373333
6lmandrile249Random execution316.61333319.88039213.566667
7apacheck193Random execution316.61333321.61057913.066667
8emmasung241Random execution316.61333322.02311612.56
9anna.tsalapova94Random execution316.61333323.23038213.473333
10sischaef132Random execution316.61333324.21992413.7
11Jon236Random execution316.61333334.42087612.626667
12jlh457232Random execution312.98666711.3804119.42
13David251Random execution39.6066678.7389328.466667
14MandanaSamiei256Tensorflow template33.10.8540871.093333
15hoiawog137TF-v033.0933330.8407461.053333

See also the extended leaderboard.

Challenge LF - Lane following (rev. 4, more samples)

RankUserSubmissionUser labelTraveled distanceSurvival timeLateral deviationMajor infractions
1heyt0ny2133Copy of #2: sub 1579 by heyt0ny (SAIC MOSCOW MML)19.2218.0.720.
2WEIGAO2132Copy of #1: sub 1633 by WEIGAO (First trial)18.618.0.60.
3Benjamin2137Copy of #6: sub 1063 by Benjamin (My ROS solution)13.6418.1.431.2
4Jon2135Copy of #4: sub 1684 by Jon (JP pipeline)11.1612.0.770.
5VincentMai2136Copy of #5: sub 1033 by VincentMai (ROS-based Lane Following)9.310.0.982.2
6hvrigazov2142Copy of #11: sub 1552 by hvrigazov (Visteon perception team)5.5818.1.020.8
7shandonguniversityInspur2145Copy of #14: sub 1482 by shandonguniversityInspur (Inspur CNN)5.5810.0.380.2
8trimcao2148Copy of #17: sub 1515 by trimcao (Tensorflow template)5.5810.0.380.2
9Allen Ou2143Copy of #12: sub 590 by Allen Ou (Tensorflow template)5.5810.0.430.2
10madelhorius2147Copy of #16: sub 1832 by madelhorius (Tensorflow template)5.5810.0.430.2
11melorian2149Copy of #18: sub 1378 by melorian (Tensorflow template)5.5810.0.430.2
12tongqiusui2144Copy of #13: sub 1363 by tongqiusui (Tensorflow template)5.5810.0.510.
13yangzm112146Copy of #15: sub 1491 by yangzm11 (Tensorflow template)5.5810.0.510.2
14orlando.m.bol@gmail.com2140Copy of #9: sub 436 by orlando.m.bol@gmail.com (PyTorch DDPG template)4.9618.1.051.8
15lavoiems2138Copy of #7: sub 819 by lavoiems (Baby Duke)4.968.0.420.2

See also the extended leaderboard.

Challenge LFV - Lane following with vehicles (rev. 2 - dynamic, 20 samples)

RankUserSubmissionUser labelTraveled distanceSurvival timeLateral deviationMajor infractions
1WEIGAO2073Copy of #1: sub 1633 by WEIGAO (First trial)17.9818.0.310.
2Benjamin2079Copy of #6: sub 1063 by Benjamin (My ROS solution)7.448.1.070.4
3sodabeta2093Copy of #20: sub 903 by sodabeta (PyTorch DDPG template)3.118.0.458.4
4pravishsainath2094Copy of #21: sub 1085 by pravishsainath (PyTorch DDPG template)3.118.0.458.4
5VincentMai2078Copy of #5: sub 1033 by VincentMai (ROS-based Lane Following)3.14.0.250.4
6miksaz2076Copy of #3: sub 1859 by miksaz (JetBrains Research)2.482.0.150.2
7anna.tsalapova2123Copy of #50: sub 396 by anna.tsalapova (simple submition)1.2418.1.710.
8Sashkasem2124Copy of #51: sub 428 by Sashkasem (Random execution)1.2418.1.710.
9haliang2108Copy of #35: sub 784 by haliang (Random execution)1.244.0.120.8
10gunshi2095Copy of #22: sub 1268 by gunshi (ROS-based Lane Following)1.244.0.260.
11Jon2077Copy of #4: sub 1684 by Jon (JP pipeline)1.242.0.060.
12placaille2083Copy of #10: sub 985 by placaille (Dolores Awakening)1.242.0.060.2
13dcharrez2092Copy of #19: sub 1479 by dcharrez (Tensorflow)1.242.0.060.4
14heyt0ny2075Copy of #2: sub 1579 by heyt0ny (SAIC MOSCOW MML)1.242.0.070.
15tongqiusui2086Copy of #13: sub 1363 by tongqiusui (Tensorflow template)1.242.0.120.

See also the extended leaderboard.