Duckietown Challenges Home Challenges Submissions

Evaluator 1501

ID1501
evaluatorp-12577
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processp-12577
versiond-c:4.0.30;d-c-r:4.0.47;d-s:4.0.22
first heard
last heard
statusinactive
# evaluating
# success5 23823
# timeout
# failed
# error3 23880
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
239923785Andrés Calderón 🇨🇴challenge-aido_LF-template-randomaido2-LFV-sim-testing-offseasonstep1-simulationsuccessyesp-125770:17:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.47061816984274774
survival_time_median2.4499999999999993
deviation-center-line_median0.09047885846140293
in-drivable-lane_median0.7999999999999974


other stats
agent_compute-ego_max0.16195418569776748
agent_compute-ego_mean0.14893476083009488
agent_compute-ego_median0.1494685725161904
agent_compute-ego_min0.13464151109967912
deviation-center-line_max0.1743913456502906
deviation-center-line_mean0.11122054568266285
deviation-center-line_min0.05649185041622204
deviation-heading_max0.8583234710357402
deviation-heading_mean0.371509805944231
deviation-heading_median0.3477435217829391
deviation-heading_min0.1892537419842091
driven_any_max1.6169932854589035
driven_any_mean0.8549136794030617
driven_any_median0.9441796518284096
driven_any_min0.28668385087664994
driven_lanedir_consec_max1.2433038249708377
driven_lanedir_consec_mean0.5622432991481365
driven_lanedir_consec_min0.24216121529728207
driven_lanedir_max1.2433038249708377
driven_lanedir_mean0.5622432991481365
driven_lanedir_median0.47061816984274774
driven_lanedir_min0.24216121529728207
in-drivable-lane_max1.3499999999999996
in-drivable-lane_mean0.5899999999999992
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.098434845606486, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.059495872921413846, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.142761265790021, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.07903329531351726, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.03538123766581217, "sim_compute_performance-ego": 0.06434091815242061, "sim_compute_robot_state-ego": 0.06348320731410274, "sim_compute_robot_state-npc0": 0.0721505880355835, "sim_compute_robot_state-npc1": 0.06678118529143157, "sim_compute_robot_state-npc2": 0.06628216637505426, "sim_compute_robot_state-npc3": 0.06355345690691913}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.4119037580118998, "sim_physics": 0.10813209533691406, "survival_time": 1.2500000000000004, "driven_lanedir": 0.3792793194470208, "sim_render-ego": 0.058473615646362304, "in-drivable-lane": 0, "agent_compute-ego": 0.13845546722412108, "deviation-heading": 0.47860435001913393, "set_robot_commands": 0.07622230529785157, "deviation-center-line": 0.09344288799285194, "driven_lanedir_consec": 0.3792793194470208, "sim_compute_sim_state": 0.03380580902099609, "sim_compute_performance-ego": 0.06336206436157227, "sim_compute_robot_state-ego": 0.06494471549987793, "sim_compute_robot_state-npc0": 0.06986496925354004, "sim_compute_robot_state-npc1": 0.06791878700256347, "sim_compute_robot_state-npc2": 0.0651912784576416, "sim_compute_robot_state-npc3": 0.0635894012451172}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.579862082650716, "sim_physics": 0.11118504561875996, "survival_time": 3.7999999999999945, "driven_lanedir": 1.2433038249708377, "sim_render-ego": 0.060922051730908845, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.14528209598440872, "deviation-heading": 0.5020407942599273, "set_robot_commands": 0.07671091744774267, "deviation-center-line": 0.1743913456502906, "driven_lanedir_consec": 1.2433038249708377, "sim_compute_sim_state": 0.035580989561582864, "sim_compute_performance-ego": 0.06507094910270289, "sim_compute_robot_state-ego": 0.06163421116377178, "sim_compute_robot_state-npc0": 0.07169790644394725, "sim_compute_robot_state-npc1": 0.06657604167335912, "sim_compute_robot_state-npc2": 0.06442023578443025, "sim_compute_robot_state-npc3": 0.06364550402289942}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6169932854589035, "sim_physics": 0.11549764041659198, "survival_time": 3.949999999999994, "driven_lanedir": 1.2126117310470583, "sim_render-ego": 0.05743617045728466, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.1423206268986569, "deviation-heading": 0.34764983431811214, "set_robot_commands": 0.07657371291631385, "deviation-center-line": 0.11519544839797546, "driven_lanedir_consec": 1.2126117310470583, "sim_compute_sim_state": 0.035107706166520904, "sim_compute_performance-ego": 0.06335013425802882, "sim_compute_robot_state-ego": 0.06214124945145619, "sim_compute_robot_state-npc0": 0.06792566444300398, "sim_compute_robot_state-npc1": 0.06451804124856297, "sim_compute_robot_state-npc2": 0.06461625763132603, "sim_compute_robot_state-npc3": 0.06397004972530317}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.5031772882846431, "sim_physics": 0.11966936077390398, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4926325270527603, "sim_render-ego": 0.05689077717917306, "in-drivable-lane": 0, "agent_compute-ego": 0.13464151109967912, "deviation-heading": 0.29868161596444953, "set_robot_commands": 0.078994163445064, "deviation-center-line": 0.09047885846140293, "driven_lanedir_consec": 0.4926325270527603, "sim_compute_sim_state": 0.03447842597961426, "sim_compute_performance-ego": 0.06233819893428257, "sim_compute_robot_state-ego": 0.06447656665529523, "sim_compute_robot_state-npc0": 0.06351843902042933, "sim_compute_robot_state-npc1": 0.06100188834326608, "sim_compute_robot_state-npc2": 0.06285929679870605, "sim_compute_robot_state-npc3": 0.06064178262438093}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0440585337030532, "sim_physics": 0.0943479714570222, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5009536316076031, "sim_render-ego": 0.058773963539688674, "in-drivable-lane": 1.0000000000000009, "agent_compute-ego": 0.14261891223766185, "deviation-heading": 0.8583234710357402, "set_robot_commands": 0.07721523002341941, "deviation-center-line": 0.1636702708892767, "driven_lanedir_consec": 0.5009536316076031, "sim_compute_sim_state": 0.03520396903709129, "sim_compute_performance-ego": 0.060763001441955566, "sim_compute_robot_state-ego": 0.060536336015771935, "sim_compute_robot_state-npc0": 0.06925196559340865, "sim_compute_robot_state-npc1": 0.06580201784769694, "sim_compute_robot_state-npc2": 0.06350919052406594, "sim_compute_robot_state-npc3": 0.06353263060251872}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.045672984989961, "sim_physics": 0.10152894037741202, "survival_time": 2.6999999999999984, "driven_lanedir": 0.636422849544542, "sim_render-ego": 0.06553931147963912, "in-drivable-lane": 0.7999999999999974, "agent_compute-ego": 0.15083931110523366, "deviation-heading": 0.39273667682101354, "set_robot_commands": 0.08015033933851454, "deviation-center-line": 0.08970073753261001, "driven_lanedir_consec": 0.636422849544542, "sim_compute_sim_state": 0.03902817655492712, "sim_compute_performance-ego": 0.06703551168794986, "sim_compute_robot_state-ego": 0.06525459554460314, "sim_compute_robot_state-npc0": 0.07541121818401196, "sim_compute_robot_state-npc1": 0.07293412862000642, "sim_compute_robot_state-npc2": 0.06643659980208785, "sim_compute_robot_state-npc3": 0.06721416667655662}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.877256228857091, "sim_physics": 0.12114576233757866, "survival_time": 2.25, "driven_lanedir": 0.38713958773684665, "sim_render-ego": 0.06778159671359592, "in-drivable-lane": 0.9999999999999996, "agent_compute-ego": 0.16195418569776748, "deviation-heading": 0.384474040685185, "set_robot_commands": 0.07862180603875055, "deviation-center-line": 0.07500609722918031, "driven_lanedir_consec": 0.38713958773684665, "sim_compute_sim_state": 0.03811475965711805, "sim_compute_performance-ego": 0.06973230044047038, "sim_compute_robot_state-ego": 0.06558503574795194, "sim_compute_robot_state-npc0": 0.07868244383070204, "sim_compute_robot_state-npc1": 0.07264242172241211, "sim_compute_robot_state-npc2": 0.06946191257900662, "sim_compute_robot_state-npc3": 0.06831898159450955}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.8433010974171573, "sim_physics": 0.0977436874223792, "survival_time": 2.3, "driven_lanedir": 0.24216121529728207, "sim_render-ego": 0.061410447825556214, "in-drivable-lane": 1.3499999999999996, "agent_compute-ego": 0.1456380097762398, "deviation-heading": 0.34566993689777425, "set_robot_commands": 0.0757247313209202, "deviation-center-line": 0.07136114712800473, "driven_lanedir_consec": 0.24216121529728207, "sim_compute_sim_state": 0.03687188936316449, "sim_compute_performance-ego": 0.06270873546600342, "sim_compute_robot_state-ego": 0.06276091285373854, "sim_compute_robot_state-npc0": 0.0712993455969769, "sim_compute_robot_state-npc1": 0.06582456568013066, "sim_compute_robot_state-npc2": 0.06528803058292555, "sim_compute_robot_state-npc3": 0.06575172362120255}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.9785013113851152, "sim_physics": 0.1155336093902588, "survival_time": 2.499999999999999, "driven_lanedir": 0.9700080716683566, "sim_render-ego": 0.06653674602508546, "in-drivable-lane": 0, "agent_compute-ego": 0.15282089233398438, "deviation-heading": 0.3477435217829391, "set_robot_commands": 0.07943522930145264, "deviation-center-line": 0.15723242276422936, "driven_lanedir_consec": 0.9700080716683566, "sim_compute_sim_state": 0.03899794101715088, "sim_compute_performance-ego": 0.06848310947418212, "sim_compute_robot_state-ego": 0.0649396800994873, "sim_compute_robot_state-npc0": 0.0738996410369873, "sim_compute_robot_state-npc1": 0.07029154300689697, "sim_compute_robot_state-npc2": 0.0699022388458252, "sim_compute_robot_state-npc3": 0.07043350219726563}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.3129292149411446, "sim_physics": 0.12855516672134398, "survival_time": 1.0000000000000002, "driven_lanedir": 0.2864579763812336, "sim_render-ego": 0.06822384595870971, "in-drivable-lane": 0, "agent_compute-ego": 0.15915240049362184, "deviation-heading": 0.38016104491567027, "set_robot_commands": 0.07915269136428833, "deviation-center-line": 0.05649185041622204, "driven_lanedir_consec": 0.2864579763812336, "sim_compute_sim_state": 0.03647351264953613, "sim_compute_performance-ego": 0.07085148096084595, "sim_compute_robot_state-ego": 0.06359115839004517, "sim_compute_robot_state-npc0": 0.06944658756256103, "sim_compute_robot_state-npc1": 0.06846989393234253, "sim_compute_robot_state-npc2": 0.0645280122756958, "sim_compute_robot_state-npc3": 0.06678223609924316}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.28668385087664994, "sim_physics": 0.10330082240857576, "survival_time": 0.9500000000000004, "driven_lanedir": 0.278237107094367, "sim_render-ego": 0.06423887453581158, "in-drivable-lane": 0, "agent_compute-ego": 0.1494685725161904, "deviation-heading": 0.19732996798555608, "set_robot_commands": 0.0770362050909745, "deviation-center-line": 0.08671144489080887, "driven_lanedir_consec": 0.278237107094367, "sim_compute_sim_state": 0.041722548635382405, "sim_compute_performance-ego": 0.06866690987034847, "sim_compute_robot_state-ego": 0.07016719015021074, "sim_compute_robot_state-npc0": 0.07581556470770585, "sim_compute_robot_state-npc1": 0.06987025863245915, "sim_compute_robot_state-npc2": 0.07053661346435547, "sim_compute_robot_state-npc3": 0.07079967699552837}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0034139953700494, "sim_physics": 0.09183900356292723, "survival_time": 2.499999999999999, "driven_lanedir": 0.47061816984274774, "sim_render-ego": 0.0654967498779297, "in-drivable-lane": 1.1499999999999986, "agent_compute-ego": 0.15745673656463624, "deviation-heading": 0.2263485117662671, "set_robot_commands": 0.07872557640075684, "deviation-center-line": 0.1594122934983136, "driven_lanedir_consec": 0.47061816984274774, "sim_compute_sim_state": 0.04023173809051514, "sim_compute_performance-ego": 0.06524239540100098, "sim_compute_robot_state-ego": 0.06293861865997315, "sim_compute_robot_state-npc0": 0.07552199840545654, "sim_compute_robot_state-npc1": 0.07293922424316407, "sim_compute_robot_state-npc2": 0.07353196144104004, "sim_compute_robot_state-npc3": 0.0702164363861084}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.351442944128162, "sim_physics": 0.11050080169330943, "survival_time": 1.1000000000000003, "driven_lanedir": 0.34114611245347204, "sim_render-ego": 0.06071734428405762, "in-drivable-lane": 0, "agent_compute-ego": 0.1528966101733121, "deviation-heading": 0.1892537419842091, "set_robot_commands": 0.07854348962957208, "deviation-center-line": 0.08918370733714856, "driven_lanedir_consec": 0.34114611245347204, "sim_compute_sim_state": 0.037165294994007454, "sim_compute_performance-ego": 0.066114219752225, "sim_compute_robot_state-ego": 0.06207285144112327, "sim_compute_robot_state-npc0": 0.07244491577148438, "sim_compute_robot_state-npc1": 0.06554003195329146, "sim_compute_robot_state-npc2": 0.06620912118391557, "sim_compute_robot_state-npc3": 0.06780147552490234}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.9441796518284096, "sim_physics": 0.11013887366470024, "survival_time": 2.4499999999999993, "driven_lanedir": 0.4203463405922858, "sim_render-ego": 0.06756215192833726, "in-drivable-lane": 1.0999999999999988, "agent_compute-ego": 0.15771481455588826, "deviation-heading": 0.4296166116434236, "set_robot_commands": 0.07924574248644771, "deviation-center-line": 0.08695038592754878, "driven_lanedir_consec": 0.4203463405922858, "sim_compute_sim_state": 0.0385951752565345, "sim_compute_performance-ego": 0.0699060109196877, "sim_compute_robot_state-ego": 0.064037950671449, "sim_compute_robot_state-npc0": 0.07558796843703912, "sim_compute_robot_state-npc1": 0.07142187137993014, "sim_compute_robot_state-npc2": 0.06934737672611159, "sim_compute_robot_state-npc3": 0.06916726365381358}}
set_robot_commands_max0.08015033933851454
set_robot_commands_mean0.07809236236103907
set_robot_commands_median0.07862180603875055
set_robot_commands_min0.0757247313209202
sim_compute_performance-ego_max0.07085148096084595
sim_compute_performance-ego_mean0.06586439601491177
sim_compute_performance-ego_median0.06524239540100098
sim_compute_performance-ego_min0.060763001441955566
sim_compute_robot_state-ego_max0.07016719015021074
sim_compute_robot_state-ego_mean0.06390428531059054
sim_compute_robot_state-ego_median0.06359115839004517
sim_compute_robot_state-ego_min0.060536336015771935
sim_compute_robot_state-npc0_max0.07868244383070204
sim_compute_robot_state-npc0_mean0.07216794775485585
sim_compute_robot_state-npc0_median0.0721505880355835
sim_compute_robot_state-npc0_min0.06351843902042933
sim_compute_robot_state-npc1_max0.07293922424316407
sim_compute_robot_state-npc1_mean0.06816879337183424
sim_compute_robot_state-npc1_median0.06791878700256347
sim_compute_robot_state-npc1_min0.06100188834326608
sim_compute_robot_state-npc2_max0.07353196144104004
sim_compute_robot_state-npc2_mean0.06680801949814587
sim_compute_robot_state-npc2_median0.06620912118391557
sim_compute_robot_state-npc2_min0.06285929679870605
sim_compute_robot_state-npc3_max0.07079967699552837
sim_compute_robot_state-npc3_mean0.06636121919175125
sim_compute_robot_state-npc3_median0.06678223609924316
sim_compute_robot_state-npc3_min0.06064178262438093
sim_compute_sim_state_max0.041722548635382405
sim_compute_sim_state_mean0.03711727824333025
sim_compute_sim_state_median0.03687188936316449
sim_compute_sim_state_min0.03380580902099609
sim_physics_max0.12855516672134398
sim_physics_mean0.10850357511921092
sim_physics_median0.11013887366470024
sim_physics_min0.09183900356292723
sim_render-ego_max0.06822384595870971
sim_render-ego_mean0.06263330134023694
sim_render-ego_median0.061410447825556214
sim_render-ego_min0.05689077717917306
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.2366666666666655
survival_time_min0.9500000000000004
No reset possible
239813780Yeong-Jia ChuBaseline solution using reinforcement learningaido2-LF-sim-validation-offseasonstep1-simulationerroryesp-125770:02:02
The container "solut [...]
The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
239713737jiang pengFinal test - TH-jpaido2-LFVI-real-validationeval1-videos-watchtowerssuccessyesp-125772:22:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
239703737jiang pengFinal test - TH-jpaido2-LFVI-real-validationeval1-videos-autobotssuccessyesp-125770:13:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
239433549Bea Baselines 🐤Running the real lane following -- without challenge containeraido2-LF-real-validationeval1-videos-watchtowerserroryesp-125773:00:32
Waited 10801 seconds [...]
Waited 10801 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
239403549Bea Baselines 🐤Running the real lane following -- without challenge containeraido2-LF-real-validationeval1-videos-autobotssuccessyesp-125770:07:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238803701jiang pengrobotarium test- cropaido2-LF-real-validationeval0-videos-autobotserroryesp-125773:00:54
Waited 10800 seconds [...]
Waited 10800 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
238233758Julian ZillyBaseline-IL-logs-tensorflowaido2-LFV-real-validationeval0-videos-watchtowerssuccessyesp-125771:57:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible