Duckietown Challenges Home Challenges Submissions

Evaluator 1605

ID1605
evaluatorip-172-31-43-40-10498
ownerI don't have one 😀
machineip-172-31-43-40
processip-172-31-43-40-10498
versiond-c:5.0.7;d-c-r:5.0.11;d-s:5.1.2
first heard
last heard
statusinactive
# evaluating
# success1 25183
# timeout
# failed
# error
# aborted
# host-error3 25151
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
251834404Alex Deliyannischallenge-aido_LF-template-tensorflowaido3-LFV-sim-testingstep1-simulationsuccessnoip-172-31-43-40-104980:14:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.23003459314400532
survival_time_median1.6500000000000008
deviation-center-line_median0.08894019456029295
in-drivable-lane_median0.05000000000000005


other stats
agent_compute-ego_max0.03327251564372669
agent_compute-ego_mean0.02252666163543059
agent_compute-ego_median0.020461422024351177
agent_compute-ego_min0.016345521189131827
deviation-center-line_max0.21481687489116805
deviation-center-line_mean0.10395169084896608
deviation-center-line_min0.059705063081282296
deviation-heading_max1.4907009526321624
deviation-heading_mean0.7294136341146421
deviation-heading_median0.6136881317742018
deviation-heading_min0.2547912413946925
driven_any_max1.430326160194243
driven_any_mean0.5106535859089196
driven_any_median0.3581472762977876
driven_any_min0.17855682642957194
driven_lanedir_consec_max0.6001749872579527
driven_lanedir_consec_mean0.2806256713919438
driven_lanedir_consec_min0.12832135084722973
driven_lanedir_max0.6001749872579527
driven_lanedir_mean0.2806256713919438
driven_lanedir_median0.23003459314400532
driven_lanedir_min0.12832135084722973
in-drivable-lane_max3.299999999999992
in-drivable-lane_mean0.5399999999999987
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.27043209897928394, "sim_physics": 0.23176004669883035, "survival_time": 1.1000000000000003, "driven_lanedir": 0.19727860143273057, "sim_render-ego": 0.013069369576194067, "in-drivable-lane": 0, "agent_compute-ego": 0.03327251564372669, "deviation-heading": 0.5608989934738143, "set_robot_commands": 0.010518789291381836, "deviation-center-line": 0.0757712585572722, "driven_lanedir_consec": 0.19727860143273057, "sim_compute_sim_state": 0.005554188381541859, "sim_compute_performance-ego": 0.007183259183710272, "sim_compute_robot_state-ego": 0.009626106782393022, "sim_compute_robot_state-npc0": 0.008667945861816406, "sim_compute_robot_state-npc1": 0.009029171683571556, "sim_compute_robot_state-npc2": 0.009210098873485218, "sim_compute_robot_state-npc3": 0.008555163036693226}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9562843751334116, "sim_physics": 0.21355517705281576, "survival_time": 3.7499999999999942, "driven_lanedir": 0.4603965528459262, "sim_render-ego": 0.01293313980102539, "in-drivable-lane": 0.849999999999997, "agent_compute-ego": 0.028295389811197915, "deviation-heading": 1.4907009526321624, "set_robot_commands": 0.010231068929036458, "deviation-center-line": 0.21481687489116805, "driven_lanedir_consec": 0.4603965528459262, "sim_compute_sim_state": 0.004871060053507487, "sim_compute_performance-ego": 0.007244462966918945, "sim_compute_robot_state-ego": 0.009503615697224936, "sim_compute_robot_state-npc0": 0.008864720662434896, "sim_compute_robot_state-npc1": 0.008379402160644532, "sim_compute_robot_state-npc2": 0.008467658360799154, "sim_compute_robot_state-npc3": 0.008602129618326822}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.24449927571696528, "sim_physics": 0.24614399671554563, "survival_time": 1.0000000000000002, "driven_lanedir": 0.1998281825264865, "sim_render-ego": 0.013545095920562744, "in-drivable-lane": 0.04999999999999993, "agent_compute-ego": 0.030669796466827392, "deviation-heading": 0.2585016887334354, "set_robot_commands": 0.01005462408065796, "deviation-center-line": 0.08148700686988777, "driven_lanedir_consec": 0.1998281825264865, "sim_compute_sim_state": 0.004851043224334717, "sim_compute_performance-ego": 0.007198107242584228, "sim_compute_robot_state-ego": 0.009596121311187745, "sim_compute_robot_state-npc0": 0.009547567367553711, "sim_compute_robot_state-npc1": 0.00855410099029541, "sim_compute_robot_state-npc2": 0.008592915534973145, "sim_compute_robot_state-npc3": 0.008868837356567382}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.19591572147057995, "sim_physics": 0.2167421516619231, "survival_time": 0.9500000000000004, "driven_lanedir": 0.14875094631717234, "sim_render-ego": 0.012614275279798004, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.02842108826888235, "deviation-heading": 0.31338636366002043, "set_robot_commands": 0.01008840611106471, "deviation-center-line": 0.07734340038522211, "driven_lanedir_consec": 0.14875094631717234, "sim_compute_sim_state": 0.005130341178492496, "sim_compute_performance-ego": 0.006916962171855725, "sim_compute_robot_state-ego": 0.008785561511391088, "sim_compute_robot_state-npc0": 0.008939906170493677, "sim_compute_robot_state-npc1": 0.008376422681306539, "sim_compute_robot_state-npc2": 0.008422914304231343, "sim_compute_robot_state-npc3": 0.008298560192710474}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.7854377291155518, "sim_physics": 0.15179517609732493, "survival_time": 3.4999999999999956, "driven_lanedir": 0.3964819556962105, "sim_render-ego": 0.011359150069100515, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.02681107521057129, "deviation-heading": 1.1851483327194934, "set_robot_commands": 0.00916989871433803, "deviation-center-line": 0.13809260253551023, "driven_lanedir_consec": 0.3964819556962105, "sim_compute_sim_state": 0.004376697540283203, "sim_compute_performance-ego": 0.006537525994437081, "sim_compute_robot_state-ego": 0.008289037431989396, "sim_compute_robot_state-npc0": 0.008011698722839355, "sim_compute_robot_state-npc1": 0.007830493790762765, "sim_compute_robot_state-npc2": 0.007698263440813337, "sim_compute_robot_state-npc3": 0.00789022786276681}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.3581472762977876, "sim_physics": 0.11585268378257751, "survival_time": 1.6000000000000008, "driven_lanedir": 0.3050786883986474, "sim_render-ego": 0.00918758660554886, "in-drivable-lane": 0, "agent_compute-ego": 0.020375967025756836, "deviation-heading": 0.6136881317742018, "set_robot_commands": 0.006845712661743164, "deviation-center-line": 0.08894019456029295, "driven_lanedir_consec": 0.3050786883986474, "sim_compute_sim_state": 0.003573484718799591, "sim_compute_performance-ego": 0.0050696879625320435, "sim_compute_robot_state-ego": 0.006944246590137482, "sim_compute_robot_state-npc0": 0.006368748843669891, "sim_compute_robot_state-npc1": 0.006526172161102295, "sim_compute_robot_state-npc2": 0.006418682634830475, "sim_compute_robot_state-npc3": 0.00605657696723938}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.17855682642957194, "sim_physics": 0.11656125386555988, "survival_time": 0.9000000000000002, "driven_lanedir": 0.12832135084722973, "sim_render-ego": 0.0077289872699313695, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.02074424425760905, "deviation-heading": 0.3305885054934995, "set_robot_commands": 0.005744258562723796, "deviation-center-line": 0.059705063081282296, "driven_lanedir_consec": 0.12832135084722973, "sim_compute_sim_state": 0.003148357073465983, "sim_compute_performance-ego": 0.004199504852294922, "sim_compute_robot_state-ego": 0.0056650108761257595, "sim_compute_robot_state-npc0": 0.005453864733378093, "sim_compute_robot_state-npc1": 0.005002511872185601, "sim_compute_robot_state-npc2": 0.0050000349680582685, "sim_compute_robot_state-npc3": 0.004908694161309136}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.5456554492578833, "sim_physics": 0.15552342191655583, "survival_time": 2.3499999999999996, "driven_lanedir": 0.2180854629444995, "sim_render-ego": 0.00989289993935443, "in-drivable-lane": 0.849999999999999, "agent_compute-ego": 0.02169288472926363, "deviation-heading": 0.6996237612227223, "set_robot_commands": 0.006597397175241024, "deviation-center-line": 0.07632642021502961, "driven_lanedir_consec": 0.2180854629444995, "sim_compute_sim_state": 0.0035354228729897355, "sim_compute_performance-ego": 0.00523861418379114, "sim_compute_robot_state-ego": 0.006452986534605635, "sim_compute_robot_state-npc0": 0.006327781271427236, "sim_compute_robot_state-npc1": 0.005834148285236765, "sim_compute_robot_state-npc2": 0.005869687871730074, "sim_compute_robot_state-npc3": 0.005682290868556246}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.5004996271261478, "sim_physics": 0.11265399397873296, "survival_time": 2.0500000000000007, "driven_lanedir": 0.1865336710518104, "sim_render-ego": 0.008773966533381765, "in-drivable-lane": 0.7500000000000002, "agent_compute-ego": 0.020432291961297755, "deviation-heading": 0.496866104331268, "set_robot_commands": 0.0062914069105939165, "deviation-center-line": 0.07383260209534934, "driven_lanedir_consec": 0.1865336710518104, "sim_compute_sim_state": 0.0035947066981617997, "sim_compute_performance-ego": 0.004867204805699791, "sim_compute_robot_state-ego": 0.006323837652439024, "sim_compute_robot_state-npc0": 0.00580070658427913, "sim_compute_robot_state-npc1": 0.00556855666928175, "sim_compute_robot_state-npc2": 0.005589479353369736, "sim_compute_robot_state-npc3": 0.005702768884054045}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.2541095922900319, "sim_physics": 0.14632003957575018, "survival_time": 1.6500000000000008, "driven_lanedir": 0.1912184846958613, "sim_render-ego": 0.009372357166174686, "in-drivable-lane": 0, "agent_compute-ego": 0.020461422024351177, "deviation-heading": 0.9266892887856656, "set_robot_commands": 0.007043065446795839, "deviation-center-line": 0.08689497187845485, "driven_lanedir_consec": 0.1912184846958613, "sim_compute_sim_state": 0.003540942163178415, "sim_compute_performance-ego": 0.005080945564038826, "sim_compute_robot_state-ego": 0.006557255080251983, "sim_compute_robot_state-npc0": 0.006194627646243934, "sim_compute_robot_state-npc1": 0.005887197725700609, "sim_compute_robot_state-npc2": 0.005924535520149, "sim_compute_robot_state-npc3": 0.006276838707201408}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.7141755496812815, "sim_physics": 0.1414728699059322, "survival_time": 2.8999999999999977, "driven_lanedir": 0.6001749872579527, "sim_render-ego": 0.009021717926551556, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.020232097855929672, "deviation-heading": 1.0678754591523183, "set_robot_commands": 0.006685848893790409, "deviation-center-line": 0.12585143260237963, "driven_lanedir_consec": 0.6001749872579527, "sim_compute_sim_state": 0.0035895396923196726, "sim_compute_performance-ego": 0.005127795811357169, "sim_compute_robot_state-ego": 0.006606961118763891, "sim_compute_robot_state-npc0": 0.006191002911534803, "sim_compute_robot_state-npc1": 0.006078021279696761, "sim_compute_robot_state-npc2": 0.005945045372535442, "sim_compute_robot_state-npc3": 0.006036092495096141}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.6397761608754972, "sim_physics": 0.10201375087102256, "survival_time": 2.9999999999999973, "driven_lanedir": 0.3018780041374076, "sim_render-ego": 0.0070811947186787925, "in-drivable-lane": 1.1499999999999964, "agent_compute-ego": 0.01654651959737142, "deviation-heading": 0.687700585391871, "set_robot_commands": 0.0048650701840718584, "deviation-center-line": 0.11074939519780017, "driven_lanedir_consec": 0.3018780041374076, "sim_compute_sim_state": 0.0027412732442220053, "sim_compute_performance-ego": 0.00388187567392985, "sim_compute_robot_state-ego": 0.005068004131317139, "sim_compute_robot_state-npc0": 0.004682012399037679, "sim_compute_robot_state-npc1": 0.004507776101430257, "sim_compute_robot_state-npc2": 0.004490617911020915, "sim_compute_robot_state-npc3": 0.004464101791381836}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.34432123501925954, "sim_physics": 0.11275298148393632, "survival_time": 1.6000000000000008, "driven_lanedir": 0.29870241418864674, "sim_render-ego": 0.007243193686008453, "in-drivable-lane": 0, "agent_compute-ego": 0.01679293066263199, "deviation-heading": 0.6122167919742211, "set_robot_commands": 0.0050978586077690125, "deviation-center-line": 0.11457496840760724, "driven_lanedir_consec": 0.29870241418864674, "sim_compute_sim_state": 0.0028450191020965576, "sim_compute_performance-ego": 0.003939248621463776, "sim_compute_robot_state-ego": 0.005200587213039398, "sim_compute_robot_state-npc0": 0.005001761019229889, "sim_compute_robot_state-npc1": 0.004530258476734161, "sim_compute_robot_state-npc2": 0.00452837347984314, "sim_compute_robot_state-npc3": 0.004540078341960907}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.24166671104629783, "sim_physics": 0.11970965428785844, "survival_time": 1.1000000000000003, "driven_lanedir": 0.23003459314400532, "sim_render-ego": 0.007147312164306641, "in-drivable-lane": 0, "agent_compute-ego": 0.016806179826909847, "deviation-heading": 0.2547912413946925, "set_robot_commands": 0.0048967491496693, "deviation-center-line": 0.09061349875132556, "driven_lanedir_consec": 0.23003459314400532, "sim_compute_sim_state": 0.002807573838667436, "sim_compute_performance-ego": 0.003877845677462491, "sim_compute_robot_state-ego": 0.005110177126797763, "sim_compute_robot_state-npc0": 0.004863804036920721, "sim_compute_robot_state-npc1": 0.004506392912431197, "sim_compute_robot_state-npc2": 0.004483786496249112, "sim_compute_robot_state-npc3": 0.004463065754283558}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 1.430326160194243, "sim_physics": 0.1011857289188313, "survival_time": 5.299999999999989, "driven_lanedir": 0.34662117539457027, "sim_render-ego": 0.006986696765107929, "in-drivable-lane": 3.299999999999992, "agent_compute-ego": 0.016345521189131827, "deviation-heading": 1.4425283109802456, "set_robot_commands": 0.004810382735054448, "deviation-center-line": 0.14427567270590896, "driven_lanedir_consec": 0.34662117539457027, "sim_compute_sim_state": 0.0027470251299300283, "sim_compute_performance-ego": 0.003868192996618883, "sim_compute_robot_state-ego": 0.005038209681241018, "sim_compute_robot_state-npc0": 0.004713227164070561, "sim_compute_robot_state-npc1": 0.004494772767120937, "sim_compute_robot_state-npc2": 0.004456164701929632, "sim_compute_robot_state-npc3": 0.004475056000475613}}
set_robot_commands_max0.010518789291381836
set_robot_commands_mean0.007262702496928784
set_robot_commands_median0.006685848893790409
set_robot_commands_min0.004810382735054448
sim_compute_performance-ego_max0.007244462966918945
sim_compute_performance-ego_mean0.00534874891391301
sim_compute_performance-ego_median0.005080945564038826
sim_compute_performance-ego_min0.003868192996618883
sim_compute_robot_state-ego_max0.009626106782393022
sim_compute_robot_state-ego_mean0.006984514582593687
sim_compute_robot_state-ego_median0.006557255080251983
sim_compute_robot_state-ego_min0.005038209681241018
sim_compute_robot_state-npc0_max0.009547567367553711
sim_compute_robot_state-npc0_mean0.006641958359661999
sim_compute_robot_state-npc0_median0.006194627646243934
sim_compute_robot_state-npc0_min0.004682012399037679
sim_compute_robot_state-npc1_max0.009029171683571556
sim_compute_robot_state-npc1_mean0.0063403599705000755
sim_compute_robot_state-npc1_median0.005887197725700609
sim_compute_robot_state-npc1_min0.004494772767120937
sim_compute_robot_state-npc2_max0.009210098873485218
sim_compute_robot_state-npc2_mean0.0063398839216012
sim_compute_robot_state-npc2_median0.005924535520149
sim_compute_robot_state-npc2_min0.004456164701929632
sim_compute_robot_state-npc3_max0.008868837356567382
sim_compute_robot_state-npc3_mean0.006321365469241532
sim_compute_robot_state-npc3_median0.006036092495096141
sim_compute_robot_state-npc3_min0.004463065754283558
sim_compute_sim_state_max0.005554188381541859
sim_compute_sim_state_mean0.003793778327466066
sim_compute_sim_state_median0.003573484718799591
sim_compute_sim_state_min0.0027412732442220053
sim_physics_max0.24614399671554563
sim_physics_mean0.15226952845421313
sim_physics_median0.1414728699059322
sim_physics_min0.1011857289188313
sim_render-ego_max0.013545095920562744
sim_render-ego_mean0.009730462894781682
sim_render-ego_median0.00918758660554886
sim_render-ego_min0.006986696765107929
simulation-passed1
survival_time_max5.299999999999989
survival_time_mean2.183333333333332
survival_time_min0.9000000000000002
No reset possible
251683257Liu Sam 🇹🇼PredictorLastaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104980:00:03
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
251653264Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104980:00:26
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
251513265Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104980:00:04
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible