Duckietown Challenges Home Challenges Submissions

Evaluator 2102

ID2102
evaluatorreg04-841736bca7b5-1
ownerI don't have one 😀
machinearchimede
processreg04-841736bca7b5-1
versiond-c:6.1.15;d-c-r:6.0.48
first heard
last heard
statusinactive
# evaluating
# success2 36053
# timeout1 36066
# failed
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
360669820Bea Baselines 🐤template-pytorchaido5-LFV-sim-testingLFVv-simtimeoutnoreg04-841736bca7b5-11:05:25
I can see how the jo [...]
I can see how the job 36066 is timeout because passed 3925 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
360649816Bea Baselines 🐤template-rosaido5-LFVI_multi-sim-validationLFVIv-simsuccessnoreg04-841736bca7b5-10:05:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.3804972906093782
survival_time_median1.825000000000001
deviation-center-line_median0.07818882197628427
in-drivable-lane_median0.07500000000000007


other stats
agent_compute-ego0_max0.04530505339304606
agent_compute-ego0_mean0.04473094736133609
agent_compute-ego0_median0.04473094736133609
agent_compute-ego0_min0.044156841329626134
agent_compute-ego1_max0.04345534943245553
agent_compute-ego1_mean0.04322228846965251
agent_compute-ego1_median0.04322228846965251
agent_compute-ego1_min0.0429892275068495
agent_compute-ego2_max0.04235444197783599
agent_compute-ego2_mean0.04058840074338713
agent_compute-ego2_median0.040588400743387126
agent_compute-ego2_min0.03882235950893826
complete-iteration_max1.6824009948306613
complete-iteration_mean1.6587676686925572
complete-iteration_median1.6587676686925574
complete-iteration_min1.6351343425544531
deviation-center-line_max0.1497749259813152
deviation-center-line_mean0.0936613611730474
deviation-center-line_min0.06525144813209902
deviation-heading_max0.5681093940199092
deviation-heading_mean0.3277449131160395
deviation-heading_median0.33460537901662224
deviation-heading_min0.04506955868582584
driven_any_max0.47298361493914426
driven_any_mean0.44399108314781
driven_any_median0.45184266805280976
driven_any_min0.3939223587431318
driven_lanedir_consec_max0.4509855441512016
driven_lanedir_consec_mean0.3770214020433342
driven_lanedir_consec_min0.23903966649837383
driven_lanedir_max0.4509855441512016
driven_lanedir_mean0.3770214020433342
driven_lanedir_median0.3804972906093782
driven_lanedir_min0.23903966649837383
get_state_dump_max0.04287772708468967
get_state_dump_mean0.042077522736053925
get_state_dump_median0.042077522736053925
get_state_dump_min0.04127731838741818
get_ui_image_max0.17444841926162308
get_ui_image_mean0.17434680891466572
get_ui_image_median0.1743468089146657
get_ui_image_min0.17424519856770834
in-drivable-lane_max0.7000000000000006
in-drivable-lane_mean0.17500000000000016
in-drivable-lane_min0.0
per-episodes
details{"udem1-0-0-ego0": {"driven_any": 0.47298361493914426, "get_ui_image": 0.17424519856770834, "step_physics": 1.136709491411845, "survival_time": 1.800000000000001, "driven_lanedir": 0.23903966649837383, "get_state_dump": 0.04287772708468967, "sim_render-ego0": 0.008394598960876465, "sim_render-ego1": 0.00795619355307685, "sim_render-ego2": 0.00744163990020752, "in-drivable-lane": 0.7000000000000006, "deviation-heading": 0.3550828965698445, "agent_compute-ego0": 0.04530505339304606, "agent_compute-ego1": 0.0429892275068495, "agent_compute-ego2": 0.03882235950893826, "complete-iteration": 1.6824009948306613, "set_robot_commands": 0.006134298112657335, "deviation-center-line": 0.0683649904756738, "driven_lanedir_consec": 0.23903966649837383, "sim_compute_sim_state": 0.05464408132765028, "sim_compute_performance-ego0": 0.006300608317057292, "sim_compute_performance-ego1": 0.005792299906412761, "sim_compute_performance-ego2": 0.0064877404106987845, "sim_compute_robot_state-ego0": 0.01009270879957411, "sim_compute_robot_state-ego1": 0.009309477276272245, "sim_compute_robot_state-ego2": 0.009090701738993326, "sim_compute_robot_state-parked0": 0.010622845755683051, "sim_compute_robot_state-parked1": 0.008724583519829644, "sim_compute_robot_state-parked2": 0.008954233593410917, "sim_compute_robot_state-parked3": 0.00921333498424954, "sim_compute_robot_state-parked4": 0.009463621510399712, "sim_compute_robot_state-parked5": 0.008925974369049072}, "udem1-0-0-ego1": {"driven_any": 0.4511608949445752, "get_ui_image": 0.17424519856770834, "step_physics": 1.136709491411845, "survival_time": 1.800000000000001, "driven_lanedir": 0.4509855441512016, "get_state_dump": 0.04287772708468967, "sim_render-ego0": 0.008394598960876465, "sim_render-ego1": 0.00795619355307685, "sim_render-ego2": 0.00744163990020752, "in-drivable-lane": 0.0, "deviation-heading": 0.04506955868582584, "agent_compute-ego0": 0.04530505339304606, "agent_compute-ego1": 0.0429892275068495, "agent_compute-ego2": 0.03882235950893826, "complete-iteration": 1.6824009948306613, "set_robot_commands": 0.006134298112657335, "deviation-center-line": 0.06525144813209902, "driven_lanedir_consec": 0.4509855441512016, "sim_compute_sim_state": 0.05464408132765028, "sim_compute_performance-ego0": 0.006300608317057292, "sim_compute_performance-ego1": 0.005792299906412761, "sim_compute_performance-ego2": 0.0064877404106987845, "sim_compute_robot_state-ego0": 0.01009270879957411, "sim_compute_robot_state-ego1": 0.009309477276272245, "sim_compute_robot_state-ego2": 0.009090701738993326, "sim_compute_robot_state-parked0": 0.010622845755683051, "sim_compute_robot_state-parked1": 0.008724583519829644, "sim_compute_robot_state-parked2": 0.008954233593410917, "sim_compute_robot_state-parked3": 0.00921333498424954, "sim_compute_robot_state-parked4": 0.009463621510399712, "sim_compute_robot_state-parked5": 0.008925974369049072}, "udem1-0-0-ego2": {"driven_any": 0.3939223587431318, "get_ui_image": 0.17424519856770834, "step_physics": 1.136709491411845, "survival_time": 1.800000000000001, "driven_lanedir": 0.3889079713465402, "get_state_dump": 0.04287772708468967, "sim_render-ego0": 0.008394598960876465, "sim_render-ego1": 0.00795619355307685, "sim_render-ego2": 0.00744163990020752, "in-drivable-lane": 0.0, "deviation-heading": 0.291991824026298, "agent_compute-ego0": 0.04530505339304606, "agent_compute-ego1": 0.0429892275068495, "agent_compute-ego2": 0.03882235950893826, "complete-iteration": 1.6824009948306613, "set_robot_commands": 0.006134298112657335, "deviation-center-line": 0.08378811091355794, "driven_lanedir_consec": 0.3889079713465402, "sim_compute_sim_state": 0.05464408132765028, "sim_compute_performance-ego0": 0.006300608317057292, "sim_compute_performance-ego1": 0.005792299906412761, "sim_compute_performance-ego2": 0.0064877404106987845, "sim_compute_robot_state-ego0": 0.01009270879957411, "sim_compute_robot_state-ego1": 0.009309477276272245, "sim_compute_robot_state-ego2": 0.009090701738993326, "sim_compute_robot_state-parked0": 0.010622845755683051, "sim_compute_robot_state-parked1": 0.008724583519829644, "sim_compute_robot_state-parked2": 0.008954233593410917, "sim_compute_robot_state-parked3": 0.00921333498424954, "sim_compute_robot_state-parked4": 0.009463621510399712, "sim_compute_robot_state-parked5": 0.008925974369049072}, "udem1-0-1-ego0": {"driven_any": 0.4310424478072667, "get_ui_image": 0.17444841926162308, "step_physics": 1.0849849726702716, "survival_time": 1.850000000000001, "driven_lanedir": 0.3720866098722162, "get_state_dump": 0.04127731838741818, "sim_render-ego0": 0.009393865997726852, "sim_render-ego1": 0.008769557282731339, "sim_render-ego2": 0.007818396027023727, "in-drivable-lane": 0.15000000000000013, "deviation-heading": 0.3920879439309596, "agent_compute-ego0": 0.044156841329626134, "agent_compute-ego1": 0.04345534943245553, "agent_compute-ego2": 0.04235444197783599, "complete-iteration": 1.6351343425544531, "set_robot_commands": 0.006269338968637827, "deviation-center-line": 0.12219915849662784, "driven_lanedir_consec": 0.3720866098722162, "sim_compute_sim_state": 0.059483534580952416, "sim_compute_performance-ego0": 0.006456516884468697, "sim_compute_performance-ego1": 0.0066970876745275545, "sim_compute_performance-ego2": 0.0059138053172343484, "sim_compute_robot_state-ego0": 0.010949141270405536, "sim_compute_robot_state-ego1": 0.010308317235998204, "sim_compute_robot_state-ego2": 0.008362261024681298, "sim_compute_robot_state-parked0": 0.008884829443854254, "sim_compute_robot_state-parked1": 0.008782380336039775, "sim_compute_robot_state-parked2": 0.008782135473715293, "sim_compute_robot_state-parked3": 0.008440765174659522, "sim_compute_robot_state-parked4": 0.007559202812813424, "sim_compute_robot_state-parked5": 0.00829756582105482}, "udem1-0-1-ego1": {"driven_any": 0.46231274129169775, "get_ui_image": 0.17444841926162308, "step_physics": 1.0849849726702716, "survival_time": 1.850000000000001, "driven_lanedir": 0.4495244631929678, "get_state_dump": 0.04127731838741818, "sim_render-ego0": 0.009393865997726852, "sim_render-ego1": 0.008769557282731339, "sim_render-ego2": 0.007818396027023727, "in-drivable-lane": 0.0, "deviation-heading": 0.31412786146340005, "agent_compute-ego0": 0.044156841329626134, "agent_compute-ego1": 0.04345534943245553, "agent_compute-ego2": 0.04235444197783599, "complete-iteration": 1.6351343425544531, "set_robot_commands": 0.006269338968637827, "deviation-center-line": 0.07258953303901061, "driven_lanedir_consec": 0.4495244631929678, "sim_compute_sim_state": 0.059483534580952416, "sim_compute_performance-ego0": 0.006456516884468697, "sim_compute_performance-ego1": 0.0066970876745275545, "sim_compute_performance-ego2": 0.0059138053172343484, "sim_compute_robot_state-ego0": 0.010949141270405536, "sim_compute_robot_state-ego1": 0.010308317235998204, "sim_compute_robot_state-ego2": 0.008362261024681298, "sim_compute_robot_state-parked0": 0.008884829443854254, "sim_compute_robot_state-parked1": 0.008782380336039775, "sim_compute_robot_state-parked2": 0.008782135473715293, "sim_compute_robot_state-parked3": 0.008440765174659522, "sim_compute_robot_state-parked4": 0.007559202812813424, "sim_compute_robot_state-parked5": 0.00829756582105482}, "udem1-0-1-ego2": {"driven_any": 0.4525244411610443, "get_ui_image": 0.17444841926162308, "step_physics": 1.0849849726702716, "survival_time": 1.850000000000001, "driven_lanedir": 0.3615841571987053, "get_state_dump": 0.04127731838741818, "sim_render-ego0": 0.009393865997726852, "sim_render-ego1": 0.008769557282731339, "sim_render-ego2": 0.007818396027023727, "in-drivable-lane": 0.20000000000000015, "deviation-heading": 0.5681093940199092, "agent_compute-ego0": 0.044156841329626134, "agent_compute-ego1": 0.04345534943245553, "agent_compute-ego2": 0.04235444197783599, "complete-iteration": 1.6351343425544531, "set_robot_commands": 0.006269338968637827, "deviation-center-line": 0.1497749259813152, "driven_lanedir_consec": 0.3615841571987053, "sim_compute_sim_state": 0.059483534580952416, "sim_compute_performance-ego0": 0.006456516884468697, "sim_compute_performance-ego1": 0.0066970876745275545, "sim_compute_performance-ego2": 0.0059138053172343484, "sim_compute_robot_state-ego0": 0.010949141270405536, "sim_compute_robot_state-ego1": 0.010308317235998204, "sim_compute_robot_state-ego2": 0.008362261024681298, "sim_compute_robot_state-parked0": 0.008884829443854254, "sim_compute_robot_state-parked1": 0.008782380336039775, "sim_compute_robot_state-parked2": 0.008782135473715293, "sim_compute_robot_state-parked3": 0.008440765174659522, "sim_compute_robot_state-parked4": 0.007559202812813424, "sim_compute_robot_state-parked5": 0.00829756582105482}}
set_robot_commands_max0.006269338968637827
set_robot_commands_mean0.006201818540647581
set_robot_commands_median0.006201818540647581
set_robot_commands_min0.006134298112657335
sim_compute_performance-ego0_max0.006456516884468697
sim_compute_performance-ego0_mean0.006378562600762993
sim_compute_performance-ego0_median0.006378562600762994
sim_compute_performance-ego0_min0.006300608317057292
sim_compute_performance-ego1_max0.0066970876745275545
sim_compute_performance-ego1_mean0.006244693790470158
sim_compute_performance-ego1_median0.006244693790470157
sim_compute_performance-ego1_min0.005792299906412761
sim_compute_performance-ego2_max0.0064877404106987845
sim_compute_performance-ego2_mean0.006200772863966566
sim_compute_performance-ego2_median0.006200772863966567
sim_compute_performance-ego2_min0.0059138053172343484
sim_compute_robot_state-ego0_max0.010949141270405536
sim_compute_robot_state-ego0_mean0.010520925034989824
sim_compute_robot_state-ego0_median0.010520925034989824
sim_compute_robot_state-ego0_min0.01009270879957411
sim_compute_robot_state-ego1_max0.010308317235998204
sim_compute_robot_state-ego1_mean0.009808897256135225
sim_compute_robot_state-ego1_median0.009808897256135225
sim_compute_robot_state-ego1_min0.009309477276272245
sim_compute_robot_state-ego2_max0.009090701738993326
sim_compute_robot_state-ego2_mean0.008726481381837311
sim_compute_robot_state-ego2_median0.008726481381837313
sim_compute_robot_state-ego2_min0.008362261024681298
sim_compute_robot_state-parked0_max0.010622845755683051
sim_compute_robot_state-parked0_mean0.009753837599768654
sim_compute_robot_state-parked0_median0.009753837599768653
sim_compute_robot_state-parked0_min0.008884829443854254
sim_compute_robot_state-parked1_max0.008782380336039775
sim_compute_robot_state-parked1_mean0.008753481927934709
sim_compute_robot_state-parked1_median0.008753481927934709
sim_compute_robot_state-parked1_min0.008724583519829644
sim_compute_robot_state-parked2_max0.008954233593410917
sim_compute_robot_state-parked2_mean0.008868184533563105
sim_compute_robot_state-parked2_median0.008868184533563105
sim_compute_robot_state-parked2_min0.008782135473715293
sim_compute_robot_state-parked3_max0.00921333498424954
sim_compute_robot_state-parked3_mean0.008827050079454531
sim_compute_robot_state-parked3_median0.008827050079454531
sim_compute_robot_state-parked3_min0.008440765174659522
sim_compute_robot_state-parked4_max0.009463621510399712
sim_compute_robot_state-parked4_mean0.008511412161606566
sim_compute_robot_state-parked4_median0.008511412161606568
sim_compute_robot_state-parked4_min0.007559202812813424
sim_compute_robot_state-parked5_max0.008925974369049072
sim_compute_robot_state-parked5_mean0.008611770095051947
sim_compute_robot_state-parked5_median0.008611770095051947
sim_compute_robot_state-parked5_min0.00829756582105482
sim_compute_sim_state_max0.059483534580952416
sim_compute_sim_state_mean0.05706380795430135
sim_compute_sim_state_median0.05706380795430135
sim_compute_sim_state_min0.05464408132765028
sim_render-ego0_max0.009393865997726852
sim_render-ego0_mean0.00889423247930166
sim_render-ego0_median0.00889423247930166
sim_render-ego0_min0.008394598960876465
sim_render-ego1_max0.008769557282731339
sim_render-ego1_mean0.008362875417904094
sim_render-ego1_median0.008362875417904096
sim_render-ego1_min0.00795619355307685
sim_render-ego2_max0.007818396027023727
sim_render-ego2_mean0.007630017963615624
sim_render-ego2_median0.007630017963615623
sim_render-ego2_min0.00744163990020752
simulation-passed1
step_physics_max1.136709491411845
step_physics_mean1.1108472320410585
step_physics_median1.1108472320410585
step_physics_min1.0849849726702716
survival_time_max1.850000000000001
survival_time_mean1.825000000000001
survival_time_min1.800000000000001
No reset possible
360539801Bea Baselines 🐤baseline-duckietownaido5-LF-sim-validationLFv-simsuccessnoreg04-841736bca7b5-10:06:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8457154748449288
survival_time_median14.950000000000076
deviation-center-line_median0.3850623967236436
in-drivable-lane_median8.125000000000094


other stats
agent_compute-ego_max0.02831503709157308
agent_compute-ego_mean0.027958674033482872
agent_compute-ego_median0.027958674033482872
agent_compute-ego_min0.02760231097539266
complete-iteration_max0.33413795948028563
complete-iteration_mean0.3311605974038442
complete-iteration_median0.3311605974038442
complete-iteration_min0.32818323532740273
deviation-center-line_max0.4045489933024265
deviation-center-line_mean0.3850623967236436
deviation-center-line_min0.3655758001448607
deviation-heading_max1.0443695283856604
deviation-heading_mean1.006641848335156
deviation-heading_median1.006641848335156
deviation-heading_min0.9689141682846516
driven_any_max1.9399405892189845
driven_any_mean1.9367574008852635
driven_any_median1.9367574008852635
driven_any_min1.9335742125515432
driven_lanedir_consec_max0.873385261017527
driven_lanedir_consec_mean0.8457154748449288
driven_lanedir_consec_min0.8180456886723306
driven_lanedir_max0.873385261017527
driven_lanedir_mean0.8457154748449288
driven_lanedir_median0.8457154748449288
driven_lanedir_min0.8180456886723306
get_state_dump_max0.016324631373087563
get_state_dump_mean0.016296174923578897
get_state_dump_median0.016296174923578897
get_state_dump_min0.01626771847407023
get_ui_image_max0.06680840651194254
get_ui_image_mean0.06443903803825378
get_ui_image_median0.06443903803825378
get_ui_image_min0.06206966956456502
in-drivable-lane_max8.300000000000093
in-drivable-lane_mean8.125000000000094
in-drivable-lane_min7.950000000000094
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.9335742125515432, "get_ui_image": 0.06206966956456502, "step_physics": 0.15675300995508828, "survival_time": 14.950000000000076, "driven_lanedir": 0.8180456886723306, "get_state_dump": 0.016324631373087563, "sim_render-ego": 0.008717842896779378, "in-drivable-lane": 8.300000000000093, "agent_compute-ego": 0.02760231097539266, "deviation-heading": 1.0443695283856604, "complete-iteration": 0.32818323532740273, "set_robot_commands": 0.006458772818247477, "deviation-center-line": 0.3655758001448607, "driven_lanedir_consec": 0.8180456886723306, "sim_compute_sim_state": 0.02389719327290853, "sim_compute_performance-ego": 0.005977274576822917, "sim_compute_robot_state-ego": 0.010085936387379963, "sim_compute_robot_state-parked0": 0.010102379322052}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 1.9399405892189845, "get_ui_image": 0.06680840651194254, "step_physics": 0.15725414276123048, "survival_time": 14.950000000000076, "driven_lanedir": 0.873385261017527, "get_state_dump": 0.01626771847407023, "sim_render-ego": 0.00862842559814453, "in-drivable-lane": 7.950000000000094, "agent_compute-ego": 0.02831503709157308, "deviation-heading": 0.9689141682846516, "complete-iteration": 0.33413795948028563, "set_robot_commands": 0.007005947430928548, "deviation-center-line": 0.4045489933024265, "driven_lanedir_consec": 0.873385261017527, "sim_compute_sim_state": 0.022983139355977376, "sim_compute_performance-ego": 0.006350212891896565, "sim_compute_robot_state-ego": 0.010277005036671956, "sim_compute_robot_state-parked0": 0.01007058064142863}}
set_robot_commands_max0.007005947430928548
set_robot_commands_mean0.006732360124588013
set_robot_commands_median0.006732360124588013
set_robot_commands_min0.006458772818247477
sim_compute_performance-ego_max0.006350212891896565
sim_compute_performance-ego_mean0.006163743734359741
sim_compute_performance-ego_median0.006163743734359741
sim_compute_performance-ego_min0.005977274576822917
sim_compute_robot_state-ego_max0.010277005036671956
sim_compute_robot_state-ego_mean0.01018147071202596
sim_compute_robot_state-ego_median0.01018147071202596
sim_compute_robot_state-ego_min0.010085936387379963
sim_compute_robot_state-parked0_max0.010102379322052
sim_compute_robot_state-parked0_mean0.010086479981740316
sim_compute_robot_state-parked0_median0.010086479981740316
sim_compute_robot_state-parked0_min0.01007058064142863
sim_compute_sim_state_max0.02389719327290853
sim_compute_sim_state_mean0.023440166314442953
sim_compute_sim_state_median0.023440166314442953
sim_compute_sim_state_min0.022983139355977376
sim_render-ego_max0.008717842896779378
sim_render-ego_mean0.008673134247461954
sim_render-ego_median0.008673134247461954
sim_render-ego_min0.00862842559814453
simulation-passed1
step_physics_max0.15725414276123048
step_physics_mean0.15700357635815937
step_physics_median0.15700357635815937
step_physics_min0.15675300995508828
survival_time_max14.950000000000076
survival_time_mean14.950000000000076
survival_time_min14.950000000000076
No reset possible