Duckietown Challenges Home Challenges Submissions

Evaluator 2254

ID2254
evaluatormont10-cdd8c9e6927b-1
ownerI don't have one 😀
machineaido-ms-7b09
processmont10-cdd8c9e6927b-1
versiond-c:6.1.23;d-c-r:6.0.59
first heard
last heard
statusinactive
# evaluating
# success2 39977
# timeout1 39987
# failed
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
3998711255Bea Baselines 🐤template-rosaido5-LFV_multi-sim-testingLFVmultibodyv-simtimeoutnomont10-cdd8c9e6927b-1----No reset possible
3998111259Bea Baselines 🐤template-rosaido5-LFVI-sim-validationLFVIv-simsuccessnomont10-cdd8c9e6927b-10:05:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6811285003996614
survival_time_median5.999999999999987
deviation-center-line_median0.14710836377715442
in-drivable-lane_median2.8999999999999897


other stats
agent_compute-ego_max0.02101993958155314
agent_compute-ego_mean0.02101993958155314
agent_compute-ego_median0.02101993958155314
agent_compute-ego_min0.02101993958155314
agent_compute-npc0_max0.06929261485735576
agent_compute-npc0_mean0.06929261485735576
agent_compute-npc0_median0.06929261485735576
agent_compute-npc0_min0.06929261485735576
agent_compute-npc1_max0.07159752647082011
agent_compute-npc1_mean0.07159752647082011
agent_compute-npc1_median0.07159752647082011
agent_compute-npc1_min0.07159752647082011
agent_compute-npc2_max0.07042269110679626
agent_compute-npc2_mean0.07042269110679626
agent_compute-npc2_median0.07042269110679626
agent_compute-npc2_min0.07042269110679626
agent_compute-npc3_max0.07063575585683186
agent_compute-npc3_mean0.07063575585683186
agent_compute-npc3_median0.07063575585683186
agent_compute-npc3_min0.07063575585683186
complete-iteration_max1.4919894595940908
complete-iteration_mean1.4919894595940908
complete-iteration_median1.4919894595940908
complete-iteration_min1.4919894595940908
deviation-center-line_max0.14710836377715442
deviation-center-line_mean0.14710836377715442
deviation-center-line_min0.14710836377715442
deviation-heading_max1.0818821135171175
deviation-heading_mean1.0818821135171175
deviation-heading_median1.0818821135171175
deviation-heading_min1.0818821135171175
driven_any_max1.6583130503295762
driven_any_mean1.6583130503295762
driven_any_median1.6583130503295762
driven_any_min1.6583130503295762
driven_lanedir_consec_max0.6811285003996614
driven_lanedir_consec_mean0.6811285003996614
driven_lanedir_consec_min0.6811285003996614
driven_lanedir_max0.7037742153801221
driven_lanedir_mean0.7037742153801221
driven_lanedir_median0.7037742153801221
driven_lanedir_min0.7037742153801221
get_duckie_state_max2.853075663248698e-06
get_duckie_state_mean2.853075663248698e-06
get_duckie_state_median2.853075663248698e-06
get_duckie_state_min2.853075663248698e-06
get_robot_state_max0.05349330306053161
get_robot_state_mean0.05349330306053161
get_robot_state_median0.05349330306053161
get_robot_state_min0.05349330306053161
get_state_dump_max0.031158175071080527
get_state_dump_mean0.031158175071080527
get_state_dump_median0.031158175071080527
get_state_dump_min0.031158175071080527
get_ui_image_max0.08638420104980468
get_ui_image_mean0.08638420104980468
get_ui_image_median0.08638420104980468
get_ui_image_min0.08638420104980468
in-drivable-lane_max2.8999999999999897
in-drivable-lane_mean2.8999999999999897
in-drivable-lane_min2.8999999999999897
per-episodes
details{"udem1-sc0-0-ego": {"driven_any": 1.6583130503295762, "get_ui_image": 0.08638420104980468, "step_physics": 0.8383803188800811, "survival_time": 5.999999999999987, "driven_lanedir": 0.7037742153801221, "get_state_dump": 0.031158175071080527, "sim_render-ego": 0.007317366202672323, "get_robot_state": 0.05349330306053161, "sim_render-npc0": 0.007898845275243123, "sim_render-npc1": 0.007410589853922526, "sim_render-npc2": 0.007639002799987793, "sim_render-npc3": 0.007573507229487101, "get_duckie_state": 2.853075663248698e-06, "in-drivable-lane": 2.8999999999999897, "agent_compute-ego": 0.02101993958155314, "deviation-heading": 1.0818821135171175, "agent_compute-npc0": 0.06929261485735576, "agent_compute-npc1": 0.07159752647082011, "agent_compute-npc2": 0.07042269110679626, "agent_compute-npc3": 0.07063575585683186, "complete-iteration": 1.4919894595940908, "set_robot_commands": 0.005168737967809042, "deviation-center-line": 0.14710836377715442, "driven_lanedir_consec": 0.6811285003996614, "sim_compute_sim_state": 0.08853009939193726, "sim_compute_performance-ego": 0.005225032567977905, "sim_compute_performance-npc0": 0.005114622910817464, "sim_compute_performance-npc1": 0.005137805143992106, "sim_compute_performance-npc2": 0.005450328191121419, "sim_compute_performance-npc3": 0.004977132876714071}}
set_robot_commands_max0.005168737967809042
set_robot_commands_mean0.005168737967809042
set_robot_commands_median0.005168737967809042
set_robot_commands_min0.005168737967809042
sim_compute_performance-ego_max0.005225032567977905
sim_compute_performance-ego_mean0.005225032567977905
sim_compute_performance-ego_median0.005225032567977905
sim_compute_performance-ego_min0.005225032567977905
sim_compute_performance-npc0_max0.005114622910817464
sim_compute_performance-npc0_mean0.005114622910817464
sim_compute_performance-npc0_median0.005114622910817464
sim_compute_performance-npc0_min0.005114622910817464
sim_compute_performance-npc1_max0.005137805143992106
sim_compute_performance-npc1_mean0.005137805143992106
sim_compute_performance-npc1_median0.005137805143992106
sim_compute_performance-npc1_min0.005137805143992106
sim_compute_performance-npc2_max0.005450328191121419
sim_compute_performance-npc2_mean0.005450328191121419
sim_compute_performance-npc2_median0.005450328191121419
sim_compute_performance-npc2_min0.005450328191121419
sim_compute_performance-npc3_max0.004977132876714071
sim_compute_performance-npc3_mean0.004977132876714071
sim_compute_performance-npc3_median0.004977132876714071
sim_compute_performance-npc3_min0.004977132876714071
sim_compute_sim_state_max0.08853009939193726
sim_compute_sim_state_mean0.08853009939193726
sim_compute_sim_state_median0.08853009939193726
sim_compute_sim_state_min0.08853009939193726
sim_render-ego_max0.007317366202672323
sim_render-ego_mean0.007317366202672323
sim_render-ego_median0.007317366202672323
sim_render-ego_min0.007317366202672323
sim_render-npc0_max0.007898845275243123
sim_render-npc0_mean0.007898845275243123
sim_render-npc0_median0.007898845275243123
sim_render-npc0_min0.007898845275243123
sim_render-npc1_max0.007410589853922526
sim_render-npc1_mean0.007410589853922526
sim_render-npc1_median0.007410589853922526
sim_render-npc1_min0.007410589853922526
sim_render-npc2_max0.007639002799987793
sim_render-npc2_mean0.007639002799987793
sim_render-npc2_median0.007639002799987793
sim_render-npc2_min0.007639002799987793
sim_render-npc3_max0.007573507229487101
sim_render-npc3_mean0.007573507229487101
sim_render-npc3_median0.007573507229487101
sim_render-npc3_min0.007573507229487101
simulation-passed1
step_physics_max0.8383803188800811
step_physics_mean0.8383803188800811
step_physics_median0.8383803188800811
step_physics_min0.8383803188800811
survival_time_max5.999999999999987
survival_time_mean5.999999999999987
survival_time_min5.999999999999987
No reset possible
3997711228Bea Baselines 🐤straightaido5-LFP-sim-validationLFP-simsuccessnomont10-cdd8c9e6927b-10:07:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median7.474999999999983
in-drivable-lane_median2.049999999999994
driven_lanedir_consec_median0.7573830578433818
deviation-center-line_median0.2404078677891369


other stats
agent_compute-ego_max0.017477520461221342
agent_compute-ego_mean0.016759351696759014
agent_compute-ego_median0.016605362368793022
agent_compute-ego_min0.016349161588228665
complete-iteration_max0.3532251688150259
complete-iteration_mean0.3439559136913769
complete-iteration_median0.3440098950649414
complete-iteration_min0.33457869582059907
deviation-center-line_max0.5442231045907214
deviation-center-line_mean0.3027171880305003
deviation-center-line_min0.185829911953006
deviation-heading_max0.9036263005525016
deviation-heading_mean0.7911931719645967
deviation-heading_median0.8021319566508551
deviation-heading_min0.656882474004175
driven_any_max1.642573882123216
driven_any_mean1.1207921547428128
driven_any_median1.1792843854366784
driven_any_min0.48202596597467895
driven_lanedir_consec_max0.8875163355320987
driven_lanedir_consec_mean0.6773693187553927
driven_lanedir_consec_min0.30719482380270824
driven_lanedir_max0.8875163355320987
driven_lanedir_mean0.6773693187553927
driven_lanedir_median0.7573830578433818
driven_lanedir_min0.30719482380270824
get_duckie_state_max0.04309947050890876
get_duckie_state_mean0.042351306476877605
get_duckie_state_median0.04241243699337342
get_duckie_state_min0.04148088141185481
get_robot_state_max0.018337369848180705
get_robot_state_mean0.018109963039710547
get_robot_state_median0.018177700701536216
get_robot_state_min0.01774708090758905
get_state_dump_max0.02229383080093949
get_state_dump_mean0.021826482101253653
get_state_dump_median0.021791570517627652
get_state_dump_min0.02142895656881981
get_ui_image_max0.06001883653494028
get_ui_image_mean0.05854754666771632
get_ui_image_median0.05940680304091425
get_ui_image_min0.0553577440540965
in-drivable-lane_max5.250000000000021
in-drivable-lane_mean2.575000000000001
in-drivable-lane_min0.9499999999999966
per-episodes
details{"ETHZ_autolab_technical_track-sc0-0-ego": {"driven_any": 1.29945636568878, "get_ui_image": 0.0553577440540965, "step_physics": 0.13348934272440469, "survival_time": 8.199999999999982, "driven_lanedir": 0.8875163355320987, "get_state_dump": 0.021766630614676128, "sim_render-ego": 0.007333267025831269, "get_robot_state": 0.01774708090758905, "get_duckie_state": 0.04148088141185481, "in-drivable-lane": 2.449999999999994, "agent_compute-ego": 0.01660668268436339, "deviation-heading": 0.656882474004175, "complete-iteration": 0.33457869582059907, "set_robot_commands": 0.005593000388726956, "deviation-center-line": 0.5442231045907214, "driven_lanedir_consec": 0.8875163355320987, "sim_compute_sim_state": 0.02984885326245936, "sim_compute_performance-ego": 0.005151735573280148}, "ETHZ_autolab_technical_track-sc1-0-ego": {"driven_any": 1.0591124051845766, "get_ui_image": 0.05934214238767271, "step_physics": 0.1323777922877559, "survival_time": 6.749999999999984, "driven_lanedir": 0.7599615841487778, "get_state_dump": 0.02229383080093949, "sim_render-ego": 0.007673832222267434, "get_robot_state": 0.018337369848180705, "get_duckie_state": 0.04218735341672544, "in-drivable-lane": 1.649999999999994, "agent_compute-ego": 0.016604042053222655, "deviation-heading": 0.9036263005525016, "complete-iteration": 0.3385315630171034, "set_robot_commands": 0.005623598451967592, "deviation-center-line": 0.24685294511397105, "driven_lanedir_consec": 0.7599615841487778, "sim_compute_sim_state": 0.028357952612417716, "sim_compute_performance-ego": 0.005523310767279731}, "ETHZ_autolab_technical_track-sc2-0-ego": {"driven_any": 1.642573882123216, "get_ui_image": 0.0594714636941558, "step_physics": 0.1380730485453189, "survival_time": 10.300000000000011, "driven_lanedir": 0.7548045315379859, "get_state_dump": 0.02142895656881981, "sim_render-ego": 0.008033505921224945, "get_robot_state": 0.01822285050327338, "get_duckie_state": 0.04309947050890876, "in-drivable-lane": 5.250000000000021, "agent_compute-ego": 0.017477520461221342, "deviation-heading": 0.8466267994542035, "complete-iteration": 0.3494882271127793, "set_robot_commands": 0.005663686585657805, "deviation-center-line": 0.23396279046430277, "driven_lanedir_consec": 0.7548045315379859, "sim_compute_sim_state": 0.03246978069972066, "sim_compute_performance-ego": 0.005333059042402842}, "ETHZ_autolab_technical_track-sc3-0-ego": {"driven_any": 0.48202596597467895, "get_ui_image": 0.06001883653494028, "step_physics": 0.13628884829007662, "survival_time": 3.2499999999999964, "driven_lanedir": 0.30719482380270824, "get_state_dump": 0.021816510420579176, "sim_render-ego": 0.007647705078125, "get_robot_state": 0.018132550899799052, "get_duckie_state": 0.04263752057002141, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.016349161588228665, "deviation-heading": 0.7576371138475069, "complete-iteration": 0.3532251688150259, "set_robot_commands": 0.0061232163355900695, "deviation-center-line": 0.185829911953006, "driven_lanedir_consec": 0.30719482380270824, "sim_compute_sim_state": 0.03844992197476901, "sim_compute_performance-ego": 0.005548818294818585}}
set_robot_commands_max0.0061232163355900695
set_robot_commands_mean0.005750875440485605
set_robot_commands_median0.0056436425188126985
set_robot_commands_min0.005593000388726956
sim_compute_performance-ego_max0.005548818294818585
sim_compute_performance-ego_mean0.005389230919445327
sim_compute_performance-ego_median0.005428184904841286
sim_compute_performance-ego_min0.005151735573280148
sim_compute_sim_state_max0.03844992197476901
sim_compute_sim_state_mean0.03228162713734169
sim_compute_sim_state_median0.031159316981090007
sim_compute_sim_state_min0.028357952612417716
sim_render-ego_max0.008033505921224945
sim_render-ego_mean0.007672077561862162
sim_render-ego_median0.007660768650196216
sim_render-ego_min0.007333267025831269
simulation-passed1
step_physics_max0.1380730485453189
step_physics_mean0.13505725796188903
step_physics_median0.13488909550724065
step_physics_min0.1323777922877559
survival_time_max10.300000000000011
survival_time_mean7.124999999999994
survival_time_min3.2499999999999964
No reset possible