Duckietown Challenges Home Challenges Submissions

Evaluator 1050

ID1050
evaluatorip-172-31-42-167-7357
ownerI don't have one 😀
machineip-172-31-42-167
processip-172-31-42-167-7357
versiond-c:4.0.28;d-c-r:4.0.39;d-s:4.0.21
first heard
last heard
statusinactive
# evaluating
# success2 21985
# timeout
# failed
# error
# aborted
# host-error1 21976
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
219923364Amaury Camus 🇨🇭challenge-aido_LF-template-randomaido2-LF-sim-validationstep1-simulationsuccessnoip-172-31-42-167-73570:03:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.13005513204193506
in-drivable-lane_median0.29999999999999893


other stats
agent_compute-ego_max0.08806822776794433
agent_compute-ego_mean0.08688868220561699
agent_compute-ego_median0.08673389285218482
agent_compute-ego_min0.08523286090177648
deviation-center-line_max0.2072381278624381
deviation-center-line_mean0.13189395308989768
deviation-center-line_min0.0788839760023766
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3206717506075888
deviation-heading_median0.23606825392187256
deviation-heading_min0.19401296908406307
driven_any_max1.618214457700942
driven_any_mean0.8613785859332687
driven_any_median0.9933867744747804
driven_any_min0.23626560261560617
driven_lanedir_consec_max0.8010406009604816
driven_lanedir_consec_mean0.5597645764431625
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max0.8010406009604816
driven_lanedir_mean0.5597645764431625
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.619999999999998
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.05683290075372766, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.035388323995802135, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.08669657618911178, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.05423364374372694, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.02252677634910301, "sim_compute_performance-ego": 0.03909821863527651, "sim_compute_robot_state-ego": 0.040584034389919706}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.05645544856202369, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.03509971674750833, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.08673389285218482, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.05533231473436542, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.02257101208555932, "sim_compute_performance-ego": 0.03813317710278081, "sim_compute_robot_state-ego": 0.04026628008075789}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.0597844544579001, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.03506813329808852, "in-drivable-lane": 0, "agent_compute-ego": 0.08523286090177648, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.05519437789916992, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.022361264509313247, "sim_compute_performance-ego": 0.03803338724024156, "sim_compute_robot_state-ego": 0.040248926948098576}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.06115565299987793, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.03545297622680664, "in-drivable-lane": 0, "agent_compute-ego": 0.08806822776794433, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.05584936141967774, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.02258556365966797, "sim_compute_performance-ego": 0.03880060195922851, "sim_compute_robot_state-ego": 0.04030109405517578}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.618214457700942, "sim_physics": 0.055160537550720985, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.03551128242589251, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.08771185331706759, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.054219194605380674, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.022499090508569644, "sim_compute_performance-ego": 0.037970829613601105, "sim_compute_robot_state-ego": 0.04084900964664507}}
set_robot_commands_max0.05584936141967774
set_robot_commands_mean0.054965778480464134
set_robot_commands_median0.05519437789916992
set_robot_commands_min0.054219194605380674
sim_compute_performance-ego_max0.03909821863527651
sim_compute_performance-ego_mean0.038407242910225695
sim_compute_performance-ego_median0.03813317710278081
sim_compute_performance-ego_min0.037970829613601105
sim_compute_robot_state-ego_max0.04084900964664507
sim_compute_robot_state-ego_mean0.0404498690241194
sim_compute_robot_state-ego_median0.04030109405517578
sim_compute_robot_state-ego_min0.040248926948098576
sim_compute_sim_state_max0.02258556365966797
sim_compute_sim_state_mean0.02250874142244264
sim_compute_sim_state_median0.02252677634910301
sim_compute_sim_state_min0.022361264509313247
sim_physics_max0.06115565299987793
sim_physics_mean0.057877798864850075
sim_physics_median0.05683290075372766
sim_physics_min0.055160537550720985
sim_render-ego_max0.03551128242589251
sim_render-ego_mean0.03530408653881963
sim_render-ego_median0.035388323995802135
sim_render-ego_min0.03506813329808852
simulation-passed1
survival_time_max3.949999999999994
survival_time_mean2.2599999999999985
survival_time_min0.8500000000000002
No reset possible
219853363Amaury Camus 🇨🇭challenge-aido_LF-template-randomaido2-LF-sim-testingstep1-simulationsuccessnoip-172-31-42-167-73570:12:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.5723310224856362
survival_time_median2.549999999999999
deviation-center-line_median0.10099507669212504
in-drivable-lane_median0.4499999999999984


other stats
agent_compute-ego_max0.1523431451232345
agent_compute-ego_mean0.13173887886134347
agent_compute-ego_median0.13763713836669922
agent_compute-ego_min0.1055595267052744
deviation-center-line_max0.2838635700333795
deviation-center-line_mean0.12786165368054708
deviation-center-line_min0.07112529346797256
deviation-heading_max0.5289022616823588
deviation-heading_mean0.3391682685652287
deviation-heading_median0.32928385103533875
deviation-heading_min0.19401296908406307
driven_any_max2.1706243860917107
driven_any_mean0.9759490378276146
driven_any_median1.0117660515775693
driven_any_min0.23626560261560617
driven_lanedir_consec_max1.919400421063832
driven_lanedir_consec_mean0.6896171461819246
driven_lanedir_consec_min0.2264965947033466
driven_lanedir_max1.919400421063832
driven_lanedir_mean0.6896171461819246
driven_lanedir_median0.5723310224856362
driven_lanedir_min0.2264965947033466
in-drivable-lane_max1.7999999999999936
in-drivable-lane_mean0.5899999999999982
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429708, "sim_physics": 0.10890038808186848, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.06313939447756167, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.1523431451232345, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.10139810155939172, "deviation-center-line": 0.15907928712407893, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.03778783921842222, "sim_compute_performance-ego": 0.06830312587596753, "sim_compute_robot_state-ego": 0.07319400045606825}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.1093299856372908, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.05999111661724016, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.15121586182538202, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.09817259451922249, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.037427149566949584, "sim_compute_performance-ego": 0.06893383287915997, "sim_compute_robot_state-ego": 0.07572823879765529}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.10710832651923684, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.05672586665433996, "in-drivable-lane": 0, "agent_compute-ego": 0.13763713836669922, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.08636784553527832, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.03819436185500201, "sim_compute_performance-ego": 0.06410574913024902, "sim_compute_robot_state-ego": 0.06851459951961741}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.10791086196899416, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.055154733657836914, "in-drivable-lane": 0, "agent_compute-ego": 0.1398337173461914, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.0927656364440918, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.03457198143005371, "sim_compute_performance-ego": 0.06513916969299316, "sim_compute_robot_state-ego": 0.06597361564636231}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.618214457700942, "sim_physics": 0.09788737417776372, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.05646851394749895, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.13674249226533913, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.08145052571839924, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.03535848327829868, "sim_compute_performance-ego": 0.061501140835918955, "sim_compute_robot_state-ego": 0.06721269631687599}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.0038514956833002, "sim_physics": 0.10488743684729752, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735944, "sim_render-ego": 0.0591925893511091, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.14127301196662748, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.0869143447097467, "deviation-center-line": 0.08968283443958727, "driven_lanedir_consec": 0.7531703123735944, "sim_compute_sim_state": 0.039204933205429386, "sim_compute_performance-ego": 0.06274346429474499, "sim_compute_robot_state-ego": 0.08801420367493921}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504375, "sim_physics": 0.09527243886675152, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.05958274006843567, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.14203348755836487, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.08847705807004656, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.036158663885934014, "sim_compute_performance-ego": 0.0667863062449864, "sim_compute_robot_state-ego": 0.07252073287963867}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.12928770837329684, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.058181717282249815, "in-drivable-lane": 0, "agent_compute-ego": 0.13949573607671828, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.09219219571068174, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.03559100060235886, "sim_compute_performance-ego": 0.06442446935744513, "sim_compute_robot_state-ego": 0.07052250135512579}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510765, "sim_physics": 0.098834482289977, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.05846549292742196, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.1403079275357521, "deviation-heading": 0.32928385103533875, "set_robot_commands": 0.0872020236516403, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.03586976406937939, "sim_compute_performance-ego": 0.06313068583860236, "sim_compute_robot_state-ego": 0.07064560308294782}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.0435481553257333, "sim_physics": 0.09325561089949176, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.05408049063249068, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.12938962416215377, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.0801390907981179, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.03275375366210938, "sim_compute_performance-ego": 0.06120073145086115, "sim_compute_robot_state-ego": 0.06026905666698109}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193265, "sim_physics": 0.07893817455737621, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848754, "sim_render-ego": 0.046852037504121855, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.11750141986004718, "deviation-heading": 0.3566810710676767, "set_robot_commands": 0.0734635235427262, "deviation-center-line": 0.22513914436082189, "driven_lanedir_consec": 1.1042463557848754, "sim_compute_sim_state": 0.030542327212048817, "sim_compute_performance-ego": 0.05111587512028682, "sim_compute_robot_state-ego": 0.051294803619384766}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.08461555412837438, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.04715371131896973, "in-drivable-lane": 0, "agent_compute-ego": 0.1180327194077628, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.07080568586077009, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.02811143227985927, "sim_compute_performance-ego": 0.05025919846126011, "sim_compute_robot_state-ego": 0.05059354645865304}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775693, "sim_physics": 0.07512141676510081, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.04112403533037971, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.1055595267052744, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.06616436266431622, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.027626472360947552, "sim_compute_performance-ego": 0.04607826120713178, "sim_compute_robot_state-ego": 0.05023399053835401}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.0836913232450132, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.050450801849365234, "in-drivable-lane": 0, "agent_compute-ego": 0.11531689431932238, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.06747661696539985, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.03364164740950973, "sim_compute_performance-ego": 0.05693459510803223, "sim_compute_robot_state-ego": 0.05026921519526729}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.07201126977509144, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063832, "sim_render-ego": 0.04296873831281475, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.1094004804012822, "deviation-heading": 0.5178939525116442, "set_robot_commands": 0.06554784961775237, "deviation-center-line": 0.2838635700333795, "driven_lanedir_consec": 1.919400421063832, "sim_compute_sim_state": 0.027462267408183975, "sim_compute_performance-ego": 0.04654982744478712, "sim_compute_robot_state-ego": 0.0490105666366278}}
set_robot_commands_max0.10139810155939172
set_robot_commands_mean0.0825691636911721
set_robot_commands_median0.08636784553527832
set_robot_commands_min0.06554784961775237
sim_compute_performance-ego_max0.06893383287915997
sim_compute_performance-ego_mean0.05981376219616178
sim_compute_performance-ego_median0.06274346429474499
sim_compute_performance-ego_min0.04607826120713178
sim_compute_robot_state-ego_max0.08801420367493921
sim_compute_robot_state-ego_mean0.06426649138963324
sim_compute_robot_state-ego_median0.06721269631687599
sim_compute_robot_state-ego_min0.0490105666366278
sim_compute_sim_state_max0.039204933205429386
sim_compute_sim_state_mean0.03402013849629911
sim_compute_sim_state_median0.03535848327829868
sim_compute_sim_state_min0.027462267408183975
sim_physics_max0.12928770837329684
sim_physics_mean0.09647015680886166
sim_physics_median0.09788737417776372
sim_physics_min0.07201126977509144
sim_render-ego_max0.06313939447756167
sim_render-ego_mean0.05396879866212241
sim_render-ego_median0.05646851394749895
sim_render-ego_min0.04112403533037971
simulation-passed1
survival_time_max5.09999999999999
survival_time_mean2.503333333333331
survival_time_min0.8500000000000002
No reset possible
219763368Amaury Camus 🇨🇭challenge-aido_LF-template-randomaido2-LFV-sim-validationstep1-simulationhost-errornoip-172-31-42-167-73570:04:43
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 1066, in run_docker
    subprocess.check_call(cmd0, cwd=cwd, stdout=tee_stdout, stderr=tee_stderr)
  File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['docker-compose', '-p', 'job21976-156474', 'down']' returned non-zero exit status 1.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 535, in get_cr
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1071, in run_docker
    raise DockerComposeFail(msg) from e
duckietown_challenges_runner.runner.DockerComposeFail: Could not run ['docker-compose', '-p', 'job21976-156474', 'down']:

   >  Command '['docker-compose', '-p', 'job21976-156474', 'down']' returned non-zero exit status 1.

stdout | 

stderr | 
stderr |         Can't find a suitable configuration file in this directory or any
stderr |         parent. Are you in the right directory?
stderr | 
stderr |         Supported filenames: docker-compose.yml, docker-compose.yaml
stderr | 
stderr |
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible