Duckietown Challenges Home Challenges Submissions

Evaluator 640

ID640
evaluatorInstance1
ownerJulian Zilly
machineidsc-rudolf
processInstance1
versiond-c:4.0.14;d-c-r:4.0.18;d-s:4.0.15
first heard
last heard
statusinactive
# evaluating
# success2 17603
# timeout
# failed
# error2 17605
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
176222440Jacopo Tanirandom_agentaido2-LF-sim-validationstep1-simulationsuccessnoInstance10:03:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8171921272237834
survival_time_median1.1000000000000003
deviation-center-line_median0.025668493904927793
in-drivable-lane_median0.04999999999999999


other stats
agent_compute-ego_max0.16978838708665636
agent_compute-ego_mean0.14173893715861743
agent_compute-ego_median0.13535587411177785
agent_compute-ego_min0.1322293592535931
deviation-center-line_max0.057019386027434615
deviation-center-line_mean0.03634226830745448
deviation-center-line_min0.02280708605632212
deviation-heading_max0.235756116193223
deviation-heading_mean0.1638165081234764
deviation-heading_median0.1411032446854075
deviation-heading_min0.118601117425238
driven_any_max1.028556309238097
driven_any_mean0.8412197852214389
driven_any_median0.956358012875939
driven_any_min0.39300074524610096
driven_lanedir_consec_max0.9648079053779924
driven_lanedir_consec_mean0.6642871273891694
driven_lanedir_consec_min0.31973792380899546
driven_lanedir_max0.9648079053779924
driven_lanedir_mean0.6642871273891694
driven_lanedir_median0.8171921272237834
driven_lanedir_min0.31973792380899546
in-drivable-lane_max0.7000000000000004
in-drivable-lane_mean0.1700000000000001
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.956358012875939, "sim_physics": 0.013548249783723251, "survival_time": 1.1500000000000004, "driven_lanedir": 0.32939421125240065, "sim_render-ego": 0.054902024891065514, "in-drivable-lane": 0.7000000000000004, "agent_compute-ego": 0.1322293592535931, "deviation-heading": 0.118601117425238, "set_robot_commands": 0.07254475095997685, "deviation-center-line": 0.02280708605632212, "driven_lanedir_consec": 0.32939421125240065, "sim_compute_sim_state": 0.03611536647962487, "sim_compute_performance-ego": 0.0602603062339451, "sim_compute_robot_state-ego": 0.060872191968171493}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.028556309238097, "sim_physics": 0.015141466389531675, "survival_time": 1.1500000000000004, "driven_lanedir": 0.8903034692826748, "sim_render-ego": 0.05692540044369905, "in-drivable-lane": 0.10000000000000007, "agent_compute-ego": 0.13618906684543777, "deviation-heading": 0.2040407077275886, "set_robot_commands": 0.07576124564461086, "deviation-center-line": 0.025365426656828987, "driven_lanedir_consec": 0.8903034692826748, "sim_compute_sim_state": 0.034527944481891136, "sim_compute_performance-ego": 0.05937832334767217, "sim_compute_robot_state-ego": 0.061205936514812966}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.976866789982963, "sim_physics": 0.013612519611011852, "survival_time": 1.1000000000000003, "driven_lanedir": 0.9648079053779924, "sim_render-ego": 0.05854190479625355, "in-drivable-lane": 0, "agent_compute-ego": 0.1351319984956221, "deviation-heading": 0.1411032446854075, "set_robot_commands": 0.07501828670501709, "deviation-center-line": 0.050850948891758854, "driven_lanedir_consec": 0.9648079053779924, "sim_compute_sim_state": 0.04010791128331965, "sim_compute_performance-ego": 0.061362288214943626, "sim_compute_robot_state-ego": 0.06581452759829434}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.39300074524610096, "sim_physics": 0.01610541343688965, "survival_time": 0.45, "driven_lanedir": 0.31973792380899546, "sim_render-ego": 0.06330249044630262, "in-drivable-lane": 0.04999999999999999, "agent_compute-ego": 0.16978838708665636, "deviation-heading": 0.11958135458592496, "set_robot_commands": 0.08112544483608669, "deviation-center-line": 0.025668493904927793, "driven_lanedir_consec": 0.31973792380899546, "sim_compute_sim_state": 0.03538603252834744, "sim_compute_performance-ego": 0.06621252165900336, "sim_compute_robot_state-ego": 0.06904061635335286}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.8513170687640946, "sim_physics": 0.013323846616243062, "survival_time": 0.9500000000000004, "driven_lanedir": 0.8171921272237834, "sim_render-ego": 0.05766690404791581, "in-drivable-lane": 0, "agent_compute-ego": 0.13535587411177785, "deviation-heading": 0.235756116193223, "set_robot_commands": 0.08274976830733449, "deviation-center-line": 0.057019386027434615, "driven_lanedir_consec": 0.8171921272237834, "sim_compute_sim_state": 0.03736544910230135, "sim_compute_performance-ego": 0.06020136883384303, "sim_compute_robot_state-ego": 0.061458361776251545}}
set_robot_commands_max0.08274976830733449
set_robot_commands_mean0.0774398992906052
set_robot_commands_median0.07576124564461086
set_robot_commands_min0.07254475095997685
sim_compute_performance-ego_max0.06621252165900336
sim_compute_performance-ego_mean0.06148296165788146
sim_compute_performance-ego_median0.0602603062339451
sim_compute_performance-ego_min0.05937832334767217
sim_compute_robot_state-ego_max0.06904061635335286
sim_compute_robot_state-ego_mean0.06367832684217664
sim_compute_robot_state-ego_median0.061458361776251545
sim_compute_robot_state-ego_min0.060872191968171493
sim_compute_sim_state_max0.04010791128331965
sim_compute_sim_state_mean0.03670054077509689
sim_compute_sim_state_median0.03611536647962487
sim_compute_sim_state_min0.034527944481891136
sim_physics_max0.01610541343688965
sim_physics_mean0.014346299167479898
sim_physics_median0.013612519611011852
sim_physics_min0.013323846616243062
sim_render-ego_max0.06330249044630262
sim_render-ego_mean0.058267744925047306
sim_render-ego_median0.05766690404791581
sim_render-ego_min0.054902024891065514
simulation-passed1
survival_time_max1.1500000000000004
survival_time_mean0.9600000000000004
survival_time_min0.45
No reset possible
176112411jiang pengBaseline solution using reinforcement learningaido2-LFV-sim-validationstep1-simulationerrornoInstance10:21:44
Timeout: Waited 120 [...]
Timeout:

Waited 1202 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
176052436Andrea Censi 🇨🇭random_agentaido2-LFVI-sim-testingstep1-simulationerrornoInstance10:00:36
The result file is n [...]
The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2436/step1-simulation-Instance1-job17605:

File '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2436/step1-simulation-Instance1-job17605/challenge-results/challenge_results.yaml' does not exist. 

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
176032438Liam Paull 🇨🇦PredictorLastaido2-PREDstep1-simulationsuccessnoInstance10:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.10927995892015312
error_L20.0192379661837549


No reset possible