Duckietown Challenges Home Challenges Submissions

Evaluator 639

ID639
evaluatoridsc-rudolf-6108
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-6108
versiond-c:4.0.14;d-c-r:4.0.18;d-s:4.0.15
first heard
last heard
statusinactive
# evaluating
# success1 17621
# timeout
# failed
# error2 17596
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
176212440Jacopo Tanirandom_agentaido2-LF-sim-validationstep1-simulationsuccessnoidsc-rudolf-61080:03:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8171921272237834
survival_time_median1.1000000000000003
deviation-center-line_median0.025668493904927793
in-drivable-lane_median0.04999999999999999


other stats
agent_compute-ego_max0.14576417521426552
agent_compute-ego_mean0.14167117110690322
agent_compute-ego_median0.142066592755525
agent_compute-ego_min0.1377150747511122
deviation-center-line_max0.057019386027434615
deviation-center-line_mean0.03634226830745448
deviation-center-line_min0.02280708605632212
deviation-heading_max0.235756116193223
deviation-heading_mean0.1638165081234764
deviation-heading_median0.1411032446854075
deviation-heading_min0.118601117425238
driven_any_max1.028556309238097
driven_any_mean0.8412197852214389
driven_any_median0.956358012875939
driven_any_min0.39300074524610096
driven_lanedir_consec_max0.9648079053779924
driven_lanedir_consec_mean0.6642871273891694
driven_lanedir_consec_min0.31973792380899546
driven_lanedir_max0.9648079053779924
driven_lanedir_mean0.6642871273891694
driven_lanedir_median0.8171921272237834
driven_lanedir_min0.31973792380899546
in-drivable-lane_max0.7000000000000004
in-drivable-lane_mean0.1700000000000001
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.956358012875939, "sim_physics": 0.013594368229741636, "survival_time": 1.1500000000000004, "driven_lanedir": 0.32939421125240065, "sim_render-ego": 0.059553664663563606, "in-drivable-lane": 0.7000000000000004, "agent_compute-ego": 0.142066592755525, "deviation-heading": 0.118601117425238, "set_robot_commands": 0.08101555575495181, "deviation-center-line": 0.02280708605632212, "driven_lanedir_consec": 0.32939421125240065, "sim_compute_sim_state": 0.038297176361083984, "sim_compute_performance-ego": 0.06105668648429539, "sim_compute_robot_state-ego": 0.065073998078056}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.028556309238097, "sim_physics": 0.01502685961516007, "survival_time": 1.1500000000000004, "driven_lanedir": 0.8903034692826748, "sim_render-ego": 0.06072327365045962, "in-drivable-lane": 0.10000000000000007, "agent_compute-ego": 0.13872130020805026, "deviation-heading": 0.2040407077275886, "set_robot_commands": 0.07402273882990298, "deviation-center-line": 0.025365426656828987, "driven_lanedir_consec": 0.8903034692826748, "sim_compute_sim_state": 0.036972149558689285, "sim_compute_performance-ego": 0.0630931335946788, "sim_compute_robot_state-ego": 0.06726934598839801}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.976866789982963, "sim_physics": 0.013589555566961117, "survival_time": 1.1000000000000003, "driven_lanedir": 0.9648079053779924, "sim_render-ego": 0.059495709159157494, "in-drivable-lane": 0, "agent_compute-ego": 0.14408871260556308, "deviation-heading": 0.1411032446854075, "set_robot_commands": 0.07600683515722101, "deviation-center-line": 0.050850948891758854, "driven_lanedir_consec": 0.9648079053779924, "sim_compute_sim_state": 0.036455967209555885, "sim_compute_performance-ego": 0.06402913006869229, "sim_compute_robot_state-ego": 0.06034669009121982}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.39300074524610096, "sim_physics": 0.01547135247124566, "survival_time": 0.45, "driven_lanedir": 0.31973792380899546, "sim_render-ego": 0.05703833368089464, "in-drivable-lane": 0.04999999999999999, "agent_compute-ego": 0.1377150747511122, "deviation-heading": 0.11958135458592496, "set_robot_commands": 0.07517602708604601, "deviation-center-line": 0.025668493904927793, "driven_lanedir_consec": 0.31973792380899546, "sim_compute_sim_state": 0.04007185830010308, "sim_compute_performance-ego": 0.0574287838406033, "sim_compute_robot_state-ego": 0.05564822090996637}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.8513170687640946, "sim_physics": 0.013293027877807615, "survival_time": 0.9500000000000004, "driven_lanedir": 0.8171921272237834, "sim_render-ego": 0.06817865371704102, "in-drivable-lane": 0, "agent_compute-ego": 0.14576417521426552, "deviation-heading": 0.235756116193223, "set_robot_commands": 0.09142845555355676, "deviation-center-line": 0.057019386027434615, "driven_lanedir_consec": 0.8171921272237834, "sim_compute_sim_state": 0.038584458200555095, "sim_compute_performance-ego": 0.0668160036990517, "sim_compute_robot_state-ego": 0.06849758248580129}}
set_robot_commands_max0.09142845555355676
set_robot_commands_mean0.0795299224763357
set_robot_commands_median0.07600683515722101
set_robot_commands_min0.07402273882990298
sim_compute_performance-ego_max0.0668160036990517
sim_compute_performance-ego_mean0.06248474753746429
sim_compute_performance-ego_median0.0630931335946788
sim_compute_performance-ego_min0.0574287838406033
sim_compute_robot_state-ego_max0.06849758248580129
sim_compute_robot_state-ego_mean0.0633671675106883
sim_compute_robot_state-ego_median0.065073998078056
sim_compute_robot_state-ego_min0.05564822090996637
sim_compute_sim_state_max0.04007185830010308
sim_compute_sim_state_mean0.03807632192599746
sim_compute_sim_state_median0.038297176361083984
sim_compute_sim_state_min0.036455967209555885
sim_physics_max0.01547135247124566
sim_physics_mean0.01419503275218322
sim_physics_median0.013594368229741636
sim_physics_min0.013293027877807615
sim_render-ego_max0.06817865371704102
sim_render-ego_mean0.06099792697422328
sim_render-ego_median0.059553664663563606
sim_render-ego_min0.05703833368089464
simulation-passed1
survival_time_max1.1500000000000004
survival_time_mean0.9600000000000004
survival_time_min0.45
No reset possible
176102413jiang pengBaseline solution using reinforcement learningaido2-LFVI-sim-validationstep1-simulationerrornoidsc-rudolf-61080:22:34
Timeout: Waited 120 [...]
Timeout:

Waited 1202 seconds for container to finish. Giving up.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
175962429Andrea Censi 🇨🇭random_agentaido2-LF-sim-validationstep1-simulationerrornoidsc-rudolf-61080:00:35
The result file is n [...]
The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-validation/submission2429/step1-simulation-idsc-rudolf-6108-job17596:

File '/tmp/duckietown/DT18/evaluator/executions/aido2-LF-sim-validation/submission2429/step1-simulation-idsc-rudolf-6108-job17596/challenge-results/challenge_results.yaml' does not exist. 

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible