Duckietown Challenges Home Challenges Submissions

Evaluator 636

ID636
evaluatoridsc-rudolf-5273
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-5273
versiond-c:4.0.14;d-c-r:4.0.18;d-s:4.0.15
first heard
last heard
statusinactive
# evaluating
# success2 17620
# timeout
# failed
# error2 17600
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
176232445Claudio RuchPredictorMeanaido2-PREDstep1-simulationsuccessnoidsc-rudolf-52730:00:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
error_L10.273147697176924
error_L20.0901067808784513


No reset possible
176202439Jacopo Tanirandom_agentaido2-LF-sim-testingstep1-simulationsuccessnoidsc-rudolf-52730:03:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8171921272237834
survival_time_median1.1000000000000003
deviation-center-line_median0.025668493904927793
in-drivable-lane_median0.04999999999999999


other stats
agent_compute-ego_max0.15972344498885305
agent_compute-ego_mean0.1457784627569181
agent_compute-ego_median0.14369964599609375
agent_compute-ego_min0.13463030690732208
deviation-center-line_max0.057019386027434615
deviation-center-line_mean0.03634226830745448
deviation-center-line_min0.02280708605632212
deviation-heading_max0.235756116193223
deviation-heading_mean0.1638165081234764
deviation-heading_median0.1411032446854075
deviation-heading_min0.118601117425238
driven_any_max1.028556309238097
driven_any_mean0.8412197852214389
driven_any_median0.956358012875939
driven_any_min0.39300074524610096
driven_lanedir_consec_max0.9648079053779924
driven_lanedir_consec_mean0.6642871273891694
driven_lanedir_consec_min0.31973792380899546
driven_lanedir_max0.9648079053779924
driven_lanedir_mean0.6642871273891694
driven_lanedir_median0.8171921272237834
driven_lanedir_min0.31973792380899546
in-drivable-lane_max0.7000000000000004
in-drivable-lane_mean0.1700000000000001
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.956358012875939, "sim_physics": 0.014317439950030784, "survival_time": 1.1500000000000004, "driven_lanedir": 0.32939421125240065, "sim_render-ego": 0.059403118879898735, "in-drivable-lane": 0.7000000000000004, "agent_compute-ego": 0.14369964599609375, "deviation-heading": 0.118601117425238, "set_robot_commands": 0.08365635249925696, "deviation-center-line": 0.02280708605632212, "driven_lanedir_consec": 0.32939421125240065, "sim_compute_sim_state": 0.03995315924934719, "sim_compute_performance-ego": 0.06061128948045814, "sim_compute_robot_state-ego": 0.06207873510277789}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.028556309238097, "sim_physics": 0.014702807302060337, "survival_time": 1.1500000000000004, "driven_lanedir": 0.8903034692826748, "sim_render-ego": 0.0549705754155698, "in-drivable-lane": 0.10000000000000007, "agent_compute-ego": 0.13463030690732208, "deviation-heading": 0.2040407077275886, "set_robot_commands": 0.07921813881915549, "deviation-center-line": 0.025365426656828987, "driven_lanedir_consec": 0.8903034692826748, "sim_compute_sim_state": 0.03661017832548722, "sim_compute_performance-ego": 0.06098047546718432, "sim_compute_robot_state-ego": 0.06400412061940068}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.976866789982963, "sim_physics": 0.013846061446449974, "survival_time": 1.1000000000000003, "driven_lanedir": 0.9648079053779924, "sim_render-ego": 0.06024413759058172, "in-drivable-lane": 0, "agent_compute-ego": 0.14903766458684747, "deviation-heading": 0.1411032446854075, "set_robot_commands": 0.07382126287980513, "deviation-center-line": 0.050850948891758854, "driven_lanedir_consec": 0.9648079053779924, "sim_compute_sim_state": 0.03950379111550071, "sim_compute_performance-ego": 0.061536062847484245, "sim_compute_robot_state-ego": 0.06333709846843373}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.39300074524610096, "sim_physics": 0.013714922799004449, "survival_time": 0.45, "driven_lanedir": 0.31973792380899546, "sim_render-ego": 0.05765769216749403, "in-drivable-lane": 0.04999999999999999, "agent_compute-ego": 0.14180125130547416, "deviation-heading": 0.11958135458592496, "set_robot_commands": 0.07337752978006999, "deviation-center-line": 0.025668493904927793, "driven_lanedir_consec": 0.31973792380899546, "sim_compute_sim_state": 0.03863538636101617, "sim_compute_performance-ego": 0.056982729170057506, "sim_compute_robot_state-ego": 0.05672846900092231}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.8513170687640946, "sim_physics": 0.01513696971692537, "survival_time": 0.9500000000000004, "driven_lanedir": 0.8171921272237834, "sim_render-ego": 0.06419869473105982, "in-drivable-lane": 0, "agent_compute-ego": 0.15972344498885305, "deviation-heading": 0.235756116193223, "set_robot_commands": 0.0855995730349892, "deviation-center-line": 0.057019386027434615, "driven_lanedir_consec": 0.8171921272237834, "sim_compute_sim_state": 0.041844493464419714, "sim_compute_performance-ego": 0.06540034946642424, "sim_compute_robot_state-ego": 0.06720020896510075}}
set_robot_commands_max0.0855995730349892
set_robot_commands_mean0.07913457140265537
set_robot_commands_median0.07921813881915549
set_robot_commands_min0.07337752978006999
sim_compute_performance-ego_max0.06540034946642424
sim_compute_performance-ego_mean0.06110218128632169
sim_compute_performance-ego_median0.06098047546718432
sim_compute_performance-ego_min0.056982729170057506
sim_compute_robot_state-ego_max0.06720020896510075
sim_compute_robot_state-ego_mean0.06266972643132708
sim_compute_robot_state-ego_median0.06333709846843373
sim_compute_robot_state-ego_min0.05672846900092231
sim_compute_sim_state_max0.041844493464419714
sim_compute_sim_state_mean0.0393094017031542
sim_compute_sim_state_median0.03950379111550071
sim_compute_sim_state_min0.03661017832548722
sim_physics_max0.01513696971692537
sim_physics_mean0.014343640242894182
sim_physics_median0.014317439950030784
sim_physics_min0.013714922799004449
sim_render-ego_max0.06419869473105982
sim_render-ego_mean0.059294843756920824
sim_render-ego_median0.059403118879898735
sim_render-ego_min0.0549705754155698
simulation-passed1
survival_time_max1.1500000000000004
survival_time_mean0.9600000000000004
survival_time_min0.45
No reset possible
176082412jiang pengBaseline solution using reinforcement learningaido2-LFVI-sim-testingstep1-simulationerrornoidsc-rudolf-52730:22:41
Timeout: Waited 120 [...]
Timeout:

Waited 1202 seconds for container to finish. Giving up. 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
176002432Andrea Censi 🇨🇭random_agentaido2-LFV-sim-testingstep1-simulationerrornoidsc-rudolf-52730:00:39
The result file is n [...]
The result file is not found in working dir /tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission2432/step1-simulation-idsc-rudolf-5273-job17600:

File '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-testing/submission2432/step1-simulation-idsc-rudolf-5273-job17600/challenge-results/challenge_results.yaml' does not exist. 

This usually means that the evaluator did not finish and some times that there was an import error.
Check the evaluator log to see what happened.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible