Duckietown Challenges Home Challenges Submissions

Evaluator 1599

ID1599
evaluatorip-172-31-43-40-10497
ownerI don't have one 😀
machineip-172-31-43-40
processip-172-31-43-40-10497
versiond-c:5.0.7;d-c-r:5.0.11;d-s:5.1.2
first heard
last heard
statusinactive
# evaluating
# success1 25188
# timeout
# failed
# error
# aborted
# host-error4 25134
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
251884402Alex Deliyannischallenge-aido_LF-template-tensorflowaido3-LF-sim-testingstep1-simulationsuccessnoip-172-31-43-40-104970:10:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.1750559498459222
survival_time_median1.1000000000000003
deviation-center-line_median0.08071324192243379
in-drivable-lane_median0.04999999999999993


other stats
agent_compute-ego_max0.035164833068847656
agent_compute-ego_mean0.02322113730354366
agent_compute-ego_median0.02131648063659668
agent_compute-ego_min0.018740253014998
deviation-center-line_max0.2888767499293823
deviation-center-line_mean0.1060789540736528
deviation-center-line_min0.05326075722330962
deviation-heading_max1.800676815487114
deviation-heading_mean0.6801833179333552
deviation-heading_median0.4562460218657986
deviation-heading_min0.2918864839563521
driven_any_max1.4125741908897125
driven_any_mean0.3997190059067721
driven_any_median0.23272759769605292
driven_any_min0.13701734804879975
driven_lanedir_consec_max0.9076454326196453
driven_lanedir_consec_mean0.26516556056678536
driven_lanedir_consec_min0.07043894237404436
driven_lanedir_max0.9076454326196453
driven_lanedir_mean0.2680635396036654
driven_lanedir_median0.1750559498459222
driven_lanedir_min0.09039622841807791
in-drivable-lane_max1.1999999999999955
in-drivable-lane_mean0.21666666666666615
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.23272759769605292, "sim_physics": 0.15325277501886542, "survival_time": 1.1000000000000003, "driven_lanedir": 0.15454062649654654, "sim_render-ego": 0.014771439812400124, "in-drivable-lane": 0, "agent_compute-ego": 0.035164833068847656, "deviation-heading": 0.6139566152574653, "set_robot_commands": 0.010696671225807884, "deviation-center-line": 0.0764295348229879, "driven_lanedir_consec": 0.15454062649654654, "sim_compute_sim_state": 0.00527246431870894, "sim_compute_performance-ego": 0.008851636539806019, "sim_compute_robot_state-ego": 0.011033903468738905}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.8640535208355179, "sim_physics": 0.09447372905791752, "survival_time": 3.149999999999997, "driven_lanedir": 0.6474136017400851, "sim_render-ego": 0.01093639646257673, "in-drivable-lane": 0.2499999999999991, "agent_compute-ego": 0.02380273834107414, "deviation-heading": 1.1818381704235277, "set_robot_commands": 0.008078155063447497, "deviation-center-line": 0.2879441170370926, "driven_lanedir_consec": 0.6474136017400851, "sim_compute_sim_state": 0.004337450814625573, "sim_compute_performance-ego": 0.0066221971360463944, "sim_compute_robot_state-ego": 0.008341372959197514}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.6747642243328454, "sim_physics": 0.13466250896453855, "survival_time": 2.499999999999999, "driven_lanedir": 0.29245256588282986, "sim_render-ego": 0.012743825912475588, "in-drivable-lane": 0.9999999999999984, "agent_compute-ego": 0.029638185501098632, "deviation-heading": 0.6310981049356266, "set_robot_commands": 0.011888041496276857, "deviation-center-line": 0.09042358970377462, "driven_lanedir_consec": 0.29245256588282986, "sim_compute_sim_state": 0.0052205705642700195, "sim_compute_performance-ego": 0.008088355064392089, "sim_compute_robot_state-ego": 0.009955000877380372}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4236177348340329, "sim_physics": 0.12404134196619834, "survival_time": 1.5500000000000007, "driven_lanedir": 0.3910309204164717, "sim_render-ego": 0.012261506049863754, "in-drivable-lane": 0, "agent_compute-ego": 0.02570909069430443, "deviation-heading": 0.5077712542310505, "set_robot_commands": 0.008871493800993889, "deviation-center-line": 0.09487392393917612, "driven_lanedir_consec": 0.3910309204164717, "sim_compute_sim_state": 0.00461609132828251, "sim_compute_performance-ego": 0.007308421596404045, "sim_compute_robot_state-ego": 0.00905371481372464}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.33430511913327876, "sim_physics": 0.12038822357471173, "survival_time": 1.3000000000000005, "driven_lanedir": 0.2852938759171679, "sim_render-ego": 0.012607767031742975, "in-drivable-lane": 0, "agent_compute-ego": 0.028697554881756123, "deviation-heading": 0.43022977056173506, "set_robot_commands": 0.01016704852764423, "deviation-center-line": 0.06518231632418017, "driven_lanedir_consec": 0.2852938759171679, "sim_compute_sim_state": 0.005145797362694373, "sim_compute_performance-ego": 0.006981400343088003, "sim_compute_robot_state-ego": 0.009164773500882665}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.4125741908897125, "sim_physics": 0.09073591463774151, "survival_time": 5.14999999999999, "driven_lanedir": 0.9076454326196453, "sim_render-ego": 0.011227904014217045, "in-drivable-lane": 1.1999999999999955, "agent_compute-ego": 0.024141705151900505, "deviation-heading": 1.489169767199274, "set_robot_commands": 0.008034754725335871, "deviation-center-line": 0.2888767499293823, "driven_lanedir_consec": 0.9076454326196453, "sim_compute_sim_state": 0.00450387278806816, "sim_compute_performance-ego": 0.006340260644560879, "sim_compute_robot_state-ego": 0.008210006269436437}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.45136345675440653, "sim_physics": 0.06686905452183314, "survival_time": 2.4499999999999993, "driven_lanedir": 0.1139086279272452, "sim_render-ego": 0.008210337891870615, "in-drivable-lane": 0.5999999999999988, "agent_compute-ego": 0.020366469208075076, "deviation-heading": 1.800676815487114, "set_robot_commands": 0.006222525421453982, "deviation-center-line": 0.10923923941874604, "driven_lanedir_consec": 0.07043894237404436, "sim_compute_sim_state": 0.0033315882390859176, "sim_compute_performance-ego": 0.004462261589205995, "sim_compute_robot_state-ego": 0.0057980041114651425}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.13701734804879975, "sim_physics": 0.07662111170151654, "survival_time": 0.8500000000000002, "driven_lanedir": 0.0908828989571706, "sim_render-ego": 0.009444909937241498, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.02071447933421415, "deviation-heading": 0.4243271001415093, "set_robot_commands": 0.006939369089463178, "deviation-center-line": 0.06856636343013063, "driven_lanedir_consec": 0.0908828989571706, "sim_compute_sim_state": 0.003778892404892865, "sim_compute_performance-ego": 0.00514831262476304, "sim_compute_robot_state-ego": 0.006669016445384306}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.1528933094070216, "sim_physics": 0.06861805032800745, "survival_time": 1.3500000000000003, "driven_lanedir": 0.09039622841807791, "sim_render-ego": 0.007623610673127351, "in-drivable-lane": 0, "agent_compute-ego": 0.019006013870239254, "deviation-heading": 0.8764395317610932, "set_robot_commands": 0.005698636726096824, "deviation-center-line": 0.08071324192243379, "driven_lanedir_consec": 0.09039622841807791, "sim_compute_sim_state": 0.003169836821379485, "sim_compute_performance-ego": 0.004210127724541558, "sim_compute_robot_state-ego": 0.005588231263337312}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.21298388640949556, "sim_physics": 0.07574615478515626, "survival_time": 1.0000000000000002, "driven_lanedir": 0.16808341278400274, "sim_render-ego": 0.008192241191864014, "in-drivable-lane": 0.04999999999999993, "agent_compute-ego": 0.02131648063659668, "deviation-heading": 0.33876230591370177, "set_robot_commands": 0.005757248401641846, "deviation-center-line": 0.06886103099000374, "driven_lanedir_consec": 0.16808341278400274, "sim_compute_sim_state": 0.0033871769905090333, "sim_compute_performance-ego": 0.004573678970336914, "sim_compute_robot_state-ego": 0.005889344215393067}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.22086591364753969, "sim_physics": 0.08074039220809937, "survival_time": 1.0000000000000002, "driven_lanedir": 0.1750559498459222, "sim_render-ego": 0.018288278579711915, "in-drivable-lane": 0.04999999999999993, "agent_compute-ego": 0.021498346328735353, "deviation-heading": 0.2918864839563521, "set_robot_commands": 0.0072426915168762205, "deviation-center-line": 0.0835238729638319, "driven_lanedir_consec": 0.1750559498459222, "sim_compute_sim_state": 0.004076635837554932, "sim_compute_performance-ego": 0.005689632892608642, "sim_compute_robot_state-ego": 0.008035588264465331}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.20838960896240927, "sim_physics": 0.07495625813802083, "survival_time": 0.9000000000000002, "driven_lanedir": 0.15367647453533428, "sim_render-ego": 0.007862448692321777, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.019774648878309466, "deviation-heading": 0.35506513134809353, "set_robot_commands": 0.005993432468838162, "deviation-center-line": 0.05326075722330962, "driven_lanedir_consec": 0.15367647453533428, "sim_compute_sim_state": 0.0032151142756144204, "sim_compute_performance-ego": 0.004481090439690484, "sim_compute_robot_state-ego": 0.005601843198140462}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.29264949221172876, "sim_physics": 0.07078739729794589, "survival_time": 1.1000000000000003, "driven_lanedir": 0.2267084531748793, "sim_render-ego": 0.009277072819796476, "in-drivable-lane": 0, "agent_compute-ego": 0.018740253014998, "deviation-heading": 0.4562460218657986, "set_robot_commands": 0.006686275655573065, "deviation-center-line": 0.05964534928555487, "driven_lanedir_consec": 0.2267084531748793, "sim_compute_sim_state": 0.0035904104059392757, "sim_compute_performance-ego": 0.004754499955610795, "sim_compute_robot_state-ego": 0.006144892085682262}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.23234067802643577, "sim_physics": 0.06914300918579101, "survival_time": 1.0000000000000002, "driven_lanedir": 0.20032761743169036, "sim_render-ego": 0.01682922840118408, "in-drivable-lane": 0, "agent_compute-ego": 0.020147204399108887, "deviation-heading": 0.4281029304082001, "set_robot_commands": 0.0056498050689697266, "deviation-center-line": 0.05343792437860645, "driven_lanedir_consec": 0.20032761743169036, "sim_compute_sim_state": 0.0032124996185302736, "sim_compute_performance-ego": 0.004231977462768555, "sim_compute_robot_state-ego": 0.005566811561584473}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.14523900741230425, "sim_physics": 0.075321626663208, "survival_time": 1.0000000000000002, "driven_lanedir": 0.12353640790791244, "sim_render-ego": 0.009176552295684814, "in-drivable-lane": 0, "agent_compute-ego": 0.019599056243896483, "deviation-heading": 0.3771797655097867, "set_robot_commands": 0.007039415836334229, "deviation-center-line": 0.11020629973558103, "driven_lanedir_consec": 0.12353640790791244, "sim_compute_sim_state": 0.0033488035202026366, "sim_compute_performance-ego": 0.005124282836914062, "sim_compute_robot_state-ego": 0.006821584701538086}}
set_robot_commands_max0.011888041496276857
set_robot_commands_mean0.00766437100165023
set_robot_commands_median0.007039415836334229
set_robot_commands_min0.0056498050689697266
sim_compute_performance-ego_max0.008851636539806019
sim_compute_performance-ego_mean0.005791209054715832
sim_compute_performance-ego_median0.00514831262476304
sim_compute_performance-ego_min0.004210127724541558
sim_compute_robot_state-ego_max0.011033903468738905
sim_compute_robot_state-ego_mean0.007458272515756732
sim_compute_robot_state-ego_median0.006821584701538086
sim_compute_robot_state-ego_min0.005566811561584473
sim_compute_sim_state_max0.00527246431870894
sim_compute_sim_state_mean0.0040138136860238946
sim_compute_sim_state_median0.003778892404892865
sim_compute_sim_state_min0.003169836821379485
sim_physics_max0.15325277501886542
sim_physics_mean0.0917571698699701
sim_physics_median0.07662111170151654
sim_physics_min0.06686905452183314
sim_render-ego_max0.018288278579711915
sim_render-ego_mean0.011296901317738584
sim_render-ego_median0.01093639646257673
sim_render-ego_min0.007623610673127351
simulation-passed1
survival_time_max5.14999999999999
survival_time_mean1.6933333333333325
survival_time_min0.8500000000000002
No reset possible
251573264Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104970:00:04
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
251473265Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104970:00:03
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
251443265Liu Sam 🇹🇼challenge-aido_LF-template-randomaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104970:00:04
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
251343257Liu Sam 🇹🇼PredictorLastaido2-PREDstep1-simulationhost-erroryesip-172-31-43-40-104970:00:24
Error while running [...]

stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible