Duckietown Challenges Home Challenges Submissions

Evaluator 69

ID69
evaluatoridsc-rudolf-7787
ownerAndrea CensiΒ πŸ‡¨πŸ‡­
machineidsc-rudolf
processidsc-rudolf-7787
version3.0.36
first heard
last heard
statusinactive
# evaluating
# success15 240
# timeout
# failed
# error2 296
# aborted1 297
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
351194Florian GolemoPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrornoidsc-rudolf-77870:00:35
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 392, in run
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/challenge_results.py", line 65, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission194/step1-simulation-idsc-rudolf-7787-job351/challenge-results/challenge_results.yaml' does not exist.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
349193Adam PacheckRandom executionaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-77870:01:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
323175Jan HakenbergPython templateaido1_amod1-v3step2-scoringsuccessyesidsc-rudolf-77870:00:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
service_quality-7822061.260794768
efficiency-5711973.757710503
fleet_size-1000000000


No reset possible
315169Dylan ChiuΒ πŸ‡ΊπŸ‡ΈRandom executionaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-77870:01:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median2
survival_time_mean10.33999999999998
good_angle_mean13.677322440395011
valid_direction_mean10.48666666666664


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.9323687732219696, "good_angle": 16.462351872231675, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 13.133333333333317}, "ep001": {"nsteps": 302, "reward": -5.86795754128734, "good_angle": 14.440286069059786, "survival_time": 9.999999999999975, "traveled_tiles": 3, "valid_direction": 8.666666666666643}, "ep002": {"nsteps": 500, "reward": -1.8576834003925324, "good_angle": 14.526231312877742, "survival_time": 9.966666666666642, "traveled_tiles": 2, "valid_direction": 12.966666666666622}, "ep003": {"nsteps": 500, "reward": -1.387599057005951, "good_angle": 17.641007521827717, "survival_time": 9.966666666666642, "traveled_tiles": 3, "valid_direction": 12.833333333333291}, "ep004": {"nsteps": 153, "reward": -7.371195160162945, "good_angle": 5.316735425978136, "survival_time": 5.099999999999992, "traveled_tiles": 1, "valid_direction": 4.833333333333325}}
good_angle_max17.641007521827717
good_angle_median14.526231312877742
good_angle_min5.316735425978136
reward_max-0.9323687732219696
reward_mean-3.483360786414148
reward_median-1.8576834003925324
reward_min-7.371195160162945
survival_time_max16.666666666666654
survival_time_median9.966666666666642
survival_time_min5.099999999999992
traveled_tiles_max3
traveled_tiles_mean2
traveled_tiles_min1
valid_direction_max13.133333333333317
valid_direction_median12.833333333333291
valid_direction_min4.833333333333325
No reset possible
313168Dylan ChiuΒ πŸ‡ΊπŸ‡ΈSolution templateaido1_luck-v3step1successyesidsc-rudolf-77870:00:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score176.6557269364963


No reset possible
311165Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r3-v3step3-videossuccessnoidsc-rudolf-77872:00:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
306165Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r3-v3step1-simulationsuccessnoidsc-rudolf-77872:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
301144Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r3-v3step3-videossuccessnoidsc-rudolf-77872:01:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
297165Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r3-v3step1-simulationabortednoidsc-rudolf-77872:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
296149Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationerrornoidsc-rudolf-77872:00:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 376, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 537, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job296-7781', 'pull']:

   >  Command '['docker-compose', '-p', 'job296-7781', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
292163Andrea CensiΒ πŸ‡¨πŸ‡­Solution templateaido1_luck-v3step1successyesidsc-rudolf-77872:00:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score162.04282575671755


No reset possible
291162Andrea CensiΒ πŸ‡¨πŸ‡­Solution templateaido1_luck-v3step1successyesidsc-rudolf-77872:00:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score119.10050606251474


No reset possible
290161Andrea CensiΒ πŸ‡¨πŸ‡­Solution templateaido1_luck-v3step1successyesidsc-rudolf-77872:00:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score125.02738323127707


No reset possible
289160Andrea CensiΒ πŸ‡¨πŸ‡­Solution templateaido1_luck-v3step1successyesidsc-rudolf-77872:00:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score165.77212966366497


No reset possible
288159Andrea CensiΒ πŸ‡¨πŸ‡­Solution templateaido1_luck-v3step1successyesidsc-rudolf-77872:00:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score16.812673246654441


No reset possible
253132Simon SchaeferΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-77872:01:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
24774Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-77872:01:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean16.613333333333316
good_angle_mean23.20707122186628
valid_direction_mean11.993333333333322


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.0535637301508105, "good_angle": 13.552013968782518, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 13.799999999999995}, "ep001": {"nsteps": 500, "reward": -1.9187823569476603, "good_angle": 6.18571844800465, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 9.36666666666667}, "ep002": {"nsteps": 500, "reward": -1.242627432912588, "good_angle": 76.05253361495349, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 13.76666666666663}, "ep003": {"nsteps": 500, "reward": -1.188151815056801, "good_angle": 6.074301128423805, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 9.499999999999996}, "ep004": {"nsteps": 500, "reward": -0.680690432714764, "good_angle": 14.17078894916694, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 13.53333333333332}}
good_angle_max76.05253361495349
good_angle_median13.552013968782518
good_angle_min6.074301128423805
reward_max-0.680690432714764
reward_mean-1.2167631535565246
reward_median-1.188151815056801
reward_min-1.9187823569476603
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min16.599999999999984
traveled_tiles_max3
traveled_tiles_mean2.6
traveled_tiles_min1
valid_direction_max13.799999999999995
valid_direction_median13.53333333333332
valid_direction_min9.36666666666667
No reset possible
24074Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-77872:01:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible