Duckietown Challenges Home Challenges Submissions

Evaluator 32

ID32
evaluatorrudolf-3
ownerAndrea CensiΒ πŸ‡¨πŸ‡­
machineidsc-rudolf
processrudolf-3
version3.0.27
first heard
last heard
statusinactive
# evaluating
# success10 123
# timeout
# failed1 150
# error2 119
# aborted2 125
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
214105Brian WangΒ πŸ‡ΊπŸ‡ΈSolution templateaido1_luck-v3step1successyesrudolf-30:00:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score195.62874167457156


No reset possible
21088Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step2-scoringsuccessyesrudolf-30:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median5
survival_time_mean11.159999999999991
good_angle_mean5.700746329790628
valid_direction_mean2.0400000000000027


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.4222462384796235, "good_angle": 27.09156883116741, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.400000000000013}, "ep001": {"nsteps": 500, "reward": -0.7161610741019249, "good_angle": 0.516020677906025, "survival_time": 16.63333333333332, "traveled_tiles": 5, "valid_direction": 1.3999999999999988}, "ep002": {"nsteps": 500, "reward": -1.0602927017807962, "good_angle": 0.27033355847307683, "survival_time": 16.63333333333332, "traveled_tiles": 5, "valid_direction": 0.49999999999999994}, "ep003": {"nsteps": 87, "reward": -13.924883039518337, "good_angle": 0.11620891323429636, "survival_time": 2.8666666666666663, "traveled_tiles": 2, "valid_direction": 0}, "ep004": {"nsteps": 90, "reward": -12.73018991748492, "good_angle": 0.5095996681723338, "survival_time": 2.999999999999999, "traveled_tiles": 2, "valid_direction": 1.9000000000000024}}
good_angle_max27.09156883116741
good_angle_median0.5095996681723338
good_angle_min0.11620891323429636
reward_max-0.4222462384796235
reward_mean-5.77075459427312
reward_median-1.0602927017807962
reward_min-13.924883039518337
survival_time_max16.666666666666654
survival_time_median16.63333333333332
survival_time_min2.8666666666666663
traveled_tiles_max5
traveled_tiles_mean3.8
traveled_tiles_min2
valid_direction_max6.400000000000013
valid_direction_median1.3999999999999988
valid_direction_min0
No reset possible
20680Bhairav MehtaROS-based Lane Following, v2aido1_LF1_r2-v3step2-scoringsuccessyesrudolf-30:00:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean16.639999999999986
good_angle_mean19.400973454716613
valid_direction_mean13.046666666666656


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.6743399429947603, "good_angle": 16.99228129378378, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 12.99999999999999}, "ep001": {"nsteps": 500, "reward": -1.0934869987164857, "good_angle": 24.784477936252543, "survival_time": 16.63333333333332, "traveled_tiles": 4, "valid_direction": 13.166666666666654}, "ep002": {"nsteps": 500, "reward": -0.877338105738163, "good_angle": 24.722114918891094, "survival_time": 16.63333333333332, "traveled_tiles": 3, "valid_direction": 13.39999999999999}, "ep003": {"nsteps": 500, "reward": -0.45057291654497383, "good_angle": 13.42465701684305, "survival_time": 16.63333333333332, "traveled_tiles": 3, "valid_direction": 13.199999999999989}, "ep004": {"nsteps": 500, "reward": -0.7861739034691128, "good_angle": 17.081336107812586, "survival_time": 16.63333333333332, "traveled_tiles": 4, "valid_direction": 12.466666666666658}}
good_angle_max24.784477936252543
good_angle_median17.081336107812586
good_angle_min13.42465701684305
reward_max-0.45057291654497383
reward_mean-0.7763823734926991
reward_median-0.7861739034691128
reward_min-1.0934869987164857
survival_time_max16.666666666666654
survival_time_median16.63333333333332
survival_time_min16.63333333333332
traveled_tiles_max4
traveled_tiles_mean3.2
traveled_tiles_min2
valid_direction_max13.39999999999999
valid_direction_median13.166666666666654
valid_direction_min12.466666666666658
No reset possible
19895Vadim VolodinΒ πŸ‡·πŸ‡ΊRandom executionaido1_LF1_r2-v3step1-simulationsuccessyesrudolf-30:01:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
18786Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationsuccessyesrudolf-30:02:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
17774Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r2-v3step1-simulationabortedyesrudolf-30:02:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
16394Anna TsalapovaΒ πŸ‡·πŸ‡ΊRandom executionaido1_LF1_r2-v3step1-simulationsuccessnorudolf-30:01:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
16192Anna TsalapovaΒ πŸ‡·πŸ‡ΊSolution templateaido1_LF1_r2-v3step1-simulationerrornorudolf-30:00:24
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 368, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 507, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job161-892', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job161-892', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
15084Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationfailednorudolf-31:40:34
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 466, in wrap_evaluator
    cie.wait_for_solution()
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 264, in wait_for_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Time out: Timeout of 6000 while waiting for /challenge-solution-output/output-solution.yaml.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14682Andy SerTensorflow templateaido1_LF1_r2-v3step1-simulationsuccessnorudolf-30:00:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
13478Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationsuccessnorudolf-30:03:46
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
13076Andrea CensiΒ πŸ‡¨πŸ‡­Tensorflow templateaido1_LF1_r2-v3step1-simulationsuccessnorudolf-30:01:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
12574Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r2-v3step1-simulationabortednorudolf-30:01:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
12372Bhairav MehtaROS-based Lane Followingaido1_LF1-v3step2-scoringsuccessyesrudolf-30:01:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
reward-3.917294915007301


other stats
ep000
details{"nsteps": 130, "reward": -10.631935761525082}
ep001
details{"nsteps": 415, "reward": -3.557544670055385}
ep002
details{"nsteps": 354, "reward": -3.951317148188413}
ep003
details{"nsteps": 500, "reward": -0.951256725908257}
ep004
details{"nsteps": 500, "reward": -0.4944202693593688}
No reset possible
11971Andrea CensiΒ πŸ‡¨πŸ‡­Random executionaido1_LF1_r2-v3step1-simulationerrornorudolf-30:00:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 458, in wrap_evaluator
    evaluator.prepare(cie)
  File "eval.py", line 75, in prepare
    raise dc.InvalidEvaluator(msg)
InvalidEvaluator: The gym terminated with return code 2 before starting.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible