Duckietown Challenges Home Challenges Submissions

Evaluator 70

ID70
evaluatoridsc-rudolf-1974
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-1974
version3.0.36
first heard
last heard
statusinactive
# evaluating
# success15 232
# timeout
# failed1 320
# error2 255
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
327178Florian GolemoTensorflow templateaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-19740:00:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean3.106666666666663
good_angle_mean0.8582415175275339
valid_direction_mean1.0866666666666662


other stats
episodes
details{"ep000": {"nsteps": 45, "reward": -22.730665498682193, "good_angle": 2.464011645649803, "survival_time": 1.500000000000001, "traveled_tiles": 3, "valid_direction": 1.400000000000001}, "ep001": {"nsteps": 108, "reward": -9.473568658999822, "good_angle": 0.21552867377610185, "survival_time": 3.599999999999997, "traveled_tiles": 2, "valid_direction": 0.5333333333333314}, "ep002": {"nsteps": 237, "reward": -4.432748170956362, "good_angle": 0.44670148280839816, "survival_time": 7.899999999999982, "traveled_tiles": 5, "valid_direction": 0.6666666666666666}, "ep003": {"nsteps": 40, "reward": -27.05611758381128, "good_angle": 0.3536391550655539, "survival_time": 1.333333333333334, "traveled_tiles": 1, "valid_direction": 0.9333333333333336}, "ep004": {"nsteps": 111, "reward": -9.86401510136286, "good_angle": 0.8113266303378122, "survival_time": 1.2000000000000004, "traveled_tiles": 5, "valid_direction": 1.8999999999999984}}
good_angle_max2.464011645649803
good_angle_median0.44670148280839816
good_angle_min0.21552867377610185
reward_max-4.432748170956362
reward_mean-14.711423002762508
reward_median-9.86401510136286
reward_min-27.05611758381128
survival_time_max7.899999999999982
survival_time_median1.500000000000001
survival_time_min1.2000000000000004
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_min1
valid_direction_max1.8999999999999984
valid_direction_median0.9333333333333336
valid_direction_min0.5333333333333314
No reset possible
322175Jan HakenbergPython templateaido1_amod1-v3step1-simulationsuccessyesidsc-rudolf-19740:02:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
passedtrue
No reset possible
321174Jan HakenbergSolution templateaido1_luck-v3step1successyesidsc-rudolf-19740:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score188.06777829289516


No reset possible
320173Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailednoidsc-rudolf-19740:00:27
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 480, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Env Duckietown-Lf-Lfv-Navv-Silent-v1 not found (valid versions include ['Duckietown-Lf-Lfv-Navv-Silent-v0'])
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
309165Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step1-simulationsuccessnoidsc-rudolf-19742:01:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
265139Julian ZillySolution templateaido1_luck-v3step1successyesidsc-rudolf-19742:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score167.93359083476635


No reset possible
264138Julian ZillySolution templateaido1_luck-v3step1erroryesidsc-rudolf-19742:00:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 376, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 537, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job264-3031', 'pull']:

   >  Command '['docker-compose', '-p', 'job264-3031', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
261136Vadim Volodin 🇷🇺Random executionaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-19742:00:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean15.04666666666663
good_angle_mean21.21323305962173
valid_direction_mean13.499999999999968


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.4831203885326395, "good_angle": 12.275620718473334, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 14.333333333333329}, "ep001": {"nsteps": 442, "reward": -4.533181337275168, "good_angle": 8.19197869993212, "survival_time": 14.666666666666623, "traveled_tiles": 3, "valid_direction": 11.399999999999968}, "ep002": {"nsteps": 500, "reward": -1.689824109852314, "good_angle": 52.24130204744942, "survival_time": 14.633333333333292, "traveled_tiles": 2, "valid_direction": 15.533333333333283}, "ep003": {"nsteps": 500, "reward": -0.7884967522611842, "good_angle": 15.05704709524352, "survival_time": 14.633333333333292, "traveled_tiles": 3, "valid_direction": 12.799999999999969}, "ep004": {"nsteps": 500, "reward": -0.9460928311357276, "good_angle": 18.300216737010253, "survival_time": 14.633333333333292, "traveled_tiles": 3, "valid_direction": 13.433333333333294}}
good_angle_max52.24130204744942
good_angle_median15.05704709524352
good_angle_min8.19197869993212
reward_max-0.4831203885326395
reward_mean-1.6881430838114064
reward_median-0.9460928311357276
reward_min-4.533181337275168
survival_time_max16.666666666666654
survival_time_median14.633333333333292
survival_time_min14.633333333333292
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_min1
valid_direction_max15.533333333333283
valid_direction_median13.433333333333294
valid_direction_min11.399999999999968
No reset possible
260136Vadim Volodin 🇷🇺Random executionaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-19742:01:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
259135Claudio RuchPython templateaido1_amod1-v3step2-scoringsuccessyesidsc-rudolf-19742:00:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
service_quality-7981498.453173093
efficiency-5828366.037980752
fleet_size-1000000000


No reset possible
257135Claudio RuchPython templateaido1_amod1-v3step1-simulationsuccessyesidsc-rudolf-19742:02:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
passedtrue
No reset possible
255133Claudio RuchPython templateaido1_amod1-v3step1-simulationerroryesidsc-rudolf-19742:00:10
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 376, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 537, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job255-7450', 'pull']:

   >  Command '['docker-compose', '-p', 'job255-7450', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
254132Simon Schaefer 🇨🇭Random executionaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-19742:00:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean16.613333333333316
good_angle_mean24.219923570778985
valid_direction_mean13.699999999999983


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.1021499887704849, "good_angle": 9.33501996979166, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.43333333333333}, "ep001": {"nsteps": 500, "reward": -1.741656614583917, "good_angle": 17.71874076718678, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 15.233333333333292}, "ep002": {"nsteps": 500, "reward": -1.5007373089790343, "good_angle": 73.50207854189686, "survival_time": 16.599999999999984, "traveled_tiles": 2, "valid_direction": 16.233333333333317}, "ep003": {"nsteps": 500, "reward": -1.2225796927213668, "good_angle": 8.012821742629477, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 12.06666666666666}, "ep004": {"nsteps": 500, "reward": -0.9013783309273422, "good_angle": 12.53095683239013, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 12.533333333333324}}
good_angle_max73.50207854189686
good_angle_median12.53095683239013
good_angle_min8.012821742629477
reward_max-0.9013783309273422
reward_mean-1.293700387196429
reward_median-1.2225796927213668
reward_min-1.741656614583917
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min16.599999999999984
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_min1
valid_direction_max16.233333333333317
valid_direction_median12.533333333333324
valid_direction_min12.06666666666666
No reset possible
252131Andrea Censi 🇨🇭Solution templateaido1_luck-v3step1successyesidsc-rudolf-19742:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score146.12506326697186


No reset possible
24674Andrea Censi 🇨🇭Random executionaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-19742:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean16.613333333333316
good_angle_mean21.153485385500257
valid_direction_mean12.039999999999992


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.8591660871505737, "good_angle": 11.257293415724876, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 8.533333333333342}, "ep001": {"nsteps": 500, "reward": -1.6598995124697686, "good_angle": 46.93787052259383, "survival_time": 16.599999999999984, "traveled_tiles": 2, "valid_direction": 13.566666666666656}, "ep002": {"nsteps": 500, "reward": -0.9550404280126096, "good_angle": 18.679399540854796, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 12.499999999999991}, "ep003": {"nsteps": 500, "reward": -1.2373386880159378, "good_angle": 20.944676094792023, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 15.499999999999986}, "ep004": {"nsteps": 500, "reward": -0.5126900258227252, "good_angle": 7.948187353535748, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 10.099999999999987}}
good_angle_max46.93787052259383
good_angle_median18.679399540854796
good_angle_min7.948187353535748
reward_max-0.5126900258227252
reward_mean-1.044826948294323
reward_median-0.9550404280126096
reward_min-1.6598995124697686
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min16.599999999999984
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_min1
valid_direction_max15.499999999999986
valid_direction_median12.499999999999991
valid_direction_min8.533333333333342
No reset possible
24274Andrea Censi 🇨🇭Random executionaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-19742:01:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
233124Andrea Censi 🇨🇭Solution templateaido1_luck-v3step1successyesidsc-rudolf-19742:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score197.09525324323911


No reset possible
232123Andrea Censi 🇨🇭Solution templateaido1_luck-v3step1successyesidsc-rudolf-19742:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score116.755246257018655


No reset possible