Duckietown Challenges Home Challenges Submissions

Evaluator 33

ID33
evaluatoridsc-rudolf-1974
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-1974
version3.0.27
first heard
last heard
statusinactive
# evaluating
# success8 120
# timeout
# failed1 124
# error3 128
# aborted2 126
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
21295Vadim Volodin 🇷🇺Random executionaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-19740:00:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean16.613333333333316
good_angle_mean10.129177905754815
valid_direction_mean11.266666666666652


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.1028703950420022, "good_angle": 14.55528886546265, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 13.099999999999996}, "ep001": {"nsteps": 500, "reward": -2.076906876723282, "good_angle": 4.463438139630227, "survival_time": 16.599999999999984, "traveled_tiles": 2, "valid_direction": 8.599999999999989}, "ep002": {"nsteps": 500, "reward": -1.987362047176808, "good_angle": 13.379992658002996, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 11.899999999999997}, "ep003": {"nsteps": 500, "reward": -1.2790714666523273, "good_angle": 10.323571334724727, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 13.16666666666664}, "ep004": {"nsteps": 500, "reward": -1.6038520691990852, "good_angle": 7.923598530953476, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 9.566666666666643}}
good_angle_max14.55528886546265
good_angle_median10.323571334724727
good_angle_min4.463438139630227
reward_max-1.1028703950420022
reward_mean-1.6100125709587012
reward_median-1.6038520691990852
reward_min-2.076906876723282
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min16.599999999999984
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_min1
valid_direction_max13.16666666666664
valid_direction_median11.899999999999997
valid_direction_min8.599999999999989
No reset possible
20782Andy SerTensorflow templateaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-19740:00:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean3.0866666666666625
good_angle_mean0.8650031657716386
valid_direction_mean1.0599999999999994


other stats
episodes
details{"ep000": {"nsteps": 45, "reward": -22.730665498682193, "good_angle": 2.464011645649803, "survival_time": 1.500000000000001, "traveled_tiles": 3, "valid_direction": 1.400000000000001}, "ep001": {"nsteps": 108, "reward": -9.473568658999822, "good_angle": 0.21552867377610185, "survival_time": 3.599999999999997, "traveled_tiles": 2, "valid_direction": 0.5333333333333314}, "ep002": {"nsteps": 237, "reward": -4.432748170956362, "good_angle": 0.44670148280839816, "survival_time": 7.899999999999982, "traveled_tiles": 5, "valid_direction": 0.6666666666666666}, "ep003": {"nsteps": 39, "reward": -27.65655777698908, "good_angle": 0.33450547468440583, "survival_time": 1.3000000000000007, "traveled_tiles": 1, "valid_direction": 0.9000000000000002}, "ep004": {"nsteps": 110, "reward": -9.79016963146119, "good_angle": 0.8642685519394845, "survival_time": 1.1333333333333335, "traveled_tiles": 5, "valid_direction": 1.799999999999998}}
good_angle_max2.464011645649803
good_angle_median0.44670148280839816
good_angle_min0.21552867377610185
reward_max-4.432748170956362
reward_mean-14.816741947417734
reward_median-9.79016963146119
reward_min-27.65655777698908
survival_time_max7.899999999999982
survival_time_median1.500000000000001
survival_time_min1.1333333333333335
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_min1
valid_direction_max1.799999999999998
valid_direction_median0.9000000000000002
valid_direction_min0.5333333333333314
No reset possible
20476Andrea Censi 🇨🇭Tensorflow templateaido1_LF1_r2-v3step2-scoringerroryesidsc-rudolf-19740:00:10
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 368, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 507, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job204-6958', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job204-6958', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
20195Vadim Volodin 🇷🇺Random executionaido1_LF1_r2-v3step2-scoringerrornoidsc-rudolf-19740:00:30
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 370, in run
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/challenge_results.py", line 66, in read_challenge_results
    data = read_yaml_file(fn)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/yaml_utils.py", line 17, in read_yaml_file
    return yaml.load(data, Loader=yaml.Loader)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/main.py", line 924, in load
    return loader._constructor.get_single_data()
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 108, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 118, in construct_document
    for _dummy in generator:
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 672, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 399, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 242, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 166, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 938, in construct_python_object_apply
    args = self.construct_sequence(node, deep=True)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 198, in construct_sequence
    return [self.construct_object(child, deep=deep) for child in node.value]
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 166, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 950, in construct_python_object_apply
    instance = self.make_python_instance(suffix, node, args, kwds, newobj)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 882, in make_python_instance
    cls = self.find_python_name(suffix, node.start_mark)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 835, in find_python_name
    mark,
ConstructorError: while constructing a Python object
cannot find module 'numpy' (No module named numpy)
  in "<byte string>", line 6, column 9:
          - &id001 !!python/object/apply:num ... 
            ^ (line: 6)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
19188Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-19740:03:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
18078Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-19740:03:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
14282Andy SerTensorflow templateaido1_LF1_r2-v3step1-simulationsuccessnoidsc-rudolf-19740:01:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
14182Andy SerTensorflow templateaido1_LF1_r2-v3step1-simulationabortednoidsc-rudolf-19740:00:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 354, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 507, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job141-4732', 'pull']:

   >  Command '['docker-compose', '-p', 'job141-4732', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
13880Bhairav MehtaROS-based Lane Following, v2aido1_LF1_r2-v3step1-simulationsuccessnoidsc-rudolf-19740:03:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
13175Bhairav MehtaROS-based Lane Followingaido1_LF1-v3step2-scoringsuccessyesidsc-rudolf-19740:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
reward-2.906480613810852


other stats
ep000
details{"nsteps": 233, "reward": -5.201444343589806}
ep001
details{"nsteps": 315, "reward": -3.792179065865893}
ep002
details{"nsteps": 500, "reward": -0.5485908435722813}
ep003
details{"nsteps": 500, "reward": -1.573633426239714}
ep004
details{"nsteps": 411, "reward": -3.4165553897865673}
No reset possible
12871Andrea Censi 🇨🇭Random executionaido1_LF1_r2-v3step2-scoringerrornoidsc-rudolf-19740:00:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 368, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 507, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job128-4885', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job128-4885', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12674Andrea Censi 🇨🇭Random executionaido1_LF1_r2-v3step2-scoringabortednoidsc-rudolf-19740:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_mean16.613333333333316
good_angle_mean4.622046586172948
valid_direction_mean11.759999999999994


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.6687548997402191, "good_angle": 2.346875555585716, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 9.099999999999994}, "ep001": {"nsteps": 500, "reward": -2.053634369403124, "good_angle": 1.1012067416734153, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 6.933333333333344}, "ep002": {"nsteps": 500, "reward": -1.511635816635564, "good_angle": 11.962119726476049, "survival_time": 16.599999999999984, "traveled_tiles": 2, "valid_direction": 14.399999999999984}, "ep003": {"nsteps": 500, "reward": -1.2118231953382492, "good_angle": 4.620920989414631, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 13.76666666666666}, "ep004": {"nsteps": 500, "reward": -0.47050132013599794, "good_angle": 3.07910991771493, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 14.599999999999987}}
good_angle_max11.962119726476049
good_angle_median3.07910991771493
good_angle_min1.1012067416734153
reward_max-0.47050132013599794
reward_mean-1.183269920250631
reward_median-1.2118231953382492
reward_min-2.053634369403124
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min16.599999999999984
valid_direction_max14.599999999999987
valid_direction_median13.76666666666666
valid_direction_min6.933333333333344
No reset possible
12473Bhairav MehtaROS-based Lane Followingaido1_LF1-v3step1-simulationfailedyesidsc-rudolf-19740:03:09
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 395, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Can't connect to the gym-duckietown-server
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
12071Andrea Censi 🇨🇭Random executionaido1_LF1_r2-v3step1-simulationsuccessnoidsc-rudolf-19740:00:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible