Duckietown Challenges Home Challenges Submissions

Evaluator 34

ID34
evaluatoridsc-rudolf-7787
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-7787
version3.0.27
first heard
last heard
statusinactive
# evaluating
# success11 127
# timeout
# failed1 186
# error4 151
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
18684Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationfailedyesidsc-rudolf-77871:40:33
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 466, in wrap_evaluator
    cie.wait_for_solution()
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 264, in wait_for_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Time out: Timeout of 6000 while waiting for /challenge-solution-output/output-solution.yaml.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18376Andrea Censi 🇨🇭Tensorflow templateaido1_LF1_r2-v3step2-scoringerrornoidsc-rudolf-77870:00:26
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 370, in run
    cr = read_challenge_results(wd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/challenge_results.py", line 66, in read_challenge_results
    data = read_yaml_file(fn)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/yaml_utils.py", line 17, in read_yaml_file
    return yaml.load(data, Loader=yaml.Loader)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/main.py", line 924, in load
    return loader._constructor.get_single_data()
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 108, in get_single_data
    return self.construct_document(node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 118, in construct_document
    for _dummy in generator:
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 672, in construct_yaml_map
    value = self.construct_mapping(node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 399, in construct_mapping
    return BaseConstructor.construct_mapping(self, node, deep=deep)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 242, in construct_mapping
    value = self.construct_object(value_node, deep=deep)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 166, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 938, in construct_python_object_apply
    args = self.construct_sequence(node, deep=True)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 198, in construct_sequence
    return [self.construct_object(child, deep=deep) for child in node.value]
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 166, in construct_object
    data = constructor(self, tag_suffix, node)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 950, in construct_python_object_apply
    instance = self.make_python_instance(suffix, node, args, kwds, newobj)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 882, in make_python_instance
    cls = self.find_python_name(suffix, node.start_mark)
  File "/usr/local/lib/python2.7/dist-packages/ruamel/yaml/constructor.py", line 835, in find_python_name
    mark,
ConstructorError: while constructing a Python object
cannot find module 'numpy' (No module named numpy)
  in "<byte string>", line 6, column 9:
          - &id001 !!python/object/apply:num ... 
            ^ (line: 6)
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
17876Andrea Censi 🇨🇭Tensorflow templateaido1_LF1_r2-v3step1-simulationsuccessyesidsc-rudolf-77870:01:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
170100????????? ???????Solution templateaido1_luck-v3step1successyesidsc-rudolf-77870:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score127.854911031762622


No reset possible
16999????????? ???????Solution templateaido1_luck-v3step1successyesidsc-rudolf-77870:00:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score138.62954447254834


No reset possible
16797????????? ???????Solution templateaido1_luck-v3step1erroryesidsc-rudolf-77870:00:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 354, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 507, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job167-5642', 'pull']:

   >  Command '['docker-compose', '-p', 'job167-5642', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
16595Vadim Volodin 🇷🇺Random executionaido1_LF1_r2-v3step1-simulationerrornoidsc-rudolf-77870:00:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 354, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 507, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job165-8033', 'pull']:

   >  Command '['docker-compose', '-p', 'job165-8033', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
16494Anna Tsalapova 🇷🇺Random executionaido1_LF1_r2-v3step2-scoringsuccessnoidsc-rudolf-77870:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean16.613333333333316
good_angle_mean-4.3748282809293855
valid_direction_mean12.90666666666666


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.2778133477866649, "good_angle": 5.113558586050537, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 15.43333333333332}, "ep001": {"nsteps": 500, "reward": -1.6999089566394687, "good_angle": -12.266352025951424, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 11.66666666666666}, "ep002": {"nsteps": 500, "reward": -1.0894966607391834, "good_angle": -19.87848219232253, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 13.899999999999988}, "ep003": {"nsteps": 500, "reward": -1.4684983215630054, "good_angle": 9.040870913580086, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 11.666666666666664}, "ep004": {"nsteps": 500, "reward": -1.117078724384308, "good_angle": -3.8837366860036, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 11.86666666666666}}
good_angle_max9.040870913580086
good_angle_median-3.8837366860036
good_angle_min-19.87848219232253
reward_max-1.0894966607391834
reward_mean-1.3305592022225263
reward_median-1.2778133477866649
reward_min-1.6999089566394687
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min16.599999999999984
traveled_tiles_max3
traveled_tiles_mean2.6
traveled_tiles_min1
valid_direction_max15.43333333333332
valid_direction_median11.86666666666666
valid_direction_min11.66666666666666
No reset possible
15990Anna Tsalapova 🇷🇺Solution templateaido1_luck-v3step1successyesidsc-rudolf-77870:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score155.67130978463395


No reset possible
15889Anna Tsalapova 🇷🇺Solution templateaido1_luck-v3step1successyesidsc-rudolf-77870:00:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
score174.05022706190263


No reset possible
15588Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationsuccessnoidsc-rudolf-77870:02:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
15286Bhairav MehtaROS-based Lane Followingaido1_LF1_r2-v3step1-simulationsuccessnoidsc-rudolf-77870:02:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
15185Anna Tsalapova 🇷🇺Solution templateaido1_luck-v3step1erroryesidsc-rudolf-77870:00:09
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 354, in run
    run_docker(wd, project, cmd)
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.27-py2.7.egg/duckietown_challenges/runner.py", line 507, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job151-6616', 'pull']:

   >  Command '['docker-compose', '-p', 'job151-6616', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
14882Andy SerTensorflow templateaido1_LF1_r2-v3step2-scoringsuccessnoidsc-rudolf-77870:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean3.099999999999996
good_angle_mean0.5203500054961638
valid_direction_mean1.0933333333333328


other stats
episodes
details{"ep000": {"nsteps": 45, "reward": -22.730665498682193, "good_angle": 0.5399229977937066, "survival_time": 1.500000000000001, "traveled_tiles": 3, "valid_direction": 1.400000000000001}, "ep001": {"nsteps": 108, "reward": -9.473568658999822, "good_angle": 0.24177168678006927, "survival_time": 3.599999999999997, "traveled_tiles": 2, "valid_direction": 0.5333333333333314}, "ep002": {"nsteps": 237, "reward": -4.432748170956362, "good_angle": 0.7527284654481572, "survival_time": 7.899999999999982, "traveled_tiles": 5, "valid_direction": 0.6666666666666666}, "ep003": {"nsteps": 39, "reward": -27.63335389510179, "good_angle": -0.43954698209119936, "survival_time": 1.3000000000000007, "traveled_tiles": 1, "valid_direction": 0.9333333333333336}, "ep004": {"nsteps": 112, "reward": -9.770550391495428, "good_angle": 1.5068738595500857, "survival_time": 1.2000000000000004, "traveled_tiles": 5, "valid_direction": 1.9333333333333316}}
good_angle_max1.5068738595500857
good_angle_median0.5399229977937066
good_angle_min-0.43954698209119936
reward_max-4.432748170956362
reward_mean-14.808177323047122
reward_median-9.770550391495428
reward_min-27.63335389510179
survival_time_max7.899999999999982
survival_time_median1.500000000000001
survival_time_min1.2000000000000004
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_min1
valid_direction_max1.9333333333333316
valid_direction_median0.9333333333333336
valid_direction_min0.5333333333333314
No reset possible
14382Andy SerTensorflow templateaido1_LF1_r2-v3step1-simulationsuccessnoidsc-rudolf-77870:01:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
12775Bhairav MehtaROS-based Lane Followingaido1_LF1-v3step1-simulationsuccessyesidsc-rudolf-77870:03:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible