Duckietown Challenges Home Challenges Submissions

Evaluator 106

ID106
evaluatornutonomy-P50-29851
ownerAndrea Censi 🇨🇭
machinenutonomy-P50
processnutonomy-P50-29851
versionDC:3.1.42;DCR:3.2.4;DTS:3.0.29
first heard
last heard
statusinactive
# evaluating
# success19 485
# timeout1 653
# failed3 576
# error9 470
# aborted1 474
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
6531Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationtimeoutnonutonomy-P50-298510:11:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
6481Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6431Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6381Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6341Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6291Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6241Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6201Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6161Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6111Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6061Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
6011Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
5961Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
5911Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:00:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
5861Andrea Censi 🇨🇭nullaido1_LF1-v3step1-simulationsuccessnonutonomy-P50-298510:01:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation"1.0"
No reset possible
584223Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationfailednonutonomy-P50-298510:00:32
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 107, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: Can't connect to the gym-duckietown-server
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
580217Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailednonutonomy-P50-298510:00:24
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Env Duckietown-Lf-Lfv-Navv-Silent-v1 not found (valid versions include ['Duckietown-Lf-Lfv-Navv-Silent-v0'])
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
576173Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailednonutonomy-P50-298510:00:25
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Env Duckietown-Lf-Lfv-Navv-Silent-v1 not found (valid versions include ['Duckietown-Lf-Lfv-Navv-Silent-v0'])
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
570286Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:15
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 488, in wrap_evaluator
    evaluator.score(cie)
  File "eval.py", line 96, in score
    raise dc.InvalidEvaluator(msg)
InvalidEvaluator: Gym exited with code 2
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
565278Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:25
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 412, in run
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 560, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job565-2885', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job565-2885', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
560273Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:24
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 412, in run
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 560, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job560-5207', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job560-5207', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
557270Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:24
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 412, in run
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 560, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job557-1754', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job557-1754', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
554195Florian GolemoPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:20
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 488, in wrap_evaluator
    evaluator.score(cie)
  File "eval.py", line 96, in score
    raise dc.InvalidEvaluator(msg)
InvalidEvaluator: Gym exited with code 2
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
524281Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:39
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 414, in run
    cr = read_challenge_results(wd)
  File "/project/src/duckietown-challenges/src/duckietown_challenges/challenge_results.py", line 63, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission281/step1-simulation-nutonomy-P50-29851-job524/challenge-results/challenge_results.yaml' does not exist.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
518275Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:27
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 412, in run
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 560, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job518-1017', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job518-1017', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
516273Anton Mashikhin 🇷🇺AI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:26
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 412, in run
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 560, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job516-7224', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job516-7224', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
510269Anton Mashikhin 🇷🇺PyTorch DDPG templateaido1_LF1_r2-v3step1-simulationsuccessyesnutonomy-P50-298510:01:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
501224Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationsuccessnonutonomy-P50-298510:04:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
495267Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step3-videossuccessnonutonomy-P50-298510:00:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
488268Andrea Censi 🇨🇭Random executionaido1_LFV_r1-v3step1-simulationsuccessnonutonomy-P50-298510:02:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
48557Andrea Censi 🇨🇭Solution templateaido1_test_multistep-v3step2successyesnutonomy-P50-298510:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
passed-step21


No reset possible
474166Andrea Censi 🇨🇭Random executionaido1_LFV_r1-v3step4-vizabortednonutonomy-P50-298510:01:27
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 530, in wrap_scorer
    evaluator.score(cie)
  File "/visualizer/videos.py", line 12, in score
    logdir = cie.get_completed_step_evaluation_file('step1-simulation', 'episodes')
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 398, in get_completed_step_evaluation_file
    return get_completed_step_evaluation_file(self.root, step_name, basename)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 427, in get_completed_step_evaluation_file
    available = get_completed_step_evaluation_files(root, step_name)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 421, in get_completed_step_evaluation_files
    raise InvalidEnvironment(msg)
InvalidEnvironment: Could not find dir /previous-steps/step1-simulation/challenge-evaluation-output
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
470195Florian GolemoPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrornonutonomy-P50-298510:00:25
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 414, in run
    cr = read_challenge_results(wd)
  File "/project/src/duckietown-challenges/src/duckietown_challenges/challenge_results.py", line 63, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission195/step1-simulation-nutonomy-P50-29851-job470/challenge-results/challenge_results.yaml' does not exist.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible