AI Driving Olympics Home Challenges Submissions Jobs

List of evaluation jobs

Job opportunities

Paper experiment p1 - Lane following (robotarium 🏎, validation 🏋)

SubmissionDateUserStepAdmin priorityUser priorityPreviousAvoidEvaluator assignment
6605JBRRussiaeval28050{'eval0': 32791, 'eval1': 32818, 'eval0-videos': 32794, 'eval1-videos': 32821, 'eval2-videos': 32829, 'eval0-visualize': 32795, 'eval1-visualize': 32820, 'eval2-visualize': 32830}No assignment yet.
6609AmaurXeval26080{'eval0': 32885, 'eval1': 32907, 'eval0-videos': 32890, 'eval1-videos': 32913, 'eval2-videos': 32863, 'eval0-visualize': 32891, 'eval1-visualize': 32910, 'eval2-visualize': 32861}No assignment yet.
6607saryazdieval26050{'eval0': 32797, 'eval1': 32805, 'eval0-videos': 32800, 'eval1-videos': 32808, 'eval0-visualize': 32801, 'eval1-visualize': 32809}No assignment yet.
6589frank_qcd_qkeval25050{'eval0': 32613, 'eval1': 32769, 'eval0-videos': 32623, 'eval1-videos': 32773, 'eval2-videos': 32656, 'eval0-visualize': 32622, 'eval1-visualize': 32770, 'eval2-visualize': 32655}No assignment yet.

Jobs being evaluated

No jobs being evaluated.

Evaluated jobs

page 2

LF 🚗 - Lane following (simulation 👾, validation 🏋)

Job IDsubmissionuseruser labelstepstatusup to dateevaluatordate starteddate completeddurationmessage
329246613brianwong174challenge-aido_LF-baseline-duckietownstep1-simulationsuccessyes16180:10:40Hidden. If you are the author, please login using the top-right link or use the dashboard.
329236613brianwong174challenge-aido_LF-baseline-duckietownstep1-simulationsuccessyes16170:09:57Hidden. If you are the author, please login using the top-right link or use the dashboard.
329156612frank_qcd_qkchallenge-aido_LF-baseline-behavior-cloningstep1-simulationsuccessyes16180:08:16Hidden. If you are the author, please login using the top-right link or use the dashboard.
329146612frank_qcd_qkchallenge-aido_LF-baseline-behavior-cloningstep1-simulationsuccessyes16170:08:45Hidden. If you are the author, please login using the top-right link or use the dashboard.

Paper experiment p1 - Lane following (robotarium 🏎, validation 🏋)

Job IDsubmissionuseruser labelstepstatusup to dateevaluatordate starteddate completeddurationmessage
329226606frank_qcd_qk2019-12-10|FrankNet|(Kay❤️Frank)eval2-videossuccessyes17000:07:37Hidden. If you are the author, please login using the top-right link or use the dashboard.
329216606frank_qcd_qk2019-12-10|FrankNet|(Kay❤️Frank)eval2-videossuccessyes16170:05:15Hidden. If you are the author, please login using the top-right link or use the dashboard.
329206606frank_qcd_qk2019-12-10|FrankNet|(Kay❤️Frank)eval2-visualizesuccessyes16990:07:37Hidden. If you are the author, please login using the top-right link or use the dashboard.
329196606frank_qcd_qk2019-12-10|FrankNet|(Kay❤️Frank)eval2-videossuccessyes16180:05:49Hidden. If you are the author, please login using the top-right link or use the dashboard.
329186606frank_qcd_qk2019-12-10|FrankNet|(Kay❤️Frank)eval2-visualizesuccessyes17010:08:14Hidden. If you are the author, please login using the top-right link or use the dashboard.
329176606frank_qcd_qk2019-12-10|FrankNet|(Kay❤️Frank)eval2successyes15950:31:30
Scores from the eval [...]
Scores from the evaluation of job: 32917
Hidden. If you are the author, please login using the top-right link or use the dashboard.
329166609AmaurXTTICeval2abortedyes15950:01:18
Scores from the eval [...]
Scores from the evaluation of job: 32916
Hidden. If you are the author, please login using the top-right link or use the dashboard.
329136609AmaurXTTICeval1-videossuccessyes16230:03:01Hidden. If you are the author, please login using the top-right link or use the dashboard.
329126609AmaurXTTICeval1-videossuccessyes16240:03:35Hidden. If you are the author, please login using the top-right link or use the dashboard.
329116609AmaurXTTICeval1-videoshost-erroryes16190:00:49
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 643, in get_cr
    server_host=server_host,
  File "/project/src/duckietown_challenges_runner/runner.py", line 720, in run_single
    prepare_dir(wd, aws_config, steps2artefacts, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1029, in prepare_dir
    aws_config, steps2artefacts, previous_steps_dir, use_ipfs=use_ipfs
  File "/project/src/duckietown_challenges_runner/runner.py", line 1404, in download_artefacts
    get_file_from_cache(fn, sha256hex)
  File "/project/src/duckietown_challenges_runner/runner_cache.py", line 25, in get_file_from_cache
    shutil.copy(have, fn)
  File "/usr/lib/python3.6/shutil.py", line 246, in copy
    copymode(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.6/shutil.py", line 143, in copymode
    st = stat_func(src)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/cache/by-value/sha256hex/58720fae954e4f4b2f06bd756df439a9ea3e10e19325e8bba94a2ca67f7d718f'
Hidden. If you are the author, please login using the top-right link or use the dashboard.
329106609AmaurXTTICeval1-visualizesuccessyes16260:05:05Hidden. If you are the author, please login using the top-right link or use the dashboard.
329096609AmaurXTTICeval1-videoshost-erroryes16180:01:50
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 1404, in download_artefacts
    get_file_from_cache(fn, sha256hex)
  File "/project/src/duckietown_challenges_runner/runner_cache.py", line 28, in get_file_from_cache
    raise KeyError(msg)
KeyError: 'Hash not in cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 643, in get_cr
    server_host=server_host,
  File "/project/src/duckietown_challenges_runner/runner.py", line 720, in run_single
    prepare_dir(wd, aws_config, steps2artefacts, use_ipfs=use_ipfs)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1029, in prepare_dir
    aws_config, steps2artefacts, previous_steps_dir, use_ipfs=use_ipfs
  File "/project/src/duckietown_challenges_runner/runner.py", line 1419, in download_artefacts
    get_object(aws_config, bucket_name, object_key, fn)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1463, in get_object
    aws_object.download_file(fn)
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 314, in object_download_file
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 172, in download_file
    extra_args=ExtraArgs, callback=Callback)
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/transfer.py", line 307, in download_file
    future.result()
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 73, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 233, in result
    raise self._exception
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 126, in __call__
    return self._execute_main(kwargs)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 150, in _execute_main
    return_value = self._main(**kwargs)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/download.py", line 582, in _main
    fileobj.seek(offset)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/utils.py", line 335, in seek
    self._open_if_needed()
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/utils.py", line 318, in _open_if_needed
    self._fileobj = self._open_function(self._filename, self._mode)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/utils.py", line 244, in open
    return open(filename, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido3-off-LF-real-validation-p1/submission6609/eval1-videos-ip-172-31-45-34-10500-job32909/previous-steps/eval2-videos/logs/challenges-runner/stderr.html.967a7d8C'
Hidden. If you are the author, please login using the top-right link or use the dashboard.
329086609AmaurXTTICeval1-visualizesuccessyes16330:05:19Hidden. If you are the author, please login using the top-right link or use the dashboard.
329076609AmaurXTTICeval1successyes16380:09:29
Scores from the eval [...]
Scores from the evaluation of job: 32907
Hidden. If you are the author, please login using the top-right link or use the dashboard.
329066611AmaurXETHZeval2-videossuccessyes16180:07:20Hidden. If you are the author, please login using the top-right link or use the dashboard.
329056611AmaurXETHZeval2-visualizesuccessyes16260:06:47Hidden. If you are the author, please login using the top-right link or use the dashboard.
329046611AmaurXETHZeval2successyes15950:28:19
Scores from the eval [...]
Scores from the evaluation of job: 32904
Hidden. If you are the author, please login using the top-right link or use the dashboard.
329036611AmaurXETHZeval1-videossuccessyes16330:07:38Hidden. If you are the author, please login using the top-right link or use the dashboard.
329026611AmaurXETHZeval1-visualizesuccessyes16170:03:07Hidden. If you are the author, please login using the top-right link or use the dashboard.
329016611AmaurXETHZeval1-videossuccessyes16230:08:30Hidden. If you are the author, please login using the top-right link or use the dashboard.
329006611AmaurXETHZeval1-visualizesuccessyes16240:07:46Hidden. If you are the author, please login using the top-right link or use the dashboard.
328996611AmaurXETHZeval1successyes15950:18:04
Scores from the eval [...]
Scores from the evaluation of job: 32899
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328986609AmaurXTTICeval1timeoutyes15950:01:44
Timeout because eval [...]
Timeout because evaluator contacted us
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328976608AmaurXTTICeval2-videossuccessyes16260:09:35Hidden. If you are the author, please login using the top-right link or use the dashboard.
328966608AmaurXTTICeval2-visualizesuccessyes16520:08:53Hidden. If you are the author, please login using the top-right link or use the dashboard.
328956611AmaurXETHZeval1abortedyes15950:04:33
Scores from the eval [...]
Scores from the evaluation of job: 32895
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328946608AmaurXTTICeval2-videossuccessyes16230:10:00Hidden. If you are the author, please login using the top-right link or use the dashboard.
328936609AmaurXTTICeval1abortedyes15950:00:04
Scores from the eval [...]
Scores from the evaluation of job: 32893
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328926608AmaurXTTICeval2-visualizesuccessyes16240:09:32Hidden. If you are the author, please login using the top-right link or use the dashboard.
328916609AmaurXTTICeval0-visualizesuccessyes16320:07:05Hidden. If you are the author, please login using the top-right link or use the dashboard.
328906609AmaurXTTICeval0-videossuccessyes16250:07:17Hidden. If you are the author, please login using the top-right link or use the dashboard.
328896610AmaurXETHZeval2-videossuccessyes16180:05:11Hidden. If you are the author, please login using the top-right link or use the dashboard.
328886610AmaurXETHZeval2-visualizesuccessyes16520:04:20Hidden. If you are the author, please login using the top-right link or use the dashboard.
328876610AmaurXETHZeval2-videossuccessyes16260:05:24Hidden. If you are the author, please login using the top-right link or use the dashboard.
328866610AmaurXETHZeval2-visualizesuccessyes16170:04:26Hidden. If you are the author, please login using the top-right link or use the dashboard.
328856609AmaurXTTICeval0successyes16380:06:43
Scores from the eval [...]
Scores from the evaluation of job: 32885
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328846610AmaurXETHZeval2successyes16380:07:50
Scores from the eval [...]
Scores from the evaluation of job: 32884
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328836608AmaurXTTICeval2successyes15950:18:16
Scores from the eval [...]
Scores from the evaluation of job: 32883
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328826610AmaurXETHZeval2abortedyes15950:00:03
Scores from the eval [...]
Scores from the evaluation of job: 32882
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328816611AmaurXETHZeval0-videossuccessyes16250:08:25Hidden. If you are the author, please login using the top-right link or use the dashboard.
328806611AmaurXETHZeval0-videossuccessyes16260:08:25Hidden. If you are the author, please login using the top-right link or use the dashboard.
328796610AmaurXETHZeval2abortedyes15950:00:08
Scores from the eval [...]
Scores from the evaluation of job: 32879
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328786611AmaurXETHZeval0-visualizesuccessyes16230:08:07Hidden. If you are the author, please login using the top-right link or use the dashboard.
328776610AmaurXETHZeval2abortedyes15950:00:10
Scores from the eval [...]
Scores from the evaluation of job: 32877
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328766610AmaurXETHZeval2successyes16380:07:16
Scores from the eval [...]
Scores from the evaluation of job: 32876
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328756610AmaurXETHZeval1-visualizesuccessyes16180:02:15Hidden. If you are the author, please login using the top-right link or use the dashboard.
328746610AmaurXETHZeval1-videossuccessyes16260:04:03Hidden. If you are the author, please login using the top-right link or use the dashboard.
328736610AmaurXETHZeval1-videossuccessyes16230:04:32Hidden. If you are the author, please login using the top-right link or use the dashboard.
328726610AmaurXETHZeval1-visualizesuccessyes16170:02:49Hidden. If you are the author, please login using the top-right link or use the dashboard.
328716608AmaurXTTICeval2timeoutyes16380:10:29
Timeout because eval [...]
Timeout because evaluator contacted us
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328706608AmaurXTTICeval1-videossuccessyes16260:04:18Hidden. If you are the author, please login using the top-right link or use the dashboard.
328696610AmaurXETHZeval1successyes16380:05:46
Scores from the eval [...]
Scores from the evaluation of job: 32869
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328686608AmaurXTTICeval1-videossuccessyes16300:04:52Hidden. If you are the author, please login using the top-right link or use the dashboard.
328676611AmaurXETHZeval0successyes15950:21:27
Scores from the eval [...]
Scores from the evaluation of job: 32867
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328666608AmaurXTTICeval2abortedyes16380:01:14
Scores from the eval [...]
Scores from the evaluation of job: 32866
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328656610AmaurXETHZeval1abortedyes15950:00:38
Scores from the eval [...]
Scores from the evaluation of job: 32865
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328646608AmaurXTTICeval2timeoutyes16380:02:54
Timeout because eval [...]
Timeout because evaluator contacted us
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328636609AmaurXTTICeval2-videossuccessyes16230:05:04Hidden. If you are the author, please login using the top-right link or use the dashboard.
328626609AmaurXTTICeval2-videossuccessyes16170:04:31Hidden. If you are the author, please login using the top-right link or use the dashboard.
328616609AmaurXTTICeval2-visualizesuccessyes16520:05:06Hidden. If you are the author, please login using the top-right link or use the dashboard.
328606608AmaurXTTICeval1-videoshost-erroryes16180:04:43
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job32860-881731', 'up', '-d', '--renew-anon-volumes']:

   >  Command '['docker-compose', '-p', 'job32860-881731', 'up', '-d', '--renew-anon-volumes']' returned non-zero exit status 1.

stdout | 

stderr | Creating network "job32860-881731_evaluation" with the default driver
stderr | Creating volume "job32860-881731_fifos" with default driver
stderr | Creating job32860-881731_evaluator_1 ...
stderr | 
stderr | ERROR: for job32860-881731_evaluator_1  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for evaluator  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
stderr | If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).
stderr | 
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328596608AmaurXTTICeval1-visualizesuccessyes16270:09:38Hidden. If you are the author, please login using the top-right link or use the dashboard.
328586608AmaurXTTICeval1successyes15950:21:20
Scores from the eval [...]
Scores from the evaluation of job: 32858
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328576610AmaurXETHZeval1abortedyes15950:02:05
Scores from the eval [...]
Scores from the evaluation of job: 32857
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328566610AmaurXETHZeval1abortedyes15950:00:03
Scores from the eval [...]
Scores from the evaluation of job: 32856
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328556608AmaurXTTICeval0-videossuccessyes16180:04:34Hidden. If you are the author, please login using the top-right link or use the dashboard.
328546610AmaurXETHZeval1abortedyes15950:00:03
Scores from the eval [...]
Scores from the evaluation of job: 32854
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328536608AmaurXTTICeval0-visualizesuccessyes16270:08:26Hidden. If you are the author, please login using the top-right link or use the dashboard.
328526608AmaurXTTICeval0-videossuccessyes16300:09:21Hidden. If you are the author, please login using the top-right link or use the dashboard.
328516610AmaurXETHZeval1abortedyes15950:00:10
Scores from the eval [...]
Scores from the evaluation of job: 32851
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328506608AmaurXTTICeval0-visualizesuccessyes16520:09:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
328496608AmaurXTTICeval0successyes15950:15:46
Scores from the eval [...]
Scores from the evaluation of job: 32849
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328486610AmaurXETHZeval1abortedyes15950:01:02
Scores from the eval [...]
Scores from the evaluation of job: 32848
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328476610AmaurXETHZeval0-videossuccessyes16270:05:00Hidden. If you are the author, please login using the top-right link or use the dashboard.
328466610AmaurXETHZeval0-visualizesuccessyes16230:04:39Hidden. If you are the author, please login using the top-right link or use the dashboard.
328456610AmaurXETHZeval0-videossuccessyes16320:05:31Hidden. If you are the author, please login using the top-right link or use the dashboard.
328446610AmaurXETHZeval0-videossuccessyes16260:05:46Hidden. If you are the author, please login using the top-right link or use the dashboard.
328436610AmaurXETHZeval0-visualizesuccessyes16520:05:30Hidden. If you are the author, please login using the top-right link or use the dashboard.
328426609AmaurXTTICeval2successyes16380:59:16
Scores from the eval [...]
Scores from the evaluation of job: 32842
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328416609AmaurXTTICeval1-videossuccessyes16270:04:39Hidden. If you are the author, please login using the top-right link or use the dashboard.
328406609AmaurXTTICeval1-visualizesuccessyes16520:04:21Hidden. If you are the author, please login using the top-right link or use the dashboard.
328396610AmaurXETHZeval0successyes16380:08:16
Scores from the eval [...]
Scores from the evaluation of job: 32839
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328386608AmaurXTTICeval0timeoutyes16380:01:24
Timeout because eval [...]
Timeout because evaluator contacted us
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328376609AmaurXTTICeval0-videossuccessyes16270:03:58Hidden. If you are the author, please login using the top-right link or use the dashboard.
328366609AmaurXTTICeval0-visualizesuccessyes16520:03:51Hidden. If you are the author, please login using the top-right link or use the dashboard.
328356609AmaurXTTICeval1successyes16380:05:37
Scores from the eval [...]
Scores from the evaluation of job: 32835
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328346609AmaurXTTICeval0abortedyes16380:08:19
Scores from the eval [...]
Scores from the evaluation of job: 32834
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328336607saryazdibluejayeval2abortedyes16380:01:50
Scores from the eval [...]
Scores from the evaluation of job: 32833
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328326607saryazdibluejayeval2timeoutyes16380:50:26
Timeout because eval [...]
Timeout because evaluator contacted us
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328316589frank_qcd_qk2019-12-10|FrankNet|(Kay❤️Frank)eval2timeoutyes16380:03:46
Timeout because eval [...]
Timeout because evaluator contacted us
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328306605JBRRussiachallenge-aido_LF-template-ros - Template solution using ROSeval2-visualizesuccessyes16180:02:47Hidden. If you are the author, please login using the top-right link or use the dashboard.
328296605JBRRussiachallenge-aido_LF-template-ros - Template solution using ROSeval2-videossuccessyes16240:15:03Hidden. If you are the author, please login using the top-right link or use the dashboard.
328286605JBRRussiachallenge-aido_LF-template-ros - Template solution using ROSeval2-visualizesuccessyes16520:15:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
328276605JBRRussiachallenge-aido_LF-template-ros - Template solution using ROSeval2timeoutyes16380:01:00
Timeout because eval [...]
Timeout because evaluator contacted us
Hidden. If you are the author, please login using the top-right link or use the dashboard.
328266605JBRRussiachallenge-aido_LF-template-ros - Template solution using ROSeval2-videossuccessyes16320:15:27Hidden. If you are the author, please login using the top-right link or use the dashboard.
328256605JBRRussiachallenge-aido_LF-template-ros - Template solution using ROSeval2-visualizesuccessyes16230:15:29Hidden. If you are the author, please login using the top-right link or use the dashboard.