Submission 4535

Submission	`4535`

Competing	yes
Challenge	`aido3-LF-real-validation`
User	Gianmarco Bernasconi
Date submitted	2019-11-21 15:47:08+00:00
Last status update	2021-12-04 20:53:33+00:00
Complete	not complete
Details	Violations... Ready to do eval0-visualize, eval0-videos, eval1-videos, eval2-videos, eval1-visualize, eval2-visualize.
Sisters
Result
Jobs	eval0: `26808` eval1: `28606` eval2: `28616`
Next	eval0-visualize eval0-videos eval1-videos eval2-videos eval1-visualize eval2-visualize
User label	challenge-aido_LF-baseline-duckietown
Admin priority	80
Blessing	n/a
User priority	80

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID step status up to date date started date completed duration message

28623

eval2-videos

success

2019-12-04 14:01:14+00:00

2019-12-04 14:06:57+00:00

0:05:43

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28622

eval2-visualize

success

2019-12-04 13:55:11+00:00

2019-12-04 14:05:35+00:00

0:10:24

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28621

eval2-videos

host-error

2019-12-04 13:55:10+00:00

2019-12-04 14:00:41+00:00

0:05:31

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 894, in run_one
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1277, in run_docker
    subprocess.check_call(cmd0, cwd=cwd, stdout=tee_stdout, stderr=tee_stderr)
  File "/usr/lib/python3.6/subprocess.py", line 306, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 287, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido3-LF-real-validation/submission4535/eval2-videos-ip-172-31-45-34-10499-job28621': '/tmp/duckietown/DT18/evaluator/executions/aido3-LF-real-validation/submission4535/eval2-videos-ip-172-31-45-34-10499-job28621'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28620

eval2-videos

error

2019-12-04 13:54:40+00:00

2019-12-04 14:00:20+00:00

0:05:40

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28619

eval2-visualize

host-error

2019-12-04 13:54:39+00:00

2019-12-04 14:00:15+00:00

0:05:36

Error while running [...]


stderr | Creating job28619-481221_evaluator_1 ... error
stderr | 
stderr | ERROR: for job28619-481221_evaluator_1  Cannot start service evaluator: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/tmp/duckietown/DT18/evaluator/executions/aido3-LF-real-validation/submission4535/eval2-visualize-ip-172-31-45-34-10501-job28619/fifos\\\" to rootfs \\\"/var/lib/docker/overlay2/e29155acd5f4e1be3ad4dfdd4fe94c11c5815e1f91f1a1f6e173cc6addb99610/merged\\\" at \\\"/fifos\\\" caused \\\"stat /tmp/duckietown/DT18/evaluator/executions/aido3-LF-real-validation/submission4535/eval2-visualize-ip-172-31-45-34-10501-job28619/fifos: no such file or directory\\\"\"": unknown
stderr | 
stderr | ERROR: for evaluator  Cannot start service evaluator: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused \"rootfs_linux.go:58: mounting \\\"/tmp/duckietown/DT18/evaluator/executions/aido3-LF-real-validation/submission4535/eval2-visualize-ip-172-31-45-34-10501-job28619/fifos\\\" to rootfs \\\"/var/lib/docker/overlay2/e29155acd5f4e1be3ad4dfdd4fe94c11c5815e1f91f1a1f6e173cc6addb99610/merged\\\" at \\\"/fifos\\\" caused \\\"stat /tmp/duckietown/DT18/evaluator/executions/aido3-LF-real-validation/submission4535/eval2-visualize-ip-172-31-45-34-10501-job28619/fifos: no such file or directory\\\"\"": unknown
stderr | Encountered errors while bringing up the project.
stderr |

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28618

eval1-visualize

success

2019-12-04 13:45:24+00:00

2019-12-04 13:49:37+00:00

0:04:13

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28617

eval1-visualize

success

2019-12-04 13:44:52+00:00

2019-12-04 13:49:08+00:00

0:04:16

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28616

eval2

success

yes

2019-12-04 13:21:01+00:00

2019-12-04 13:53:54+00:00

0:32:53

Scores from the eval [...]

Scores from the evaluation of job: 28616

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

other stats

driven_lanedir_consec_median	7.5054652305586655
survival_time_median	58.923

No reset possible

28615

eval1-visualize

success

2019-12-04 13:20:53+00:00

2019-12-04 13:44:50+00:00

0:23:57

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28614

eval1-videos

success

2019-12-04 13:20:48+00:00

2019-12-04 13:46:55+00:00

0:26:07

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28613

eval1-visualize

success

2019-12-04 13:20:21+00:00

2019-12-04 13:44:50+00:00

0:24:29

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28612

eval1-videos

success

2019-12-04 13:20:14+00:00

2019-12-04 13:46:56+00:00

0:26:42

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28606

eval1

success

yes

2019-12-04 12:58:44+00:00

2019-12-04 13:19:42+00:00

0:20:58

Scores from the eval [...]

Scores from the evaluation of job: 28606

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

other stats

driven_lanedir_consec_median	8.761913927741482
survival_time_median	69.618

No reset possible

28600

eval1

aborted

yes

2019-12-04 12:26:31+00:00

2019-12-04 12:51:33+00:00

0:25:02

Scores from the eval [...]

Scores from the evaluation of job: 28600

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28571

eval1

aborted

yes

2019-12-04 12:02:45+00:00

2019-12-04 12:06:49+00:00

0:04:04

Scores from the eval [...]

Scores from the evaluation of job: 28571

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

28568

eval1

timeout

yes

2019-12-04 11:47:01+00:00

2019-12-04 11:50:21+00:00

0:03:20

Timeout because eval [...]

Timeout because evaluator contacted us

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26830

eval0-videos

success

2019-12-02 16:12:39+00:00

2019-12-02 16:17:47+00:00

0:05:08

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26829

eval0-videos

success

2019-12-02 16:12:34+00:00

2019-12-02 16:20:43+00:00

0:08:09

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26828

eval0-videos

success

2019-12-02 16:11:48+00:00

2019-12-02 16:22:44+00:00

0:10:56

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26827

eval0-videos

success

2019-12-02 16:11:45+00:00

2019-12-02 16:16:50+00:00

0:05:05

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26826

eval0-visualize

success

2019-12-02 16:11:30+00:00

2019-12-02 16:20:52+00:00

0:09:22

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26825

eval0-visualize

success

2019-12-02 16:11:04+00:00

2019-12-02 16:20:40+00:00

0:09:36

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26808

eval0

success

yes

2019-12-02 15:52:28+00:00

2019-12-02 16:10:09+00:00

0:17:41

Scores from the eval [...]

Scores from the evaluation of job: 26808

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

other stats

driven_lanedir_consec_median	7.914067929141605
survival_time_median	45.475

No reset possible

26806

eval0

timeout

yes

2019-12-02 14:41:35+00:00

2019-12-02 15:44:38+00:00

1:03:03

Timeout because eval [...]

Timeout because evaluator contacted us

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26023

eval0-visualize

error

2019-11-25 11:02:29+00:00

2019-11-25 11:03:12+00:00

0:00:43

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "main.py", line 19, in wrap
    stats = evaluate_robotarium_log(logdir, outd=outd)
  File "/project/evaluate_robotarium_log.py", line 41, in evaluate_robotarium_log
    d = yaml.load(open(mapfile).read())
FileNotFoundError: [Errno 2] No such file or directory: '/challenges/previous-steps/eval0/trajectories/map.yaml'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 676, in scoring_context
    yield cie
  File "main.py", line 34, in <module>
    wrap(cie)
  File "main.py", line 24, in wrap
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

26020

eval0-visualize

error

2019-11-25 11:02:11+00:00

2019-11-25 11:03:05+00:00

0:00:54

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "main.py", line 19, in wrap
    stats = evaluate_robotarium_log(logdir, outd=outd)
  File "/project/evaluate_robotarium_log.py", line 41, in evaluate_robotarium_log
    d = yaml.load(open(mapfile).read())
FileNotFoundError: [Errno 2] No such file or directory: '/challenges/previous-steps/eval0/trajectories/map.yaml'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 676, in scoring_context
    yield cie
  File "main.py", line 34, in <module>
    wrap(cie)
  File "main.py", line 24, in wrap
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25971

eval0

aborted

yes

2019-11-22 13:24:25+00:00

2019-11-22 13:50:48+00:00

0:26:23

Scores from the eval [...]

Scores from the evaluation of job: 25971

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25970 eval0 aborted yes 2019-11-22 13:18:51+00:00 - - - - No reset possible

25968

eval0

aborted

yes

2019-11-22 13:09:10+00:00

2019-11-22 13:18:08+00:00

0:08:58

Timeout because eval [...]

Timeout because evaluator contacted us

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25966

eval0

aborted

yes

2019-11-22 12:58:41+00:00

2019-11-22 13:08:00+00:00

0:09:19

Scores from the eval [...]

Scores from the evaluation of job: 25966

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25964

eval0

aborted

yes

2019-11-22 12:36:47+00:00

2019-11-22 12:57:08+00:00

0:20:21

Scores from the eval [...]

Scores from the evaluation of job: 25964

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25962

eval0

aborted

yes

2019-11-22 12:32:30+00:00

2019-11-22 12:35:13+00:00

0:02:43

Scores from the eval [...]

Scores from the evaluation of job: 25962

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25961

eval0

aborted

yes

2019-11-22 12:16:12+00:00

2019-11-22 12:21:07+00:00

0:04:55

Scores from the eval [...]

Scores from the evaluation of job: 25961

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25959

eval0

aborted

yes

2019-11-22 12:13:22+00:00

2019-11-22 12:13:29+00:00

0:00:07

Scores from the eval [...]

Scores from the evaluation of job: 25959

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25957

eval0

aborted

yes

2019-11-22 11:55:40+00:00

2019-11-22 11:56:24+00:00

0:00:44

Scores from the eval [...]

Scores from the evaluation of job: 25957

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25954

eval0

aborted

yes

2019-11-21 18:29:32+00:00

2019-11-21 18:43:54+00:00

0:14:22

Scores from the eval [...]

Scores from the evaluation of job: 25954

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25952

eval0

aborted

yes

2019-11-21 17:59:41+00:00

2019-11-21 18:29:02+00:00

0:29:21

Scores from the eval [...]

Scores from the evaluation of job: 25952

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25951

eval1

aborted

yes

2019-11-21 17:48:01+00:00

2019-11-21 17:59:35+00:00

0:11:34

Scores from the eval [...]

Scores from the evaluation of job: 25951

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25949

eval0

aborted

yes

2019-11-21 17:30:17+00:00

2019-11-21 17:47:34+00:00

0:17:17

Scores from the eval [...]

Scores from the evaluation of job: 25949

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25947

eval0

aborted

yes

2019-11-21 16:44:42+00:00

2019-11-21 16:45:04+00:00

0:00:22

Scores from the eval [...]

Scores from the evaluation of job: 25947

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25942

eval0

aborted

yes

2019-11-21 15:56:59+00:00

2019-11-21 17:27:31+00:00

1:30:32

Scores from the eval [...]

Scores from the evaluation of job: 25942

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25941

eval0

aborted

yes

2019-11-21 15:50:10+00:00

2019-11-21 15:52:33+00:00

0:02:23

Scores from the eval [...]

Scores from the evaluation of job: 25941

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible