Submission 17783

Submission

17783

Competing

yes

Challenge

aido-hello-sim-validation

User

Bea Baselines 🐤

Date submitted

2021-11-03 20:16:41+00:00

Last status update

2022-01-25 13:09:38+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

sanity-check: 128365

User label

challenge-aido_LF-template-pytorch:daffy:0c7f47987b4675b26902025da29aae7bb75771fd

Admin priority

Blessing

n/a

User priority

128365

Click the images to see detailed statistics about the episode.

scenarios

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID

step

status

up to date

date started

date completed

duration

message

128365

sanity-check

success

yes

2022-01-25 13:03:15+00:00

2022-01-25 13:04:18+00:00

0:01:03

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	0.49999999999999994
in-drivable-lane_median	0.0
driven_lanedir_consec_median	0.021388380795927864
deviation-center-line_median	0.0053394608355529305

other stats

deviation-center-line_max	0.0053394608355529305
deviation-center-line_mean	0.0053394608355529305
deviation-center-line_min	0.0053394608355529305
deviation-heading_max	0.3085092391119281
deviation-heading_mean	0.3085092391119281
deviation-heading_median	0.3085092391119281
deviation-heading_min	0.3085092391119281
distance-from-start_max	0.0282659725843045
distance-from-start_mean	0.0282659725843045
distance-from-start_median	0.0282659725843045
distance-from-start_min	0.0282659725843045
driven_any_max	0.02841899828306222
driven_any_mean	0.02841899828306222
driven_any_median	0.02841899828306222
driven_any_min	0.02841899828306222
driven_lanedir_consec_max	0.021388380795927864
driven_lanedir_consec_mean	0.021388380795927864
driven_lanedir_consec_min	0.021388380795927864
driven_lanedir_max	0.021388380795927864
driven_lanedir_mean	0.021388380795927864
driven_lanedir_median	0.021388380795927864
driven_lanedir_min	0.021388380795927864
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"scenarios-ego0": {"driven_any": 0.02841899828306222, "survival_time": 0.49999999999999994, "driven_lanedir": 0.021388380795927864, "in-drivable-lane": 0.0, "deviation-heading": 0.3085092391119281, "distance-from-start": 0.0282659725843045, "deviation-center-line": 0.0053394608355529305, "driven_lanedir_consec": 0.021388380795927864}}
simulation-passed	1
survival_time_max	0.49999999999999994
survival_time_mean	0.49999999999999994
survival_time_min	0.49999999999999994

No reset possible

128357

sanity-check

success

yes

2022-01-25 13:01:37+00:00

2022-01-25 13:02:42+00:00

0:01:05

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

128353

sanity-check

success

yes

2022-01-25 12:59:59+00:00

2022-01-25 13:01:02+00:00

0:01:03

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

128347

sanity-check

success

yes

2022-01-25 12:58:24+00:00

2022-01-25 12:59:23+00:00

0:00:59

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

128313

sanity-check

host-error

yes

2022-01-25 12:48:55+00:00

2022-01-25 12:52:56+00:00

0:04:01

The container "simul [...]

The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

128302

sanity-check

success

yes

2022-01-25 12:45:39+00:00

2022-01-25 12:48:09+00:00

0:02:30

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

108444

sanity-check

failed

2021-11-04 01:49:51+00:00

2021-11-04 01:50:35+00:00

0:00:44

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

108441

sanity-check

failed

2021-11-04 01:48:41+00:00

2021-11-04 01:49:19+00:00

0:00:38

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

108439

sanity-check

failed

2021-11-04 01:47:32+00:00

2021-11-04 01:48:15+00:00

0:00:43

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

108438

sanity-check

failed

2021-11-04 01:46:33+00:00

2021-11-04 01:47:11+00:00

0:00:38

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

108437

sanity-check

failed

2021-11-04 01:45:31+00:00

2021-11-04 01:46:13+00:00

0:00:42

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

108430

sanity-check

host-error

2021-11-04 01:36:25+00:00

2021-11-04 01:40:05+00:00

0:03:40

Error while running [...]

Error while running Docker Compose:

Could not run command
│    cmd: [docker-compose, -p, gpu-production-spot-0-04-job108430-596659, up, --no-start,
│          --renew-anon-volumes]
│ stdout: ''
│  sderr: ''
│      e: Command '['docker-compose', '-p', 'gpu-production-spot-0-04-job108430-596659', 'up', '--no-start', '--renew-anon-volumes']' returned non-zero exit status 1.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible