Duckietown Challenges Home Challenges Submissions

Submission 13601

Submission13601
Competingyes
Challengeaido-LFI-sim-validation
UserAndras Beres
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 116786 sim-1of4: 105368 sim-2of4: 105356 sim-3of4: 105350
Next
User labelfsf+il
Admin priority50
Blessingn/a
User priority50

116786

Click the images to see detailed statistics about the episode.

udem1-000

105368

Click the images to see detailed statistics about the episode.

4way-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
116786sim-0of4successyes0:18:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median26.150000000000237
in-drivable-lane_median3.700000000000044
driven_lanedir_consec_median10.025385364108905
deviation-center-line_median1.79988824534035


other stats
deviation-center-line_max1.79988824534035
deviation-center-line_mean1.79988824534035
deviation-center-line_min1.79988824534035
deviation-heading_max4.579655851707952
deviation-heading_mean4.579655851707952
deviation-heading_median4.579655851707952
deviation-heading_min4.579655851707952
distance-from-start_max2.263822943862504
distance-from-start_mean2.263822943862504
distance-from-start_median2.263822943862504
distance-from-start_min2.263822943862504
driven_any_max12.057880662432456
driven_any_mean12.057880662432456
driven_any_median12.057880662432456
driven_any_min12.057880662432456
driven_lanedir_consec_max10.025385364108905
driven_lanedir_consec_mean10.025385364108905
driven_lanedir_consec_min10.025385364108905
driven_lanedir_max10.430924607522147
driven_lanedir_mean10.430924607522147
driven_lanedir_median10.430924607522147
driven_lanedir_min10.430924607522147
in-drivable-lane_max3.700000000000044
in-drivable-lane_mean3.700000000000044
in-drivable-lane_min3.700000000000044
per-episodes
details{"udem1-000-ego0": {"driven_any": 12.057880662432456, "survival_time": 26.150000000000237, "driven_lanedir": 10.430924607522147, "in-drivable-lane": 3.700000000000044, "deviation-heading": 4.579655851707952, "distance-from-start": 2.263822943862504, "deviation-center-line": 1.79988824534035, "driven_lanedir_consec": 10.025385364108905}}
simulation-passed1
survival_time_max26.150000000000237
survival_time_mean26.150000000000237
survival_time_min26.150000000000237
No reset possible
105368sim-1of4successyes0:02:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median11.950000000000037
in-drivable-lane_median0.5000000000000071
driven_lanedir_consec_median4.215364946095
deviation-center-line_median0.8778272483686288


other stats
deviation-center-line_max0.8778272483686288
deviation-center-line_mean0.8778272483686288
deviation-center-line_min0.8778272483686288
deviation-heading_max2.1711734934635016
deviation-heading_mean2.1711734934635016
deviation-heading_median2.1711734934635016
deviation-heading_min2.1711734934635016
distance-from-start_max2.020788232918576
distance-from-start_mean2.020788232918576
distance-from-start_median2.020788232918576
distance-from-start_min2.020788232918576
driven_any_max5.307490845620776
driven_any_mean5.307490845620776
driven_any_median5.307490845620776
driven_any_min5.307490845620776
driven_lanedir_consec_max4.215364946095
driven_lanedir_consec_mean4.215364946095
driven_lanedir_consec_min4.215364946095
driven_lanedir_max4.970094299779473
driven_lanedir_mean4.970094299779473
driven_lanedir_median4.970094299779473
driven_lanedir_min4.970094299779473
in-drivable-lane_max0.5000000000000071
in-drivable-lane_mean0.5000000000000071
in-drivable-lane_min0.5000000000000071
per-episodes
details{"4way-000-ego0": {"driven_any": 5.307490845620776, "survival_time": 11.950000000000037, "driven_lanedir": 4.970094299779473, "in-drivable-lane": 0.5000000000000071, "deviation-heading": 2.1711734934635016, "distance-from-start": 2.020788232918576, "deviation-center-line": 0.8778272483686288, "driven_lanedir_consec": 4.215364946095}}
simulation-passed1
survival_time_max11.950000000000037
survival_time_mean11.950000000000037
survival_time_min11.950000000000037
No reset possible
105361sim-1of4successyes0:02:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105356sim-2of4successyes0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105350sim-3of4successyes0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105348sim-2of4successyes0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105343sim-3of4successyes0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105341sim-2of4successyes0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105335sim-2of4successyes0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105328sim-2of4successyes0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100400sim-0of4host-errorno0:01:04
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100390sim-0of4failedno0:01:06
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100376sim-0of4host-errorno0:01:12
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100367sim-0of4failedno0:01:08
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100363sim-1of4failedno0:01:05
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100354sim-1of4host-errorno0:01:07
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100353sim-0of4failedno0:01:17
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100345sim-0of4host-errorno0:01:08
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100343sim-1of4host-errorno0:01:15
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100333sim-1of4host-errorno0:00:59
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 268, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 953, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.40/containers/821e4c6592fc635da7a7b930c8367cc69e71bcbd2abd6dca9a71d70f6f322210/json

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1265, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 897, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/container.py", line 771, in inspect_container
    return self._result(
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 274, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 270, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.NotFound: 404 Client Error for http+docker://localhost/v1.40/containers/821e4c6592fc635da7a7b930c8367cc69e71bcbd2abd6dca9a71d70f6f322210/json: Not Found ("No such container: 821e4c6592fc635da7a7b930c8367cc69e71bcbd2abd6dca9a71d70f6f322210")
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100331sim-0of4failedno0:01:17
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100321sim-3of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100304sim-3of4successno0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100291sim-3of4successno0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100283sim-3of4successno0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100268sim-3of4successno0:01:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100259sim-1of4host-errorno0:01:05
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100256sim-0of4host-errorno0:01:11
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100255sim-2of4successno0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100245sim-0of4host-errorno0:01:08
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100243sim-2of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100242sim-1of4host-errorno0:01:10
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100232sim-0of4host-errorno0:01:09
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
100226sim-1of4failedno0:01:07
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98057sim-3of4successno0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98052sim-2of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98050sim-1of4successno0:06:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98049sim-3of4successno0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98046sim-2of4successno0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98044sim-0of4successno0:06:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98043sim-3of4successno0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98040sim-2of4successno0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98035sim-3of4successno0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98033sim-0of4successno0:02:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98032sim-2of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98029sim-1of4successno0:04:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
98027sim-3of4successno0:01:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
90406sim-1of4successno0:18:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
90388sim-0of4successno0:09:36
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
90386sim-2of4successno0:02:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible