Duckietown Challenges Home Challenges Submissions

Submission 13509

Submission13509
Competingyes
Challengeaido-LFI-sim-validation
UserAndrás Kalapos 🇭🇺
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 106127 sim-1of4: 106094 sim-2of4: 106100 sim-3of4: 106098
Next
User labelreal-v1.0-3091-310
Admin priority50
Blessingn/a
User priority50

106127

Click the images to see detailed statistics about the episode.

udem1-000

106094

Click the images to see detailed statistics about the episode.

4way-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
106127sim-0of4successyes0:02:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median9.950000000000006
in-drivable-lane_median1.050000000000015
driven_lanedir_consec_median3.948472045449301
deviation-center-line_median0.32343709465081966


other stats
deviation-center-line_max0.32343709465081966
deviation-center-line_mean0.32343709465081966
deviation-center-line_min0.32343709465081966
deviation-heading_max1.591377056804706
deviation-heading_mean1.591377056804706
deviation-heading_median1.591377056804706
deviation-heading_min1.591377056804706
distance-from-start_max2.246757950100424
distance-from-start_mean2.246757950100424
distance-from-start_median2.246757950100424
distance-from-start_min2.246757950100424
driven_any_max4.311582310812604
driven_any_mean4.311582310812604
driven_any_median4.311582310812604
driven_any_min4.311582310812604
driven_lanedir_consec_max3.948472045449301
driven_lanedir_consec_mean3.948472045449301
driven_lanedir_consec_min3.948472045449301
driven_lanedir_max4.05545872537013
driven_lanedir_mean4.05545872537013
driven_lanedir_median4.05545872537013
driven_lanedir_min4.05545872537013
in-drivable-lane_max1.050000000000015
in-drivable-lane_mean1.050000000000015
in-drivable-lane_min1.050000000000015
per-episodes
details{"udem1-000-ego0": {"driven_any": 4.311582310812604, "survival_time": 9.950000000000006, "driven_lanedir": 4.05545872537013, "in-drivable-lane": 1.050000000000015, "deviation-heading": 1.591377056804706, "distance-from-start": 2.246757950100424, "deviation-center-line": 0.32343709465081966, "driven_lanedir_consec": 3.948472045449301}}
simulation-passed1
survival_time_max9.950000000000006
survival_time_mean9.950000000000006
survival_time_min9.950000000000006
No reset possible
106119sim-0of4successyes0:02:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
106109sim-0of4successyes0:02:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
106100sim-2of4successyes0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
106098sim-3of4successyes0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
106097sim-0of4successyes0:02:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
106094sim-1of4successyes0:04:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median35.9000000000001
in-drivable-lane_median13.74999999999996
driven_lanedir_consec_median9.850412039446482
deviation-center-line_median1.4417970642593658


other stats
deviation-center-line_max1.4417970642593658
deviation-center-line_mean1.4417970642593658
deviation-center-line_min1.4417970642593658
deviation-heading_max6.04948380305133
deviation-heading_mean6.04948380305133
deviation-heading_median6.04948380305133
deviation-heading_min6.04948380305133
distance-from-start_max1.2959701693887355
distance-from-start_mean1.2959701693887355
distance-from-start_median1.2959701693887355
distance-from-start_min1.2959701693887355
driven_any_max16.72095619830474
driven_any_mean16.72095619830474
driven_any_median16.72095619830474
driven_any_min16.72095619830474
driven_lanedir_consec_max9.850412039446482
driven_lanedir_consec_mean9.850412039446482
driven_lanedir_consec_min9.850412039446482
driven_lanedir_max10.21629330047332
driven_lanedir_mean10.21629330047332
driven_lanedir_median10.21629330047332
driven_lanedir_min10.21629330047332
in-drivable-lane_max13.74999999999996
in-drivable-lane_mean13.74999999999996
in-drivable-lane_min13.74999999999996
per-episodes
details{"4way-000-ego0": {"driven_any": 16.72095619830474, "survival_time": 35.9000000000001, "driven_lanedir": 10.21629330047332, "in-drivable-lane": 13.74999999999996, "deviation-heading": 6.04948380305133, "distance-from-start": 1.2959701693887355, "deviation-center-line": 1.4417970642593658, "driven_lanedir_consec": 9.850412039446482}}
simulation-passed1
survival_time_max35.9000000000001
survival_time_mean35.9000000000001
survival_time_min35.9000000000001
No reset possible
106092sim-2of4successyes0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
106089sim-3of4successyes0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
106085sim-2of4successyes0:01:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101312sim-0of4failedno0:01:07
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101311sim-3of4successno0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101310sim-1of4failedno0:01:06
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101308sim-2of4successno0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101303sim-1of4host-errorno0:01:06
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101302sim-0of4failedno0:01:07
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101299sim-2of4successno0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101298sim-3of4successno0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101294sim-0of4host-errorno0:01:07
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101293sim-3of4successno0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101290sim-2of4successno0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101289sim-1of4failedno0:01:05
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101282sim-1of4host-errorno0:01:06
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101280sim-3of4successno0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101279sim-0of4host-errorno0:01:08
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101278sim-2of4successno0:01:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101271sim-0of4failedno0:01:10
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101270sim-1of4host-errorno0:01:08
The container "simul [...]
The container "simulator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101269sim-3of4successno0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101267sim-2of4successno0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible