Duckietown Challenges Home Challenges Submissions

Submission 17658

Submission17658
Competingyes
Challengeaido-LFVI_multi-sim-validation
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 105311 sim-1of4: 105346 sim-2of4: 105314 sim-3of4: 105359
Next
User labeltemplate-random
Admin priority50
Blessingn/a
User priority50

105346

Click the images to see detailed statistics about the episode.

udem-000

105311

Click the images to see detailed statistics about the episode.

4way-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
105359sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105353sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105346sim-1of4successyes0:10:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median33.75000000000022
in-drivable-lane_median29.825000000000212
driven_lanedir_consec_median0.21731228142323933
deviation-center-line_median0.3331807431846997


other stats
deviation-center-line_max0.7655512378412563
deviation-center-line_mean0.379245390698747
deviation-center-line_min0.0850688385843324
deviation-heading_max5.708723015945328
deviation-heading_mean2.8582267855416443
deviation-heading_median2.247219992369361
deviation-heading_min1.229744141482529
distance-from-start_max0.9340577697896516
distance-from-start_mean0.6073071039932331
distance-from-start_median0.5461575079396492
distance-from-start_min0.4028556303039823
driven_any_max8.310583890769195
driven_any_mean3.410525124545609
driven_any_median2.453720078564638
driven_any_min0.4240764502839678
driven_lanedir_consec_max0.3926134494082678
driven_lanedir_consec_mean0.2327684277313727
driven_lanedir_consec_min0.10383569867074448
driven_lanedir_max0.3926134494082678
driven_lanedir_mean0.2281267458150863
driven_lanedir_median0.21016777279894416
driven_lanedir_min0.09955798825418904
in-drivable-lane_max31.650000000000222
in-drivable-lane_mean29.125000000000213
in-drivable-lane_min25.2000000000002
per-episodes
details{"udem-000-ego0": {"driven_any": 8.310583890769195, "survival_time": 33.75000000000022, "driven_lanedir": 0.25382856508768814, "in-drivable-lane": 25.2000000000002, "deviation-heading": 5.708723015945328, "distance-from-start": 0.9340577697896516, "deviation-center-line": 0.7655512378412563, "driven_lanedir_consec": 0.2718370953578422}, "udem-000-ego1": {"driven_any": 0.4240764502839678, "survival_time": 33.75000000000022, "driven_lanedir": 0.3926134494082678, "in-drivable-lane": 31.650000000000222, "deviation-heading": 1.229744141482529, "distance-from-start": 0.4028556303039823, "deviation-center-line": 0.0850688385843324, "driven_lanedir_consec": 0.3926134494082678}, "udem-000-ego2": {"driven_any": 2.4390062983425578, "survival_time": 33.75000000000022, "driven_lanedir": 0.1665069805102002, "in-drivable-lane": 29.350000000000215, "deviation-heading": 2.144336915325389, "distance-from-start": 0.5461575079396477, "deviation-center-line": 0.3777614073769809, "driven_lanedir_consec": 0.16278746748863648}, "udem-000-ego3": {"driven_any": 2.4684338587867183, "survival_time": 33.75000000000022, "driven_lanedir": 0.09955798825418904, "in-drivable-lane": 30.30000000000021, "deviation-heading": 2.350103069413333, "distance-from-start": 0.5461575079396509, "deviation-center-line": 0.2886000789924186, "driven_lanedir_consec": 0.10383569867074448}}
simulation-passed1
survival_time_max33.75000000000022
survival_time_mean33.75000000000022
survival_time_min33.75000000000022
No reset possible
105345sim-3of4successyes0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105314sim-2of4successyes0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105311sim-0of4successyes0:09:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median27.85000000000026
in-drivable-lane_median23.575000000000244
driven_lanedir_consec_median0.2152178291743677
deviation-center-line_median0.34607680035313926


other stats
deviation-center-line_max0.7555295762526522
deviation-center-line_mean0.3831880038858158
deviation-center-line_min0.0850688385843324
deviation-heading_max5.523299013646982
deviation-heading_mean2.976848653346106
deviation-heading_median2.577175729127456
deviation-heading_min1.229744141482529
distance-from-start_max0.8413577961802242
distance-from-start_mean0.6171534020652945
distance-from-start_median0.6122000908884857
distance-from-start_min0.4028556303039823
driven_any_max6.724613878580574
driven_any_mean3.448516774187835
driven_any_median3.322688383943399
driven_any_min0.4240764502839678
driven_lanedir_consec_max0.3926134494082678
driven_lanedir_consec_mean0.23172102279300744
driven_lanedir_consec_min0.10383498341502671
driven_lanedir_max0.3926134494082678
driven_lanedir_mean0.22283655208020933
driven_lanedir_median0.1995874449338097
driven_lanedir_min0.09955786904495012
in-drivable-lane_max25.750000000000263
in-drivable-lane_mean23.05000000000025
in-drivable-lane_min19.30000000000024
per-episodes
details{"4way-000-ego0": {"driven_any": 6.724613878580574, "survival_time": 27.85000000000026, "driven_lanedir": 0.24963678127672728, "in-drivable-lane": 19.30000000000024, "deviation-heading": 5.523299013646982, "distance-from-start": 0.8413577961802242, "deviation-center-line": 0.7555295762526522, "driven_lanedir_consec": 0.26764984149988424}, "4way-000-ego1": {"driven_any": 0.4240764502839678, "survival_time": 27.85000000000026, "driven_lanedir": 0.3926134494082678, "in-drivable-lane": 25.750000000000263, "deviation-heading": 1.229744141482529, "distance-from-start": 0.4028556303039823, "deviation-center-line": 0.0850688385843324, "driven_lanedir_consec": 0.3926134494082678}, "4way-000-ego2": {"driven_any": 4.117732239446978, "survival_time": 27.85000000000026, "driven_lanedir": 0.14953810859089212, "in-drivable-lane": 22.75000000000024, "deviation-heading": 2.8042483888415783, "distance-from-start": 0.6736010379604935, "deviation-center-line": 0.4035531580478341, "driven_lanedir_consec": 0.16278581684885107}, "4way-000-ego3": {"driven_any": 2.527644528439821, "survival_time": 27.85000000000026, "driven_lanedir": 0.09955786904495012, "in-drivable-lane": 24.40000000000025, "deviation-heading": 2.350103069413334, "distance-from-start": 0.5507991438164779, "deviation-center-line": 0.2886004426584445, "driven_lanedir_consec": 0.10383498341502671}}
simulation-passed1
survival_time_max27.85000000000026
survival_time_mean27.85000000000026
survival_time_min27.85000000000026
No reset possible
103013sim-1of4failedno0:00:46
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
103012sim-2of4successno0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
103008sim-3of4successno0:01:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
103006sim-0of4failedno0:00:46
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
103003sim-1of4failedno0:00:54
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102999sim-2of4successno0:01:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102998sim-0of4failedno0:00:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102993sim-3of4successno0:01:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102992sim-1of4failedno0:00:50
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102990sim-0of4failedno0:00:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102986sim-1of4failedno0:00:42
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102984sim-2of4successno0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible