Duckietown Challenges Home Challenges Submissions

Submission 17676

Submission17676
Competingyes
Challengeaido-LFVI_multi-sim-validation
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 104323 sim-1of4: 104337 sim-2of4: 104333 sim-3of4: 104332
Next
User labelminimal-agent
Admin priority50
Blessingn/a
User priority50

104337

Click the images to see detailed statistics about the episode.

udem-000

104323

Click the images to see detailed statistics about the episode.

4way-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
104337sim-1of4successyes0:02:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median9.750000000000004
in-drivable-lane_median6.05000000000001
driven_lanedir_consec_median0.30101962152421446
deviation-center-line_median0.33331321378753215


other stats
deviation-center-line_max0.9867208312321796
deviation-center-line_mean0.444755703855315
deviation-center-line_min0.12567555661401583
deviation-heading_max2.548187013405534
deviation-heading_mean1.64877502395785
deviation-heading_median1.6495518057392644
deviation-heading_min0.7478094709473372
distance-from-start_max1.438162383761261
distance-from-start_mean0.7289714152785263
distance-from-start_median0.5564612149057249
distance-from-start_min0.3648008475413942
driven_any_max1.4382693230332029
driven_any_mean0.729004624492714
driven_any_median0.556470991690694
driven_any_min0.3648071915562648
driven_lanedir_consec_max0.7375240370941809
driven_lanedir_consec_mean0.3748149029942042
driven_lanedir_consec_min0.15969633183420706
driven_lanedir_max0.746168337032741
driven_lanedir_mean0.37697597797884425
driven_lanedir_median0.30101962152421446
driven_lanedir_min0.15969633183420706
in-drivable-lane_max8.300000000000004
in-drivable-lane_mean5.100000000000006
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 0.6981436944804056, "survival_time": 9.750000000000004, "driven_lanedir": 0.199551480959018, "in-drivable-lane": 8.000000000000004, "deviation-heading": 0.9488219555580464, "distance-from-start": 0.6981319454565775, "deviation-center-line": 0.12567555661401583, "driven_lanedir_consec": 0.199551480959018}, "udem-000-ego1": {"driven_any": 0.3648071915562648, "survival_time": 9.750000000000004, "driven_lanedir": 0.15969633183420706, "in-drivable-lane": 8.300000000000004, "deviation-heading": 0.7478094709473372, "distance-from-start": 0.3648008475413942, "deviation-center-line": 0.1356249877645468, "driven_lanedir_consec": 0.15969633183420706}, "udem-000-ego2": {"driven_any": 0.4147982889009824, "survival_time": 9.750000000000004, "driven_lanedir": 0.40248776208941095, "in-drivable-lane": 0.0, "deviation-heading": 2.350281655920482, "distance-from-start": 0.41479048435487215, "deviation-center-line": 0.9867208312321796, "driven_lanedir_consec": 0.40248776208941095}, "udem-000-ego3": {"driven_any": 1.4382693230332029, "survival_time": 9.750000000000004, "driven_lanedir": 0.746168337032741, "in-drivable-lane": 4.100000000000016, "deviation-heading": 2.548187013405534, "distance-from-start": 1.438162383761261, "deviation-center-line": 0.5310014398105175, "driven_lanedir_consec": 0.7375240370941809}}
simulation-passed1
survival_time_max9.750000000000004
survival_time_mean9.750000000000004
survival_time_min9.750000000000004
No reset possible
104333sim-2of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
104332sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
104329sim-2of4successyes0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104327sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104323sim-0of4successyes0:03:28
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median14.250000000000068
in-drivable-lane_median11.000000000000073
driven_lanedir_consec_median0.3959648974675054
deviation-center-line_median0.2727383614229232


other stats
deviation-center-line_max0.3711792475730154
deviation-center-line_mean0.2630702395458521
deviation-center-line_min0.1356249877645468
deviation-heading_max2.5383505748046193
deviation-heading_mean1.5839274061600843
deviation-heading_median1.530547580682912
deviation-heading_min0.7362638884698944
distance-from-start_max2.1822334065150604
distance-from-start_mean1.3494658762364755
distance-from-start_median1.4254146254447235
distance-from-start_min0.3648008475413942
driven_any_max2.182581278372698
driven_any_mean1.3495961616000156
driven_any_median1.4254980882355446
driven_any_min0.3648071915562748
driven_lanedir_consec_max0.6388981877279706
driven_lanedir_consec_mean0.39763107862429714
driven_lanedir_consec_min0.15969633183420706
driven_lanedir_max0.6388981877279706
driven_lanedir_mean0.39763107862429714
driven_lanedir_median0.3959648974675054
driven_lanedir_min0.15969633183420706
in-drivable-lane_max12.800000000000068
in-drivable-lane_mean11.012500000000074
in-drivable-lane_min9.250000000000078
per-episodes
details{"4way-000-ego0": {"driven_any": 1.1588419001142332, "survival_time": 14.250000000000068, "driven_lanedir": 0.331229062365487, "in-drivable-lane": 10.950000000000072, "deviation-heading": 2.5383505748046193, "distance-from-start": 1.1587848457868677, "deviation-center-line": 0.26973530940385876, "driven_lanedir_consec": 0.331229062365487}, "4way-000-ego1": {"driven_any": 0.3648071915562748, "survival_time": 14.250000000000068, "driven_lanedir": 0.15969633183420706, "in-drivable-lane": 12.800000000000068, "deviation-heading": 0.7478094709473372, "distance-from-start": 0.3648008475413942, "deviation-center-line": 0.1356249877645468, "driven_lanedir_consec": 0.15969633183420706}, "4way-000-ego2": {"driven_any": 1.692154276356856, "survival_time": 14.250000000000068, "driven_lanedir": 0.6388981877279706, "in-drivable-lane": 9.250000000000078, "deviation-heading": 2.3132856904184864, "distance-from-start": 1.6920444051025798, "deviation-center-line": 0.3711792475730154, "driven_lanedir_consec": 0.6388981877279706}, "4way-000-ego3": {"driven_any": 2.182581278372698, "survival_time": 14.250000000000068, "driven_lanedir": 0.4607007325695238, "in-drivable-lane": 11.050000000000072, "deviation-heading": 0.7362638884698944, "distance-from-start": 2.1822334065150604, "deviation-center-line": 0.2757414134419876, "driven_lanedir_consec": 0.4607007325695238}}
simulation-passed1
survival_time_max14.250000000000068
survival_time_mean14.250000000000068
survival_time_min14.250000000000068
No reset possible
104322sim-1of4successyes0:02:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104321sim-2of4successyes0:01:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104320sim-3of4successyes0:01:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101739sim-1of4failedno0:01:04
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101733sim-3of4successno0:01:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101725sim-1of4failedno0:00:54
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101723sim-3of4successno0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101605sim-2of4successno0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101603sim-0of4failedno0:01:07
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible