Duckietown Challenges Home Challenges Submissions

Submission 17630

Submission17630
Competingyes
Challengeaido-LFVI-full-sim-testing
UserBea Baselines 🐤
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 104556 sim-1of4: 104554 sim-2of4: 104581 sim-3of4: 104566
Next
User labelminimal-agent-full
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
104581sim-2of4successyes0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
104578sim-2of4successyes0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104573sim-2of4successyes0:01:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104568sim-2of4successyes0:01:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104566sim-3of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
104560sim-2of4successyes0:01:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104559sim-3of4successyes0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104556sim-0of4successyes0:07:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median49.9499999999993
in-drivable-lane_median0.0
driven_lanedir_consec_median2.903013185974951
deviation-center-line_median3.78943987331601


other stats
deviation-center-line_max3.78943987331601
deviation-center-line_mean3.78943987331601
deviation-center-line_min3.78943987331601
deviation-heading_max4.334690003953373
deviation-heading_mean4.334690003953373
deviation-heading_median4.334690003953373
deviation-heading_min4.334690003953373
distance-from-start_max1.5738876609453012
distance-from-start_mean1.5738876609453012
distance-from-start_median1.5738876609453012
distance-from-start_min1.5738876609453012
driven_any_max2.934594724791732
driven_any_mean2.934594724791732
driven_any_median2.934594724791732
driven_any_min2.934594724791732
driven_lanedir_consec_max2.903013185974951
driven_lanedir_consec_mean2.903013185974951
driven_lanedir_consec_min2.903013185974951
driven_lanedir_max2.9099324876731436
driven_lanedir_mean2.9099324876731436
driven_lanedir_median2.9099324876731436
driven_lanedir_min2.9099324876731436
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 2.934594724791732, "survival_time": 49.9499999999993, "driven_lanedir": 2.9099324876731436, "in-drivable-lane": 0.0, "deviation-heading": 4.334690003953373, "distance-from-start": 1.5738876609453012, "deviation-center-line": 3.78943987331601, "driven_lanedir_consec": 2.903013185974951}}
simulation-passed1
survival_time_max49.9499999999993
survival_time_mean49.9499999999993
survival_time_min49.9499999999993
No reset possible
104554sim-1of4successyes0:03:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median14.800000000000075
in-drivable-lane_median0.0
driven_lanedir_consec_median0.6855199039137942
deviation-center-line_median1.1151363222094262


other stats
deviation-center-line_max1.1151363222094262
deviation-center-line_mean1.1151363222094262
deviation-center-line_min1.1151363222094262
deviation-heading_max1.8997107406045024
deviation-heading_mean1.8997107406045024
deviation-heading_median1.8997107406045024
deviation-heading_min1.8997107406045024
distance-from-start_max0.6890064417552494
distance-from-start_mean0.6890064417552494
distance-from-start_median0.6890064417552494
distance-from-start_min0.6890064417552494
driven_any_max0.8255923756486474
driven_any_mean0.8255923756486474
driven_any_median0.8255923756486474
driven_any_min0.8255923756486474
driven_lanedir_consec_max0.6855199039137942
driven_lanedir_consec_mean0.6855199039137942
driven_lanedir_consec_min0.6855199039137942
driven_lanedir_max0.8113565710261739
driven_lanedir_mean0.8113565710261739
driven_lanedir_median0.8113565710261739
driven_lanedir_min0.8113565710261739
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_min0.0
per-episodes
details{"4way-000-ego0": {"driven_any": 0.8255923756486474, "survival_time": 14.800000000000075, "driven_lanedir": 0.8113565710261739, "in-drivable-lane": 0.0, "deviation-heading": 1.8997107406045024, "distance-from-start": 0.6890064417552494, "deviation-center-line": 1.1151363222094262, "driven_lanedir_consec": 0.6855199039137942}}
simulation-passed1
survival_time_max14.800000000000075
survival_time_mean14.800000000000075
survival_time_min14.800000000000075
No reset possible
104551sim-3of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
104546sim-3of4successyes0:01:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102468sim-3of4successno0:01:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102467sim-0of4failedno0:01:02
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102455sim-0of4failedno0:01:13
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102449sim-3of4successno0:01:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102444sim-0of4failedno0:01:17
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102439sim-3of4successno0:01:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102432sim-0of4failedno0:01:25
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102426sim-3of4successno0:01:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
102425sim-0of4failedno0:01:11
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101558sim-2of4successno0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
101557sim-1of4failedno0:01:29
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible