Submission 13609

Submission	`13609`

Competing	yes
Challenge	`aido-LFV_multi-sim-validation`
User	Andras Beres
Date submitted	2021-01-31 22:17:50+00:00
Last status update	2021-12-04 20:47:23+00:00
Complete	complete
Details	Evaluation is complete.
Sisters
Result	💚
Jobs	sim-0of4: `105006` sim-1of4: `105001` sim-2of4: `105008` sim-3of4: `105012`
Next
User label	fsf+il
Admin priority	50
Blessing	n/a
User priority	50

105012

Click the images to see detailed statistics about the episode.

zigzag-000

105008

Click the images to see detailed statistics about the episode.

loop-000

105006

Click the images to see detailed statistics about the episode.

loop2-000

105001

Click the images to see detailed statistics about the episode.

autolab-000

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID step status up to date date started date completed duration message

105012

sim-3of4

success

yes

2021-11-02 23:27:11+00:00

2021-11-02 23:39:54+00:00

0:12:43

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	48.04999999999941
in-drivable-lane_median	1.400000000000002
driven_lanedir_consec_median	12.321453207979236
deviation-center-line_median	3.4864195191239116

other stats

deviation-center-line_max	5.994432738138446
deviation-center-line_mean	3.897847194685539
deviation-center-line_min	2.624117002355886
deviation-heading_max	16.825936194772645
deviation-heading_mean	9.228888405392246
deviation-heading_median	7.85076934488408
deviation-heading_min	4.388078737028178
distance-from-start_max	4.545915690786742
distance-from-start_mean	3.840131580575844
distance-from-start_median	3.7397349552661
distance-from-start_min	3.3351407209844317
driven_any_max	22.64339562096856
driven_any_mean	14.255101158688875
driven_any_median	13.073846602207626
driven_any_min	8.229315809371698
driven_lanedir_consec_max	21.49522399055956
driven_lanedir_consec_mean	13.489351136554584
driven_lanedir_consec_min	7.819274139700303
driven_lanedir_max	21.49522399055956
driven_lanedir_mean	13.489351136554584
driven_lanedir_median	12.321453207979236
driven_lanedir_min	7.819274139700303
in-drivable-lane_max	2.549999999999984
in-drivable-lane_mean	1.337499999999997
in-drivable-lane_min	0.0
per-episodes	details {"zigzag-000-ego0": {"driven_any": 8.488527303229343, "survival_time": 48.04999999999941, "driven_lanedir": 8.335988699510631, "in-drivable-lane": 0.0, "deviation-heading": 16.825936194772645, "distance-from-start": 3.752748021412601, "deviation-center-line": 5.994432738138446, "driven_lanedir_consec": 8.335988699510631}, "zigzag-000-ego1": {"driven_any": 8.229315809371698, "survival_time": 48.04999999999941, "driven_lanedir": 7.819274139700303, "in-drivable-lane": 0.70000000000001, "deviation-heading": 4.388078737028178, "distance-from-start": 3.3351407209844317, "deviation-center-line": 2.624117002355886, "driven_lanedir_consec": 7.819274139700303}, "zigzag-000-ego2": {"driven_any": 22.64339562096856, "survival_time": 48.04999999999941, "driven_lanedir": 21.49522399055956, "in-drivable-lane": 2.0999999999999943, "deviation-heading": 7.089642638603622, "distance-from-start": 3.7267218891196, "deviation-center-line": 3.5672860463129195, "driven_lanedir_consec": 21.49522399055956}, "zigzag-000-ego3": {"driven_any": 17.659165901185908, "survival_time": 48.04999999999941, "driven_lanedir": 16.306917716447842, "in-drivable-lane": 2.549999999999984, "deviation-heading": 8.611896051164537, "distance-from-start": 4.545915690786742, "deviation-center-line": 3.405552991934904, "driven_lanedir_consec": 16.306917716447842}}
simulation-passed	1
survival_time_max	48.04999999999941
survival_time_mean	48.04999999999941
survival_time_min	48.04999999999941

No reset possible

105008

sim-2of4

success

yes

2021-11-02 23:24:33+00:00

2021-11-02 23:30:02+00:00

0:05:29

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	20.05000000000015
in-drivable-lane_median	0.04999999999999982
driven_lanedir_consec_median	3.2879455948370335
deviation-center-line_median	1.435760538235538

other stats

deviation-center-line_max	2.3155954911986583
deviation-center-line_mean	1.6105324207837906
deviation-center-line_min	1.2550131154654272
deviation-heading_max	2.669908179498758
deviation-heading_mean	1.4951173171378178
deviation-heading_median	1.260268433648053
deviation-heading_min	0.7900242217564071
distance-from-start_max	3.183126437909208
distance-from-start_mean	2.3008182593488704
distance-from-start_median	2.313084096388629
distance-from-start_min	1.3939784067090155
driven_any_max	9.897704859572052
driven_any_mean	4.588581302885249
driven_any_median	3.319834435925789
driven_any_min	1.8169514801173643
driven_lanedir_consec_max	9.63221452891225
driven_lanedir_consec_mean	4.488307484016873
driven_lanedir_consec_min	1.745124217481175
driven_lanedir_max	9.63221452891225
driven_lanedir_mean	4.488307484016873
driven_lanedir_median	3.2879455948370335
driven_lanedir_min	1.745124217481175
in-drivable-lane_max	0.34999999999999876
in-drivable-lane_mean	0.1124999999999996
in-drivable-lane_min	0.0
per-episodes	details {"loop-000-ego0": {"driven_any": 1.8169514801173643, "survival_time": 20.05000000000015, "driven_lanedir": 1.745124217481175, "in-drivable-lane": 0.09999999999999964, "deviation-heading": 0.7900242217564071, "distance-from-start": 1.3939784067090155, "deviation-center-line": 2.3155954911986583, "driven_lanedir_consec": 1.745124217481175}, "loop-000-ego1": {"driven_any": 9.897704859572052, "survival_time": 20.05000000000015, "driven_lanedir": 9.63221452891225, "in-drivable-lane": 0.34999999999999876, "deviation-heading": 2.669908179498758, "distance-from-start": 3.183126437909208, "deviation-center-line": 1.2550131154654272, "driven_lanedir_consec": 9.63221452891225}, "loop-000-ego2": {"driven_any": 4.76183525314148, "survival_time": 20.05000000000015, "driven_lanedir": 4.714995530289445, "in-drivable-lane": 0.0, "deviation-heading": 1.6291790398337922, "distance-from-start": 3.1603853345088666, "deviation-center-line": 1.2833386272455098, "driven_lanedir_consec": 4.714995530289445}, "loop-000-ego3": {"driven_any": 1.877833618710098, "survival_time": 20.05000000000015, "driven_lanedir": 1.8608956593846224, "in-drivable-lane": 0.0, "deviation-heading": 0.8913578274623137, "distance-from-start": 1.4657828582683914, "deviation-center-line": 1.5881824492255665, "driven_lanedir_consec": 1.8608956593846224}}
simulation-passed	1
survival_time_max	20.05000000000015
survival_time_mean	20.05000000000015
survival_time_min	20.05000000000015

No reset possible

105006

sim-0of4

success

yes

2021-11-02 23:24:01+00:00

2021-11-02 23:31:15+00:00

0:07:14

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	59.99999999999873
in-drivable-lane_median	0.0
driven_lanedir_consec_median	29.531951370457456
deviation-center-line_median	4.176548488915756

other stats

deviation-center-line_max	4.216209458301931
deviation-center-line_mean	4.176548488915756
deviation-center-line_min	4.136887519529582
deviation-heading_max	9.929001403221555
deviation-heading_mean	9.804603382231864
deviation-heading_median	9.804603382231864
deviation-heading_min	9.680205361242171
distance-from-start_max	1.5576228270125034
distance-from-start_mean	1.5379231965822755
distance-from-start_median	1.5379231965822755
distance-from-start_min	1.5182235661520471
driven_any_max	30.157533317024193
driven_any_mean	30.11450720571249
driven_any_median	30.11450720571249
driven_any_min	30.071481094400784
driven_lanedir_consec_max	29.586291967178088
driven_lanedir_consec_mean	29.531951370457456
driven_lanedir_consec_min	29.477610773736828
driven_lanedir_max	29.586291967178088
driven_lanedir_mean	29.531951370457456
driven_lanedir_median	29.531951370457456
driven_lanedir_min	29.477610773736828
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"loop2-000-ego0": {"driven_any": 30.157533317024193, "survival_time": 59.99999999999873, "driven_lanedir": 29.586291967178088, "in-drivable-lane": 0.0, "deviation-heading": 9.680205361242171, "distance-from-start": 1.5182235661520471, "deviation-center-line": 4.136887519529582, "driven_lanedir_consec": 29.586291967178088}, "loop2-000-ego1": {"driven_any": 30.071481094400784, "survival_time": 59.99999999999873, "driven_lanedir": 29.477610773736828, "in-drivable-lane": 0.0, "deviation-heading": 9.929001403221555, "distance-from-start": 1.5576228270125034, "deviation-center-line": 4.216209458301931, "driven_lanedir_consec": 29.477610773736828}}
simulation-passed	1
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

105001

sim-1of4

success

yes

2021-11-02 23:22:01+00:00

2021-11-02 23:28:17+00:00

0:06:16

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	13.00000000000005
in-drivable-lane_median	0.825000000000006
driven_lanedir_consec_median	2.635621499928275
deviation-center-line_median	0.7759472610789162

other stats

deviation-center-line_max	1.2335049036579908
deviation-center-line_mean	0.7810890486031707
deviation-center-line_min	0.3389567685968598
deviation-heading_max	4.1818343078918465
deviation-heading_mean	2.4501856648058076
deviation-heading_median	2.194073098796678
deviation-heading_min	1.2307621537380278
distance-from-start_max	2.8108091583767627
distance-from-start_mean	2.0978107681442437
distance-from-start_median	1.9415663000803671
distance-from-start_min	1.697301314039478
driven_any_max	5.874694482531834
driven_any_mean	3.5518377028740376
driven_any_median	2.9199466857990037
driven_any_min	2.4927629573663075
driven_lanedir_consec_max	5.689340871416261
driven_lanedir_consec_mean	3.300178647040193
driven_lanedir_consec_min	2.240130716887959
driven_lanedir_max	5.689340871416261
driven_lanedir_mean	3.300178647040193
driven_lanedir_median	2.635621499928275
driven_lanedir_min	2.240130716887959
in-drivable-lane_max	7.650000000000061
in-drivable-lane_mean	2.4750000000000174
in-drivable-lane_min	0.5999999999999979
per-episodes	details {"autolab-000-ego0": {"driven_any": 2.5290324199932144, "survival_time": 13.00000000000005, "driven_lanedir": 2.240130716887959, "in-drivable-lane": 0.5999999999999979, "deviation-heading": 2.720439072535883, "distance-from-start": 1.8980016679374911, "deviation-center-line": 0.77427421258511, "driven_lanedir_consec": 2.240130716887959}, "autolab-000-ego1": {"driven_any": 2.4927629573663075, "survival_time": 13.00000000000005, "driven_lanedir": 2.2643951399834745, "in-drivable-lane": 7.650000000000061, "deviation-heading": 1.2307621537380278, "distance-from-start": 1.9851309322232429, "deviation-center-line": 0.3389567685968598, "driven_lanedir_consec": 2.2643951399834745}, "autolab-000-ego2": {"driven_any": 5.874694482531834, "survival_time": 13.00000000000005, "driven_lanedir": 5.689340871416261, "in-drivable-lane": 1.0000000000000142, "deviation-heading": 1.6677071250574729, "distance-from-start": 2.8108091583767627, "deviation-center-line": 0.7776203095727224, "driven_lanedir_consec": 5.689340871416261}, "autolab-000-ego3": {"driven_any": 3.3108609516047935, "survival_time": 13.00000000000005, "driven_lanedir": 3.0068478598730755, "in-drivable-lane": 0.6499999999999977, "deviation-heading": 4.1818343078918465, "distance-from-start": 1.697301314039478, "deviation-center-line": 1.2335049036579908, "driven_lanedir_consec": 3.0068478598730755}}
simulation-passed	1
survival_time_max	13.00000000000005
survival_time_mean	13.00000000000005
survival_time_min	13.00000000000005

No reset possible

100101

sim-1of4

failed

2021-11-02 05:26:23+00:00

2021-11-02 05:27:22+00:00

0:00:59

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100097

sim-0of4

failed

2021-11-02 05:25:23+00:00

2021-11-02 05:26:26+00:00

0:01:03

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100094

sim-1of4

failed

2021-11-02 05:25:20+00:00

2021-11-02 05:26:11+00:00

0:00:51

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100093

sim-2of4

failed

2021-11-02 05:25:20+00:00

2021-11-02 05:26:19+00:00

0:00:59

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100084

sim-2of4

failed

2021-11-02 05:23:47+00:00

2021-11-02 05:24:49+00:00

0:01:02

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100083

sim-0of4

failed

2021-11-02 05:23:45+00:00

2021-11-02 05:24:46+00:00

0:01:01

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100082

sim-3of4

failed

2021-11-02 05:23:45+00:00

2021-11-02 05:24:37+00:00

0:00:52

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100081

sim-1of4

failed

2021-11-02 05:23:43+00:00

2021-11-02 05:24:44+00:00

0:01:01

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100074

sim-3of4

failed

2021-11-02 05:22:10+00:00

2021-11-02 05:23:19+00:00

0:01:09

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100072

sim-2of4

failed

2021-11-02 05:21:41+00:00

2021-11-02 05:23:18+00:00

0:01:37

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100069

sim-1of4

failed

2021-11-02 05:21:06+00:00

2021-11-02 05:23:21+00:00

0:02:15

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

100065

sim-0of4

failed

2021-11-02 05:20:41+00:00

2021-11-02 05:23:21+00:00

0:02:40

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 293, in main
    sim_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 328, in read_reply
    raise RemoteNodeAborted(msg) from None
zuper_nodes.structures.RemoteNodeAborted: Remote node closed communication (None)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97934

sim-0of4

success

2021-11-01 20:31:18+00:00

2021-11-01 20:38:26+00:00

0:07:08

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97933

sim-3of4

success

2021-11-01 20:30:47+00:00

2021-11-01 20:44:11+00:00

0:13:24

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97918

sim-0of4

success

2021-11-01 20:23:56+00:00

2021-11-01 20:31:05+00:00

0:07:09

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97895

sim-3of4

success

2021-11-01 20:17:00+00:00

2021-11-01 20:30:12+00:00

0:13:12

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97894

sim-0of4

success

2021-11-01 20:16:23+00:00

2021-11-01 20:23:28+00:00

0:07:05

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97893

sim-1of4

success

2021-11-01 20:15:53+00:00

2021-11-01 20:20:21+00:00

0:04:28

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97883

sim-2of4

success

2021-11-01 20:12:21+00:00

2021-11-01 20:17:40+00:00

0:05:19

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

97869

sim-2of4

success

2021-11-01 20:04:05+00:00

2021-11-01 20:10:45+00:00

0:06:40

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

90195

403

success

yes

2021-10-21 16:17:25+00:00

2021-10-21 16:44:19+00:00

0:26:54

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

90187

402

success

yes

2021-10-21 15:42:41+00:00

2021-10-21 16:17:11+00:00

0:34:30

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

88816 402 timeout yes 2021-10-19 10:23:49+00:00 - - - - No reset possible

71458

402

success

yes

2021-04-21 16:15:25+00:00

2021-04-21 16:29:08+00:00

0:13:43

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

71457

403

success

yes

2021-04-21 16:15:22+00:00

2021-04-21 16:30:05+00:00

0:14:43

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

70770

402

success

2021-04-21 13:47:41+00:00

2021-04-21 14:00:15+00:00

0:12:34

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

70769

403

success

2021-04-21 13:47:15+00:00

2021-04-21 14:00:34+00:00

0:13:19

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

65742

403

success

2021-04-09 09:13:55+00:00

2021-04-09 09:28:55+00:00

0:15:00

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

63134

402

success

2021-04-08 23:04:30+00:00

2021-04-08 23:46:04+00:00

0:41:34

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

62331

356

success

yes

2021-03-23 07:18:01+00:00

2021-03-23 08:44:12+00:00

1:26:11

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

62075

356

success

2021-02-01 01:07:11+00:00

2021-02-01 02:35:38+00:00

1:28:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

62051

356

host-error

2021-01-31 22:19:34+00:00

2021-01-31 22:28:37+00:00

0:09:03

InvalidEnvironment: [...]

InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego3" aborted with the following error:

error in ego3 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "submission.py", line 60, in init
              ||     self.agent = self.create_agent(self.env)
              ||   File "submission.py", line 162, in create_agent_dagger
              ||     agent = DaggerAgent.load_from_checkpoint(
              ||   File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/utilities/device_dtype_mixin.py", line 124, in cuda
              ||     return super().cuda(device=device)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 248, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 242, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego3" aborted with the following error:

error in ego3 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "submission.py", line 60, in init
              ||     self.agent = self.create_agent(self.env)
              ||   File "submission.py", line 162, in create_agent_dagger
              ||     agent = DaggerAgent.load_from_checkpoint(
              ||   File "/usr/local/lib/python3.8/dist-packages/pytorch_lightning/utilities/device_dtype_mixin.py", line 124, in cuda
              ||     return super().cuda(device=device)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible