Duckietown Challenges Home Challenges Submissions

Submission 11543

Submission11543
Competingyes
Challengeaido5-LF-sim-validation
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54437
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54437

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
54437LFv-simsuccessyes0:26:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median6.229995277573435
survival_time_median44.1999999999995
deviation-center-line_median2.182068153318206
in-drivable-lane_median3.725000000000053


other stats
agent_compute-ego0_max0.013610877973962123
agent_compute-ego0_mean0.013175286533413242
agent_compute-ego0_median0.01311502738479571
agent_compute-ego0_min0.012860213390099417
complete-iteration_max0.2283041068258929
complete-iteration_mean0.19695918121253717
complete-iteration_median0.1927205133334545
complete-iteration_min0.17409159135734678
deviation-center-line_max4.398214432037851
deviation-center-line_mean2.258744777995552
deviation-center-line_min0.2726283733079452
deviation-heading_max14.300149212732972
deviation-heading_mean7.780347288218778
deviation-heading_median8.063353571567086
deviation-heading_min0.6945327970079658
driven_any_max12.03990369032132
driven_any_mean7.92673626622937
driven_any_median7.990752704263581
driven_any_min3.685535966069
driven_lanedir_consec_max11.539878592945154
driven_lanedir_consec_mean6.186246469242104
driven_lanedir_consec_min0.7451167288763929
driven_lanedir_max11.539878592945154
driven_lanedir_mean6.186246469242104
driven_lanedir_median6.229995277573435
driven_lanedir_min0.7451167288763929
get_duckie_state_max1.5017437390158172e-06
get_duckie_state_mean1.4399841900348497e-06
get_duckie_state_median1.476269578258759e-06
get_duckie_state_min1.305653864606063e-06
get_robot_state_max0.00385250902762522
get_robot_state_mean0.003822895989749199
get_robot_state_median0.003820011756700044
get_robot_state_min0.00379905141797149
get_state_dump_max0.00568071386760607
get_state_dump_mean0.005045706571826384
get_state_dump_median0.004877562529300727
get_state_dump_min0.004746987361098011
get_ui_image_max0.036630949509530934
get_ui_image_mean0.031219869181224477
get_ui_image_median0.0306170864022709
get_ui_image_min0.02701435441082517
in-drivable-lane_max24.55000000000027
in-drivable-lane_mean8.000000000000096
in-drivable-lane_min0.0
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 12.03990369032132, "get_ui_image": 0.02914718684308435, "step_physics": 0.11828811182566824, "survival_time": 59.99999999999873, "driven_lanedir": 11.539878592945154, "get_state_dump": 0.004746987361098011, "get_robot_state": 0.0038294949003500702, "sim_render-ego0": 0.003906328017070431, "get_duckie_state": 1.4805376877097862e-06, "in-drivable-lane": 0.25000000000000355, "deviation-heading": 13.370134023692817, "agent_compute-ego0": 0.012860213390099417, "complete-iteration": 0.18687098210896183, "set_robot_commands": 0.0023350781147724185, "deviation-center-line": 3.5384731247714183, "driven_lanedir_consec": 11.539878592945154, "sim_compute_sim_state": 0.009553401694508218, "sim_compute_performance-ego0": 0.002114846049300042}, "LF-norm-zigzag-000-ego0": {"driven_any": 10.625171265047664, "get_ui_image": 0.036630949509530934, "step_physics": 0.1482141006002815, "survival_time": 59.99999999999873, "driven_lanedir": 10.168415679697215, "get_state_dump": 0.00476457375868671, "get_robot_state": 0.0038105286130500175, "sim_render-ego0": 0.003957769058824678, "get_duckie_state": 1.4720014688077318e-06, "in-drivable-lane": 0.0, "deviation-heading": 14.300149212732972, "agent_compute-ego0": 0.012908558762143, "complete-iteration": 0.2283041068258929, "set_robot_commands": 0.0023317364828473425, "deviation-center-line": 4.398214432037851, "driven_lanedir_consec": 10.168415679697215, "sim_compute_sim_state": 0.013468949423543024, "sim_compute_performance-ego0": 0.002125104996286562}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.685535966069, "get_ui_image": 0.03208698596145744, "step_physics": 0.12460312819540352, "survival_time": 20.00000000000015, "driven_lanedir": 2.291574875449653, "get_state_dump": 0.00568071386760607, "get_robot_state": 0.00379905141797149, "sim_render-ego0": 0.003796828952513431, "get_duckie_state": 1.305653864606063e-06, "in-drivable-lane": 7.200000000000102, "deviation-heading": 2.7565731194413563, "agent_compute-ego0": 0.01332149600744842, "complete-iteration": 0.19857004455794716, "set_robot_commands": 0.0023632293330166404, "deviation-center-line": 0.8256631818649931, "driven_lanedir_consec": 2.291574875449653, "sim_compute_sim_state": 0.0108395092505172, "sim_compute_performance-ego0": 0.0019930645711998688}, "LF-norm-small_loop-000-ego0": {"driven_any": 5.356334143479498, "get_ui_image": 0.02701435441082517, "step_physics": 0.10942441340280544, "survival_time": 28.40000000000027, "driven_lanedir": 0.7451167288763929, "get_state_dump": 0.004990551299914744, "get_robot_state": 0.00385250902762522, "sim_render-ego0": 0.003974992487258777, "get_duckie_state": 1.5017437390158172e-06, "in-drivable-lane": 24.55000000000027, "deviation-heading": 0.6945327970079658, "agent_compute-ego0": 0.013610877973962123, "complete-iteration": 0.17409159135734678, "set_robot_commands": 0.0023378278333399125, "deviation-center-line": 0.2726283733079452, "driven_lanedir_consec": 0.7451167288763929, "sim_compute_sim_state": 0.006745796840513528, "sim_compute_performance-ego0": 0.0020502535446247443}}
set_robot_commands_max0.0023632293330166404
set_robot_commands_mean0.002341967940994078
set_robot_commands_median0.0023364529740561657
set_robot_commands_min0.0023317364828473425
sim_compute_performance-ego0_max0.002125104996286562
sim_compute_performance-ego0_mean0.0020708172903528043
sim_compute_performance-ego0_median0.002082549796962393
sim_compute_performance-ego0_min0.0019930645711998688
sim_compute_sim_state_max0.013468949423543024
sim_compute_sim_state_mean0.010151914302270491
sim_compute_sim_state_median0.01019645547251271
sim_compute_sim_state_min0.006745796840513528
sim_render-ego0_max0.003974992487258777
sim_render-ego0_mean0.003908979628916829
sim_render-ego0_median0.003932048537947554
sim_render-ego0_min0.003796828952513431
simulation-passed1
step_physics_max0.1482141006002815
step_physics_mean0.12513243850603967
step_physics_median0.12144562001053588
step_physics_min0.10942441340280544
survival_time_max59.99999999999873
survival_time_mean42.09999999999947
survival_time_min20.00000000000015
No reset possible
54435LFv-simsuccessyes0:30:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54434LFv-simsuccessyes0:22:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54433LFv-simsuccessyes0:36:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54432LFv-simsuccessyes0:30:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
54431LFv-simsuccessyes0:29:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49590LFv-simerrorno0:09:43
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139942211294640
- M:video_aido:cmdline(in:/;out:/) 139942211293248
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49580LFv-simerrorno0:08:12
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139831664354112
- M:video_aido:cmdline(in:/;out:/) 139831664355168
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
49578LFv-simerrorno0:09:32
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140353940596096
- M:video_aido:cmdline(in:/;out:/) 140353940597968
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42843LFv-simfailedno0:03:48
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 164 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42842LFv-simfailedno0:09:06
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 838 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42841LFv-simsuccessno0:13:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
42840LFv-simsuccessno0:08:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible