Duckietown Challenges Home Challenges Submissions

Submission 11056

Submission11056
Competingyes
Challengeaido5-LF-sim-validation
UserBhavya Patwa 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 56937
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

56937

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
56937LFv-simsuccessyes0:20:27
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.8374319595497
survival_time_median26.925000000000068
deviation-center-line_median0.7480026792103054
in-drivable-lane_median7.4249999999998


other stats
agent_compute-ego0_max0.014914569936596784
agent_compute-ego0_mean0.013052555278917789
agent_compute-ego0_median0.01284728631832903
agent_compute-ego0_min0.011601078542416309
complete-iteration_max0.20801141129031203
complete-iteration_mean0.1782702934341005
complete-iteration_median0.17882389669046406
complete-iteration_min0.14742196906516197
deviation-center-line_max3.0503832421142247
deviation-center-line_mean1.202436867998091
deviation-center-line_min0.26335887145752873
deviation-heading_max11.929297592885716
deviation-heading_mean4.88428231341549
deviation-heading_median2.908067604942091
deviation-heading_min1.7916964508920614
driven_any_max6.840827848418442
driven_any_mean3.748405653174149
driven_any_median3.3326950846680994
driven_any_min1.487404594941954
driven_lanedir_consec_max5.838766620560801
driven_lanedir_consec_mean2.5340030274079153
driven_lanedir_consec_min0.6223815699714597
driven_lanedir_max5.838766620560801
driven_lanedir_mean2.5352235599703876
driven_lanedir_median1.8374319595497
driven_lanedir_min0.6272637002213495
get_duckie_state_max1.2593028807144844e-06
get_duckie_state_mean1.202404688207296e-06
get_duckie_state_median1.1984668683720551e-06
get_duckie_state_min1.1533821353705896e-06
get_robot_state_max0.003679310886371737
get_robot_state_mean0.003565196910569934
get_robot_state_median0.0035761428286429295
get_robot_state_min0.00342919109862214
get_state_dump_max0.004985185983675307
get_state_dump_mean0.004817331038426446
get_state_dump_median0.004861917176726814
get_state_dump_min0.004560303816576852
get_ui_image_max0.03551010307835919
get_ui_image_mean0.029679241705644243
get_ui_image_median0.02931301466702254
get_ui_image_min0.024580834410172715
in-drivable-lane_max24.199999999999985
in-drivable-lane_mean10.862499999999912
in-drivable-lane_min4.4000000000000625
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.420133121713322, "get_ui_image": 0.02676723138341364, "step_physics": 0.1021522492411323, "survival_time": 37.05000000000003, "driven_lanedir": 2.0271931287774714, "get_state_dump": 0.004560303816576852, "get_robot_state": 0.0035527064151198396, "sim_render-ego0": 0.003744996140266686, "get_duckie_state": 1.1567478231342655e-06, "in-drivable-lane": 24.199999999999985, "deviation-heading": 2.8935233689360667, "agent_compute-ego0": 0.011601078542416309, "complete-iteration": 0.1668201003755842, "set_robot_commands": 0.0021158482507875344, "deviation-center-line": 0.688877213622229, "driven_lanedir_consec": 2.0271931287774714, "sim_compute_sim_state": 0.010310387996971768, "sim_compute_performance-ego0": 0.0019366789378245889}, "LF-norm-zigzag-000-ego0": {"driven_any": 1.487404594941954, "get_ui_image": 0.03551010307835919, "step_physics": 0.130880964672105, "survival_time": 11.60000000000003, "driven_lanedir": 0.6272637002213495, "get_state_dump": 0.004980960117389204, "get_robot_state": 0.00359957924216602, "sim_render-ego0": 0.003902023953941247, "get_duckie_state": 1.2401859136098445e-06, "in-drivable-lane": 7.150000000000038, "deviation-heading": 1.7916964508920614, "agent_compute-ego0": 0.014914569936596784, "complete-iteration": 0.20801141129031203, "set_robot_commands": 0.0021549458155816206, "deviation-center-line": 0.26335887145752873, "driven_lanedir_consec": 0.6223815699714597, "sim_compute_sim_state": 0.010020381902932095, "sim_compute_performance-ego0": 0.0019644473243680635}, "LF-norm-techtrack-000-ego0": {"driven_any": 2.2452570476228764, "get_ui_image": 0.031858797950631436, "step_physics": 0.12049036209944096, "survival_time": 16.800000000000104, "driven_lanedir": 1.6476707903219283, "get_state_dump": 0.004742874236064424, "get_robot_state": 0.003679310886371737, "sim_render-ego0": 0.003856890279390691, "get_duckie_state": 1.2593028807144844e-06, "in-drivable-lane": 4.4000000000000625, "deviation-heading": 2.922611840948116, "agent_compute-ego0": 0.012264228362357936, "complete-iteration": 0.1908276930053439, "set_robot_commands": 0.002164421166439792, "deviation-center-line": 0.8071281447983819, "driven_lanedir_consec": 1.6476707903219283, "sim_compute_sim_state": 0.009739864473880576, "sim_compute_performance-ego0": 0.001949328697046116}, "LF-norm-small_loop-000-ego0": {"driven_any": 6.840827848418442, "get_ui_image": 0.024580834410172715, "step_physics": 0.08752152901902782, "survival_time": 59.99999999999873, "driven_lanedir": 5.838766620560801, "get_state_dump": 0.004985185983675307, "get_robot_state": 0.00342919109862214, "sim_render-ego0": 0.003710683636820187, "get_duckie_state": 1.1533821353705896e-06, "in-drivable-lane": 7.699999999999562, "deviation-heading": 11.929297592885716, "agent_compute-ego0": 0.013430344274300125, "complete-iteration": 0.14742196906516197, "set_robot_commands": 0.0021558238703642757, "deviation-center-line": 3.0503832421142247, "driven_lanedir_consec": 5.838766620560801, "sim_compute_sim_state": 0.00567038430460883, "sim_compute_performance-ego0": 0.001864381674227369}}
set_robot_commands_max0.002164421166439792
set_robot_commands_mean0.002147759775793306
set_robot_commands_median0.002155384842972948
set_robot_commands_min0.0021158482507875344
sim_compute_performance-ego0_max0.0019644473243680635
sim_compute_performance-ego0_mean0.0019287091583665343
sim_compute_performance-ego0_median0.0019430038174353525
sim_compute_performance-ego0_min0.001864381674227369
sim_compute_sim_state_max0.010310387996971768
sim_compute_sim_state_mean0.008935254669598318
sim_compute_sim_state_median0.009880123188406337
sim_compute_sim_state_min0.00567038430460883
sim_render-ego0_max0.003902023953941247
sim_render-ego0_mean0.0038036485026047024
sim_render-ego0_median0.003800943209828689
sim_render-ego0_min0.003710683636820187
simulation-passed1
step_physics_max0.130880964672105
step_physics_mean0.11026127625792652
step_physics_median0.1113213056702866
step_physics_min0.08752152901902782
survival_time_max59.99999999999873
survival_time_mean31.36249999999972
survival_time_min11.60000000000003
No reset possible
56926LFv-simfailedyes0:20:08
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 2270 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
56925LFv-simsuccessyes0:31:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
56922LFv-simfailedyes0:05:03
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 477 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
56920LFv-simfailedyes0:21:03
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 218, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 341, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego0".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 563, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 135, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 281, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego0-out after 1770 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 567, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50773LFv-simerrorno0:06:55
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139719501028272
- M:video_aido:cmdline(in:/;out:/) 139719500328816
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40340LFv-simsuccessno0:09:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39705LFv-simfailedno0:09:24
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 903 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39704LFv-simsuccessno0:12:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38820LFv-simsuccessno0:09:25
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38819LFv-simsuccessno0:09:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible