Duckietown Challenges Home Challenges Submissions

Submission 11114

Submission11114
Competingyes
Challengeaido5-LF-sim-validation
UserBhavya Patwa 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 56492
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

56492

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
56492LFv-simsuccessyes0:17:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.282318518203379
survival_time_median23.7000000000002
deviation-center-line_median1.063479027860971
in-drivable-lane_median6.700000000000063


other stats
agent_compute-ego0_max0.012218195910847517
agent_compute-ego0_mean0.011934147177532883
agent_compute-ego0_median0.012094533545381463
agent_compute-ego0_min0.011329325708521095
complete-iteration_max0.19664465560825592
complete-iteration_mean0.16986330472230135
complete-iteration_median0.16865663819591475
complete-iteration_min0.14549528688911992
deviation-center-line_max4.320604688594504
deviation-center-line_mean1.8265261616100203
deviation-center-line_min0.8585419021236356
deviation-heading_max8.629579143252165
deviation-heading_mean5.208578688770153
deviation-heading_median4.8040338359675125
deviation-heading_min2.5966679398934205
driven_any_max9.006198951387148
driven_any_mean4.49490880669607
driven_any_median3.283383436139678
driven_any_min2.4066694031177738
driven_lanedir_consec_max8.368585003662405
driven_lanedir_consec_mean3.5993375351356187
driven_lanedir_consec_min1.464128100473311
driven_lanedir_max8.368585003662405
driven_lanedir_mean3.5993375351356187
driven_lanedir_median2.282318518203379
driven_lanedir_min1.464128100473311
get_duckie_state_max1.4574176465456015e-06
get_duckie_state_mean1.303861160285415e-06
get_duckie_state_median1.2919003018834825e-06
get_duckie_state_min1.1742263908290942e-06
get_robot_state_max0.00353273029985099
get_robot_state_mean0.003492395715743441
get_robot_state_median0.003506151194174645
get_robot_state_min0.003424550174773484
get_state_dump_max0.004619445920364236
get_state_dump_mean0.00439533784005962
get_state_dump_median0.004406835154702687
get_state_dump_min0.004148235130468872
get_ui_image_max0.03326116406589473
get_ui_image_mean0.0282171880688531
get_ui_image_median0.027850925695558366
get_ui_image_min0.023905736818400945
in-drivable-lane_max11.800000000000136
in-drivable-lane_mean6.975000000000076
in-drivable-lane_min2.7000000000000384
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 2.4066694031177738, "get_ui_image": 0.025861834283906465, "step_physics": 0.09564605476714227, "survival_time": 15.900000000000093, "driven_lanedir": 2.1441137444821714, "get_state_dump": 0.004619445920364236, "get_robot_state": 0.00353273029985099, "sim_render-ego0": 0.0036737567578737263, "get_duckie_state": 1.4574176465456015e-06, "in-drivable-lane": 2.7000000000000384, "deviation-heading": 2.5966679398934205, "agent_compute-ego0": 0.012164476149508198, "complete-iteration": 0.15826410410172514, "set_robot_commands": 0.0020725383280212977, "deviation-center-line": 0.8585419021236356, "driven_lanedir_consec": 2.1441137444821714, "sim_compute_sim_state": 0.00875159490818514, "sim_compute_performance-ego0": 0.0018596933179514535}, "LF-norm-zigzag-000-ego0": {"driven_any": 3.159281056858016, "get_ui_image": 0.03326116406589473, "step_physics": 0.12458143485795468, "survival_time": 21.750000000000178, "driven_lanedir": 1.464128100473311, "get_state_dump": 0.004337204705684557, "get_robot_state": 0.00352977229914534, "sim_render-ego0": 0.0036995569500354454, "get_duckie_state": 1.3490335656962264e-06, "in-drivable-lane": 11.800000000000136, "deviation-heading": 3.4953313596182545, "agent_compute-ego0": 0.012218195910847517, "complete-iteration": 0.19664465560825592, "set_robot_commands": 0.0020262773977507147, "deviation-center-line": 0.9479509548569937, "driven_lanedir_consec": 1.464128100473311, "sim_compute_sim_state": 0.010958682506456287, "sim_compute_performance-ego0": 0.0019468965880367736}, "LF-norm-techtrack-000-ego0": {"driven_any": 3.407485815421339, "get_ui_image": 0.029840017107210272, "step_physics": 0.111742826751234, "survival_time": 25.65000000000023, "driven_lanedir": 2.4205232919245865, "get_state_dump": 0.004476465603720817, "get_robot_state": 0.0034825300892039495, "sim_render-ego0": 0.003669625126434207, "get_duckie_state": 1.234767038070738e-06, "in-drivable-lane": 8.100000000000115, "deviation-heading": 6.1127363123167715, "agent_compute-ego0": 0.01202459094125473, "complete-iteration": 0.17904917229010436, "set_robot_commands": 0.001989488471806745, "deviation-center-line": 1.179007100864948, "driven_lanedir_consec": 2.4205232919245865, "sim_compute_sim_state": 0.009882846694975977, "sim_compute_performance-ego0": 0.0018600174425177072}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.006198951387148, "get_ui_image": 0.023905736818400945, "step_physics": 0.08954477766769117, "survival_time": 59.99999999999873, "driven_lanedir": 8.368585003662405, "get_state_dump": 0.004148235130468872, "get_robot_state": 0.003424550174773484, "sim_render-ego0": 0.00357884451511202, "get_duckie_state": 1.1742263908290942e-06, "in-drivable-lane": 5.300000000000011, "deviation-heading": 8.629579143252165, "agent_compute-ego0": 0.011329325708521095, "complete-iteration": 0.14549528688911992, "set_robot_commands": 0.0020333759790653987, "deviation-center-line": 4.320604688594504, "driven_lanedir_consec": 8.368585003662405, "sim_compute_sim_state": 0.005597103446052036, "sim_compute_performance-ego0": 0.001858076187692812}}
set_robot_commands_max0.0020725383280212977
set_robot_commands_mean0.0020304200441610388
set_robot_commands_median0.0020298266884080565
set_robot_commands_min0.001989488471806745
sim_compute_performance-ego0_max0.0019468965880367736
sim_compute_performance-ego0_mean0.001881170884049687
sim_compute_performance-ego0_median0.0018598553802345805
sim_compute_performance-ego0_min0.001858076187692812
sim_compute_sim_state_max0.010958682506456287
sim_compute_sim_state_mean0.00879755688891736
sim_compute_sim_state_median0.009317220801580556
sim_compute_sim_state_min0.005597103446052036
sim_render-ego0_max0.0036995569500354454
sim_render-ego0_mean0.00365544583736385
sim_render-ego0_median0.003671690942153967
sim_render-ego0_min0.00357884451511202
simulation-passed1
step_physics_max0.12458143485795468
step_physics_mean0.10537877351100552
step_physics_median0.10369444075918814
step_physics_min0.08954477766769117
survival_time_max59.99999999999873
survival_time_mean30.824999999999804
survival_time_min15.900000000000093
No reset possible
50529LFv-simerrorno0:10:14
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140283534282816
- M:video_aido:cmdline(in:/;out:/) 140283534286224
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50511LFv-simhost-errorno0:02:50
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40197LFv-simfailedno0:08:23
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 823 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39630LFv-simfailedno0:10:15
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 540, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 854 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 678, in scoring_context
    yield cie
  File "experiment_manager.py", line 683, in go
    wrap(cie)
  File "experiment_manager.py", line 668, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 281, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 544, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38937LFv-simsuccessno0:09:39
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible