Duckietown Challenges Home Challenges Submissions

Submission 11752

Submission11752
Competingyes
Challengeaido5-LF-sim-validation
UserOlga (Ge Ya) Xu 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 53835
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

53835

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
53835LFv-simsuccessyes0:23:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9529201434024994
survival_time_median44.09999999999963
deviation-center-line_median0.3622815861098768
in-drivable-lane_median39.07499999999946


other stats
agent_compute-ego0_max0.013075901911808897
agent_compute-ego0_mean0.012900953260199272
agent_compute-ego0_median0.012863867872275978
agent_compute-ego0_min0.01280017538443624
complete-iteration_max0.19948571746548016
complete-iteration_mean0.18878657800403464
complete-iteration_median0.1898220171731687
complete-iteration_min0.1760165602043209
deviation-center-line_max1.9785263128001949
deviation-center-line_mean0.7083628281004295
deviation-center-line_min0.1303618273817695
deviation-heading_max2.356270208346236
deviation-heading_mean1.4353595733559632
deviation-heading_median1.436331668170631
deviation-heading_min0.5125047487363545
driven_any_max14.716379960641586
driven_any_mean9.266020275640882
driven_any_median10.24472024194428
driven_any_min1.858260658033379
driven_lanedir_consec_max4.7704446303693375
driven_lanedir_consec_mean1.83636056465997
driven_lanedir_consec_min0.6691573414655436
driven_lanedir_max4.7704446303693375
driven_lanedir_mean1.83636056465997
driven_lanedir_median0.9529201434024994
driven_lanedir_min0.6691573414655436
get_duckie_state_max1.6857909209674996e-06
get_duckie_state_mean1.6213409216650985e-06
get_duckie_state_median1.635108460915498e-06
get_duckie_state_min1.529355843861898e-06
get_robot_state_max0.0040638606641545645
get_robot_state_mean0.004015623884033263
get_robot_state_median0.004013860712830837
get_robot_state_min0.00397091344631681
get_state_dump_max0.005200993415363674
get_state_dump_mean0.005098816772995658
get_state_dump_median0.0051551940968773205
get_state_dump_min0.004883885482864316
get_ui_image_max0.036352879057327904
get_ui_image_mean0.032110638199815786
get_ui_image_median0.031760455883597806
get_ui_image_min0.028568761974739627
in-drivable-lane_max42.399999999999395
in-drivable-lane_mean31.58749999999958
in-drivable-lane_min5.7999999999999945
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 14.716379960641586, "get_ui_image": 0.02937486169737245, "step_physics": 0.10577941059966964, "survival_time": 59.99999999999873, "driven_lanedir": 4.7704446303693375, "get_state_dump": 0.004883885482864316, "get_robot_state": 0.00397091344631681, "sim_render-ego0": 0.0040189062526680645, "get_duckie_state": 1.5932951839043636e-06, "in-drivable-lane": 41.79999999999908, "deviation-heading": 2.356270208346236, "agent_compute-ego0": 0.012822107510403928, "complete-iteration": 0.1760165602043209, "set_robot_commands": 0.0023012679383518494, "deviation-center-line": 1.9785263128001949, "driven_lanedir_consec": 4.7704446303693375, "sim_compute_sim_state": 0.01053118963821246, "sim_compute_performance-ego0": 0.0022351672309919956}, "LF-norm-zigzag-000-ego0": {"driven_any": 11.0601596573647, "get_ui_image": 0.036352879057327904, "step_physics": 0.11724767585595448, "survival_time": 47.94999999999941, "driven_lanedir": 1.185532287181698, "get_state_dump": 0.00517999604344368, "get_robot_state": 0.00397435799241066, "sim_render-ego0": 0.004038471480210622, "get_duckie_state": 1.529355843861898e-06, "in-drivable-lane": 42.399999999999395, "deviation-heading": 1.8646035455277323, "agent_compute-ego0": 0.012905628234148026, "complete-iteration": 0.19948571746548016, "set_robot_commands": 0.00237425168355306, "deviation-center-line": 0.5344338459495331, "driven_lanedir_consec": 1.185532287181698, "sim_compute_sim_state": 0.015052698800961176, "sim_compute_performance-ego0": 0.0022634617984294892}, "LF-norm-techtrack-000-ego0": {"driven_any": 9.429280826523858, "get_ui_image": 0.03414605006982316, "step_physics": 0.11920854501037977, "survival_time": 40.24999999999985, "driven_lanedir": 0.7203079996233009, "get_state_dump": 0.005130392150310961, "get_robot_state": 0.004053363433251014, "sim_render-ego0": 0.00412097048227012, "get_duckie_state": 1.6857909209674996e-06, "in-drivable-lane": 36.34999999999985, "deviation-heading": 1.0080597908135298, "agent_compute-ego0": 0.013075901911808897, "complete-iteration": 0.19894234033731312, "set_robot_commands": 0.002378130964842387, "deviation-center-line": 0.19012932627022044, "driven_lanedir_consec": 0.7203079996233009, "sim_compute_sim_state": 0.014405418566379594, "sim_compute_performance-ego0": 0.0023218703920728813}, "LF-norm-small_loop-000-ego0": {"driven_any": 1.858260658033379, "get_ui_image": 0.028568761974739627, "step_physics": 0.11522054805435948, "survival_time": 8.899999999999991, "driven_lanedir": 0.6691573414655436, "get_state_dump": 0.005200993415363674, "get_robot_state": 0.0040638606641545645, "sim_render-ego0": 0.004315835803580683, "get_duckie_state": 1.6769217379266324e-06, "in-drivable-lane": 5.7999999999999945, "deviation-heading": 0.5125047487363545, "agent_compute-ego0": 0.01280017538443624, "complete-iteration": 0.18070169400902433, "set_robot_commands": 0.002412323179191717, "deviation-center-line": 0.1303618273817695, "driven_lanedir_consec": 0.6691573414655436, "sim_compute_sim_state": 0.005798455723171127, "sim_compute_performance-ego0": 0.002221770792700059}}
set_robot_commands_max0.002412323179191717
set_robot_commands_mean0.002366493441484753
set_robot_commands_median0.0023761913241977235
set_robot_commands_min0.0023012679383518494
sim_compute_performance-ego0_max0.0023218703920728813
sim_compute_performance-ego0_mean0.002260567553548606
sim_compute_performance-ego0_median0.0022493145147107424
sim_compute_performance-ego0_min0.002221770792700059
sim_compute_sim_state_max0.015052698800961176
sim_compute_sim_state_mean0.01144694068218109
sim_compute_sim_state_median0.012468304102296029
sim_compute_sim_state_min0.005798455723171127
sim_render-ego0_max0.004315835803580683
sim_render-ego0_mean0.004123546004682373
sim_render-ego0_median0.004079720981240371
sim_render-ego0_min0.0040189062526680645
simulation-passed1
step_physics_max0.11920854501037977
step_physics_mean0.11436404488009085
step_physics_median0.11623411195515698
step_physics_min0.10577941059966964
survival_time_max59.99999999999873
survival_time_mean39.274999999999494
survival_time_min8.899999999999991
No reset possible
53834LFv-simsuccessyes0:18:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
53830LFv-simsuccessyes0:19:34
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48197LFv-simerrorno0:04:36
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140249473378672
- M:video_aido:cmdline(in:/;out:/) 140249473591952
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48172LFv-simabortedno0:00:30
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1169, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 628, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
48154LFv-simabortedno0:00:36
KeyboardInterrupt: T [...]
KeyboardInterrupt:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1169, in run_one
    heartbeat()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 628, in heartbeat
    raise KeyboardInterrupt(msg_)
KeyboardInterrupt: The server told us to abort the job because: The challenge has been updated.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43823LFv-simsuccessno0:10:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43822LFv-simsuccessno0:09:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43821LFv-simhost-errorno0:00:29
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43820LFv-simsuccessno0:10:10
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43819LFv-simsuccessno0:10:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43818LFv-simtimeoutno0:00:26
Timeout because eval [...]
Timeout because evaluator contacted us
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43817LFv-simsuccessno0:10:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43816LFv-simsuccessno0:09:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
43815LFv-simsuccessno0:11:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible