Duckietown Challenges Home Challenges Submissions

Submission 10058

Submission10058
Competingyes
Challengeaido5-LF-sim-validation
UserJean-Sébastien Grondin 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 58023
Next
User labelreal-exercise-1
Admin priority50
Blessingn/a
User priority50

58023

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
58023LFv-simsuccessyes0:26:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.7276566033342902
survival_time_median36.32499999999993
deviation-center-line_median0.8462482650207317
in-drivable-lane_median19.84999999999956


other stats
agent_compute-ego0_max0.01410690373829738
agent_compute-ego0_mean0.013377033974802558
agent_compute-ego0_median0.013385355485182285
agent_compute-ego0_min0.012630521190548282
complete-iteration_max0.2227446433407977
complete-iteration_mean0.1969612986498993
complete-iteration_median0.2013089789256946
complete-iteration_min0.1624825934074103
deviation-center-line_max3.049728121184737
deviation-center-line_mean1.3539402131501863
deviation-center-line_min0.6735362013745455
deviation-heading_max12.873643416971245
deviation-heading_mean6.707299171828435
deviation-heading_median5.060377715188015
deviation-heading_min3.834797839966464
driven_any_max9.03027665616199
driven_any_mean5.8295396567515905
driven_any_median5.266483608301326
driven_any_min3.7549147542417223
driven_lanedir_consec_max5.420236513178551
driven_lanedir_consec_mean2.4698976208664867
driven_lanedir_consec_min1.004040763618815
driven_lanedir_max5.420236513178551
driven_lanedir_mean2.4698976208664867
driven_lanedir_median1.7276566033342902
driven_lanedir_min1.004040763618815
get_duckie_state_max2.17059691836025e-06
get_duckie_state_mean2.107243096817547e-06
get_duckie_state_median2.151968522781571e-06
get_duckie_state_min1.9544384233467944e-06
get_robot_state_max0.004043545728301148
get_robot_state_mean0.003839069494646312
get_robot_state_median0.00381666112796737
get_robot_state_min0.003679409994349361
get_state_dump_max0.004878063618402598
get_state_dump_mean0.004736204890811235
get_state_dump_median0.004760092880437279
get_state_dump_min0.00454657018396778
get_ui_image_max0.03856261286707742
get_ui_image_mean0.03252661114474234
get_ui_image_median0.033053034703667376
get_ui_image_min0.025437762304557177
in-drivable-lane_max32.34999999999952
in-drivable-lane_mean21.69999999999971
in-drivable-lane_min14.75000000000021
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 4.282528832065514, "get_ui_image": 0.03055095313408056, "step_physics": 0.12142356595975082, "survival_time": 26.50000000000024, "driven_lanedir": 1.903189462261989, "get_state_dump": 0.004820076534752567, "get_robot_state": 0.003843855274150152, "sim_render-ego0": 0.004075309875545753, "get_duckie_state": 2.144869213499367e-06, "in-drivable-lane": 14.75000000000021, "deviation-heading": 3.834797839966464, "agent_compute-ego0": 0.013248734555001989, "complete-iteration": 0.19325675174116863, "set_robot_commands": 0.002350910459759096, "deviation-center-line": 0.7833783883966013, "driven_lanedir_consec": 1.903189462261989, "sim_compute_sim_state": 0.010626800989700575, "sim_compute_performance-ego0": 0.0022255103691374064}, "LF-norm-zigzag-000-ego0": {"driven_any": 9.03027665616199, "get_ui_image": 0.03856261286707742, "step_physics": 0.1403978267974599, "survival_time": 59.99999999999873, "driven_lanedir": 5.420236513178551, "get_state_dump": 0.00470010922612199, "get_robot_state": 0.003789466981784589, "sim_render-ego0": 0.0040770597402301064, "get_duckie_state": 2.159067832063775e-06, "in-drivable-lane": 20.899999999998883, "deviation-heading": 12.873643416971245, "agent_compute-ego0": 0.013521976415362584, "complete-iteration": 0.2227446433407977, "set_robot_commands": 0.0022820574754878545, "deviation-center-line": 3.049728121184737, "driven_lanedir_consec": 5.420236513178551, "sim_compute_sim_state": 0.01318679642022202, "sim_compute_performance-ego0": 0.0021325132431932333}, "LF-norm-techtrack-000-ego0": {"driven_any": 6.250438384537137, "get_ui_image": 0.03555511627325419, "step_physics": 0.12992723017497132, "survival_time": 44.5999999999996, "driven_lanedir": 1.5521237444065914, "get_state_dump": 0.004878063618402598, "get_robot_state": 0.004043545728301148, "sim_render-ego0": 0.0042934086661856755, "get_duckie_state": 2.17059691836025e-06, "in-drivable-lane": 32.34999999999952, "deviation-heading": 4.827984245864417, "agent_compute-ego0": 0.01410690373829738, "complete-iteration": 0.2093612061102206, "set_robot_commands": 0.002445726084949455, "deviation-center-line": 0.9091181416448622, "driven_lanedir_consec": 1.5521237444065914, "sim_compute_sim_state": 0.011792822621036885, "sim_compute_performance-ego0": 0.0022229499539228995}, "LF-norm-small_loop-000-ego0": {"driven_any": 3.7549147542417223, "get_ui_image": 0.025437762304557177, "step_physics": 0.10194824853401592, "survival_time": 28.050000000000264, "driven_lanedir": 1.004040763618815, "get_state_dump": 0.00454657018396778, "get_robot_state": 0.003679409994349361, "sim_render-ego0": 0.003778533154959356, "get_duckie_state": 1.9544384233467944e-06, "in-drivable-lane": 18.80000000000024, "deviation-heading": 5.292771184511614, "agent_compute-ego0": 0.012630521190548282, "complete-iteration": 0.1624825934074103, "set_robot_commands": 0.0021108354962169062, "deviation-center-line": 0.6735362013745455, "driven_lanedir_consec": 1.004040763618815, "sim_compute_sim_state": 0.006326475601603552, "sim_compute_performance-ego0": 0.001938209839138696}}
set_robot_commands_max0.002445726084949455
set_robot_commands_mean0.002297382379103328
set_robot_commands_median0.0023164839676234755
set_robot_commands_min0.0021108354962169062
sim_compute_performance-ego0_max0.0022255103691374064
sim_compute_performance-ego0_mean0.0021297958513480588
sim_compute_performance-ego0_median0.0021777315985580664
sim_compute_performance-ego0_min0.001938209839138696
sim_compute_sim_state_max0.01318679642022202
sim_compute_sim_state_mean0.010483223908140755
sim_compute_sim_state_median0.01120981180536873
sim_compute_sim_state_min0.006326475601603552
sim_render-ego0_max0.0042934086661856755
sim_render-ego0_mean0.0040560778592302225
sim_render-ego0_median0.00407618480788793
sim_render-ego0_min0.003778533154959356
simulation-passed1
step_physics_max0.1403978267974599
step_physics_mean0.12342421786654947
step_physics_median0.12567539806736103
step_physics_min0.10194824853401592
survival_time_max59.99999999999873
survival_time_mean39.78749999999971
survival_time_min26.50000000000024
No reset possible
58019LFv-simhost-erroryes0:01:56
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 53, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 777, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 991, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 120, in write_logs
    services2id: Dict[ServiceName, ContainerID] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/docker_compose.py", line 63, in get_services_id
    raise DockerComposeFail(msg, output=output.decode(), names=names) from e
duckietown_challenges_runner.docker_compose.DockerComposeFail: Cannot get process ids
│ output: ''
│  names: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51947LFv-simerrorno0:03:49
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 140307771509104
- M:video_aido:cmdline(in:/;out:/) 140307771509392
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
51933LFv-simhost-errorno0:07:39
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
41520LFv-simsuccessno0:07:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
37179LFv-simsuccessno0:09:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36587LFv-simabortedno0:08:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
36586LFv-simabortedno0:10:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible