Duckietown Challenges Home Challenges Submissions

Submission 11106

Submission11106
Competingyes
Challengeaido5-LF-sim-validation
UserMelisande Teng
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 56556
Next
User labelexercise_ros_template
Admin priority50
Blessingn/a
User priority50

56556

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
56556LFv-simsuccessyes0:31:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median4.546502562990541
survival_time_median55.54999999999898
deviation-center-line_median2.1972679635793915
in-drivable-lane_median12.849999999999683


other stats
agent_compute-ego0_max0.016763237497367627
agent_compute-ego0_mean0.013562868429138885
agent_compute-ego0_median0.012542479720714437
agent_compute-ego0_min0.01240327677775904
complete-iteration_max0.2025331606234807
complete-iteration_mean0.1771413094238185
complete-iteration_median0.17768121294184447
complete-iteration_min0.15066965118810444
deviation-center-line_max2.5243006193318394
deviation-center-line_mean2.190395879323039
deviation-center-line_min1.8427469708015325
deviation-heading_max12.152295661293923
deviation-heading_mean10.623883015592613
deviation-heading_median11.062949798302975
deviation-heading_min8.217336804470564
driven_any_max10.694567878648169
driven_any_mean8.907837136275802
driven_any_median9.521065489094092
driven_any_min5.8946496882668535
driven_lanedir_consec_max6.204393849741938
driven_lanedir_consec_mean4.519544141409851
driven_lanedir_consec_min2.7807775899163865
driven_lanedir_max8.249970413161623
driven_lanedir_mean5.794611339096432
driven_lanedir_median5.8560072077458685
driven_lanedir_min3.216460527732369
get_duckie_state_max1.3592439726131543e-06
get_duckie_state_mean1.2373633507996737e-06
get_duckie_state_median1.2094646100147743e-06
get_duckie_state_min1.1712802105559906e-06
get_robot_state_max0.0036192124738304255
get_robot_state_mean0.0035290439494151315
get_robot_state_median0.003543282928848217
get_robot_state_min0.003410397466133665
get_state_dump_max0.004640188741247223
get_state_dump_mean0.0044453973405098515
get_state_dump_median0.0044355887810449855
get_state_dump_min0.00427022305870221
get_ui_image_max0.03361578443503983
get_ui_image_mean0.02904297932025516
get_ui_image_median0.02941574042494954
get_ui_image_min0.023724651996081723
in-drivable-lane_max28.39999999999889
in-drivable-lane_mean16.149999999999494
in-drivable-lane_min10.499999999999712
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 10.694567878648169, "get_ui_image": 0.02780056773971062, "step_physics": 0.1032984141604688, "survival_time": 59.99999999999873, "driven_lanedir": 8.249970413161623, "get_state_dump": 0.004640188741247223, "get_robot_state": 0.0036138888699724513, "sim_render-ego0": 0.0038346527617340977, "get_duckie_state": 1.3592439726131543e-06, "in-drivable-lane": 11.699999999999434, "deviation-heading": 11.792662406416047, "agent_compute-ego0": 0.012518480159559416, "complete-iteration": 0.1702340016456369, "set_robot_commands": 0.0021439035369593537, "deviation-center-line": 2.5243006193318394, "driven_lanedir_consec": 6.204393849741938, "sim_compute_sim_state": 0.01034130760275454, "sim_compute_performance-ego0": 0.001956734629495257}, "LF-norm-zigzag-000-ego0": {"driven_any": 8.905632927351675, "get_ui_image": 0.03361578443503983, "step_physics": 0.12980701952418547, "survival_time": 53.29999999999911, "driven_lanedir": 3.6918703360983223, "get_state_dump": 0.004483620474689344, "get_robot_state": 0.0034726769877239825, "sim_render-ego0": 0.003724563646897492, "get_duckie_state": 1.2325369093947692e-06, "in-drivable-lane": 28.39999999999889, "deviation-heading": 10.333237190189909, "agent_compute-ego0": 0.01240327677775904, "complete-iteration": 0.2025331606234807, "set_robot_commands": 0.0020518108518076702, "deviation-center-line": 1.8914638836324429, "driven_lanedir_consec": 3.140809891852626, "sim_compute_sim_state": 0.011017202995822269, "sim_compute_performance-ego0": 0.001874760812761187}, "LF-norm-techtrack-000-ego0": {"driven_any": 5.8946496882668535, "get_ui_image": 0.03103091311018846, "step_physics": 0.1144857835836719, "survival_time": 35.50000000000012, "driven_lanedir": 3.216460527732369, "get_state_dump": 0.004387557087400627, "get_robot_state": 0.0036192124738304255, "sim_render-ego0": 0.003913555131850531, "get_duckie_state": 1.1863923106347794e-06, "in-drivable-lane": 13.999999999999932, "deviation-heading": 8.217336804470564, "agent_compute-ego0": 0.012566479281869452, "complete-iteration": 0.18512842423805204, "set_robot_commands": 0.0021964172438394837, "deviation-center-line": 1.8427469708015325, "driven_lanedir_consec": 2.7807775899163865, "sim_compute_sim_state": 0.01086243362534063, "sim_compute_performance-ego0": 0.0019832438725124095}, "LF-norm-small_loop-000-ego0": {"driven_any": 10.136498050836511, "get_ui_image": 0.023724651996081723, "step_physics": 0.08918701788425033, "survival_time": 57.79999999999885, "driven_lanedir": 8.020144079393415, "get_state_dump": 0.00427022305870221, "get_robot_state": 0.003410397466133665, "sim_render-ego0": 0.0036113472364025065, "get_duckie_state": 1.1712802105559906e-06, "in-drivable-lane": 10.499999999999712, "deviation-heading": 12.152295661293923, "agent_compute-ego0": 0.016763237497367627, "complete-iteration": 0.15066965118810444, "set_robot_commands": 0.002033582924766178, "deviation-center-line": 2.5030720435263407, "driven_lanedir_consec": 5.952195234128455, "sim_compute_sim_state": 0.005771890630367612, "sim_compute_performance-ego0": 0.0018194974653661096}}
set_robot_commands_max0.0021964172438394837
set_robot_commands_mean0.0021064286393431713
set_robot_commands_median0.0020978571943835117
set_robot_commands_min0.002033582924766178
sim_compute_performance-ego0_max0.0019832438725124095
sim_compute_performance-ego0_mean0.0019085591950337409
sim_compute_performance-ego0_median0.001915747721128222
sim_compute_performance-ego0_min0.0018194974653661096
sim_compute_sim_state_max0.011017202995822269
sim_compute_sim_state_mean0.009498208713571262
sim_compute_sim_state_median0.010601870614047585
sim_compute_sim_state_min0.005771890630367612
sim_render-ego0_max0.003913555131850531
sim_render-ego0_mean0.0037710296942211567
sim_render-ego0_median0.0037796082043157946
sim_render-ego0_min0.0036113472364025065
simulation-passed1
step_physics_max0.12980701952418547
step_physics_mean0.10919455878814412
step_physics_median0.10889209887207035
step_physics_min0.08918701788425033
survival_time_max59.99999999999873
survival_time_mean51.6499999999992
survival_time_min35.50000000000012
No reset possible
56546LFv-simsuccessyes0:23:45
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
56531LFv-simsuccessyes0:23:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50648LFv-simerrorno0:08:11
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
    im = Image.open(filename)
  File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
    raise UnidentifiedImageError(
PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
    result = block.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
    image = imread(self.config.file)
  File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
    raise ValueError(msg) from e
ValueError: Could not open filename "banner1.png".

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 329, in main
    make_video2(
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_video.py", line 149, in make_video2
    pg("video_aido", params)
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 280, in pg
    raise e
  File "/usr/local/lib/python3.8/site-packages/procgraph/scripts/pgmain.py", line 277, in pg
    model.update()
  File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 321, in update
    raise BadMethodCall("update", block, traceback.format_exc())
procgraph.core.exceptions.BadMethodCall: User-thrown exception while calling update() in block 'static_image'.
- B:StaticImage:static_image(in:/;out:rgb) 139939591167472
- M:video_aido:cmdline(in:/;out:/) 139939906659664
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 51, in imread
>     im = Image.open(filename)
>   File "/usr/local/lib/python3.8/site-packages/PIL/Image.py", line 2943, in open
>     raise UnidentifiedImageError(
> PIL.UnidentifiedImageError: cannot identify image file 'banner1.png'
> 
> The above exception was the direct cause of the following exception:
> 
> Traceback (most recent call last):
>   File "/usr/local/lib/python3.8/site-packages/procgraph/core/model.py", line 316, in update
>     result = block.update()
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 31, in update
>     image = imread(self.config.file)
>   File "/usr/local/lib/python3.8/site-packages/procgraph_pil/imread_imp.py", line 54, in imread
>     raise ValueError(msg) from e
> ValueError: Could not open filename "banner1.png".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
50641LFv-simhost-errorno0:05:11
The container "evalu [...]
The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40236LFv-simsuccessno0:09:22
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40233LFv-simsuccessno0:09:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40230LFv-simsuccessno0:09:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40227LFv-simsuccessno0:09:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40222LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40220LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40219LFv-simsuccessno0:08:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40218LFv-simsuccessno0:08:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40214LFv-simhost-errorno0:00:09
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40213LFv-simhost-errorno0:00:09
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40212LFv-simsuccessno0:08:48
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40211LFv-simsuccessno0:08:41
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40208LFv-simhost-errorno0:00:12
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40207LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40206LFv-simhost-errorno0:00:11
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 687, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 888, in run_single
    write_logs(wd, project, services=config["services"])
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1388, in write_logs
    services2id: Dict[str, str] = get_services_id(wd, project, services)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 937, in get_services_id
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/models/containers.py", line 880, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.8/dist-packages/docker/utils/decorators.py", line 16, in wrapped
    raise errors.NullResource(
docker.errors.NullResource: Resource ID was not provided
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
40205LFv-simsuccessno0:08:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
39638LFv-simsuccessno0:09:15
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38922LFv-simsuccessno0:09:29
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
38921LFv-simsuccessno0:09:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible