Duckietown Challenges Home Challenges Submissions

Submission 13576

Submission13576
Competingyes
Challengeaido-LFVI_multi-sim-validation
UserMárton Tim 🇭🇺
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of4: 105567 sim-1of4: 105536 sim-2of4: 105555 sim-3of4: 105557
Next
User label3626
Admin priority50
Blessingn/a
User priority50

105567

Click the images to see detailed statistics about the episode.

udem-000

challenge-solution-output

105536

Click the images to see detailed statistics about the episode.

4way-000

challenge-solution-output

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
105567sim-0of4successyes0:06:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median17.30000000000011
in-drivable-lane_median0.9500000000000072
driven_lanedir_consec_median4.595679228515795
deviation-center-line_median1.2031099735550883


other stats
deviation-center-line_max2.5173242687643107
deviation-center-line_mean1.4352429772162971
deviation-center-line_min0.8174276929907005
deviation-heading_max8.488767093733564
deviation-heading_mean4.009814164763104
deviation-heading_median3.275958583175495
deviation-heading_min0.9985723989678624
distance-from-start_max3.1773726759750174
distance-from-start_mean1.9327646315689035
distance-from-start_median1.7512158119565902
distance-from-start_min1.051254226387417
driven_any_max8.029006774970583
driven_any_mean5.107181904828435
driven_any_median5.640159529581741
driven_any_min1.1194017851796758
driven_lanedir_consec_max7.326255309831163
driven_lanedir_consec_mean4.393182461452074
driven_lanedir_consec_min1.0551160789455454
driven_lanedir_max7.347159484846827
driven_lanedir_mean4.523527336196031
driven_lanedir_median4.845870171736838
driven_lanedir_min1.0552095164636184
in-drivable-lane_max7.150000000000087
in-drivable-lane_mean2.2625000000000255
in-drivable-lane_min0.0
per-episodes
details{"udem-000-ego0": {"driven_any": 5.6415363280258966, "survival_time": 17.30000000000011, "driven_lanedir": 5.261195007662264, "in-drivable-lane": 0.25000000000000355, "deviation-heading": 8.488767093733564, "distance-from-start": 3.1773726759750174, "deviation-center-line": 1.5502736328209956, "driven_lanedir_consec": 5.182633467661285}, "udem-000-ego1": {"driven_any": 1.1194017851796758, "survival_time": 17.30000000000011, "driven_lanedir": 1.0552095164636184, "in-drivable-lane": 0.0, "deviation-heading": 0.9985723989678624, "distance-from-start": 1.051254226387417, "deviation-center-line": 2.5173242687643107, "driven_lanedir_consec": 1.0551160789455454}, "udem-000-ego2": {"driven_any": 8.029006774970583, "survival_time": 17.30000000000011, "driven_lanedir": 7.347159484846827, "in-drivable-lane": 1.6500000000000106, "deviation-heading": 3.1939858228122673, "distance-from-start": 1.8084739415920228, "deviation-center-line": 0.8174276929907005, "driven_lanedir_consec": 7.326255309831163}, "udem-000-ego3": {"driven_any": 5.638782731137585, "survival_time": 17.30000000000011, "driven_lanedir": 4.430545335811413, "in-drivable-lane": 7.150000000000087, "deviation-heading": 3.357931343538723, "distance-from-start": 1.6939576823211575, "deviation-center-line": 0.8559463142891816, "driven_lanedir_consec": 4.008724989370305}}
simulation-passed1
survival_time_max17.30000000000011
survival_time_mean17.30000000000011
survival_time_min17.30000000000011
No reset possible
105557sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105555sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
skipped1
No reset possible
105551sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105550sim-3of4successyes0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105547sim-0of4successyes0:05:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105543sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105542sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105541sim-0of4host-erroryes0:01:21
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 338, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego3" aborted with the following error:

error in ego3 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(0, experiment_idx=0, checkpoint_idx=0, logger=context)
              ||   File "/submission/model.py", line 42, in __init__
              ||     dummy_env = wrap_env(config["env_config"], extra_config={
              ||   File "/submission/duckietown_utils/env.py", line 46, in wrap_env
              ||     env = SegmentObsWrapper(env, model=extra_config['model'])
              ||   File "/submission/duckietown_utils/wrappers/SegmentObsWrapper.py", line 43, in __init__
              ||     self.model.cuda()
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 306, in main
    raise InvalidEnvironment(msg) from e
duckietown_challenges.exceptions.InvalidEnvironment: Detected out of CUDA memory:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 298, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 156, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 338, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego3" aborted with the following error:

error in ego3 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(0, experiment_idx=0, checkpoint_idx=0, logger=context)
              ||   File "/submission/model.py", line 42, in __init__
              ||     dummy_env = wrap_env(config["env_config"], extra_config={
              ||   File "/submission/duckietown_utils/env.py", line 46, in wrap_env
              ||     env = SegmentObsWrapper(env, model=extra_config['model'])
              ||   File "/submission/duckietown_utils/wrappers/SegmentObsWrapper.py", line 43, in __init__
              ||     self.model.cuda()
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in cuda
              ||     return self._apply(lambda t: t.cuda(device))
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 359, in _apply
              ||     module._apply(fn)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 381, in _apply
              ||     param_applied = fn(param)
              ||   File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 463, in <lambda>
              ||     return self._apply(lambda t: t.cuda(device))
              || RuntimeError: CUDA error: out of memory
              ||
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105540sim-3of4successyes0:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105539sim-2of4successyes0:01:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
105536sim-1of4successyes0:03:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median4.749999999999991
in-drivable-lane_median0.37499999999999867
driven_lanedir_consec_median1.3461137381912445
deviation-center-line_median0.3100673308427664


other stats
deviation-center-line_max0.36581936795974845
deviation-center-line_mean0.31013941571096204
deviation-center-line_min0.2546036331985669
deviation-heading_max2.217872295390038
deviation-heading_mean1.4285781618671891
deviation-heading_median1.3977234883105054
deviation-heading_min0.700993375457709
distance-from-start_max1.633203569285496
distance-from-start_mean1.3357370528540304
distance-from-start_median1.35432001222902
distance-from-start_min1.0011046176725855
driven_any_max1.8717790822321103
driven_any_mean1.777766659112212
driven_any_median1.7995605112294657
driven_any_min1.6401665317578062
driven_lanedir_consec_max1.807055301393011
driven_lanedir_consec_mean1.3451175071332344
driven_lanedir_consec_min0.8811872507574376
driven_lanedir_max1.8507604770965893
driven_lanedir_mean1.430103991090534
driven_lanedir_median1.4821209349845224
driven_lanedir_min0.9054136172965016
in-drivable-lane_max1.2999999999999956
in-drivable-lane_mean0.5124999999999982
in-drivable-lane_min0.0
per-episodes
details{"4way-000-ego0": {"driven_any": 1.841767784048266, "survival_time": 4.749999999999991, "driven_lanedir": 1.8070553214161336, "in-drivable-lane": 0.0, "deviation-heading": 0.9342163633862156, "distance-from-start": 1.3036912861820276, "deviation-center-line": 0.2546036331985669, "driven_lanedir_consec": 1.807055301393011}, "4way-000-ego1": {"driven_any": 1.7573532384106654, "survival_time": 4.749999999999991, "driven_lanedir": 1.1571865485529107, "in-drivable-lane": 0.7499999999999973, "deviation-heading": 2.217872295390038, "distance-from-start": 1.633203569285496, "deviation-center-line": 0.36581936795974845, "driven_lanedir_consec": 1.1421372937613687}, "4way-000-ego2": {"driven_any": 1.8717790822321103, "survival_time": 4.749999999999991, "driven_lanedir": 1.8507604770965893, "in-drivable-lane": 0.0, "deviation-heading": 0.700993375457709, "distance-from-start": 1.4049487382760126, "deviation-center-line": 0.29416794940876584, "driven_lanedir_consec": 1.55009018262112}, "4way-000-ego3": {"driven_any": 1.6401665317578062, "survival_time": 4.749999999999991, "driven_lanedir": 0.9054136172965016, "in-drivable-lane": 1.2999999999999956, "deviation-heading": 1.861230613234795, "distance-from-start": 1.0011046176725855, "deviation-center-line": 0.3259667122767669, "driven_lanedir_consec": 0.8811872507574376}}
simulation-passed1
survival_time_max4.749999999999991
survival_time_mean4.749999999999991
survival_time_min4.749999999999991
No reset possible
105524sim-1of4successyes0:05:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible