Evaluator 5099

Job ID

submission

user

user label

challenge

step

status

up to date

evaluator

date started

date completed

duration

message

66351

14119

Andrea Censi 🇨🇭

exercises_braitenberg

mooc-BV1

417

failed

yes

gpu-production-spot-2-01

2021-04-10 11:59:13+00:00

2021-04-10 12:01:45+00:00

0:02:32

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 581, in run_episode
    agent_ci.write_topic_and_expect_zero("observations", obs_plus)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 331, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego0" aborted:

error in ego0 |Exception while handling a message on topic "observations".
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "/agent/solution/agent.py", line 65, in on_received_observations
              ||     context.info("received first observations", data=data)
              || TypeError: info() got an unexpected keyword argument 'data'
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 355, in loop
              ||     handle_message_node(parsed, receiver0, context0)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 531, in handle_message_node
              ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 24, in call_if_fun_exists
              ||     raise ZTypeError(msg, f=f, args=kwargs, argspec=a) from e
              || zuper_commons.types.exceptions.ZTypeError: Cannot call function <bound method BraitenbergAgent.on_received_observations of <__main__.BraitenbergAgent object at 0x7fbbc40cc880>>.
              || │       f: <bound method BraitenbergAgent.on_received_observations of <__main__.BraitenbergAgent object at 0x7fbbc40cc880>>
              || │    args: dict[2]
              || │          │ data:
              || │          │ DB20Observations
              || │          │ │ camera: JPGImage(jpg_data=58263 bytes b'\xff\xd8\xff\xe0\x00\x10JFIF')
              || │          │ │ odometry:
              || │          │ │ DB20Odometry
              || │          │ │ │ resolution_rad: 0.046542113386515455
              || │          │ │ │ axis_left_rad: 0.0
              || │          │ │ │ axis_right_rad: 0.0
              || │          │ context: <zuper_nodes_wrapper.wrapper.ConcreteContext object at 0x7fbbb1b0e7f0>
              || │ argspec: <class 'inspect.FullArgSpec'>[7]
              || │          #0 [self, context, data]
              || │          #1 None
              || │          #2 None
              || │          #3 None
              || │          #4 []
              || │          #5 None
              || │          #6 dict[2]
              || │             │ context: <class 'zuper_nodes_wrapper.interface.Context'>
              || │             │ data:
              || │             │ dataclass aido_schemas.schemas.DB20Observations
              || │             │  field   camera : dataclass aido_schemas.protocol_simulator.JPGImage
              || │             │                    field jpg_data : bytes
              || │             │                           __doc__
              || │             │                                             An image in JPG format.
              || │             │
              || │             │                                             jpg_data
              || │             │  field odometry : dataclass aido_schemas.schemas.DB20Odometry
              || │             │                    field resolution_rad : float
              || │             │                    field  axis_left_rad : float
              || │             │                    field axis_right_rad : float
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 312, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 593, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

66340

14115

Ashish Singh

template-random

aido-hello-sim-validation

370

success

gpu-production-spot-2-01

2021-04-09 17:53:19+00:00

2021-04-09 17:55:35+00:00

0:02:16

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	2.1500000000000004
in-drivable-lane_median	1.15
driven_lanedir_consec_median	0.2357331065993169
deviation-center-line_median	0.05863921966719952

other stats

agent_compute-ego0_max	0.012226987968791616
agent_compute-ego0_mean	0.012226987968791616
agent_compute-ego0_median	0.012226987968791616
agent_compute-ego0_min	0.012226987968791616
complete-iteration_max	0.13426973602988504
complete-iteration_mean	0.13426973602988504
complete-iteration_median	0.13426973602988504
complete-iteration_min	0.13426973602988504
deviation-center-line_max	0.05863921966719952
deviation-center-line_mean	0.05863921966719952
deviation-center-line_min	0.05863921966719952
deviation-heading_max	0.46108604520141144
deviation-heading_mean	0.46108604520141144
deviation-heading_median	0.46108604520141144
deviation-heading_min	0.46108604520141144
driven_any_max	0.4603380961060899
driven_any_mean	0.4603380961060899
driven_any_median	0.4603380961060899
driven_any_min	0.4603380961060899
driven_lanedir_consec_max	0.2357331065993169
driven_lanedir_consec_mean	0.2357331065993169
driven_lanedir_consec_min	0.2357331065993169
driven_lanedir_max	0.2357331065993169
driven_lanedir_mean	0.2357331065993169
driven_lanedir_median	0.2357331065993169
driven_lanedir_min	0.2357331065993169
get_duckie_state_max	0.004268131472847678
get_duckie_state_mean	0.004268131472847678
get_duckie_state_median	0.004268131472847678
get_duckie_state_min	0.004268131472847678
get_robot_state_max	0.003648167306726629
get_robot_state_mean	0.003648167306726629
get_robot_state_median	0.003648167306726629
get_robot_state_min	0.003648167306726629
get_state_dump_max	0.0054606416008689184
get_state_dump_mean	0.0054606416008689184
get_state_dump_median	0.0054606416008689184
get_state_dump_min	0.0054606416008689184
get_ui_image_max	0.026631496169350365
get_ui_image_mean	0.026631496169350365
get_ui_image_median	0.026631496169350365
get_ui_image_min	0.026631496169350365
in-drivable-lane_max	1.15
in-drivable-lane_mean	1.15
in-drivable-lane_min	1.15
per-episodes	details {"hello-norm-small_loop-000-ego0": {"driven_any": 0.4603380961060899, "get_ui_image": 0.026631496169350365, "step_physics": 0.06825311617417769, "survival_time": 2.1500000000000004, "driven_lanedir": 0.2357331065993169, "get_state_dump": 0.0054606416008689184, "get_robot_state": 0.003648167306726629, "sim_render-ego0": 0.0039868300611322575, "get_duckie_state": 0.004268131472847678, "in-drivable-lane": 1.15, "deviation-heading": 0.46108604520141144, "agent_compute-ego0": 0.012226987968791616, "complete-iteration": 0.13426973602988504, "set_robot_commands": 0.002363963560624556, "deviation-center-line": 0.05863921966719952, "driven_lanedir_consec": 0.2357331065993169, "sim_compute_sim_state": 0.005127402869137851, "sim_compute_performance-ego0": 0.002204927531155673}}
set_robot_commands_max	0.002363963560624556
set_robot_commands_mean	0.002363963560624556
set_robot_commands_median	0.002363963560624556
set_robot_commands_min	0.002363963560624556
sim_compute_performance-ego0_max	0.002204927531155673
sim_compute_performance-ego0_mean	0.002204927531155673
sim_compute_performance-ego0_median	0.002204927531155673
sim_compute_performance-ego0_min	0.002204927531155673
sim_compute_sim_state_max	0.005127402869137851
sim_compute_sim_state_mean	0.005127402869137851
sim_compute_sim_state_median	0.005127402869137851
sim_compute_sim_state_min	0.005127402869137851
sim_render-ego0_max	0.0039868300611322575
sim_render-ego0_mean	0.0039868300611322575
sim_render-ego0_median	0.0039868300611322575
sim_render-ego0_min	0.0039868300611322575
simulation-passed	1
step_physics_max	0.06825311617417769
step_physics_mean	0.06825311617417769
step_physics_median	0.06825311617417769
step_physics_min	0.06825311617417769
survival_time_max	2.1500000000000004
survival_time_mean	2.1500000000000004
survival_time_min	2.1500000000000004

No reset possible

66328

13798

Nicholas Kostelnik

template-random

aido-hello-sim-validation

370

aborted

gpu-production-spot-2-01

2021-04-09 15:06:02+00:00

2021-04-09 15:06:25+00:00

0:00:23

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 261, in _raise_for_status
    response.raise_for_status()
  File "/usr/local/lib/python3.8/dist-packages/requests/models.py", line 941, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.35/images/create?tag=sha256%3Ab13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691&fromImage=docker.io%2Fnitaigao%2Faido-submissions

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 65, in docker_pull
    pulling = client.api.pull(repository=repository, tag=br.tag, stream=True, decode=True)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/image.py", line 415, in pull
    self._raise_for_status(response)
  File "/usr/local/lib/python3.8/dist-packages/docker/api/client.py", line 263, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python3.8/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
docker.errors.ImageNotFound: 404 Client Error: Not Found ("pull access denied for nitaigao/aido-submissions, repository does not exist or may require 'docker login': denied: requested access to the resource is denied")

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 38, in docker_pull_retry
    return docker_pull(client, image_name, quiet=quiet)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 84, in docker_pull
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: Cannot pull repo  docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691  tag  None

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 745, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 944, in run_single
    docker_pull_retry(client, image, ntimes=4, wait=5)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_build_utils/docker_pulling.py", line 42, in docker_pull_retry
    raise PullError(msg) from e
duckietown_build_utils.docker_pulling.PullError: After trying 4 I still could not pull docker.io/nitaigao/aido-submissions@sha256:b13078d04947eb3a802ebc4e9db985f6a60c2a3cae145d65fbd44ef0177e1691

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

66092

13518

András Kalapos 🇭🇺

real-v1.0-3091-310

aido-LFV_multi-sim-validation

402

success

gpu-production-spot-2-01

2021-04-09 13:59:04+00:00

2021-04-09 14:58:45+00:00

0:59:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	53.67499999999909
in-drivable-lane_median	0.0
driven_lanedir_consec_median	26.10822710875899
deviation-center-line_median	2.825203233185186

other stats

agent_compute-ego0_max	0.014855135688178247
agent_compute-ego0_mean	0.014607897083407262
agent_compute-ego0_median	0.014607897083407262
agent_compute-ego0_min	0.014360658478636278
agent_compute-ego1_max	0.015061449349472466
agent_compute-ego1_mean	0.014696943331767073
agent_compute-ego1_median	0.014696943331767073
agent_compute-ego1_min	0.014332437314061676
agent_compute-ego2_max	0.01466609893690835
agent_compute-ego2_mean	0.014439527421161844
agent_compute-ego2_median	0.014439527421161844
agent_compute-ego2_min	0.014212955905415338
agent_compute-ego3_max	0.014464903632171148
agent_compute-ego3_mean	0.014348806157833969
agent_compute-ego3_median	0.014348806157833969
agent_compute-ego3_min	0.014232708683496789
complete-iteration_max	1.1059453384167546
complete-iteration_mean	0.9086627229550476
complete-iteration_median	0.9086627229550476
complete-iteration_min	0.7113801074933402
deviation-center-line_max	3.0404233972008377
deviation-center-line_mean	2.6384168018473613
deviation-center-line_min	1.943451741399122
deviation-heading_max	8.413519108662621
deviation-heading_mean	6.743807373589924
deviation-heading_median	7.043718959539019
deviation-heading_min	4.139959537823334
driven_any_max	29.191860465783133
driven_any_mean	24.688203455946784
driven_any_median	26.436328267156597
driven_any_min	18.39222920613022
driven_lanedir_consec_max	28.762974518627427
driven_lanedir_consec_mean	24.37769668947243
driven_lanedir_consec_min	18.184062185872012
driven_lanedir_max	28.762974518627427
driven_lanedir_mean	24.37769668947243
driven_lanedir_median	26.10822710875899
driven_lanedir_min	18.184062185872012
get_duckie_state_max	1.8173129246707707e-06
get_duckie_state_mean	1.724857951074831e-06
get_duckie_state_median	1.724857951074831e-06
get_duckie_state_min	1.6324029774788914e-06
get_robot_state_max	0.015542537346172097
get_robot_state_mean	0.01549257117736398
get_robot_state_median	0.01549257117736398
get_robot_state_min	0.015442605008555868
get_state_dump_max	0.010418326291353894
get_state_dump_mean	0.01032228025093703
get_state_dump_median	0.01032228025093703
get_state_dump_min	0.010226234210520165
get_ui_image_max	0.05282177972753876
get_ui_image_mean	0.04958839930743876
get_ui_image_median	0.04958839930743876
get_ui_image_min	0.04635501888733875
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"LFV_multi-norm-loop-000-ego0": {"driven_any": 18.39222920613022, "get_ui_image": 0.04635501888733875, "step_physics": 0.5181451278899792, "survival_time": 47.34999999999945, "driven_lanedir": 18.184062185872012, "get_state_dump": 0.010418326291353894, "get_robot_state": 0.015442605008555868, "sim_render-ego0": 0.004079996030541915, "sim_render-ego1": 0.004084680905321983, "sim_render-ego2": 0.004057251702884079, "sim_render-ego3": 0.004124280018142507, "get_duckie_state": 1.8173129246707707e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.936428825590696, "agent_compute-ego0": 0.014360658478636278, "agent_compute-ego1": 0.014332437314061676, "agent_compute-ego2": 0.014212955905415338, "agent_compute-ego3": 0.014232708683496789, "complete-iteration": 0.7113801074933402, "set_robot_commands": 0.002451138405860225, "deviation-center-line": 2.928223682612597, "driven_lanedir_consec": 18.184062185872012, "sim_compute_sim_state": 0.028681757329385493, "sim_compute_performance-ego0": 0.0022415272294217525, "sim_compute_performance-ego1": 0.0021783442939887067, "sim_compute_performance-ego2": 0.002162161758680384, "sim_compute_performance-ego3": 0.002182982893432746}, "LFV_multi-norm-loop-000-ego1": {"driven_any": 20.664627585435408, "get_ui_image": 0.04635501888733875, "step_physics": 0.5181451278899792, "survival_time": 47.34999999999945, "driven_lanedir": 20.408384905127047, "get_state_dump": 0.010418326291353894, "get_robot_state": 0.015442605008555868, "sim_render-ego0": 0.004079996030541915, "sim_render-ego1": 0.004084680905321983, "sim_render-ego2": 0.004057251702884079, "sim_render-ego3": 0.004124280018142507, "get_duckie_state": 1.8173129246707707e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.029378888918278, "agent_compute-ego0": 0.014360658478636278, "agent_compute-ego1": 0.014332437314061676, "agent_compute-ego2": 0.014212955905415338, "agent_compute-ego3": 0.014232708683496789, "complete-iteration": 0.7113801074933402, "set_robot_commands": 0.002451138405860225, "deviation-center-line": 2.526022471852477, "driven_lanedir_consec": 20.408384905127047, "sim_compute_sim_state": 0.028681757329385493, "sim_compute_performance-ego0": 0.0022415272294217525, "sim_compute_performance-ego1": 0.0021783442939887067, "sim_compute_performance-ego2": 0.002162161758680384, "sim_compute_performance-ego3": 0.002182982893432746}, "LFV_multi-norm-loop-000-ego2": {"driven_any": 18.393394162791022, "get_ui_image": 0.04635501888733875, "step_physics": 0.5181451278899792, "survival_time": 47.34999999999945, "driven_lanedir": 18.27339368357645, "get_state_dump": 0.010418326291353894, "get_robot_state": 0.015442605008555868, "sim_render-ego0": 0.004079996030541915, "sim_render-ego1": 0.004084680905321983, "sim_render-ego2": 0.004057251702884079, "sim_render-ego3": 0.004124280018142507, "get_duckie_state": 1.8173129246707707e-06, "in-drivable-lane": 0.0, "deviation-heading": 4.139959537823334, "agent_compute-ego0": 0.014360658478636278, "agent_compute-ego1": 0.014332437314061676, "agent_compute-ego2": 0.014212955905415338, "agent_compute-ego3": 0.014232708683496789, "complete-iteration": 0.7113801074933402, "set_robot_commands": 0.002451138405860225, "deviation-center-line": 1.999802535811586, "driven_lanedir_consec": 18.27339368357645, "sim_compute_sim_state": 0.028681757329385493, "sim_compute_performance-ego0": 0.0022415272294217525, "sim_compute_performance-ego1": 0.0021783442939887067, "sim_compute_performance-ego2": 0.002162161758680384, "sim_compute_performance-ego3": 0.002182982893432746}, "LFV_multi-norm-loop-000-ego3": {"driven_any": 24.2103436086508, "get_ui_image": 0.04635501888733875, "step_physics": 0.5181451278899792, "survival_time": 47.34999999999945, "driven_lanedir": 23.932229365747357, "get_state_dump": 0.010418326291353894, "get_robot_state": 0.015442605008555868, "sim_render-ego0": 0.004079996030541915, "sim_render-ego1": 0.004084680905321983, "sim_render-ego2": 0.004057251702884079, "sim_render-ego3": 0.004124280018142507, "get_duckie_state": 1.8173129246707707e-06, "in-drivable-lane": 0.0, "deviation-heading": 5.732363277206346, "agent_compute-ego0": 0.014360658478636278, "agent_compute-ego1": 0.014332437314061676, "agent_compute-ego2": 0.014212955905415338, "agent_compute-ego3": 0.014232708683496789, "complete-iteration": 0.7113801074933402, "set_robot_commands": 0.002451138405860225, "deviation-center-line": 1.943451741399122, "driven_lanedir_consec": 23.932229365747357, "sim_compute_sim_state": 0.028681757329385493, "sim_compute_performance-ego0": 0.0022415272294217525, "sim_compute_performance-ego1": 0.0021783442939887067, "sim_compute_performance-ego2": 0.002162161758680384, "sim_compute_performance-ego3": 0.002182982893432746}, "LFV_multi-norm-zigzag-000-ego0": {"driven_any": 29.023642277757418, "get_ui_image": 0.05282177972753876, "step_physics": 0.8808210501563638, "survival_time": 59.99999999999873, "driven_lanedir": 28.616900914167793, "get_state_dump": 0.010226234210520165, "get_robot_state": 0.015542537346172097, "sim_render-ego0": 0.004083105963930103, "sim_render-ego1": 0.004024506012267812, "sim_render-ego2": 0.003999846861026964, "sim_render-ego3": 0.003981972217162781, "get_duckie_state": 1.6324029774788914e-06, "in-drivable-lane": 0.0, "deviation-heading": 8.16544849692759, "agent_compute-ego0": 0.014855135688178247, "agent_compute-ego1": 0.015061449349472466, "agent_compute-ego2": 0.01466609893690835, "agent_compute-ego3": 0.014464903632171148, "complete-iteration": 1.1059453384167546, "set_robot_commands": 0.0024113923087902213, "deviation-center-line": 2.9530149190223027, "driven_lanedir_consec": 28.616900914167793, "sim_compute_sim_state": 0.05268830562213577, "sim_compute_performance-ego0": 0.002254861479099347, "sim_compute_performance-ego1": 0.002136046443751809, "sim_compute_performance-ego2": 0.002175721201074808, "sim_compute_performance-ego3": 0.0021292992575182505}, "LFV_multi-norm-zigzag-000-ego1": {"driven_any": 28.96721741536388, "get_ui_image": 0.05282177972753876, "step_physics": 0.8808210501563638, "survival_time": 59.99999999999873, "driven_lanedir": 28.55940309089073, "get_state_dump": 0.010226234210520165, "get_robot_state": 0.015542537346172097, "sim_render-ego0": 0.004083105963930103, "sim_render-ego1": 0.004024506012267812, "sim_render-ego2": 0.003999846861026964, "sim_render-ego3": 0.003981972217162781, "get_duckie_state": 1.6324029774788914e-06, "in-drivable-lane": 0.0, "deviation-heading": 8.38235176010318, "agent_compute-ego0": 0.014855135688178247, "agent_compute-ego1": 0.015061449349472466, "agent_compute-ego2": 0.01466609893690835, "agent_compute-ego3": 0.014464903632171148, "complete-iteration": 1.1059453384167546, "set_robot_commands": 0.0024113923087902213, "deviation-center-line": 3.0404233972008377, "driven_lanedir_consec": 28.55940309089073, "sim_compute_sim_state": 0.05268830562213577, "sim_compute_performance-ego0": 0.002254861479099347, "sim_compute_performance-ego1": 0.002136046443751809, "sim_compute_performance-ego2": 0.002175721201074808, "sim_compute_performance-ego3": 0.0021292992575182505}, "LFV_multi-norm-zigzag-000-ego2": {"driven_any": 29.191860465783133, "get_ui_image": 0.05282177972753876, "step_physics": 0.8808210501563638, "survival_time": 59.99999999999873, "driven_lanedir": 28.762974518627427, "get_state_dump": 0.010226234210520165, "get_robot_state": 0.015542537346172097, "sim_render-ego0": 0.004083105963930103, "sim_render-ego1": 0.004024506012267812, "sim_render-ego2": 0.003999846861026964, "sim_render-ego3": 0.003981972217162781, "get_duckie_state": 1.6324029774788914e-06, "in-drivable-lane": 0.0, "deviation-heading": 8.413519108662621, "agent_compute-ego0": 0.014855135688178247, "agent_compute-ego1": 0.015061449349472466, "agent_compute-ego2": 0.01466609893690835, "agent_compute-ego3": 0.014464903632171148, "complete-iteration": 1.1059453384167546, "set_robot_commands": 0.0024113923087902213, "deviation-center-line": 2.9942128831221932, "driven_lanedir_consec": 28.762974518627427, "sim_compute_sim_state": 0.05268830562213577, "sim_compute_performance-ego0": 0.002254861479099347, "sim_compute_performance-ego1": 0.002136046443751809, "sim_compute_performance-ego2": 0.002175721201074808, "sim_compute_performance-ego3": 0.0021292992575182505}, "LFV_multi-norm-zigzag-000-ego3": {"driven_any": 28.6623129256624, "get_ui_image": 0.05282177972753876, "step_physics": 0.8808210501563638, "survival_time": 59.99999999999873, "driven_lanedir": 28.28422485177062, "get_state_dump": 0.010226234210520165, "get_robot_state": 0.015542537346172097, "sim_render-ego0": 0.004083105963930103, "sim_render-ego1": 0.004024506012267812, "sim_render-ego2": 0.003999846861026964, "sim_render-ego3": 0.003981972217162781, "get_duckie_state": 1.6324029774788914e-06, "in-drivable-lane": 0.0, "deviation-heading": 8.151009093487342, "agent_compute-ego0": 0.014855135688178247, "agent_compute-ego1": 0.015061449349472466, "agent_compute-ego2": 0.01466609893690835, "agent_compute-ego3": 0.014464903632171148, "complete-iteration": 1.1059453384167546, "set_robot_commands": 0.0024113923087902213, "deviation-center-line": 2.722182783757775, "driven_lanedir_consec": 28.28422485177062, "sim_compute_sim_state": 0.05268830562213577, "sim_compute_performance-ego0": 0.002254861479099347, "sim_compute_performance-ego1": 0.002136046443751809, "sim_compute_performance-ego2": 0.002175721201074808, "sim_compute_performance-ego3": 0.0021292992575182505}}
set_robot_commands_max	0.002451138405860225
set_robot_commands_mean	0.0024312653573252234
set_robot_commands_median	0.0024312653573252234
set_robot_commands_min	0.0024113923087902213
sim_compute_performance-ego0_max	0.002254861479099347
sim_compute_performance-ego0_mean	0.00224819435426055
sim_compute_performance-ego0_median	0.00224819435426055
sim_compute_performance-ego0_min	0.0022415272294217525
sim_compute_performance-ego1_max	0.0021783442939887067
sim_compute_performance-ego1_mean	0.002157195368870258
sim_compute_performance-ego1_median	0.002157195368870258
sim_compute_performance-ego1_min	0.002136046443751809
sim_compute_performance-ego2_max	0.002175721201074808
sim_compute_performance-ego2_mean	0.0021689414798775957
sim_compute_performance-ego2_median	0.0021689414798775957
sim_compute_performance-ego2_min	0.002162161758680384
sim_compute_performance-ego3_max	0.002182982893432746
sim_compute_performance-ego3_mean	0.0021561410754754986
sim_compute_performance-ego3_median	0.0021561410754754986
sim_compute_performance-ego3_min	0.0021292992575182505
sim_compute_sim_state_max	0.05268830562213577
sim_compute_sim_state_mean	0.04068503147576064
sim_compute_sim_state_median	0.04068503147576064
sim_compute_sim_state_min	0.028681757329385493
sim_render-ego0_max	0.004083105963930103
sim_render-ego0_mean	0.004081550997236009
sim_render-ego0_median	0.004081550997236009
sim_render-ego0_min	0.004079996030541915
sim_render-ego1_max	0.004084680905321983
sim_render-ego1_mean	0.0040545934587948975
sim_render-ego1_median	0.0040545934587948975
sim_render-ego1_min	0.004024506012267812
sim_render-ego2_max	0.004057251702884079
sim_render-ego2_mean	0.004028549281955521
sim_render-ego2_median	0.004028549281955521
sim_render-ego2_min	0.003999846861026964
sim_render-ego3_max	0.004124280018142507
sim_render-ego3_mean	0.0040531261176526435
sim_render-ego3_median	0.0040531261176526435
sim_render-ego3_min	0.003981972217162781
simulation-passed	1
step_physics_max	0.8808210501563638
step_physics_mean	0.6994830890231716
step_physics_median	0.6994830890231716
step_physics_min	0.5181451278899792
survival_time_max	59.99999999999873
survival_time_mean	53.67499999999909
survival_time_min	47.34999999999945

No reset possible

66084

13518

András Kalapos 🇭🇺

real-v1.0-3091-310

aido-LFV_multi-sim-validation

403

failed

gpu-production-spot-2-01

2021-04-09 13:57:36+00:00

2021-04-09 13:58:38+00:00

0:01:02

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 268, in main
    robot_ci.write_topic_and_expect_zero("seed", config.seed)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 309, in read_reply
    raise RemoteNodeAborted(msg)
zuper_nodes.structures.RemoteNodeAborted: The remote node "ego1" aborted with the following error:

error in ego1 |Unexpected error:
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1365, in _do_call
              ||     return fn(*args)
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1349, in _run_fn
              ||     return self._call_tf_sessionrun(options, feed_dict, fetch_list,
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1441, in _call_tf_sessionrun
              ||     return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
              || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
              ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || 	 [[{{node default_policy/functional_1_1/conv_value_1/Relu}}]]
              ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || 	 [[{{node default_policy/functional_1_1/conv_value_1/Relu}}]]
              || 	 [[default_policy/strided_slice_1/_3]]
              || 0 successful operations.
              || 0 derived errors ignored.
              ||
              || During handling of the above exception, another exception occurred:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(SEED,experiment_idx=0,checkpoint_idx=0,logger=context)
              ||   File "/submission/model.py", line 55, in __init__
              ||     self.model = PPOTrainer(config=config["rllib_config"])
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
              ||     Trainer.__init__(self, config, env, logger_creator)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 455, in __init__
              ||     super().__init__(config, logger_creator)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/tune/trainable.py", line 174, in __init__
              ||     self._setup(copy.deepcopy(self.config))
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 596, in _setup
              ||     self._init(self.config, self.env_creator)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 115, in _init
              ||     self.workers = self._make_workers(env_creator, self._policy,
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 662, in _make_workers
              ||     return WorkerSet(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 61, in __init__
              ||     self._local_worker = self._make_worker(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 237, in _make_worker
              ||     worker = cls(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 360, in __init__
              ||     self._build_policy_map(policy_dict, policy_config)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 842, in _build_policy_map
              ||     policy_map[name] = cls(obs_space, act_space, merged_conf)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 129, in __init__
              ||     DynamicTFPolicy.__init__(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 237, in __init__
              ||     self._initialize_loss()
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 324, in _initialize_loss
              ||     postprocessed_batch = self.postprocess_trajectory(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 155, in postprocess_trajectory
              ||     return postprocess_fn(self, sample_batch, other_agent_batches,
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/ppo/ppo_tf_policy.py", line 182, in postprocess_ppo_gae
              ||     last_r = policy._value(sample_batch[SampleBatch.NEXT_OBS][-1],
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/utils/tf_ops.py", line 86, in call
              ||     return session_or_none.run(symbolic_out[0], feed_dict)
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 957, in run
              ||     result = self._run(None, fetches, feed_dict, options_ptr,
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1180, in _run
              ||     results = self._do_run(handle, final_targets, final_fetches,
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1358, in _do_run
              ||     return self._do_call(_run_fn, feeds, fetches, targets, options,
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1384, in _do_call
              ||     raise type(e)(node_def, op, message)
              || tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
              ||   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || 	 [[node default_policy/functional_1_1/conv_value_1/Relu (defined at /usr/local/lib/python3.8/dist-packages/ray/rllib/models/tf/visionnet_v2.py:103) ]]
              ||   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || 	 [[node default_policy/functional_1_1/conv_value_1/Relu (defined at /usr/local/lib/python3.8/dist-packages/ray/rllib/models/tf/visionnet_v2.py:103) ]]
              || 	 [[default_policy/strided_slice_1/_3]]
              || 0 successful operations.
              || 0 derived errors ignored.
              ||
              || Original stack trace for 'default_policy/functional_1_1/conv_value_1/Relu':
              ||   File "solution.py", line 127, in <module>
              ||     main()
              ||   File "solution.py", line 123, in main
              ||     wrap_direct(node=node, protocol=protocol)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/interface.py", line 24, in wrap_direct
              ||     run_loop(node, protocol, args)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 243, in run_loop
              ||     loop(node_name, fi, fo, node, protocol, tin, tout, config=config, fi_desc=fin, fo_desc=fout)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              ||     call_if_fun_exists(node, "init", context=context_data)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "solution.py", line 29, in init
              ||     self.model = RLlibModel(SEED,experiment_idx=0,checkpoint_idx=0,logger=context)
              ||   File "/submission/model.py", line 55, in __init__
              ||     self.model = PPOTrainer(config=config["rllib_config"])
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
              ||     Trainer.__init__(self, config, env, logger_creator)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 455, in __init__
              ||     super().__init__(config, logger_creator)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/tune/trainable.py", line 174, in __init__
              ||     self._setup(copy.deepcopy(self.config))
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 596, in _setup
              ||     self._init(self.config, self.env_creator)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 115, in _init
              ||     self.workers = self._make_workers(env_creator, self._policy,
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 662, in _make_workers
              ||     return WorkerSet(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 61, in __init__
              ||     self._local_worker = self._make_worker(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 237, in _make_worker
              ||     worker = cls(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 360, in __init__
              ||     self._build_policy_map(policy_dict, policy_config)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 842, in _build_policy_map
              ||     policy_map[name] = cls(obs_space, act_space, merged_conf)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 129, in __init__
              ||     DynamicTFPolicy.__init__(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 237, in __init__
              ||     self._initialize_loss()
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 324, in _initialize_loss
              ||     postprocessed_batch = self.postprocess_trajectory(
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 155, in postprocess_trajectory
              ||     return postprocess_fn(self, sample_batch, other_agent_batches,
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/ppo/ppo_tf_policy.py", line 182, in postprocess_ppo_gae
              ||     last_r = policy._value(sample_batch[SampleBatch.NEXT_OBS][-1],
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/utils/tf_ops.py", line 84, in call
              ||     symbolic_out[0] = fn(*placeholders)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/ppo/ppo_tf_policy.py", line 235, in value
              ||     model_out, _ = self.model({
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/models/modelv2.py", line 150, in __call__
              ||     res = self.forward(restored, state or [], seq_lens)
              ||   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/models/tf/visionnet_v2.py", line 103, in forward
              ||     model_out, self._value_out = self.base_model(
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 776, in __call__
              ||     outputs = call_fn(cast_inputs, *args, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/functional.py", line 385, in call
              ||     return self._run_internal_graph(
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/functional.py", line 508, in _run_internal_graph
              ||     outputs = node.layer(*args, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 776, in __call__
              ||     outputs = call_fn(cast_inputs, *args, **kwargs)
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/layers/convolutional.py", line 269, in call
              ||     return self.activation(outputs)
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 10435, in relu
              ||     _, _, _op, _outputs = _op_def_library._apply_op_helper(
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 742, in _apply_op_helper
              ||     op = g._create_op_internal(op_type_name, inputs, dtypes=None,
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 3477, in _create_op_internal
              ||     ret = Operation(
              ||   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1949, in __init__
              ||     self._traceback = tf_stack.extract_stack()
              ||
              ||
              || The above exception was the direct cause of the following exception:
              ||
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 339, in loop
              ||     raise Exception(msg) from e
              || Exception: Exception while calling the node's init() function.
              ||
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1365, in _do_call
              || |     return fn(*args)
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1349, in _run_fn
              || |     return self._call_tf_sessionrun(options, feed_dict, fetch_list,
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1441, in _call_tf_sessionrun
              || |     return tf_session.TF_SessionRun_wrapper(self._session, options, feed_dict,
              || | tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
              || |   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || | 	 [[{{node default_policy/functional_1_1/conv_value_1/Relu}}]]
              || |   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || | 	 [[{{node default_policy/functional_1_1/conv_value_1/Relu}}]]
              || | 	 [[default_policy/strided_slice_1/_3]]
              || | 0 successful operations.
              || | 0 derived errors ignored.
              || |
              || | During handling of the above exception, another exception occurred:
              || |
              || | Traceback (most recent call last):
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 29, in init
              || |     self.model = RLlibModel(SEED,experiment_idx=0,checkpoint_idx=0,logger=context)
              || |   File "/submission/model.py", line 55, in __init__
              || |     self.model = PPOTrainer(config=config["rllib_config"])
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
              || |     Trainer.__init__(self, config, env, logger_creator)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 455, in __init__
              || |     super().__init__(config, logger_creator)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/tune/trainable.py", line 174, in __init__
              || |     self._setup(copy.deepcopy(self.config))
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 596, in _setup
              || |     self._init(self.config, self.env_creator)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 115, in _init
              || |     self.workers = self._make_workers(env_creator, self._policy,
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 662, in _make_workers
              || |     return WorkerSet(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 61, in __init__
              || |     self._local_worker = self._make_worker(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 237, in _make_worker
              || |     worker = cls(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 360, in __init__
              || |     self._build_policy_map(policy_dict, policy_config)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 842, in _build_policy_map
              || |     policy_map[name] = cls(obs_space, act_space, merged_conf)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 129, in __init__
              || |     DynamicTFPolicy.__init__(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 237, in __init__
              || |     self._initialize_loss()
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 324, in _initialize_loss
              || |     postprocessed_batch = self.postprocess_trajectory(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 155, in postprocess_trajectory
              || |     return postprocess_fn(self, sample_batch, other_agent_batches,
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/ppo/ppo_tf_policy.py", line 182, in postprocess_ppo_gae
              || |     last_r = policy._value(sample_batch[SampleBatch.NEXT_OBS][-1],
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/utils/tf_ops.py", line 86, in call
              || |     return session_or_none.run(symbolic_out[0], feed_dict)
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 957, in run
              || |     result = self._run(None, fetches, feed_dict, options_ptr,
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1180, in _run
              || |     results = self._do_run(handle, final_targets, final_fetches,
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1358, in _do_run
              || |     return self._do_call(_run_fn, feeds, fetches, targets, options,
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/client/session.py", line 1384, in _do_call
              || |     raise type(e)(node_def, op, message)
              || | tensorflow.python.framework.errors_impl.UnknownError: 2 root error(s) found.
              || |   (0) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || | 	 [[node default_policy/functional_1_1/conv_value_1/Relu (defined at /usr/local/lib/python3.8/dist-packages/ray/rllib/models/tf/visionnet_v2.py:103) ]]
              || |   (1) Unknown: Failed to get convolution algorithm. This is probably because cuDNN failed to initialize, so try looking to see if a warning log message was printed above.
              || | 	 [[node default_policy/functional_1_1/conv_value_1/Relu (defined at /usr/local/lib/python3.8/dist-packages/ray/rllib/models/tf/visionnet_v2.py:103) ]]
              || | 	 [[default_policy/strided_slice_1/_3]]
              || | 0 successful operations.
              || | 0 derived errors ignored.
              || |
              || | Original stack trace for 'default_policy/functional_1_1/conv_value_1/Relu':
              || |   File "solution.py", line 127, in <module>
              || |     main()
              || |   File "solution.py", line 123, in main
              || |     wrap_direct(node=node, protocol=protocol)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/interface.py", line 24, in wrap_direct
              || |     run_loop(node, protocol, args)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 243, in run_loop
              || |     loop(node_name, fi, fo, node, protocol, tin, tout, config=config, fi_desc=fin, fo_desc=fout)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 322, in loop
              || |     call_if_fun_exists(node, "init", context=context_data)
              || |   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              || |     f(**kwargs)
              || |   File "solution.py", line 29, in init
              || |     self.model = RLlibModel(SEED,experiment_idx=0,checkpoint_idx=0,logger=context)
              || |   File "/submission/model.py", line 55, in __init__
              || |     self.model = PPOTrainer(config=config["rllib_config"])
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 90, in __init__
              || |     Trainer.__init__(self, config, env, logger_creator)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 455, in __init__
              || |     super().__init__(config, logger_creator)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/tune/trainable.py", line 174, in __init__
              || |     self._setup(copy.deepcopy(self.config))
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 596, in _setup
              || |     self._init(self.config, self.env_creator)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer_template.py", line 115, in _init
              || |     self.workers = self._make_workers(env_creator, self._policy,
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/trainer.py", line 662, in _make_workers
              || |     return WorkerSet(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 61, in __init__
              || |     self._local_worker = self._make_worker(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/worker_set.py", line 237, in _make_worker
              || |     worker = cls(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 360, in __init__
              || |     self._build_policy_map(policy_dict, policy_config)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/evaluation/rollout_worker.py", line 842, in _build_policy_map
              || |     policy_map[name] = cls(obs_space, act_space, merged_conf)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 129, in __init__
              || |     DynamicTFPolicy.__init__(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 237, in __init__
              || |     self._initialize_loss()
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/dynamic_tf_policy.py", line 324, in _initialize_loss
              || |     postprocessed_batch = self.postprocess_trajectory(
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/policy/tf_policy_template.py", line 155, in postprocess_trajectory
              || |     return postprocess_fn(self, sample_batch, other_agent_batches,
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/ppo/ppo_tf_policy.py", line 182, in postprocess_ppo_gae
              || |     last_r = policy._value(sample_batch[SampleBatch.NEXT_OBS][-1],
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/utils/tf_ops.py", line 84, in call
              || |     symbolic_out[0] = fn(*placeholders)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/agents/ppo/ppo_tf_policy.py", line 235, in value
              || |     model_out, _ = self.model({
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/models/modelv2.py", line 150, in __call__
              || |     res = self.forward(restored, state or [], seq_lens)
              || |   File "/usr/local/lib/python3.8/dist-packages/ray/rllib/models/tf/visionnet_v2.py", line 103, in forward
              || |     model_out, self._value_out = self.base_model(
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 776, in __call__
              || |     outputs = call_fn(cast_inputs, *args, **kwargs)
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/functional.py", line 385, in call
              || |     return self._run_internal_graph(
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/functional.py", line 508, in _run_internal_graph
              || |     outputs = node.layer(*args, **kwargs)
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/engine/base_layer_v1.py", line 776, in __call__
              || |     outputs = call_fn(cast_inputs, *args, **kwargs)
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/keras/layers/convolutional.py", line 269, in call
              || |     return self.activation(outputs)
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/ops/gen_nn_ops.py", line 10435, in relu
              || |     _, _, _op, _outputs = _op_def_library._apply_op_helper(
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/op_def_library.py", line 742, in _apply_op_helper
              || |     op = g._create_op_internal(op_type_name, inputs, dtypes=None,
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 3477, in _create_op_internal
              || |     ret = Operation(
              || |   File "/usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/ops.py", line 1949, in __init__
              || |     self._traceback = tf_stack.extract_stack()
              || |
              || |
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 275, in main
    raise InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Getting agent protocol

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

66066

13543

András Kalapos 🇭🇺

3090

aido-LFV-sim-validation

sim-2of4

success

gpu-production-spot-2-01

2021-04-09 13:52:19+00:00

2021-04-09 13:57:17+00:00

0:04:58

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	10.90000000000002
in-drivable-lane_median	0.0
driven_lanedir_consec_median	4.531282120023431
deviation-center-line_median	0.49819210008591025

other stats

agent_compute-ego0_max	0.015203788400240684
agent_compute-ego0_mean	0.015203788400240684
agent_compute-ego0_median	0.015203788400240684
agent_compute-ego0_min	0.015203788400240684
agent_compute-npc0_max	0.03373588492336883
agent_compute-npc0_mean	0.03373588492336883
agent_compute-npc0_median	0.03373588492336883
agent_compute-npc0_min	0.03373588492336883
agent_compute-npc1_max	0.03402949903653637
agent_compute-npc1_mean	0.03402949903653637
agent_compute-npc1_median	0.03402949903653637
agent_compute-npc1_min	0.03402949903653637
agent_compute-npc2_max	0.03814444258877132
agent_compute-npc2_mean	0.03814444258877132
agent_compute-npc2_median	0.03814444258877132
agent_compute-npc2_min	0.03814444258877132
complete-iteration_max	0.6091962209030918
complete-iteration_mean	0.6091962209030918
complete-iteration_median	0.6091962209030918
complete-iteration_min	0.6091962209030918
deviation-center-line_max	0.49819210008591025
deviation-center-line_mean	0.49819210008591025
deviation-center-line_min	0.49819210008591025
deviation-heading_max	2.1624613318495545
deviation-heading_mean	2.1624613318495545
deviation-heading_median	2.1624613318495545
deviation-heading_min	2.1624613318495545
driven_any_max	4.669215645029459
driven_any_mean	4.669215645029459
driven_any_median	4.669215645029459
driven_any_min	4.669215645029459
driven_lanedir_consec_max	4.531282120023431
driven_lanedir_consec_mean	4.531282120023431
driven_lanedir_consec_min	4.531282120023431
driven_lanedir_max	4.531282120023431
driven_lanedir_mean	4.531282120023431
driven_lanedir_median	4.531282120023431
driven_lanedir_min	4.531282120023431
get_duckie_state_max	1.714654164771511e-06
get_duckie_state_mean	1.714654164771511e-06
get_duckie_state_median	1.714654164771511e-06
get_duckie_state_min	1.714654164771511e-06
get_robot_state_max	0.01600030676959312
get_robot_state_mean	0.01600030676959312
get_robot_state_median	0.01600030676959312
get_robot_state_min	0.01600030676959312
get_state_dump_max	0.010879561237004247
get_state_dump_mean	0.010879561237004247
get_state_dump_median	0.010879561237004247
get_state_dump_min	0.010879561237004247
get_ui_image_max	0.049091240042420825
get_ui_image_mean	0.049091240042420825
get_ui_image_median	0.049091240042420825
get_ui_image_min	0.049091240042420825
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"LFV-norm-loop-000-ego0": {"driven_any": 4.669215645029459, "get_ui_image": 0.049091240042420825, "step_physics": 0.34854665956540737, "survival_time": 10.90000000000002, "driven_lanedir": 4.531282120023431, "get_state_dump": 0.010879561237004247, "get_robot_state": 0.01600030676959312, "sim_render-ego0": 0.004107389275886153, "sim_render-npc0": 0.004109737535589906, "sim_render-npc1": 0.004255208794929122, "sim_render-npc2": 0.004283731930876431, "get_duckie_state": 1.714654164771511e-06, "in-drivable-lane": 0.0, "deviation-heading": 2.1624613318495545, "agent_compute-ego0": 0.015203788400240684, "agent_compute-npc0": 0.03373588492336883, "agent_compute-npc1": 0.03402949903653637, "agent_compute-npc2": 0.03814444258877132, "complete-iteration": 0.6091962209030918, "set_robot_commands": 0.0026807795920872797, "deviation-center-line": 0.49819210008591025, "driven_lanedir_consec": 4.531282120023431, "sim_compute_sim_state": 0.027358257607237937, "sim_compute_performance-ego0": 0.0022633783349163454, "sim_compute_performance-npc0": 0.0021657453824396, "sim_compute_performance-npc1": 0.0022411694809726383, "sim_compute_performance-npc2": 0.0022336141159545343}}
set_robot_commands_max	0.0026807795920872797
set_robot_commands_mean	0.0026807795920872797
set_robot_commands_median	0.0026807795920872797
set_robot_commands_min	0.0026807795920872797
sim_compute_performance-ego0_max	0.0022633783349163454
sim_compute_performance-ego0_mean	0.0022633783349163454
sim_compute_performance-ego0_median	0.0022633783349163454
sim_compute_performance-ego0_min	0.0022633783349163454
sim_compute_performance-npc0_max	0.0021657453824396
sim_compute_performance-npc0_mean	0.0021657453824396
sim_compute_performance-npc0_median	0.0021657453824396
sim_compute_performance-npc0_min	0.0021657453824396
sim_compute_performance-npc1_max	0.0022411694809726383
sim_compute_performance-npc1_mean	0.0022411694809726383
sim_compute_performance-npc1_median	0.0022411694809726383
sim_compute_performance-npc1_min	0.0022411694809726383
sim_compute_performance-npc2_max	0.0022336141159545343
sim_compute_performance-npc2_mean	0.0022336141159545343
sim_compute_performance-npc2_median	0.0022336141159545343
sim_compute_performance-npc2_min	0.0022336141159545343
sim_compute_sim_state_max	0.027358257607237937
sim_compute_sim_state_mean	0.027358257607237937
sim_compute_sim_state_median	0.027358257607237937
sim_compute_sim_state_min	0.027358257607237937
sim_render-ego0_max	0.004107389275886153
sim_render-ego0_mean	0.004107389275886153
sim_render-ego0_median	0.004107389275886153
sim_render-ego0_min	0.004107389275886153
sim_render-npc0_max	0.004109737535589906
sim_render-npc0_mean	0.004109737535589906
sim_render-npc0_median	0.004109737535589906
sim_render-npc0_min	0.004109737535589906
sim_render-npc1_max	0.004255208794929122
sim_render-npc1_mean	0.004255208794929122
sim_render-npc1_median	0.004255208794929122
sim_render-npc1_min	0.004255208794929122
sim_render-npc2_max	0.004283731930876431
sim_render-npc2_mean	0.004283731930876431
sim_render-npc2_median	0.004283731930876431
sim_render-npc2_min	0.004283731930876431
simulation-passed	1
step_physics_max	0.34854665956540737
step_physics_mean	0.34854665956540737
step_physics_median	0.34854665956540737
step_physics_min	0.34854665956540737
survival_time_max	10.90000000000002
survival_time_mean	10.90000000000002
survival_time_min	10.90000000000002

No reset possible

66058

13585

Andras Beres

202-1

aido-LFP-sim-testing

sim-2of4

success

gpu-production-spot-2-01

2021-04-09 13:50:26+00:00

2021-04-09 13:51:39+00:00

0:01:13

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

survival_time_median	1.7500000000000009
in-drivable-lane_median	0.0
driven_lanedir_consec_median	0.3883835482779336
deviation-center-line_median	0.11364547250871249

other stats

agent_compute-ego0_max	0.02044061819712321
agent_compute-ego0_mean	0.02044061819712321
agent_compute-ego0_median	0.02044061819712321
agent_compute-ego0_min	0.02044061819712321
complete-iteration_max	0.2238454951180352
complete-iteration_mean	0.2238454951180352
complete-iteration_median	0.2238454951180352
complete-iteration_min	0.2238454951180352
deviation-center-line_max	0.11364547250871249
deviation-center-line_mean	0.11364547250871249
deviation-center-line_min	0.11364547250871249
deviation-heading_max	0.1871769445970333
deviation-heading_mean	0.1871769445970333
deviation-heading_median	0.1871769445970333
deviation-heading_min	0.1871769445970333
driven_any_max	0.3926179402569031
driven_any_mean	0.3926179402569031
driven_any_median	0.3926179402569031
driven_any_min	0.3926179402569031
driven_lanedir_consec_max	0.3883835482779336
driven_lanedir_consec_mean	0.3883835482779336
driven_lanedir_consec_min	0.3883835482779336
driven_lanedir_max	0.3883835482779336
driven_lanedir_mean	0.3883835482779336
driven_lanedir_median	0.3883835482779336
driven_lanedir_min	0.3883835482779336
get_duckie_state_max	0.02796103556950887
get_duckie_state_mean	0.02796103556950887
get_duckie_state_median	0.02796103556950887
get_duckie_state_min	0.02796103556950887
get_robot_state_max	0.004113217194875081
get_robot_state_mean	0.004113217194875081
get_robot_state_median	0.004113217194875081
get_robot_state_min	0.004113217194875081
get_state_dump_max	0.009923073980543347
get_state_dump_mean	0.009923073980543347
get_state_dump_median	0.009923073980543347
get_state_dump_min	0.009923073980543347
get_ui_image_max	0.03654735618167453
get_ui_image_mean	0.03654735618167453
get_ui_image_median	0.03654735618167453
get_ui_image_min	0.03654735618167453
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"LFP-norm-loop-000-ego0": {"driven_any": 0.3926179402569031, "get_ui_image": 0.03654735618167453, "step_physics": 0.10906379090415108, "survival_time": 1.7500000000000009, "driven_lanedir": 0.3883835482779336, "get_state_dump": 0.009923073980543347, "get_robot_state": 0.004113217194875081, "sim_render-ego0": 0.004406895902421739, "get_duckie_state": 0.02796103556950887, "in-drivable-lane": 0.0, "deviation-heading": 0.1871769445970333, "agent_compute-ego0": 0.02044061819712321, "complete-iteration": 0.2238454951180352, "set_robot_commands": 0.002865135669708252, "deviation-center-line": 0.11364547250871249, "driven_lanedir_consec": 0.3883835482779336, "sim_compute_sim_state": 0.006155894862280952, "sim_compute_performance-ego0": 0.002266228199005127}}
set_robot_commands_max	0.002865135669708252
set_robot_commands_mean	0.002865135669708252
set_robot_commands_median	0.002865135669708252
set_robot_commands_min	0.002865135669708252
sim_compute_performance-ego0_max	0.002266228199005127
sim_compute_performance-ego0_mean	0.002266228199005127
sim_compute_performance-ego0_median	0.002266228199005127
sim_compute_performance-ego0_min	0.002266228199005127
sim_compute_sim_state_max	0.006155894862280952
sim_compute_sim_state_mean	0.006155894862280952
sim_compute_sim_state_median	0.006155894862280952
sim_compute_sim_state_min	0.006155894862280952
sim_render-ego0_max	0.004406895902421739
sim_render-ego0_mean	0.004406895902421739
sim_render-ego0_median	0.004406895902421739
sim_render-ego0_min	0.004406895902421739
simulation-passed	1
step_physics_max	0.10906379090415108
step_physics_mean	0.10906379090415108
step_physics_median	0.10906379090415108
step_physics_min	0.10906379090415108
survival_time_max	1.7500000000000009
survival_time_mean	1.7500000000000009
survival_time_min	1.7500000000000009

No reset possible

66036

13940

YU CHEN

CBC Net v2 test - added mar 31 anomaly + mar 28 bc

aido-LF-sim-validation

sim-2of4

success

gpu-production-spot-2-01

2021-04-09 13:39:19+00:00

2021-04-09 13:49:47+00:00

0:10:28

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	11.456219261950729
survival_time_median	59.99999999999873
deviation-center-line_median	2.2034489052180306
in-drivable-lane_median	30.54999999999933

other stats

agent_compute-ego0_max	0.09230621093318028
agent_compute-ego0_mean	0.09230621093318028
agent_compute-ego0_median	0.09230621093318028
agent_compute-ego0_min	0.09230621093318028
complete-iteration_max	0.2845131840733664
complete-iteration_mean	0.2845131840733664
complete-iteration_median	0.2845131840733664
complete-iteration_min	0.2845131840733664
deviation-center-line_max	2.2034489052180306
deviation-center-line_mean	2.2034489052180306
deviation-center-line_min	2.2034489052180306
deviation-heading_max	13.595216085110332
deviation-heading_mean	13.595216085110332
deviation-heading_median	13.595216085110332
deviation-heading_min	13.595216085110332
driven_any_max	24.53180667913561
driven_any_mean	24.53180667913561
driven_any_median	24.53180667913561
driven_any_min	24.53180667913561
driven_lanedir_consec_max	11.456219261950729
driven_lanedir_consec_mean	11.456219261950729
driven_lanedir_consec_min	11.456219261950729
driven_lanedir_max	11.456219261950729
driven_lanedir_mean	11.456219261950729
driven_lanedir_median	11.456219261950729
driven_lanedir_min	11.456219261950729
get_duckie_state_max	1.454333480847666e-06
get_duckie_state_mean	1.454333480847666e-06
get_duckie_state_median	1.454333480847666e-06
get_duckie_state_min	1.454333480847666e-06
get_robot_state_max	0.004049748802661499
get_robot_state_mean	0.004049748802661499
get_robot_state_median	0.004049748802661499
get_robot_state_min	0.004049748802661499
get_state_dump_max	0.004971010103313056
get_state_dump_mean	0.004971010103313056
get_state_dump_median	0.004971010103313056
get_state_dump_min	0.004971010103313056
get_ui_image_max	0.029060507694152273
get_ui_image_mean	0.029060507694152273
get_ui_image_median	0.029060507694152273
get_ui_image_min	0.029060507694152273
in-drivable-lane_max	30.54999999999933
in-drivable-lane_mean	30.54999999999933
in-drivable-lane_min	30.54999999999933
per-episodes	details {"LF-norm-small_loop-000-ego0": {"driven_any": 24.53180667913561, "get_ui_image": 0.029060507694152273, "step_physics": 0.1382845337444499, "survival_time": 59.99999999999873, "driven_lanedir": 11.456219261950729, "get_state_dump": 0.004971010103313056, "get_robot_state": 0.004049748802661499, "sim_render-ego0": 0.004116422429271384, "get_duckie_state": 1.454333480847666e-06, "in-drivable-lane": 30.54999999999933, "deviation-heading": 13.595216085110332, "agent_compute-ego0": 0.09230621093318028, "complete-iteration": 0.2845131840733664, "set_robot_commands": 0.002568428959080222, "deviation-center-line": 2.2034489052180306, "driven_lanedir_consec": 11.456219261950729, "sim_compute_sim_state": 0.006873335667593493, "sim_compute_performance-ego0": 0.0021919378333048064}}
set_robot_commands_max	0.002568428959080222
set_robot_commands_mean	0.002568428959080222
set_robot_commands_median	0.002568428959080222
set_robot_commands_min	0.002568428959080222
sim_compute_performance-ego0_max	0.0021919378333048064
sim_compute_performance-ego0_mean	0.0021919378333048064
sim_compute_performance-ego0_median	0.0021919378333048064
sim_compute_performance-ego0_min	0.0021919378333048064
sim_compute_sim_state_max	0.006873335667593493
sim_compute_sim_state_mean	0.006873335667593493
sim_compute_sim_state_median	0.006873335667593493
sim_compute_sim_state_min	0.006873335667593493
sim_render-ego0_max	0.004116422429271384
sim_render-ego0_mean	0.004116422429271384
sim_render-ego0_median	0.004116422429271384
sim_render-ego0_min	0.004116422429271384
simulation-passed	1
step_physics_max	0.1382845337444499
step_physics_mean	0.1382845337444499
step_physics_median	0.1382845337444499
step_physics_min	0.1382845337444499
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

65993

13995

Frank (Chude) Qian 🇨🇦

baseline-behavior-cloning

aido-LF-sim-validation

sim-0of4

success

gpu-production-spot-2-01

2021-04-09 13:29:38+00:00

2021-04-09 13:38:40+00:00

0:09:02

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	14.361867298994408
survival_time_median	59.99999999999873
deviation-center-line_median	2.581193378299536
in-drivable-lane_median	0.0

other stats

agent_compute-ego0_max	0.03901072028872373
agent_compute-ego0_mean	0.03901072028872373
agent_compute-ego0_median	0.03901072028872373
agent_compute-ego0_min	0.03901072028872373
complete-iteration_max	0.20156955044037295
complete-iteration_mean	0.20156955044037295
complete-iteration_median	0.20156955044037295
complete-iteration_min	0.20156955044037295
deviation-center-line_max	2.581193378299536
deviation-center-line_mean	2.581193378299536
deviation-center-line_min	2.581193378299536
deviation-heading_max	7.875243637056451
deviation-heading_mean	7.875243637056451
deviation-heading_median	7.875243637056451
deviation-heading_min	7.875243637056451
driven_any_max	14.590333604345618
driven_any_mean	14.590333604345618
driven_any_median	14.590333604345618
driven_any_min	14.590333604345618
driven_lanedir_consec_max	14.361867298994408
driven_lanedir_consec_mean	14.361867298994408
driven_lanedir_consec_min	14.361867298994408
driven_lanedir_max	14.361867298994408
driven_lanedir_mean	14.361867298994408
driven_lanedir_median	14.361867298994408
driven_lanedir_min	14.361867298994408
get_duckie_state_max	1.3550751215214534e-06
get_duckie_state_mean	1.3550751215214534e-06
get_duckie_state_median	1.3550751215214534e-06
get_duckie_state_min	1.3550751215214534e-06
get_robot_state_max	0.004004270210551978
get_robot_state_mean	0.004004270210551978
get_robot_state_median	0.004004270210551978
get_robot_state_min	0.004004270210551978
get_state_dump_max	0.004998388536566005
get_state_dump_mean	0.004998388536566005
get_state_dump_median	0.004998388536566005
get_state_dump_min	0.004998388536566005
get_ui_image_max	0.031165173806119818
get_ui_image_mean	0.031165173806119818
get_ui_image_median	0.031165173806119818
get_ui_image_min	0.031165173806119818
in-drivable-lane_max	0.0
in-drivable-lane_mean	0.0
in-drivable-lane_min	0.0
per-episodes	details {"LF-norm-loop-000-ego0": {"driven_any": 14.590333604345618, "get_ui_image": 0.031165173806119818, "step_physics": 0.1028437326591676, "survival_time": 59.99999999999873, "driven_lanedir": 14.361867298994408, "get_state_dump": 0.004998388536566005, "get_robot_state": 0.004004270210551978, "sim_render-ego0": 0.004185505651812272, "get_duckie_state": 1.3550751215214534e-06, "in-drivable-lane": 0.0, "deviation-heading": 7.875243637056451, "agent_compute-ego0": 0.03901072028872373, "complete-iteration": 0.20156955044037295, "set_robot_commands": 0.0025242441004261584, "deviation-center-line": 2.581193378299536, "driven_lanedir_consec": 14.361867298994408, "sim_compute_sim_state": 0.01052136683245682, "sim_compute_performance-ego0": 0.0022177140381215116}}
set_robot_commands_max	0.0025242441004261584
set_robot_commands_mean	0.0025242441004261584
set_robot_commands_median	0.0025242441004261584
set_robot_commands_min	0.0025242441004261584
sim_compute_performance-ego0_max	0.0022177140381215116
sim_compute_performance-ego0_mean	0.0022177140381215116
sim_compute_performance-ego0_median	0.0022177140381215116
sim_compute_performance-ego0_min	0.0022177140381215116
sim_compute_sim_state_max	0.01052136683245682
sim_compute_sim_state_mean	0.01052136683245682
sim_compute_sim_state_median	0.01052136683245682
sim_compute_sim_state_min	0.01052136683245682
sim_render-ego0_max	0.004185505651812272
sim_render-ego0_mean	0.004185505651812272
sim_render-ego0_median	0.004185505651812272
sim_render-ego0_min	0.004185505651812272
simulation-passed	1
step_physics_max	0.1028437326591676
step_physics_mean	0.1028437326591676
step_physics_median	0.1028437326591676
step_physics_min	0.1028437326591676
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

65982

13997

Frank (Chude) Qian 🇨🇦

baseline-behavior-cloning New Dataset

aido-LF-sim-validation

sim-0of4

success

gpu-production-spot-2-01

2021-04-09 13:27:38+00:00

2021-04-09 13:29:25+00:00

0:01:47

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.26255736557409315
survival_time_median	6.699999999999984
deviation-center-line_median	0.12389442478088084
in-drivable-lane_median	5.499999999999986

other stats

agent_compute-ego0_max	0.050870628710146304
agent_compute-ego0_mean	0.050870628710146304
agent_compute-ego0_median	0.050870628710146304
agent_compute-ego0_min	0.050870628710146304
complete-iteration_max	0.1891111550507722
complete-iteration_mean	0.1891111550507722
complete-iteration_median	0.1891111550507722
complete-iteration_min	0.1891111550507722
deviation-center-line_max	0.12389442478088084
deviation-center-line_mean	0.12389442478088084
deviation-center-line_min	0.12389442478088084
deviation-heading_max	0.8716923119513416
deviation-heading_mean	0.8716923119513416
deviation-heading_median	0.8716923119513416
deviation-heading_min	0.8716923119513416
driven_any_max	2.9433433924486385
driven_any_mean	2.9433433924486385
driven_any_median	2.9433433924486385
driven_any_min	2.9433433924486385
driven_lanedir_consec_max	0.26255736557409315
driven_lanedir_consec_mean	0.26255736557409315
driven_lanedir_consec_min	0.26255736557409315
driven_lanedir_max	0.26255736557409315
driven_lanedir_mean	0.26255736557409315
driven_lanedir_median	0.26255736557409315
driven_lanedir_min	0.26255736557409315
get_duckie_state_max	2.0821889241536456e-06
get_duckie_state_mean	2.0821889241536456e-06
get_duckie_state_median	2.0821889241536456e-06
get_duckie_state_min	2.0821889241536456e-06
get_robot_state_max	0.0042897648281521265
get_robot_state_mean	0.0042897648281521265
get_robot_state_median	0.0042897648281521265
get_robot_state_min	0.0042897648281521265
get_state_dump_max	0.0053401187614158345
get_state_dump_mean	0.0053401187614158345
get_state_dump_median	0.0053401187614158345
get_state_dump_min	0.0053401187614158345
get_ui_image_max	0.03314558841564037
get_ui_image_mean	0.03314558841564037
get_ui_image_median	0.03314558841564037
get_ui_image_min	0.03314558841564037
in-drivable-lane_max	5.499999999999986
in-drivable-lane_mean	5.499999999999986
in-drivable-lane_min	5.499999999999986
per-episodes	details {"LF-norm-loop-000-ego0": {"driven_any": 2.9433433924486385, "get_ui_image": 0.03314558841564037, "step_physics": 0.07522098753187391, "survival_time": 6.699999999999984, "driven_lanedir": 0.26255736557409315, "get_state_dump": 0.0053401187614158345, "get_robot_state": 0.0042897648281521265, "sim_render-ego0": 0.004316340552435981, "get_duckie_state": 2.0821889241536456e-06, "in-drivable-lane": 5.499999999999986, "deviation-heading": 0.8716923119513416, "agent_compute-ego0": 0.050870628710146304, "complete-iteration": 0.1891111550507722, "set_robot_commands": 0.002661625544230143, "deviation-center-line": 0.12389442478088084, "driven_lanedir_consec": 0.26255736557409315, "sim_compute_sim_state": 0.010846159193250868, "sim_compute_performance-ego0": 0.0023211302580656828}}
set_robot_commands_max	0.002661625544230143
set_robot_commands_mean	0.002661625544230143
set_robot_commands_median	0.002661625544230143
set_robot_commands_min	0.002661625544230143
sim_compute_performance-ego0_max	0.0023211302580656828
sim_compute_performance-ego0_mean	0.0023211302580656828
sim_compute_performance-ego0_median	0.0023211302580656828
sim_compute_performance-ego0_min	0.0023211302580656828
sim_compute_sim_state_max	0.010846159193250868
sim_compute_sim_state_mean	0.010846159193250868
sim_compute_sim_state_median	0.010846159193250868
sim_compute_sim_state_min	0.010846159193250868
sim_render-ego0_max	0.004316340552435981
sim_render-ego0_mean	0.004316340552435981
sim_render-ego0_median	0.004316340552435981
sim_render-ego0_min	0.004316340552435981
simulation-passed	1
step_physics_max	0.07522098753187391
step_physics_mean	0.07522098753187391
step_physics_median	0.07522098753187391
step_physics_min	0.07522098753187391
survival_time_max	6.699999999999984
survival_time_mean	6.699999999999984
survival_time_min	6.699999999999984

No reset possible

65964

14013

YU CHEN

CBC Net v2 test - APR 6 anomaly + mar 28 bc

aido-LF-sim-validation

sim-2of4

success

gpu-production-spot-2-01

2021-04-09 13:25:03+00:00

2021-04-09 13:27:17+00:00

0:02:14

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	1.1161932582014853
survival_time_median	8.899999999999991
deviation-center-line_median	0.17401751187742404
in-drivable-lane_median	5.749999999999995

other stats

agent_compute-ego0_max	0.0940575479795147
agent_compute-ego0_mean	0.0940575479795147
agent_compute-ego0_median	0.0940575479795147
agent_compute-ego0_min	0.0940575479795147
complete-iteration_max	0.2696088639051555
complete-iteration_mean	0.2696088639051555
complete-iteration_median	0.2696088639051555
complete-iteration_min	0.2696088639051555
deviation-center-line_max	0.17401751187742404
deviation-center-line_mean	0.17401751187742404
deviation-center-line_min	0.17401751187742404
deviation-heading_max	1.3957613105112066
deviation-heading_mean	1.3957613105112066
deviation-heading_median	1.3957613105112066
deviation-heading_min	1.3957613105112066
driven_any_max	2.9175767610504213
driven_any_mean	2.9175767610504213
driven_any_median	2.9175767610504213
driven_any_min	2.9175767610504213
driven_lanedir_consec_max	1.1161932582014853
driven_lanedir_consec_mean	1.1161932582014853
driven_lanedir_consec_min	1.1161932582014853
driven_lanedir_max	1.1161932582014853
driven_lanedir_mean	1.1161932582014853
driven_lanedir_median	1.1161932582014853
driven_lanedir_min	1.1161932582014853
get_duckie_state_max	1.543727000998385e-06
get_duckie_state_mean	1.543727000998385e-06
get_duckie_state_median	1.543727000998385e-06
get_duckie_state_min	1.543727000998385e-06
get_robot_state_max	0.0040641843273653
get_robot_state_mean	0.0040641843273653
get_robot_state_median	0.0040641843273653
get_robot_state_min	0.0040641843273653
get_state_dump_max	0.00531633339780669
get_state_dump_mean	0.00531633339780669
get_state_dump_median	0.00531633339780669
get_state_dump_min	0.00531633339780669
get_ui_image_max	0.02893207859060618
get_ui_image_mean	0.02893207859060618
get_ui_image_median	0.02893207859060618
get_ui_image_min	0.02893207859060618
in-drivable-lane_max	5.749999999999995
in-drivable-lane_mean	5.749999999999995
in-drivable-lane_min	5.749999999999995
per-episodes	details {"LF-norm-small_loop-000-ego0": {"driven_any": 2.9175767610504213, "get_ui_image": 0.02893207859060618, "step_physics": 0.12167752388469334, "survival_time": 8.899999999999991, "driven_lanedir": 1.1161932582014853, "get_state_dump": 0.00531633339780669, "get_robot_state": 0.0040641843273653, "sim_render-ego0": 0.004282073601664112, "get_duckie_state": 1.543727000998385e-06, "in-drivable-lane": 5.749999999999995, "deviation-heading": 1.3957613105112066, "agent_compute-ego0": 0.0940575479795147, "complete-iteration": 0.2696088639051555, "set_robot_commands": 0.002627954802699595, "deviation-center-line": 0.17401751187742404, "driven_lanedir_consec": 1.1161932582014853, "sim_compute_sim_state": 0.006254110922360554, "sim_compute_performance-ego0": 0.0022863835595839516}}
set_robot_commands_max	0.002627954802699595
set_robot_commands_mean	0.002627954802699595
set_robot_commands_median	0.002627954802699595
set_robot_commands_min	0.002627954802699595
sim_compute_performance-ego0_max	0.0022863835595839516
sim_compute_performance-ego0_mean	0.0022863835595839516
sim_compute_performance-ego0_median	0.0022863835595839516
sim_compute_performance-ego0_min	0.0022863835595839516
sim_compute_sim_state_max	0.006254110922360554
sim_compute_sim_state_mean	0.006254110922360554
sim_compute_sim_state_median	0.006254110922360554
sim_compute_sim_state_min	0.006254110922360554
sim_render-ego0_max	0.004282073601664112
sim_render-ego0_mean	0.004282073601664112
sim_render-ego0_median	0.004282073601664112
sim_render-ego0_min	0.004282073601664112
simulation-passed	1
step_physics_max	0.12167752388469334
step_physics_mean	0.12167752388469334
step_physics_median	0.12167752388469334
step_physics_min	0.12167752388469334
survival_time_max	8.899999999999991
survival_time_mean	8.899999999999991
survival_time_min	8.899999999999991

No reset possible

65891

13579

Andras Beres

202-1

aido-LF-sim-testing

sim-3of4

success

gpu-production-spot-2-01

2021-04-09 13:09:51+00:00

2021-04-09 13:24:30+00:00

0:14:39

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	26.15037315032919
survival_time_median	59.99999999999873
deviation-center-line_median	3.94793933381943
in-drivable-lane_median	2.8999999999999453

other stats

agent_compute-ego0_max	0.022348838285244473
agent_compute-ego0_mean	0.022348838285244473
agent_compute-ego0_median	0.022348838285244473
agent_compute-ego0_min	0.022348838285244473
complete-iteration_max	0.3162781237364808
complete-iteration_mean	0.3162781237364808
complete-iteration_median	0.3162781237364808
complete-iteration_min	0.3162781237364808
deviation-center-line_max	3.94793933381943
deviation-center-line_mean	3.94793933381943
deviation-center-line_min	3.94793933381943
deviation-heading_max	10.07033777531635
deviation-heading_mean	10.07033777531635
deviation-heading_median	10.07033777531635
deviation-heading_min	10.07033777531635
driven_any_max	27.80948031289369
driven_any_mean	27.80948031289369
driven_any_median	27.80948031289369
driven_any_min	27.80948031289369
driven_lanedir_consec_max	26.15037315032919
driven_lanedir_consec_mean	26.15037315032919
driven_lanedir_consec_min	26.15037315032919
driven_lanedir_max	26.15037315032919
driven_lanedir_mean	26.15037315032919
driven_lanedir_median	26.15037315032919
driven_lanedir_min	26.15037315032919
get_duckie_state_max	1.39080813087889e-06
get_duckie_state_mean	1.39080813087889e-06
get_duckie_state_median	1.39080813087889e-06
get_duckie_state_min	1.39080813087889e-06
get_robot_state_max	0.004268216650055211
get_robot_state_mean	0.004268216650055211
get_robot_state_median	0.004268216650055211
get_robot_state_min	0.004268216650055211
get_state_dump_max	0.005257742093266496
get_state_dump_mean	0.005257742093266496
get_state_dump_median	0.005257742093266496
get_state_dump_min	0.005257742093266496
get_ui_image_max	0.0410284561281101
get_ui_image_mean	0.0410284561281101
get_ui_image_median	0.0410284561281101
get_ui_image_min	0.0410284561281101
in-drivable-lane_max	2.8999999999999453
in-drivable-lane_mean	2.8999999999999453
in-drivable-lane_min	2.8999999999999453
per-episodes	details {"LF-norm-zigzag-000-ego0": {"driven_any": 27.80948031289369, "get_ui_image": 0.0410284561281101, "step_physics": 0.21850506907994305, "survival_time": 59.99999999999873, "driven_lanedir": 26.15037315032919, "get_state_dump": 0.005257742093266496, "get_robot_state": 0.004268216650055211, "sim_render-ego0": 0.0043129234091626115, "get_duckie_state": 1.39080813087889e-06, "in-drivable-lane": 2.8999999999999453, "deviation-heading": 10.07033777531635, "agent_compute-ego0": 0.022348838285244473, "complete-iteration": 0.3162781237364808, "set_robot_commands": 0.002696837910406794, "deviation-center-line": 3.94793933381943, "driven_lanedir_consec": 26.15037315032919, "sim_compute_sim_state": 0.015409340568625064, "sim_compute_performance-ego0": 0.0023550848281949288}}
set_robot_commands_max	0.002696837910406794
set_robot_commands_mean	0.002696837910406794
set_robot_commands_median	0.002696837910406794
set_robot_commands_min	0.002696837910406794
sim_compute_performance-ego0_max	0.0023550848281949288
sim_compute_performance-ego0_mean	0.0023550848281949288
sim_compute_performance-ego0_median	0.0023550848281949288
sim_compute_performance-ego0_min	0.0023550848281949288
sim_compute_sim_state_max	0.015409340568625064
sim_compute_sim_state_mean	0.015409340568625064
sim_compute_sim_state_median	0.015409340568625064
sim_compute_sim_state_min	0.015409340568625064
sim_render-ego0_max	0.0043129234091626115
sim_render-ego0_mean	0.0043129234091626115
sim_render-ego0_median	0.0043129234091626115
sim_render-ego0_min	0.0043129234091626115
simulation-passed	1
step_physics_max	0.21850506907994305
step_physics_mean	0.21850506907994305
step_physics_median	0.21850506907994305
step_physics_min	0.21850506907994305
survival_time_max	59.99999999999873
survival_time_mean	59.99999999999873
survival_time_min	59.99999999999873

No reset possible

ID	5099
evaluator	`gpu-production-spot-2-01`
owner	I don't have one 😀
machine	gpu-production-spot-2_c6d12c133877
process	gpu-production-spot-2-01_c6d12c133877
version	6.2.7
first heard	2021-04-09 13:09:18+00:00
last heard	2021-04-10 15:33:00+00:00
status	inactive
# evaluating
# success	9 `65891`
# timeout
# failed	2 `66084`
# error
# aborted	1 `66328`
# host-error
arm	0
x86_64	1
Mac	0
gpu available	1
Number of processors	64
Processor frequency (MHz)	0.0 GHz
Free % of processors	100%
RAM total (MB)	249.0 GB
RAM free (MB)	238.0 GB
Disk (MB)	969.3 GB
Disk available (MB)	866.7 GB
Docker Hub
P1	1
P2
Cloud simulations	1
PI Camera	0
# Duckiebots	0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator 5099

Evaluator jobs