Duckietown Challenges Home Challenges Submissions

Submission 5267

Submission5267
Competingyes
Challengeaido3-LF-sim-validation
UserCharles Cossette 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 28239
Next
User labelchallenge-aido_LF-baseline-duckietown
Admin priority50
Blessingn/a
User priority50

28239

Click the images to see detailed statistics about the episode.

ETHZ_autolab_technical_track-0-0

ETHZ_autolab_technical_track-1-0

ETHZ_autolab_technical_track-2-0

ETHZ_autolab_technical_track-3-0

ETHZ_autolab_technical_track-4-0

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
28239step1-simulationsuccessyes0:03:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6781823981374886
survival_time_median2.1000000000000005
deviation-center-line_median0.1057618641131284
in-drivable-lane_median0.20000000000000015


other stats
agent_compute-ego_max0.0530880159801907
agent_compute-ego_mean0.0472941079693888
agent_compute-ego_median0.0522722314144003
agent_compute-ego_min0.032399393894054274
deviation-center-line_max0.2996044847699386
deviation-center-line_mean0.14544209294706784
deviation-center-line_min0.08615744033137747
deviation-heading_max0.9300041455452376
deviation-heading_mean0.4737896961707406
deviation-heading_median0.3283074399476135
deviation-heading_min0.21099526121704065
driven_any_max2.879620882904639
driven_any_mean1.2194159779442582
driven_any_median0.9254938960651375
driven_any_min0.35683861678067713
driven_lanedir_consec_max2.444034659200923
driven_lanedir_consec_mean1.008127310702766
driven_lanedir_consec_min0.3503817471664201
driven_lanedir_max2.444034659200923
driven_lanedir_mean1.008127310702766
driven_lanedir_median0.6781823981374886
driven_lanedir_min0.3503817471664201
in-drivable-lane_max0.8499999999999974
in-drivable-lane_mean0.359999999999999
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.1569592723199114, "sim_physics": 0.10433738761478, "survival_time": 2.6999999999999984, "driven_lanedir": 0.6781823981374886, "sim_render-ego": 0.01179867320590549, "in-drivable-lane": 0.8499999999999974, "agent_compute-ego": 0.032399393894054274, "deviation-heading": 0.6769234998237628, "set_robot_commands": 0.012366237463774504, "deviation-center-line": 0.10460827804906989, "driven_lanedir_consec": 0.6781823981374886, "sim_compute_sim_state": 0.005740642547607422, "sim_compute_performance-ego": 0.007442889390168367, "sim_compute_robot_state-ego": 0.008733864183779116}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.35683861678067713, "sim_physics": 0.09027502536773682, "survival_time": 1.0000000000000002, "driven_lanedir": 0.3503817471664201, "sim_render-ego": 0.009754753112792967, "in-drivable-lane": 0, "agent_compute-ego": 0.046026432514190675, "deviation-heading": 0.21099526121704065, "set_robot_commands": 0.00896517038345337, "deviation-center-line": 0.08615744033137747, "driven_lanedir_consec": 0.3503817471664201, "sim_compute_sim_state": 0.0043465614318847655, "sim_compute_performance-ego": 0.006066286563873291, "sim_compute_robot_state-ego": 0.006433796882629394}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.7781672216509258, "sim_physics": 0.08655565314822727, "survival_time": 1.800000000000001, "driven_lanedir": 0.6489707885656197, "sim_render-ego": 0.014470040798187256, "in-drivable-lane": 0.20000000000000015, "agent_compute-ego": 0.0530880159801907, "deviation-heading": 0.3283074399476135, "set_robot_commands": 0.008852369255489774, "deviation-center-line": 0.1057618641131284, "driven_lanedir_consec": 0.6489707885656197, "sim_compute_sim_state": 0.004740019639333089, "sim_compute_performance-ego": 0.005541967021094428, "sim_compute_robot_state-ego": 0.007246183024512397}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.9254938960651375, "sim_physics": 0.07664239406585693, "survival_time": 2.1000000000000005, "driven_lanedir": 0.9190669604433784, "sim_render-ego": 0.0089242004212879, "in-drivable-lane": 0, "agent_compute-ego": 0.05268446604410807, "deviation-heading": 0.22271813432004875, "set_robot_commands": 0.007480388595944359, "deviation-center-line": 0.13107839747182476, "driven_lanedir_consec": 0.9190669604433784, "sim_compute_sim_state": 0.003947104726518903, "sim_compute_performance-ego": 0.005020243780953544, "sim_compute_robot_state-ego": 0.006307533809116909}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.879620882904639, "sim_physics": 0.0763874259488336, "survival_time": 5.799999999999987, "driven_lanedir": 2.444034659200923, "sim_render-ego": 0.009191954958027809, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.0522722314144003, "deviation-heading": 0.9300041455452376, "set_robot_commands": 0.008227611410206762, "deviation-center-line": 0.2996044847699386, "driven_lanedir_consec": 2.444034659200923, "sim_compute_sim_state": 0.004423297684768151, "sim_compute_performance-ego": 0.005182950661100191, "sim_compute_robot_state-ego": 0.0065095897378592655}}
set_robot_commands_max0.012366237463774504
set_robot_commands_mean0.009178355421773754
set_robot_commands_median0.008852369255489774
set_robot_commands_min0.007480388595944359
sim_compute_performance-ego_max0.007442889390168367
sim_compute_performance-ego_mean0.005850867483437964
sim_compute_performance-ego_median0.005541967021094428
sim_compute_performance-ego_min0.005020243780953544
sim_compute_robot_state-ego_max0.008733864183779116
sim_compute_robot_state-ego_mean0.007046193527579415
sim_compute_robot_state-ego_median0.0065095897378592655
sim_compute_robot_state-ego_min0.006307533809116909
sim_compute_sim_state_max0.005740642547607422
sim_compute_sim_state_mean0.004639525206022467
sim_compute_sim_state_median0.004423297684768151
sim_compute_sim_state_min0.003947104726518903
sim_physics_max0.10433738761478
sim_physics_mean0.08683957722908693
sim_physics_median0.08655565314822727
sim_physics_min0.0763874259488336
sim_render-ego_max0.014470040798187256
sim_render-ego_mean0.010827924499240284
sim_render-ego_median0.009754753112792967
sim_render-ego_min0.0089242004212879
simulation-passed1
survival_time_max5.799999999999987
survival_time_mean2.6799999999999975
survival_time_min1.0000000000000002
No reset possible
28238step1-simulationsuccessyes0:05:23
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
28237step1-simulationfailedyes0:03:47
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 296, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 301, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 239, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 237 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 676, in scoring_context
    yield cie
  File "experiment_manager.py", line 446, in <module>
    wrap(cie)
  File "experiment_manager.py", line 434, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 138, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 305, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible