Duckietown Challenges Home Challenges Submissions

Submission 3344

Submission3344
Competingyes
Challengeaido2-LFV-sim-validation
UserAnastasiya Nikolskaya 🇷🇺
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 22388
Next
User labelJetBrains Research
Admin priority50
Blessingn/a
User priority50

22388

Click the images to see detailed statistics about the episode.

ETHZ_autolab_technical_track-0-0

ETHZ_autolab_technical_track-1-0

ETHZ_autolab_technical_track-2-0

ETHZ_autolab_technical_track-3-0

ETHZ_autolab_technical_track-4-0

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
22388step1-simulationsuccessyes0:41:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median2.5528718052464026
survival_time_median14.950000000000076
deviation-center-line_median1.0013510186572738
in-drivable-lane_median0


other stats
agent_compute-ego_max0.10190905253092448
agent_compute-ego_mean0.09873348135727746
agent_compute-ego_median0.09922749891768406
agent_compute-ego_min0.09561718384424846
deviation-center-line_max1.2083534012513113
deviation-center-line_mean0.9837684505486972
deviation-center-line_min0.7793499477662331
deviation-heading_max2.6934656062975613
deviation-heading_mean2.0715631450970093
deviation-heading_median1.8763653376599545
deviation-heading_min1.8396032412496717
driven_any_max2.846374679535044
driven_any_mean2.615106079673468
driven_any_median2.592767586378671
driven_any_min2.4471188628876672
driven_lanedir_consec_max2.641913136272697
driven_lanedir_consec_mean2.497226719119475
driven_lanedir_consec_min2.335429427736802
driven_lanedir_max2.641913136272697
driven_lanedir_mean2.497226719119475
driven_lanedir_median2.5528718052464026
driven_lanedir_min2.335429427736802
in-drivable-lane_max1.9500000000000275
in-drivable-lane_mean0.6700000000000095
in-drivable-lane_min0
per-episodes
details{"ETHZ_autolab_technical_track-0-0": {"driven_any": 2.519306794706149, "sim_physics": 0.21668468554814657, "survival_time": 14.950000000000076, "driven_lanedir": 2.335429427736802, "sim_render-ego": 0.06614806572596232, "in-drivable-lane": 1.40000000000002, "agent_compute-ego": 0.10057286580403646, "deviation-heading": 2.0920169163572933, "set_robot_commands": 0.12371594587961832, "deviation-center-line": 0.8675375942880417, "driven_lanedir_consec": 2.335429427736802, "sim_compute_sim_state": 0.04312630335489909, "sim_compute_performance-ego": 0.07451350450515747, "sim_compute_robot_state-ego": 0.08328669548034667, "sim_compute_robot_state-npc0": 0.0824186642964681, "sim_compute_robot_state-npc1": 0.07939558823903402, "sim_compute_robot_state-npc2": 0.07484789609909058, "sim_compute_robot_state-npc3": 0.07628479560216268}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.846374679535044, "sim_physics": 0.22288448492685953, "survival_time": 14.950000000000076, "driven_lanedir": 2.5528718052464026, "sim_render-ego": 0.07209946632385254, "in-drivable-lane": 1.9500000000000275, "agent_compute-ego": 0.10190905253092448, "deviation-heading": 1.8396032412496717, "set_robot_commands": 0.13091478983561197, "deviation-center-line": 1.2083534012513113, "driven_lanedir_consec": 2.5528718052464026, "sim_compute_sim_state": 0.04580481290817261, "sim_compute_performance-ego": 0.08058137178421021, "sim_compute_robot_state-ego": 0.09226917346318562, "sim_compute_robot_state-npc0": 0.08640130043029785, "sim_compute_robot_state-npc1": 0.08363185326258342, "sim_compute_robot_state-npc2": 0.08087248961130777, "sim_compute_robot_state-npc3": 0.08132250467936197}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.592767586378671, "sim_physics": 0.21631168762842817, "survival_time": 14.950000000000076, "driven_lanedir": 2.5679472246138033, "sim_render-ego": 0.07132217327753702, "in-drivable-lane": 0, "agent_compute-ego": 0.0963408056894938, "deviation-heading": 1.8563646239205664, "set_robot_commands": 0.12679818630218506, "deviation-center-line": 1.0013510186572738, "driven_lanedir_consec": 2.5679472246138033, "sim_compute_sim_state": 0.04451848109563192, "sim_compute_performance-ego": 0.08086414178212484, "sim_compute_robot_state-ego": 0.08719781398773194, "sim_compute_robot_state-npc0": 0.08691550652186075, "sim_compute_robot_state-npc1": 0.0823401133219401, "sim_compute_robot_state-npc2": 0.07960812012354533, "sim_compute_robot_state-npc3": 0.07937823216120402}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 2.669962474859809, "sim_physics": 0.20524457772572835, "survival_time": 14.950000000000076, "driven_lanedir": 2.641913136272697, "sim_render-ego": 0.06998920917510987, "in-drivable-lane": 0, "agent_compute-ego": 0.09561718384424846, "deviation-heading": 1.8763653376599545, "set_robot_commands": 0.12379836241404216, "deviation-center-line": 1.0622502907806248, "driven_lanedir_consec": 2.641913136272697, "sim_compute_sim_state": 0.044194118181864424, "sim_compute_performance-ego": 0.07810549815495808, "sim_compute_robot_state-ego": 0.08523449977238973, "sim_compute_robot_state-npc0": 0.0839932107925415, "sim_compute_robot_state-npc1": 0.08120386838912964, "sim_compute_robot_state-npc2": 0.07836428721745808, "sim_compute_robot_state-npc3": 0.07845160563786825}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.4471188628876672, "sim_physics": 0.18992267994985093, "survival_time": 13.70000000000006, "driven_lanedir": 2.38797200172767, "sim_render-ego": 0.06915012415308151, "in-drivable-lane": 0, "agent_compute-ego": 0.09922749891768406, "deviation-heading": 2.6934656062975613, "set_robot_commands": 0.12304565245217652, "deviation-center-line": 0.7793499477662331, "driven_lanedir_consec": 2.38797200172767, "sim_compute_sim_state": 0.0432339523830553, "sim_compute_performance-ego": 0.07635629090079427, "sim_compute_robot_state-ego": 0.08543991172400704, "sim_compute_robot_state-npc0": 0.08324357801980346, "sim_compute_robot_state-npc1": 0.08204155159692694, "sim_compute_robot_state-npc2": 0.07886221112996122, "sim_compute_robot_state-npc3": 0.07927904076819872}}
set_robot_commands_max0.13091478983561197
set_robot_commands_mean0.1256545873767268
set_robot_commands_median0.12379836241404216
set_robot_commands_min0.12304565245217652
sim_compute_performance-ego_max0.08086414178212484
sim_compute_performance-ego_mean0.07808416142544897
sim_compute_performance-ego_median0.07810549815495808
sim_compute_performance-ego_min0.07451350450515747
sim_compute_robot_state-ego_max0.09226917346318562
sim_compute_robot_state-ego_mean0.0866856188855322
sim_compute_robot_state-ego_median0.08543991172400704
sim_compute_robot_state-ego_min0.08328669548034667
sim_compute_robot_state-npc0_max0.08691550652186075
sim_compute_robot_state-npc0_mean0.08459445201219434
sim_compute_robot_state-npc0_median0.0839932107925415
sim_compute_robot_state-npc0_min0.0824186642964681
sim_compute_robot_state-npc1_max0.08363185326258342
sim_compute_robot_state-npc1_mean0.08172259496192283
sim_compute_robot_state-npc1_median0.08204155159692694
sim_compute_robot_state-npc1_min0.07939558823903402
sim_compute_robot_state-npc2_max0.08087248961130777
sim_compute_robot_state-npc2_mean0.0785110008362726
sim_compute_robot_state-npc2_median0.07886221112996122
sim_compute_robot_state-npc2_min0.07484789609909058
sim_compute_robot_state-npc3_max0.08132250467936197
sim_compute_robot_state-npc3_mean0.07894323576975912
sim_compute_robot_state-npc3_median0.07927904076819872
sim_compute_robot_state-npc3_min0.07628479560216268
sim_compute_sim_state_max0.04580481290817261
sim_compute_sim_state_mean0.04417553358472466
sim_compute_sim_state_median0.044194118181864424
sim_compute_sim_state_min0.04312630335489909
sim_physics_max0.22288448492685953
sim_physics_mean0.2102096231558027
sim_physics_median0.21631168762842817
sim_physics_min0.18992267994985093
sim_render-ego_max0.07209946632385254
sim_render-ego_mean0.06974180773110866
sim_render-ego_median0.06998920917510987
sim_render-ego_min0.06614806572596232
simulation-passed1
survival_time_max14.950000000000076
survival_time_mean14.700000000000074
survival_time_min13.70000000000006
No reset possible
22386step1-simulationerroryes0:01:21
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "experiment_manager.py", line 87, in main
    sim_ci.write_topic_and_expect_zero('seed', config.seed)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
    wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
    wait_for_data(f, timeout, waiting_for)
  File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
    raise TimeoutError(msg)
TimeoutError: Timeout after 30.0 s.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 441, in <module>
    wrap(cie)
  File "experiment_manager.py", line 429, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 186, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:

 > Traceback (most recent call last):
 >   File "experiment_manager.py", line 87, in main
 >     sim_ci.write_topic_and_expect_zero('seed', config.seed)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero
 >     nickname=self.nickname)
 >   File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 247, in read_reply
 >     wm: WireMessage = read_next_cbor(fpout, timeout=timeout, waiting_for=waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 161, in read_next_cbor
 >     wait_for_data(f, timeout, waiting_for)
 >   File "/project/src/zuper-utils/src/zuper_json/json2cbor.py", line 146, in wait_for_data
 >     raise TimeoutError(msg)
 > TimeoutError: Timeout after 30.0 s.
 >
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21904step1-simulationsuccessno0:17:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
21898step1-simulationhost-errorno0:25:15
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 488, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 347, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in convert
    with open(fn, 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-validation/submission3344/step1-simulation-ip-172-31-42-167-7357-job21898/logs/challenges-runner/stdout.html'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible