Duckietown Challenges Home Challenges Submissions

Submission 11773

Submission11773
Competingyes
Challengeaido5-LF-sim-validation
UserDishank Bansal 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 53770
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

53770

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
53770LFv-simsuccessyes0:33:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.812007175230265
survival_time_median59.99999999999873
deviation-center-line_median3.414571244404917
in-drivable-lane_median9.79999999999982


other stats
agent_compute-ego0_max0.012190167207895598
agent_compute-ego0_mean0.012059014492883331
agent_compute-ego0_median0.01207037541788881
agent_compute-ego0_min0.011905139927860103
complete-iteration_max0.2183600194705939
complete-iteration_mean0.19309426043997463
complete-iteration_median0.19288528739760857
complete-iteration_min0.16824644749408757
deviation-center-line_max3.797294228634931
deviation-center-line_mean2.9488748890722816
deviation-center-line_min1.1690628388443616
deviation-heading_max13.768216731277525
deviation-heading_mean11.056954665001427
deviation-heading_median12.589048547622568
deviation-heading_min5.281504833483047
driven_any_max19.065374490549843
driven_any_mean15.307978545220292
driven_any_median16.111530881961205
driven_any_min9.943477926408915
driven_lanedir_consec_max14.335309814908632
driven_lanedir_consec_mean8.785230432686893
driven_lanedir_consec_min3.181597565378407
driven_lanedir_max16.108051118285758
driven_lanedir_mean12.501586859263508
driven_lanedir_median14.360903859684148
driven_lanedir_min5.17648859939998
get_duckie_state_max1.5208365815962283e-06
get_duckie_state_mean1.4265704861860403e-06
get_duckie_state_median1.4332501422106104e-06
get_duckie_state_min1.3189450787267122e-06
get_robot_state_max0.0038335347155746473
get_robot_state_mean0.0037505512715074016
get_robot_state_median0.003772819270500037
get_robot_state_min0.003623031829454884
get_state_dump_max0.004986336388059897
get_state_dump_mean0.004825032730148309
get_state_dump_median0.004856271708041405
get_state_dump_min0.004601251116450529
get_ui_image_max0.034948116207715145
get_ui_image_mean0.03051160675930361
get_ui_image_median0.030160034129661287
get_ui_image_min0.02677824257017671
in-drivable-lane_max21.450000000000124
in-drivable-lane_mean11.46249999999992
in-drivable-lane_min4.799999999999914
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 19.065374490549843, "get_ui_image": 0.02914577479366458, "step_physics": 0.1180557577338048, "survival_time": 59.99999999999873, "driven_lanedir": 16.108051118285758, "get_state_dump": 0.004904864729691504, "get_robot_state": 0.0038335347155746473, "sim_render-ego0": 0.003898129276590085, "get_duckie_state": 1.5208365815962283e-06, "in-drivable-lane": 11.99999999999985, "deviation-heading": 11.543387600849607, "agent_compute-ego0": 0.01208431297893032, "complete-iteration": 0.18622203849932237, "set_robot_commands": 0.002302022897432885, "deviation-center-line": 3.1909949427980493, "driven_lanedir_consec": 14.335309814908632, "sim_compute_sim_state": 0.009847030155267644, "sim_compute_performance-ego0": 0.0020640875080245222}, "LF-norm-zigzag-000-ego0": {"driven_any": 9.943477926408915, "get_ui_image": 0.034948116207715145, "step_physics": 0.1437730978734745, "survival_time": 40.199999999999854, "driven_lanedir": 5.17648859939998, "get_state_dump": 0.004601251116450529, "get_robot_state": 0.003623031829454884, "sim_render-ego0": 0.003776024291233987, "get_duckie_state": 1.3538028882897418e-06, "in-drivable-lane": 21.450000000000124, "deviation-heading": 5.281504833483047, "agent_compute-ego0": 0.012190167207895598, "complete-iteration": 0.2183600194705939, "set_robot_commands": 0.0022102373727360125, "deviation-center-line": 1.1690628388443616, "driven_lanedir_consec": 3.181597565378407, "sim_compute_sim_state": 0.0111733457316523, "sim_compute_performance-ego0": 0.001982124695866745}, "LF-norm-techtrack-000-ego0": {"driven_any": 15.134588094714148, "get_ui_image": 0.031174293465657994, "step_physics": 0.1273921006525883, "survival_time": 59.99999999999873, "driven_lanedir": 13.60159218399965, "get_state_dump": 0.004807678686391305, "get_robot_state": 0.003719341149437338, "sim_render-ego0": 0.003871043655497149, "get_duckie_state": 1.512697396131479e-06, "in-drivable-lane": 4.799999999999914, "deviation-heading": 13.634709494395526, "agent_compute-ego0": 0.0120564378568473, "complete-iteration": 0.1995485362958948, "set_robot_commands": 0.002218110078975223, "deviation-center-line": 3.638147546011785, "driven_lanedir_consec": 13.60159218399965, "sim_compute_sim_state": 0.012152093137729972, "sim_compute_performance-ego0": 0.0020656283947947025}, "LF-norm-small_loop-000-ego0": {"driven_any": 17.088473669208263, "get_ui_image": 0.02677824257017671, "step_physics": 0.1060813271334328, "survival_time": 59.99999999999873, "driven_lanedir": 15.120215535368644, "get_state_dump": 0.004986336388059897, "get_robot_state": 0.003826297391562736, "sim_render-ego0": 0.003892746495763825, "get_duckie_state": 1.3189450787267122e-06, "in-drivable-lane": 7.599999999999788, "deviation-heading": 13.768216731277525, "agent_compute-ego0": 0.011905139927860103, "complete-iteration": 0.16824644749408757, "set_robot_commands": 0.002335986328760254, "deviation-center-line": 3.797294228634931, "driven_lanedir_consec": 4.02242216646088, "sim_compute_sim_state": 0.006324983854079425, "sim_compute_performance-ego0": 0.002029871960464465}}
set_robot_commands_max0.002335986328760254
set_robot_commands_mean0.0022665891694760936
set_robot_commands_median0.0022600664882040543
set_robot_commands_min0.0022102373727360125
sim_compute_performance-ego0_max0.0020656283947947025
sim_compute_performance-ego0_mean0.0020354281397876083
sim_compute_performance-ego0_median0.0020469797342444933
sim_compute_performance-ego0_min0.001982124695866745
sim_compute_sim_state_max0.012152093137729972
sim_compute_sim_state_mean0.009874363219682335
sim_compute_sim_state_median0.010510187943459972
sim_compute_sim_state_min0.006324983854079425
sim_render-ego0_max0.003898129276590085
sim_render-ego0_mean0.003859485929771261
sim_render-ego0_median0.003881895075630487
sim_render-ego0_min0.003776024291233987
simulation-passed1
step_physics_max0.1437730978734745
step_physics_mean0.12382557084832511
step_physics_median0.12272392919319657
step_physics_min0.1060813271334328
survival_time_max59.99999999999873
survival_time_mean55.04999999999901
survival_time_min40.199999999999854
No reset possible
53766LFv-simerroryes0:09:53
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 577, in run_episode
    await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 331, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "step".
                   |
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 73, in belongs
                   ||     self.SOn.belongs(R)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_orthogonal_group.py", line 39, in belongs
                   ||     assert_allclose(det, 1, err_msg="I expect the determinant to be +1.")
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
                   ||     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 764, in assert_array_compare
                   ||     flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 740, in func_assert_same_pos
                   ||     raise AssertionError(msg)
                   || AssertionError:
                   || Not equal to tolerance rtol=1e-07, atol=0
                   || I expect the determinant to be +1.
                   || x and y nan location mismatch:
                   ||  x: array(nan)
                   ||  y: array(1)
                   ||
                   || During handling of the above exception, another exception occurred:
                   ||
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 355, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 531, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 510, in on_received_step
                   ||     self.update_physics_and_observations(until=data.until, context=context)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 560, in update_physics_and_observations
                   ||     pc.integrate(delta_time)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 217, in integrate
                   ||     self.state = self.state.integrate(dt, self.last_commands.wheels)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/dynamics_delay.py", line 74, in integrate
                   ||     state2 = self.state.integrate(dt, use_commands)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/pwm_dynamics.py", line 199, in integrate
                   ||     s1 = GenericKinematicsSE2.integrate(self, dt, commands_se2)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 48, in integrate
                   ||     return GenericKinematicsSE2(c1, t1)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 26, in __init__
                   ||     geo.SE2.belongs(q0)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 76, in belongs
                   ||     raise ValueError(msg)
                   || ValueError: The rotation is not a rotation:
                   || [[nan nan]
                   ||  [nan nan]]
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible