Duckietown Challenges Home Challenges Submissions

Submission 11695

Submission11695
Competingyes
Challengeaido5-LF-sim-validation
UserDishank Bansal 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFv-sim: 54008
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

54008

Click the images to see detailed statistics about the episode.

LF-norm-loop-000

LF-norm-small_loop-000

LF-norm-techtrack-000

LF-norm-zigzag-000

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
54008LFv-simsuccessyes0:36:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median11.137365838211297
survival_time_median59.99999999999873
deviation-center-line_median4.331118304308271
in-drivable-lane_median2.37499999999992


other stats
agent_compute-ego0_max0.01249874243629068
agent_compute-ego0_mean0.011933104815782986
agent_compute-ego0_median0.0118030324298955
agent_compute-ego0_min0.011627611967050266
complete-iteration_max0.24499258907708796
complete-iteration_mean0.21581257871022935
complete-iteration_median0.21424613899183725
complete-iteration_min0.18976544778015492
deviation-center-line_max4.597575472640299
deviation-center-line_mean3.8877282875522217
deviation-center-line_min2.291101068952043
deviation-heading_max15.680038228801264
deviation-heading_mean13.836557267465157
deviation-heading_median14.741442141849788
deviation-heading_min10.18330655735978
driven_any_max19.128553959479436
driven_any_mean14.994445450435364
driven_any_median15.42787651628182
driven_any_min9.99347480969838
driven_lanedir_consec_max18.01922925970912
driven_lanedir_consec_mean12.140785762180425
driven_lanedir_consec_min8.269182112589986
driven_lanedir_max18.01922925970912
driven_lanedir_mean13.541567088558702
driven_lanedir_median13.932575422103303
driven_lanedir_min8.281888250319078
get_duckie_state_max1.3713925487087548e-06
get_duckie_state_mean1.3399479857886569e-06
get_duckie_state_median1.3364145499681256e-06
get_duckie_state_min1.315570294509621e-06
get_robot_state_max0.004096133822108387
get_robot_state_mean0.003840003894512024
get_robot_state_median0.0037699446491555903
get_robot_state_min0.003723992457628528
get_state_dump_max0.005063038483745947
get_state_dump_mean0.004833537902626295
get_state_dump_median0.004805185577781525
get_state_dump_min0.004660741971196184
get_ui_image_max0.035380577068344736
get_ui_image_mean0.030916276325240737
get_ui_image_median0.030812134949194204
get_ui_image_min0.026660258334229796
in-drivable-lane_max6.2999999999997165
in-drivable-lane_mean2.937499999999879
in-drivable-lane_min0.6999999999999602
per-episodes
details{"LF-norm-loop-000-ego0": {"driven_any": 19.128553959479436, "get_ui_image": 0.02803537887300083, "step_physics": 0.12376556388543708, "survival_time": 59.99999999999873, "driven_lanedir": 18.01922925970912, "get_state_dump": 0.0048532996149881, "get_robot_state": 0.003787783758527134, "sim_render-ego0": 0.0038538826792365206, "get_duckie_state": 1.315570294509621e-06, "in-drivable-lane": 0.849999999999973, "deviation-heading": 14.378720718155824, "agent_compute-ego0": 0.011627611967050266, "complete-iteration": 0.18976544778015492, "set_robot_commands": 0.0022854838740517954, "deviation-center-line": 4.597575472640299, "driven_lanedir_consec": 18.01922925970912, "sim_compute_sim_state": 0.009364393529645808, "sim_compute_performance-ego0": 0.002101897002259063}, "LF-norm-zigzag-000-ego0": {"driven_any": 14.856654581832473, "get_ui_image": 0.035380577068344736, "step_physics": 0.1672908912391885, "survival_time": 59.99999999999873, "driven_lanedir": 13.824512833163368, "get_state_dump": 0.004660741971196184, "get_robot_state": 0.0037521055397840464, "sim_render-ego0": 0.0038107765604316144, "get_duckie_state": 1.3250990970049374e-06, "in-drivable-lane": 0.6999999999999602, "deviation-heading": 15.680038228801264, "agent_compute-ego0": 0.01191877584274762, "complete-iteration": 0.24499258907708796, "set_robot_commands": 0.0022905988558245935, "deviation-center-line": 4.231308107450675, "driven_lanedir_consec": 13.824512833163368, "sim_compute_sim_state": 0.013707155788272349, "sim_compute_performance-ego0": 0.0020916007341294364}, "LF-norm-techtrack-000-ego0": {"driven_any": 15.999098450731168, "get_ui_image": 0.03358889102538758, "step_physics": 0.16091292644916824, "survival_time": 59.99999999999873, "driven_lanedir": 14.04063801104324, "get_state_dump": 0.005063038483745947, "get_robot_state": 0.004096133822108387, "sim_render-ego0": 0.004078816017640024, "get_duckie_state": 1.3477300029313138e-06, "in-drivable-lane": 3.899999999999867, "deviation-heading": 15.10416356554375, "agent_compute-ego0": 0.01249874243629068, "complete-iteration": 0.23791535132929845, "set_robot_commands": 0.002459636238790571, "deviation-center-line": 4.430928501165867, "driven_lanedir_consec": 8.45021884325922, "sim_compute_sim_state": 0.012840099279132116, "sim_compute_performance-ego0": 0.0022762044085551064}, "LF-norm-small_loop-000-ego0": {"driven_any": 9.99347480969838, "get_ui_image": 0.026660258334229796, "step_physics": 0.1295021801527776, "survival_time": 42.8999999999997, "driven_lanedir": 8.281888250319078, "get_state_dump": 0.00475707154057495, "get_robot_state": 0.003723992457628528, "sim_render-ego0": 0.003769671403487431, "get_duckie_state": 1.3713925487087548e-06, "in-drivable-lane": 6.2999999999997165, "deviation-heading": 10.18330655735978, "agent_compute-ego0": 0.011687289017043375, "complete-iteration": 0.19057692665437603, "set_robot_commands": 0.002260456540393052, "deviation-center-line": 2.291101068952043, "driven_lanedir_consec": 8.269182112589986, "sim_compute_sim_state": 0.006137276972547538, "sim_compute_performance-ego0": 0.0019896768718715044}}
set_robot_commands_max0.002459636238790571
set_robot_commands_mean0.002324043877265003
set_robot_commands_median0.0022880413649381945
set_robot_commands_min0.002260456540393052
sim_compute_performance-ego0_max0.0022762044085551064
sim_compute_performance-ego0_mean0.002114844754203778
sim_compute_performance-ego0_median0.00209674886819425
sim_compute_performance-ego0_min0.0019896768718715044
sim_compute_sim_state_max0.013707155788272349
sim_compute_sim_state_mean0.010512231392399452
sim_compute_sim_state_median0.01110224640438896
sim_compute_sim_state_min0.006137276972547538
sim_render-ego0_max0.004078816017640024
sim_render-ego0_mean0.0038782866651988976
sim_render-ego0_median0.003832329619834067
sim_render-ego0_min0.003769671403487431
simulation-passed1
step_physics_max0.1672908912391885
step_physics_mean0.14536789043164283
step_physics_median0.1452075533009729
step_physics_min0.12376556388543708
survival_time_max59.99999999999873
survival_time_mean55.72499999999897
survival_time_min42.8999999999997
No reset possible
54005LFv-simerroryes0:25:00
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 286, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 577, in run_episode
    await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 143, in write_topic_and_expect_zero
    msgs = read_reply(self.fpout, timeout=timeout, nickname=self.nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 304, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 331, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted:

error in simulator |Exception while handling a message on topic "step".
                   |
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 73, in belongs
                   ||     self.SOn.belongs(R)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_orthogonal_group.py", line 39, in belongs
                   ||     assert_allclose(det, 1, err_msg="I expect the determinant to be +1.")
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
                   ||     assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 764, in assert_array_compare
                   ||     flagged = func_assert_same_pos(x, y, func=isnan, hasval='nan')
                   ||   File "/usr/local/lib/python3.8/site-packages/numpy/testing/_private/utils.py", line 740, in func_assert_same_pos
                   ||     raise AssertionError(msg)
                   || AssertionError:
                   || Not equal to tolerance rtol=1e-07, atol=0
                   || I expect the determinant to be +1.
                   || x and y nan location mismatch:
                   ||  x: array(nan)
                   ||  y: array(1)
                   ||
                   || During handling of the above exception, another exception occurred:
                   ||
                   || Traceback (most recent call last):
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 355, in loop
                   ||     handle_message_node(parsed, receiver0, context0)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper.py", line 531, in handle_message_node
                   ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
                   ||   File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
                   ||     f(**kwargs)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 510, in on_received_step
                   ||     self.update_physics_and_observations(until=data.until, context=context)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 560, in update_physics_and_observations
                   ||     pc.integrate(delta_time)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_simulator_gym/code.py", line 217, in integrate
                   ||     self.state = self.state.integrate(dt, self.last_commands.wheels)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/dynamics_delay.py", line 74, in integrate
                   ||     state2 = self.state.integrate(dt, use_commands)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/pwm_dynamics.py", line 199, in integrate
                   ||     s1 = GenericKinematicsSE2.integrate(self, dt, commands_se2)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 48, in integrate
                   ||     return GenericKinematicsSE2(c1, t1)
                   ||   File "/usr/local/lib/python3.8/site-packages/duckietown_world/world_duckietown/generic_kinematics.py", line 26, in __init__
                   ||     geo.SE2.belongs(q0)
                   ||   File "/usr/local/lib/python3.8/site-packages/geometry/manifolds/special_euclidean_group.py", line 76, in belongs
                   ||     raise ValueError(msg)
                   || ValueError: The rotation is not a rotation:
                   || [[nan nan]
                   ||  [nan nan]]
                   ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 383, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible