Submission 24242

Submission	`24242`

Competing	yes
Challenge	`lx22-planning-cc-dynamic-vali`
User	user7823
Date submitted	2023-11-28 21:37:08+00:00
Last status update	2023-11-29 05:32:42+00:00
Complete	complete
Details	Evaluation is complete.
Sisters
Result	💜
Jobs	step1-0of4: `172463` step1-1of4: `172466` step1-2of4: `170512` step1-3of4: `172490`
Next
User label	planning-exercise
Admin priority	50
Blessing	n/a
User priority	50

Evaluation jobs for this submission

See previous jobs for previous versions of challenges

Job ID step status up to date date started date completed duration message

172490

step1-3of4

success

yes

2023-11-29 05:29:24+00:00

2023-11-29 05:29:51+00:00

0:00:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

mistakes	0.0
success_ratio	0.0
duration	1000.0
complexity	1000.0
avg_min_distance	0.0

other stats

nfeasible	0
nsamples	0

No reset possible

172485

step1-3of4

success

yes

2023-11-29 05:28:16+00:00

2023-11-29 05:29:05+00:00

0:00:49

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172476

step1-3of4

success

yes

2023-11-29 05:27:10+00:00

2023-11-29 05:27:37+00:00

0:00:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172475

step1-3of4

success

yes

2023-11-29 05:26:33+00:00

2023-11-29 05:27:00+00:00

0:00:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172468

step1-3of4

success

yes

2023-11-29 05:25:28+00:00

2023-11-29 05:26:09+00:00

0:00:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172466

step1-1of4

success

yes

2023-11-29 05:25:08+00:00

2023-11-29 05:25:51+00:00

0:00:43

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

mistakes	0.0
success_ratio	0.0
duration	1000.0
complexity	1000.0
avg_min_distance	0.0

other stats

nfeasible	0
nsamples	0

No reset possible

172463

step1-0of4

error

yes

2023-11-29 05:25:03+00:00

2023-11-29 05:25:37+00:00

0:00:34

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 104, in run_checker
    scores.append(scoring(inside.params, q, r, response))
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 45, in scoring
    s = get_stats(g, pq, b, dt=EVAL_DT, collision_checker=get_ground_truth2)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/statistics.py", line 77, in get_stats
    obstacles = move_environment(ps, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 87, in move_environment
    traj = get_trajectory(pp.pose, pp.motion.steps, pp.motion.periodic, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 127, in get_trajectory
    assert_allclose(errors, 0.0, atol=1e-8)
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-08

Mismatched elements: 6 / 221 (2.71%)
Max absolute difference: 2.45839029e-06
Max relative difference: inf
 x: array([0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,...
 y: array(0.)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 102, in planning_checker_main
    sessions = run_checker(
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 119, in run_checker
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172454

step1-0of4

error

yes

2023-11-29 05:23:41+00:00

2023-11-29 05:24:17+00:00

0:00:36

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 104, in run_checker
    scores.append(scoring(inside.params, q, r, response))
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 45, in scoring
    s = get_stats(g, pq, b, dt=EVAL_DT, collision_checker=get_ground_truth2)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/statistics.py", line 77, in get_stats
    obstacles = move_environment(ps, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 87, in move_environment
    traj = get_trajectory(pp.pose, pp.motion.steps, pp.motion.periodic, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 127, in get_trajectory
    assert_allclose(errors, 0.0, atol=1e-8)
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-08

Mismatched elements: 6 / 221 (2.71%)
Max absolute difference: 2.45839029e-06
Max relative difference: inf
 x: array([0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,...
 y: array(0.)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 102, in planning_checker_main
    sessions = run_checker(
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 119, in run_checker
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172452

step1-1of4

success

yes

2023-11-29 05:23:13+00:00

2023-11-29 05:24:27+00:00

0:01:14

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172446

step1-0of4

error

yes

2023-11-29 05:22:12+00:00

2023-11-29 05:22:57+00:00

0:00:45

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 104, in run_checker
    scores.append(scoring(inside.params, q, r, response))
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 45, in scoring
    s = get_stats(g, pq, b, dt=EVAL_DT, collision_checker=get_ground_truth2)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/statistics.py", line 77, in get_stats
    obstacles = move_environment(ps, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 87, in move_environment
    traj = get_trajectory(pp.pose, pp.motion.steps, pp.motion.periodic, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 127, in get_trajectory
    assert_allclose(errors, 0.0, atol=1e-8)
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-08

Mismatched elements: 6 / 221 (2.71%)
Max absolute difference: 2.45839029e-06
Max relative difference: inf
 x: array([0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,
       0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00, 0.00000e+00,...
 y: array(0.)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 102, in planning_checker_main
    sessions = run_checker(
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 119, in run_checker
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

170512

step1-2of4

success

yes

2023-11-28 21:51:43+00:00

2023-11-28 21:52:09+00:00

0:00:26

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

mistakes	0.0
success_ratio	0.0
duration	1000.0
complexity	1000.0
avg_min_distance	0.0

other stats

nfeasible	0
nsamples	0

No reset possible

170510

step1-2of4

success

yes

2023-11-28 21:50:16+00:00

2023-11-28 21:50:42+00:00

0:00:26

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

170506

step1-2of4

success

yes

2023-11-28 21:49:14+00:00

2023-11-28 21:49:40+00:00

0:00:26

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible