Submission 24296

Submission	`24296`

Competing	yes
Challenge	`lx22-planning-cc-dynamic-test`
User	user7810
Date submitted	2023-11-28 22:09:31+00:00
Last status update	2023-11-29 05:35:25+00:00
Complete	complete
Details	Evaluation is complete.
Sisters
Result	💜
Jobs	step1-0of4: `172507` step1-1of4: `172516` step1-2of4: `172517` step1-3of4: `172514`
Next
User label	planning-exercise
Admin priority	50
Blessing	n/a
User priority	50

Evaluation jobs for this submission

Show only up-to-date jobs

Job ID step status up to date date started date completed duration message

172517

step1-2of4

success

yes

2023-11-29 05:33:09+00:00

2023-11-29 05:33:49+00:00

0:00:40

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

mistakes	0.0
success_ratio	0.0
duration	1000.0
complexity	1000.0
avg_min_distance	0.0

other stats

nfeasible	0
nsamples	0

No reset possible

172516

step1-1of4

success

yes

2023-11-29 05:33:04+00:00

2023-11-29 05:33:54+00:00

0:00:50

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

mistakes	0.0
success_ratio	0.0
duration	1000.0
complexity	1000.0
avg_min_distance	0.0

other stats

nfeasible	0
nsamples	0

No reset possible

172514

step1-3of4

success

yes

2023-11-29 05:33:03+00:00

2023-11-29 05:33:44+00:00

0:00:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

mistakes	0.0
success_ratio	0.0
duration	1000.0
complexity	1000.0
avg_min_distance	0.0

other stats

nfeasible	0
nsamples	0

No reset possible

172511

step1-1of4

success

yes

2023-11-29 05:32:11+00:00

2023-11-29 05:32:36+00:00

0:00:25

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172509

step1-3of4

success

yes

2023-11-29 05:32:10+00:00

2023-11-29 05:32:49+00:00

0:00:39

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172507

step1-0of4

error

yes

2023-11-29 05:32:06+00:00

2023-11-29 05:33:22+00:00

0:01:16

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 104, in run_checker
    scores.append(scoring(inside.params, q, r, response))
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 45, in scoring
    s = get_stats(g, pq, b, dt=EVAL_DT, collision_checker=get_ground_truth2)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/statistics.py", line 77, in get_stats
    obstacles = move_environment(ps, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 87, in move_environment
    traj = get_trajectory(pp.pose, pp.motion.steps, pp.motion.periodic, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 127, in get_trajectory
    assert_allclose(errors, 0.0, atol=1e-8)
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-08

Mismatched elements: 1 / 86 (1.16%)
Max absolute difference: 2.58072573e-07
Max relative difference: inf
 x: array([0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,
       0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,
       0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,...
 y: array(0.)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 102, in planning_checker_main
    sessions = run_checker(
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 119, in run_checker
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172506

step1-2of4

success

yes

2023-11-29 05:32:05+00:00

2023-11-29 05:32:54+00:00

0:00:49

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172501

step1-3of4

success

yes

2023-11-29 05:30:57+00:00

2023-11-29 05:31:38+00:00

0:00:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172500

step1-2of4

success

yes

2023-11-29 05:30:54+00:00

2023-11-29 05:31:35+00:00

0:00:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172498

step1-1of4

success

yes

2023-11-29 05:30:51+00:00

2023-11-29 05:31:33+00:00

0:00:42

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172497

step1-0of4

error

yes

2023-11-29 05:30:28+00:00

2023-11-29 05:31:43+00:00

0:01:15

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 104, in run_checker
    scores.append(scoring(inside.params, q, r, response))
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 45, in scoring
    s = get_stats(g, pq, b, dt=EVAL_DT, collision_checker=get_ground_truth2)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/statistics.py", line 77, in get_stats
    obstacles = move_environment(ps, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 87, in move_environment
    traj = get_trajectory(pp.pose, pp.motion.steps, pp.motion.periodic, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 127, in get_trajectory
    assert_allclose(errors, 0.0, atol=1e-8)
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-08

Mismatched elements: 1 / 86 (1.16%)
Max absolute difference: 2.58072573e-07
Max relative difference: inf
 x: array([0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,
       0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,
       0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,...
 y: array(0.)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 102, in planning_checker_main
    sessions = run_checker(
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 119, in run_checker
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172493

step1-3of4

success

yes

2023-11-29 05:29:29+00:00

2023-11-29 05:29:56+00:00

0:00:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172492

step1-1of4

success

yes

2023-11-29 05:29:28+00:00

2023-11-29 05:30:10+00:00

0:00:42

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172491

step1-2of4

success

yes

2023-11-29 05:29:25+00:00

2023-11-29 05:30:14+00:00

0:00:49

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172486

step1-2of4

success

yes

2023-11-29 05:28:19+00:00

2023-11-29 05:28:45+00:00

0:00:26

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172484

step1-1of4

success

yes

2023-11-29 05:28:15+00:00

2023-11-29 05:29:05+00:00

0:00:50

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172483

step1-0of4

error

yes

2023-11-29 05:28:15+00:00

2023-11-29 05:29:32+00:00

0:01:17

InvalidEvaluator: Tr [...]

InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 104, in run_checker
    scores.append(scoring(inside.params, q, r, response))
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 45, in scoring
    s = get_stats(g, pq, b, dt=EVAL_DT, collision_checker=get_ground_truth2)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/statistics.py", line 77, in get_stats
    obstacles = move_environment(ps, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 87, in move_environment
    traj = get_trajectory(pp.pose, pp.motion.steps, pp.motion.periodic, ts, dt)
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/utils.py", line 127, in get_trajectory
    assert_allclose(errors, 0.0, atol=1e-8)
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 1527, in assert_allclose
    assert_array_compare(compare, actual, desired, err_msg=str(err_msg),
  File "/usr/local/lib/python3.8/dist-packages/numpy/testing/_private/utils.py", line 844, in assert_array_compare
    raise AssertionError(msg)
AssertionError: 
Not equal to tolerance rtol=1e-07, atol=1e-08

Mismatched elements: 1 / 86 (1.16%)
Max absolute difference: 2.58072573e-07
Max relative difference: inf
 x: array([0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,
       0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,
       0.000000e+00, 0.000000e+00, 0.000000e+00, 0.000000e+00,...
 y: array(0.)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/dt_planning/planning_checker.py", line 102, in planning_checker_main
    sessions = run_checker(
  File "/usr/local/lib/python3.8/dist-packages/dt_protocols/checker.py", line 119, in run_checker
    raise dc.InvalidEvaluator() from e
duckietown_challenges.exceptions.InvalidEvaluator:

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

172482

step1-3of4

success

yes

2023-11-29 05:28:14+00:00

2023-11-29 05:29:03+00:00

0:00:49

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible