Duckietown Challenges Home Challenges Submissions

Submission 14461

Submission14461
Competingyes
Challengemooc-BV1
UserJeff Wong
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
Jobssim-0of5: 113265 sim-1of5: 113296 sim-2of5: 113267 sim-3of5: 113264 sim-4of5: 113283
Next
User labelquickturn_v1
Admin priority50
Blessingn/a
User priority50

113296

Click the images to see detailed statistics about the episode.

d60

113283

Click the images to see detailed statistics about the episode.

d60

113267

Click the images to see detailed statistics about the episode.

d40

113265

Click the images to see detailed statistics about the episode.

d45

113264

Click the images to see detailed statistics about the episode.

d30

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
113296sim-1of5successyes0:03:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean3.3949554252861818


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max3.3949554252861818
distance-from-start_median3.3949554252861818
distance-from-start_min3.3949554252861818
driven_any_max4.027859048132885
driven_any_mean4.027859048132885
driven_any_median4.027859048132885
driven_any_min4.027859048132885
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max32.85000000000027
in-drivable-lane_mean32.85000000000027
in-drivable-lane_median32.85000000000027
in-drivable-lane_min32.85000000000027
per-episodes
details{"d60-ego0": {"driven_any": 4.027859048132885, "survival_time": 32.85000000000027, "driven_lanedir": 0.0, "in-drivable-lane": 32.85000000000027, "deviation-heading": 0.0, "distance-from-start": 3.3949554252861818, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max32.85000000000027
survival_time_mean32.85000000000027
survival_time_median32.85000000000027
survival_time_min32.85000000000027
No reset possible
113283sim-4of5successyes0:04:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean3.5838652373598925


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max3.5838652373598925
distance-from-start_median3.5838652373598925
distance-from-start_min3.5838652373598925
driven_any_max4.103943708644593
driven_any_mean4.103943708644593
driven_any_median4.103943708644593
driven_any_min4.103943708644593
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max33.850000000000215
in-drivable-lane_mean33.850000000000215
in-drivable-lane_median33.850000000000215
in-drivable-lane_min33.850000000000215
per-episodes
details{"d60-ego0": {"driven_any": 4.103943708644593, "survival_time": 33.850000000000215, "driven_lanedir": 0.0, "in-drivable-lane": 33.850000000000215, "deviation-heading": 0.0, "distance-from-start": 3.5838652373598925, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max33.850000000000215
survival_time_mean33.850000000000215
survival_time_median33.850000000000215
survival_time_min33.850000000000215
No reset possible
113278sim-1of5successyes0:04:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
113269sim-4of5successyes0:03:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
113267sim-2of5successyes0:05:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean3.871615018797206


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max3.871615018797206
distance-from-start_median3.871615018797206
distance-from-start_min3.871615018797206
driven_any_max6.574702008867787
driven_any_mean6.574702008867787
driven_any_median6.574702008867787
driven_any_min6.574702008867787
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max55.04999999999901
in-drivable-lane_mean55.04999999999901
in-drivable-lane_median55.04999999999901
in-drivable-lane_min55.04999999999901
per-episodes
details{"d40-ego0": {"driven_any": 6.574702008867787, "survival_time": 55.04999999999901, "driven_lanedir": 0.0, "in-drivable-lane": 55.04999999999901, "deviation-heading": 0.0, "distance-from-start": 3.871615018797206, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max55.04999999999901
survival_time_mean55.04999999999901
survival_time_median55.04999999999901
survival_time_min55.04999999999901
No reset possible
113265sim-0of5successyes0:04:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean4.6892298099872365


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max4.6892298099872365
distance-from-start_median4.6892298099872365
distance-from-start_min4.6892298099872365
driven_any_max5.1184267987255785
driven_any_mean5.1184267987255785
driven_any_median5.1184267987255785
driven_any_min5.1184267987255785
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max41.89999999999976
in-drivable-lane_mean41.89999999999976
in-drivable-lane_median41.89999999999976
in-drivable-lane_min41.89999999999976
per-episodes
details{"d45-ego0": {"driven_any": 5.1184267987255785, "survival_time": 41.89999999999976, "driven_lanedir": 0.0, "in-drivable-lane": 41.89999999999976, "deviation-heading": 0.0, "distance-from-start": 4.6892298099872365, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max41.89999999999976
survival_time_mean41.89999999999976
survival_time_median41.89999999999976
survival_time_min41.89999999999976
No reset possible
113264sim-3of5successyes0:04:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean5.022077900851516


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max5.022077900851516
distance-from-start_median5.022077900851516
distance-from-start_min5.022077900851516
driven_any_max5.808764923492466
driven_any_mean5.808764923492466
driven_any_median5.808764923492466
driven_any_min5.808764923492466
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max48.34999999999939
in-drivable-lane_mean48.34999999999939
in-drivable-lane_median48.34999999999939
in-drivable-lane_min48.34999999999939
per-episodes
details{"d30-ego0": {"driven_any": 5.808764923492466, "survival_time": 48.34999999999939, "driven_lanedir": 0.0, "in-drivable-lane": 48.34999999999939, "deviation-heading": 0.0, "distance-from-start": 5.022077900851516, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max48.34999999999939
survival_time_mean48.34999999999939
survival_time_median48.34999999999939
survival_time_min48.34999999999939
No reset possible
93588sim-2of5successno0:11:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
93587sim-1of5successno0:06:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
93586sim-0of5successno0:08:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
93585sim-4of5successno0:06:43
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
93584sim-3of5successno0:08:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74001sim-1of5successno0:08:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
74000sim-2of5successno0:08:00
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
73997sim-0of5successno0:07:07
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
73996sim-4of5successno0:06:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
73993sim-3of5successno0:07:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
70221sim-4of5errorno0:05:31
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 352, in main
    evaluated = read_and_draw(fn, dn_i, pc_name)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 257, in read_and_draw
    log0 = read_simulator_log_cbor(ld, main_robot_name=robot_main)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 223, in read_simulator_log_cbor
    pose2 = SE2Transform.from_SE2(pose)
  File "/usr/local/lib/python3.8/site-packages/duckietown_world/geo/transforms.py", line 102, in from_SE2
    translation, angle = geo.translation_angle_from_SE2(q)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 235, in translation_angle_from_SE2
    R, t, _, _ = extract_pieces(pose)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 144, in extract_pieces
    M = x.shape[0] - 1
AttributeError: 'FriendlyPose' object has no attribute 'shape'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 412, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
70220sim-0of5errorno0:05:56
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 352, in main
    evaluated = read_and_draw(fn, dn_i, pc_name)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 257, in read_and_draw
    log0 = read_simulator_log_cbor(ld, main_robot_name=robot_main)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 223, in read_simulator_log_cbor
    pose2 = SE2Transform.from_SE2(pose)
  File "/usr/local/lib/python3.8/site-packages/duckietown_world/geo/transforms.py", line 102, in from_SE2
    translation, angle = geo.translation_angle_from_SE2(q)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 235, in translation_angle_from_SE2
    R, t, _, _ = extract_pieces(pose)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 144, in extract_pieces
    M = x.shape[0] - 1
AttributeError: 'FriendlyPose' object has no attribute 'shape'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 412, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
70219sim-3of5errorno0:06:06
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 352, in main
    evaluated = read_and_draw(fn, dn_i, pc_name)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 257, in read_and_draw
    log0 = read_simulator_log_cbor(ld, main_robot_name=robot_main)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 223, in read_simulator_log_cbor
    pose2 = SE2Transform.from_SE2(pose)
  File "/usr/local/lib/python3.8/site-packages/duckietown_world/geo/transforms.py", line 102, in from_SE2
    translation, angle = geo.translation_angle_from_SE2(q)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 235, in translation_angle_from_SE2
    R, t, _, _ = extract_pieces(pose)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 144, in extract_pieces
    M = x.shape[0] - 1
AttributeError: 'FriendlyPose' object has no attribute 'shape'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 412, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
70218sim-2of5errorno0:07:14
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 352, in main
    evaluated = read_and_draw(fn, dn_i, pc_name)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 257, in read_and_draw
    log0 = read_simulator_log_cbor(ld, main_robot_name=robot_main)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 223, in read_simulator_log_cbor
    pose2 = SE2Transform.from_SE2(pose)
  File "/usr/local/lib/python3.8/site-packages/duckietown_world/geo/transforms.py", line 102, in from_SE2
    translation, angle = geo.translation_angle_from_SE2(q)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 235, in translation_angle_from_SE2
    R, t, _, _ = extract_pieces(pose)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 144, in extract_pieces
    M = x.shape[0] - 1
AttributeError: 'FriendlyPose' object has no attribute 'shape'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 412, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
70217sim-1of5errorno0:06:36
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 352, in main
    evaluated = read_and_draw(fn, dn_i, pc_name)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 257, in read_and_draw
    log0 = read_simulator_log_cbor(ld, main_robot_name=robot_main)
  File "/usr/local/lib/python3.8/site-packages/aido_analyze/utils_drawing.py", line 223, in read_simulator_log_cbor
    pose2 = SE2Transform.from_SE2(pose)
  File "/usr/local/lib/python3.8/site-packages/duckietown_world/geo/transforms.py", line 102, in from_SE2
    translation, angle = geo.translation_angle_from_SE2(q)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 235, in translation_angle_from_SE2
    R, t, _, _ = extract_pieces(pose)
  File "/usr/local/lib/python3.8/site-packages/geometry/poses.py", line 144, in extract_pieces
    M = x.shape[0] - 1
AttributeError: 'FriendlyPose' object has no attribute 'shape'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 412, in main
    raise dc.InvalidEvaluator(msg) from e
duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes:
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
68285sim-0of5successno0:01:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
68284sim-0of5successno0:01:57
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
68283sim-3of5successno0:06:32
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
68282sim-4of5successno0:07:56
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
68281sim-1of5successno0:09:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
68280sim-0of5successno0:02:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
68279sim-2of5successno0:06:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible