Duckietown Challenges Home Challenges Submissions

Submission 551

Submission551
Competingyes
Challengeaido1_LF1_r3-v3
UserJonathan Plante 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💔
Jobsstep1-simulation: 12061 step2-scoring: 9995 step3-videos: 9999 step4-viz: 10018
Next
User labelJP pipeline
Admin priority50
Blessingn/a
User priority50

10018

Click the images to see detailed statistics about the episode.

ep000

ep001

ep002

ep003

ep004

9999

ep000

ep001

ep002

ep003

ep004

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
12061step1-simulationfailedyes0:08:53
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 63, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: Giving up to connect to the gym duckietown server at host: evaluator

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
10018step4-vizsuccessyes0:03:53
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_median0.3576978746005717
deviation-center-line_median0.2233841557282331
in-drivable-lane_median0


other stats
deviation-center-line_max0.3503940953154788
deviation-center-line_mean0.19738717890852345
deviation-center-line_min0
deviation-heading_max12.77547708270898
deviation-heading_mean5.865458865467552
deviation-heading_median7.097049538801558
deviation-heading_min0
driven_any_max1.2739283137934605
driven_any_mean0.6107395487412631
driven_any_median0.5166434073868348
driven_any_min0.01531878084989824
driven_lanedir_max0.8619271176621262
driven_lanedir_mean0.3497392406430405
driven_lanedir_min0
in-drivable-lane_max16.63333333333332
in-drivable-lane_mean3.333333333333331
in-drivable-lane_min0
per-episodes
details{"ep000": {"driven_any": 1.2739283137934605, "driven_lanedir": 0, "in-drivable-lane": 16.63333333333332, "deviation-heading": 0, "deviation-center-line": 0}, "ep001": {"driven_any": 0.3719349073541322, "driven_lanedir": 0.3576978746005717, "in-drivable-lane": 0.03333333333333344, "deviation-heading": 0.2824915590704668, "deviation-center-line": 0.1192082148314856}, "ep002": {"driven_any": 0.5166434073868348, "driven_lanedir": 0.5149982021318316, "in-drivable-lane": 0, "deviation-heading": 12.77547708270898, "deviation-center-line": 0.3503940953154788}, "ep003": {"driven_any": 0.01531878084989824, "driven_lanedir": 0.014073008820673303, "in-drivable-lane": 0, "deviation-heading": 7.097049538801558, "deviation-center-line": 0.2939494286674196}, "ep004": {"driven_any": 0.8758723343219902, "driven_lanedir": 0.8619271176621262, "in-drivable-lane": 0, "deviation-heading": 9.17227614675676, "deviation-center-line": 0.2233841557282331}}
No reset possible
9999step3-videossuccessyes0:02:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
9995step2-scoringsuccessyes0:00:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.9915735043883325, "good_angle": 2.953277850433357, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 7.033333333333329}, "ep001": {"nsteps": 41, "reward": -24.947025884215424, "good_angle": 0.3621553754560622, "survival_time": 1.3666666666666676, "traveled_tiles": 1, "valid_direction": 0.2666666666666675}, "ep002": {"nsteps": 500, "reward": -0.15607701748982072, "good_angle": 10.913104204852193, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 15.13333333333332}, "ep003": {"nsteps": 500, "reward": -0.05172294469177723, "good_angle": 3.312909787869147, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 8.36666666666666}, "ep004": {"nsteps": 500, "reward": -0.026097689567599447, "good_angle": 5.930381242691954, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 14.099999999999987}}
good_angle_max10.913104204852193
good_angle_mean4.694365692260542
good_angle_median3.312909787869147
good_angle_min0.3621553754560622
reward_max-0.026097689567599447
reward_mean-5.434499408070591
reward_median-0.15607701748982072
reward_min-24.947025884215424
survival_time_max16.666666666666654
survival_time_mean13.606666666666658
survival_time_min1.3666666666666676
traveled_tiles_max3
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max15.13333333333332
valid_direction_mean8.979999999999993
valid_direction_median8.36666666666666
valid_direction_min0.2666666666666675
No reset possible
9994step2-scoringabortedyes0:00:13
Error while running [...]

Creating job9994-9515_scorer_1 ... error
stderr | ERROR: for job9994-9515_scorer_1  Cannot start service scorer: network job9994-9515_evaluation not found
stderr | 
stderr | ERROR: for scorer  Cannot start service scorer: network job9994-9515_evaluation not found
stderr | Encountered errors while bringing up the project.
stderr | 
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
9959step1-simulationsuccessno0:07:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4695step4-vizsuccessno0:01:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4694step3-videossuccessno0:02:04
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4693step3-videossuccessno0:02:05
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4692step3-videoserrorno0:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job4692-5034', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job4692-5034', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4691step2-scoringsuccessno0:00:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
4690step1-simulationsuccessno0:07:11
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible