Duckietown Challenges Home Challenges Submissions

Submission 1004

Submission1004
Competingyes
Challengeaido1_LF1_r3-v3
UserLaurent Mandrile
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💔
Jobsstep1-simulation: 14027 step2-scoring: 8164 step3-videos: 8167 step4-viz: 8171
Next
User labelTensorflow template
Admin priority50
Blessingn/a
User priority50

8171

Click the images to see detailed statistics about the episode.

ep000

ep001

ep002

ep003

ep004

8167

ep000

ep001

ep002

ep003

ep004

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
14027step1-simulationfailedyes0:05:51
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 86, in run
    solve(params, cis)  # let's try to solve the challenge,
  File "solution.py", line 52, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator


Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8171step4-vizsuccessyes0:08:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_median0.13312858444658682
deviation-center-line_median0.08744010848352336
in-drivable-lane_median8.366666666666639


other stats
deviation-center-line_max0.2951251593343002
deviation-center-line_mean0.12729673166499883
deviation-center-line_min0.047860468716545944
deviation-heading_max1.2058790583016687
deviation-heading_mean0.7178061254891885
deviation-heading_median0.7905924332634966
deviation-heading_min0.1745193388357215
driven_any_max4.454184189442348
driven_any_mean2.5257450968093726
driven_any_median2.3537329967677016
driven_any_min1.0662869877338133
driven_lanedir_max0.4531051551321528
driven_lanedir_mean0.193667880939661
driven_lanedir_min0.07969926819667117
in-drivable-lane_max15.999999999999988
in-drivable-lane_mean9.086666666666648
in-drivable-lane_min3.766666666666661
per-episodes
details{"ep000": {"driven_any": 1.995026722965619, "driven_lanedir": 0.07969926819667117, "in-drivable-lane": 7.866666666666648, "deviation-heading": 0.1745193388357215, "deviation-center-line": 0.047860468716545944}, "ep001": {"driven_any": 2.3537329967677016, "driven_lanedir": 0.13312858444658682, "in-drivable-lane": 9.433333333333303, "deviation-heading": 1.2058790583016687, "deviation-center-line": 0.1298299749331926}, "ep002": {"driven_any": 4.454184189442348, "driven_lanedir": 0.18029873261228624, "in-drivable-lane": 15.999999999999988, "deviation-heading": 0.47847911565363, "deviation-center-line": 0.07622794685743216}, "ep003": {"driven_any": 1.0662869877338133, "driven_lanedir": 0.12210766431060804, "in-drivable-lane": 3.766666666666661, "deviation-heading": 0.7905924332634966, "deviation-center-line": 0.08744010848352336}, "ep004": {"driven_any": 2.7594945871373833, "driven_lanedir": 0.4531051551321528, "in-drivable-lane": 8.366666666666639, "deviation-heading": 0.9395606813914256, "deviation-center-line": 0.2951251593343002}}
No reset possible
8170step4-vizsuccessyes0:02:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8169step4-vizsuccessyes0:02:13
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8168step4-vizsuccessyes0:08:12
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8167step3-videossuccessyes0:01:14
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
8166step3-videossuccessyes0:01:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8165step3-videossuccessyes0:01:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8164step2-scoringsuccessyes0:03:42
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median10.533333333333308


other stats
episodes
details{"ep000": {"nsteps": 244, "reward": -6.2869903129815565, "good_angle": 1.776589935962943, "survival_time": 8.133333333333315, "traveled_tiles": 4, "valid_direction": 2.6999999999999997}, "ep001": {"nsteps": 316, "reward": -5.700926186605297, "good_angle": 40.99984006033056, "survival_time": 10.533333333333308, "traveled_tiles": 3, "valid_direction": 7.099999999999983}, "ep002": {"nsteps": 500, "reward": -2.864125187820755, "good_angle": 41.203790317142946, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 7.699999999999982}, "ep003": {"nsteps": 138, "reward": -9.236848439602856, "good_angle": 1.853637092777763, "survival_time": 4.599999999999993, "traveled_tiles": 2, "valid_direction": 2.5}, "ep004": {"nsteps": 316, "reward": -4.6504965261512, "good_angle": 1.752446115197312, "survival_time": 10.533333333333308, "traveled_tiles": 4, "valid_direction": 3.166666666666659}}
good_angle_max41.203790317142946
good_angle_mean17.517260704282304
good_angle_median1.853637092777763
good_angle_min1.752446115197312
reward_max-2.864125187820755
reward_mean-5.747877330632333
reward_median-5.700926186605297
reward_min-9.236848439602856
survival_time_max16.666666666666654
survival_time_mean10.093333333333314
survival_time_min4.599999999999993
traveled_tiles_max6
traveled_tiles_mean3.8
traveled_tiles_median4
traveled_tiles_min2
valid_direction_max7.699999999999982
valid_direction_mean4.633333333333324
valid_direction_median3.166666666666659
valid_direction_min2.5
No reset possible
8163step2-scoringsuccessyes0:00:20
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8162step2-scoringsuccessyes0:00:21
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible