AI Driving Olympics Home Challenges Submissions Jobs

Submission 447

Submission447
Competingyes
Challengeaido1_LF1_r3-v3
UserJon
Date submitted
Complete
DetailsEvaluation is complete.
Sisters
Result💚
Jobsstep1-simulation: 13521 step2-scoring: 13523 step3-videos: 13524 step4-viz: 13527
Next
User labelRandom execution
Admin priority50
Blessing50
User priority

13527

Click the images to see detailed statistics about the episode.

13524

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
13527447JonRandom executionaido1_LF1_r3-v3step4-vizsuccessyes-0:01:11Hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_median0.7306653400468083
deviation-center-line_median0.08201115257522808
in-drivable-lane_median0


other stats
deviation-center-line_max0.215939221592487
deviation-center-line_mean0.10178852005874656
deviation-center-line_min0.011768106226475384
deviation-heading_max0.513189919816418
deviation-heading_mean0.3544820551127265
deviation-heading_median0.3865854188587085
deviation-heading_min0.11997895386315036
driven_any_max1.331297152418986
driven_any_mean0.9478600930548136
driven_any_median1.0868014363064689
driven_any_min0.3891380660067644
driven_lanedir_max1.301116436710665
driven_lanedir_mean0.7061320064103966
driven_lanedir_min0.001837600141386897
in-drivable-lane_max2.366666666666668
in-drivable-lane_mean0.4733333333333336
in-drivable-lane_min0
per-episodes
details{"ep000": {"driven_any": 1.0868014363064689, "driven_lanedir": 0.001837600141386897, "in-drivable-lane": 2.366666666666668, "deviation-heading": 0.36135322563051214, "deviation-center-line": 0.011768106226475384}, "ep001": {"driven_any": 1.331297152418986, "driven_lanedir": 1.301116436710665, "in-drivable-lane": 0, "deviation-heading": 0.3865854188587085, "deviation-center-line": 0.215939221592487}, "ep002": {"driven_any": 0.7764086899333893, "driven_lanedir": 0.7306653400468083, "in-drivable-lane": 0, "deviation-heading": 0.3913027573948435, "deviation-center-line": 0.07085353277849211}, "ep003": {"driven_any": 0.3891380660067644, "driven_lanedir": 0.3855598500432331, "in-drivable-lane": 0, "deviation-heading": 0.11997895386315036, "deviation-center-line": 0.08201115257522808}, "ep004": {"driven_any": 1.15565512060846, "driven_lanedir": 1.1114808051098897, "in-drivable-lane": 0, "deviation-heading": 0.513189919816418, "deviation-center-line": 0.12837058712105023}}
13524447JonRandom executionaido1_LF1_r3-v3step3-videossuccessyes-0:00:40Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
13523447JonRandom executionaido1_LF1_r3-v3step2-scoringsuccessyes-0:00:22Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median2.6333333333333337


other stats
episodes
details{"ep000": {"nsteps": 79, "reward": -13.554760562326711, "good_angle": 1.3638601821504437, "survival_time": 2.6333333333333337, "traveled_tiles": 2, "valid_direction": 0.9666666666666652}, "ep001": {"nsteps": 98, "reward": -9.94825712462165, "good_angle": 0.23857113399144075, "survival_time": 3.266666666666665, "traveled_tiles": 3, "valid_direction": 0.5666666666666647}, "ep002": {"nsteps": 57, "reward": -17.320575883089187, "good_angle": 0.4006333285845668, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 0.766666666666669}, "ep003": {"nsteps": 30, "reward": -32.97934064753353, "good_angle": 0.019262225232788623, "survival_time": 1, "traveled_tiles": 1, "valid_direction": 0}, "ep004": {"nsteps": 85, "reward": -11.443003183351282, "good_angle": 0.41845204641929784, "survival_time": 2.833333333333333, "traveled_tiles": 3, "valid_direction": 0.899999999999997}}
good_angle_max1.3638601821504437
good_angle_mean0.4881557832757076
good_angle_median0.4006333285845668
good_angle_min0.019262225232788623
reward_max-9.94825712462165
reward_mean-17.04918748018447
reward_median-13.554760562326711
reward_min-32.97934064753353
survival_time_max3.266666666666665
survival_time_mean2.3266666666666667
survival_time_min1
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max0.9666666666666652
valid_direction_mean0.6399999999999991
valid_direction_median0.766666666666669
valid_direction_min0
13521447JonRandom executionaido1_LF1_r3-v3step1-simulationsuccessyes-0:01:14Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
10286447JonRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:02:23
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator


Hidden. If you are the author, please login using the top-right link or use the dashboard.
4110447JonRandom executionaido1_LF1_r3-v3step4-vizsuccessno-0:01:09Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
4086447JonRandom executionaido1_LF1_r3-v3step4-viztimeoutno-0:31:04Hidden. If you are the author, please login using the top-right link or use the dashboard.
4074447JonRandom executionaido1_LF1_r3-v3step4-viztimeoutno-0:33:00Hidden. If you are the author, please login using the top-right link or use the dashboard.
4072447JonRandom executionaido1_LF1_r3-v3step3-videossuccessno-0:01:49Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
4071447JonRandom executionaido1_LF1_r3-v3step3-videossuccessno-0:01:50Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
4070447JonRandom executionaido1_LF1_r3-v3step2-scoringsuccessno-0:00:17Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.999999999999979


other stats
episodes
details{"ep000": {"nsteps": 171, "reward": -7.510828453372096, "good_angle": 24.91932558762904, "survival_time": 5.6999999999999895, "traveled_tiles": 2, "valid_direction": 3.9333333333333207}, "ep001": {"nsteps": 378, "reward": -4.285664803767331, "good_angle": 25.14263250704381, "survival_time": 12.599999999999966, "traveled_tiles": 3, "valid_direction": 3.899999999999986}, "ep002": {"nsteps": 320, "reward": -3.2170385880021057, "good_angle": 0.12739485498542205, "survival_time": 10.66666666666664, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 258, "reward": -6.213949680328369, "good_angle": 25.60182527651299, "survival_time": 8.59999999999998, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 270, "reward": -5.481325739820453, "good_angle": 4.40064870360421, "survival_time": 8.999999999999979, "traveled_tiles": 3, "valid_direction": 3.1666666666666563}}
good_angle_max25.60182527651299
good_angle_mean16.038365385955096
good_angle_median24.91932558762904
good_angle_min0.12739485498542205
reward_max-3.2170385880021057
reward_mean-5.341761453058071
reward_median-5.481325739820453
reward_min-7.510828453372096
survival_time_max12.599999999999966
survival_time_mean9.313333333333311
survival_time_min5.6999999999999895
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.9333333333333207
valid_direction_mean2.9866666666666566
valid_direction_median3.899999999999986
valid_direction_min0
4068447JonRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-0:02:03Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1