AI Driving Olympics Home Challenges Submissions Jobs

Submission 144

Submission144
Competingyes
Challengeaido1_LF1_r3-v3
UserAndrea Censi
Date submitted
Complete
DetailsEvaluation is complete.
Sisters
Result💔
Jobsstep1-simulation: 13068
Next
User labelRandom execution
Admin priority50
Blessing50
User priority

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
13068144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedyes-0:00:55
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 60, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
10641144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:30
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 60, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
10640144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationabortedno-0:00:14
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job10640-4781', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job10640-4781', 'up', '-d']' returned non-zero exit status 1

stdout | 

stderr | Creating network "job10640-4781_evaluation" with the default driver
stderr | Creating job10640-4781_evaluator_1 ...
stderr | Creating job10640-4781_solution_1  ...
stderr | 
Creating job10640-4781_evaluator_1 ... error

stderr | ERROR: for job10640-4781_evaluator_1  Cannot start service evaluator: network job10640-4781_evaluation not found
stderr | 
Creating job10640-4781_solution_1  ... error

stderr | ERROR: for job10640-4781_solution_1  Cannot start service solution: network job10640-4781_evaluation not found
stderr | 
stderr | ERROR: for evaluator  Cannot start service evaluator: network job10640-4781_evaluation not found
stderr | 
stderr | ERROR: for solution  Cannot start service solution: network job10640-4781_evaluation not found
stderr | Encountered errors while bringing up the project.
stderr | 
Hidden. If you are the author, please login using the top-right link or use the dashboard.
10639144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:31
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 60, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
3053144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:23
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 60, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
3017144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationtimeoutno-0:11:02Hidden. If you are the author, please login using the top-right link or use the dashboard.
3016144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationtimeoutno-0:11:01Hidden. If you are the author, please login using the top-right link or use the dashboard.
2919144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:02:18
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 60, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
2879144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationtimeoutno-0:18:05Hidden. If you are the author, please login using the top-right link or use the dashboard.
2743144Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizsuccessno-0:04:00Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
2740144Andrea CensiRandom executionaido1_LF1_r3-v3step3-videossuccessno-0:01:06Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
2739144Andrea CensiRandom executionaido1_LF1_r3-v3step2-scoringsuccessno-0:00:18Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.1140329493880272, "good_angle": 13.18464621525707, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.533333333333296}, "ep001": {"nsteps": 500, "reward": -1.121210604220629, "good_angle": 42.666415284472265, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 15.999999999999988}, "ep002": {"nsteps": 500, "reward": -1.242578337728919, "good_angle": 62.961938462111625, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 13.266666666666657}, "ep003": {"nsteps": 500, "reward": -0.26420358337838845, "good_angle": 4.162725173361043, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 9.33333333333332}, "ep004": {"nsteps": 500, "reward": -2.210636956365779, "good_angle": 12.298695301801644, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 13.499999999999996}}
good_angle_max62.961938462111625
good_angle_mean27.05488408740073
good_angle_median13.18464621525707
good_angle_min4.162725173361043
reward_max-0.26420358337838845
reward_mean-1.1905324862163489
reward_median-1.121210604220629
reward_min-2.210636956365779
survival_time_max16.666666666666654
survival_time_mean16.666666666666654
survival_time_min16.666666666666654
traveled_tiles_max2
traveled_tiles_mean1.4
traveled_tiles_median1
traveled_tiles_min1
valid_direction_max15.999999999999988
valid_direction_mean12.926666666666652
valid_direction_median13.266666666666657
valid_direction_min9.33333333333332
2723144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-0:04:41Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
2722144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationerrorno-0:00:13
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job2722-1605', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job2722-1605', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
Hidden. If you are the author, please login using the top-right link or use the dashboard.
2089144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:01:10
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 60, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
573144Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizerrorno-0:00:14
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 530, in wrap_scorer
    evaluator.score(cie)
  File "/visualizer/videos.py", line 12, in score
    logdir = cie.get_completed_step_evaluation_file('step1-simulation', 'episodes')
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 398, in get_completed_step_evaluation_file
    return get_completed_step_evaluation_file(self.root, step_name, basename)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 427, in get_completed_step_evaluation_file
    available = get_completed_step_evaluation_files(root, step_name)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 421, in get_completed_step_evaluation_files
    raise InvalidEnvironment(msg)
InvalidEnvironment: Could not find dir /previous-steps/step1-simulation/challenge-evaluation-output
Hidden. If you are the author, please login using the top-right link or use the dashboard.
572144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:36
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 60, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 35, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
301144Andrea CensiRandom executionaido1_LF1_r3-v3step3-videossuccessno-2:01:01Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
299144Andrea CensiRandom executionaido1_LF1_r3-v3step2-scoringsuccessno-2:01:33Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.8184365517195549, "good_angle": 50.795966412804106, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 13.033333333333324}, "ep001": {"nsteps": 500, "reward": -0.9623222973942755, "good_angle": 10.983964048937157, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.13333333333333}, "ep002": {"nsteps": 500, "reward": -0.6675195905930014, "good_angle": 40.925411343247376, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 15.266666666666657}, "ep003": {"nsteps": 500, "reward": -1.0838831386975945, "good_angle": 60.591540085165526, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 11.699999999999989}, "ep004": {"nsteps": 500, "reward": -0.22700107742531692, "good_angle": 13.626091124112616, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.433333333333332}}
good_angle_max60.591540085165526
good_angle_mean35.384594602853355
good_angle_median40.925411343247376
good_angle_min10.983964048937157
reward_max-0.22700107742531692
reward_mean-0.9518325311659488
reward_median-0.9623222973942755
reward_min-1.8184365517195549
survival_time_max16.666666666666654
survival_time_mean16.666666666666654
survival_time_min16.666666666666654
traveled_tiles_max2
traveled_tiles_mean1.4
traveled_tiles_median1
traveled_tiles_min1
valid_direction_max15.266666666666657
valid_direction_mean12.913333333333329
valid_direction_median12.433333333333332
valid_direction_min11.699999999999989
294144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:01:39Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
272144Andrea CensiRandom executionaido1_LF1_r3-v3step2-scoringsuccessno-2:02:19Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median9.699999999999976


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.0368080102205277, "good_angle": 9.42087440065101, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.16666666666666}, "ep001": {"nsteps": 291, "reward": -4.621755821700768, "good_angle": 7.638457668088401, "survival_time": 9.699999999999976, "traveled_tiles": 1, "valid_direction": 7.499999999999982}, "ep002": {"nsteps": 212, "reward": -6.904545187367234, "good_angle": 3.0420365784389394, "survival_time": 7.066666666666651, "traveled_tiles": 2, "valid_direction": 4.499999999999989}, "ep003": {"nsteps": 226, "reward": -6.114864278156145, "good_angle": 2.2360597559888977, "survival_time": 7.533333333333316, "traveled_tiles": 1, "valid_direction": 3.1666666666666594}, "ep004": {"nsteps": 500, "reward": -1.393642165839672, "good_angle": 24.11772828936881, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 14.333333333333323}}
good_angle_max24.11772828936881
good_angle_mean9.29103133850721
good_angle_median7.638457668088401
good_angle_min2.2360597559888977
reward_max-1.0368080102205277
reward_mean-4.014323092656869
reward_median-4.621755821700768
reward_min-6.904545187367234
survival_time_max16.666666666666654
survival_time_mean11.526666666666651
survival_time_min7.066666666666651
traveled_tiles_max2
traveled_tiles_mean1.2
traveled_tiles_median1
traveled_tiles_min1
valid_direction_max14.333333333333323
valid_direction_mean8.333333333333323
valid_direction_median7.499999999999982
valid_direction_min3.1666666666666594
270144Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:02:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1