AI Driving Olympics Home Challenges Submissions Jobs

Submission 165

Submission165
Competingyes
Challengeaido1_LF1_r3-v3
UserAndrea Censi
Date submitted
Complete
DetailsEvaluation is complete.
Sisters
Result💔
Jobsstep1-simulation: 13252
Next
User labelRandom execution
Admin priority50
Blessing50
User priority

Evaluation jobs for this submission

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
13252165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedyes-0:01:17
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
10633165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:34
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
3052165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:24
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
3015165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationtimeoutno-0:11:04Hidden. If you are the author, please login using the top-right link or use the dashboard.
3014165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationtimeoutno-0:11:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
2878165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:24
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
2755165Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizsuccessno-0:01:06Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
2753165Andrea CensiRandom executionaido1_LF1_r3-v3step3-videossuccessno-0:00:41Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
2732165Andrea CensiRandom executionaido1_LF1_r3-v3step3-videostimeoutno-0:11:04Hidden. If you are the author, please login using the top-right link or use the dashboard.
2730165Andrea CensiRandom executionaido1_LF1_r3-v3step2-scoringsuccessno-0:00:16Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median8.633333333333313


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.812373652177699, "good_angle": 24.087746930908946, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 376, "reward": -3.3345901525840707, "good_angle": 22.08298366314928, "survival_time": 12.5333333333333, "traveled_tiles": 3, "valid_direction": 3.9333333333333194}, "ep002": {"nsteps": 259, "reward": -3.910537747261708, "good_angle": 0.16612215408925146, "survival_time": 8.633333333333313, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 257, "reward": -6.108992447648995, "good_angle": 24.958607361599043, "survival_time": 8.566666666666647, "traveled_tiles": 3, "valid_direction": 3.899999999999987}, "ep004": {"nsteps": 271, "reward": -5.544229294305729, "good_angle": 4.7541386616447125, "survival_time": 9.033333333333312, "traveled_tiles": 3, "valid_direction": 2.9999999999999902}}
good_angle_max24.958607361599043
good_angle_mean15.209919754278246
good_angle_median22.08298366314928
good_angle_min0.16612215408925146
reward_max-3.3345901525840707
reward_mean-5.3421446587956405
reward_median-5.544229294305729
reward_min-7.812373652177699
survival_time_max12.5333333333333
survival_time_mean8.886666666666645
survival_time_min5.666666666666656
traveled_tiles_max3
traveled_tiles_mean2.8
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.933333333333321
valid_direction_mean2.9533333333333234
valid_direction_median3.899999999999987
valid_direction_min0
2721165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-0:02:00Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
2720165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:32
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
2636165Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizsuccessno-0:02:46Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
2102165Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizerrorno-0:00:54
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 530, in wrap_scorer
    evaluator.score(cie)
  File "/visualizer/videos.py", line 27, in score
    cie.set_evaluation_file(f, f)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 309, in set_evaluation_file
    raise_wrapped(InvalidEvaluator, e, msg)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/utils.py", line 214, in raise_wrapped
    raise e
InvalidEvaluator: Invalid set_evaluation_file()
| Traceback (most recent call last):
|   File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 306, in set_evaluation_file
|     self.evaluation_files.add(basename, from_file, description)
|   File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 42, in add
|     raise ValueError(msg)
| ValueError: The file does not exist: ep001/log/drawing-interactive.html
Hidden. If you are the author, please login using the top-right link or use the dashboard.
2088165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-0:03:29Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
466165Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizerrorno-0:01:09
InvalidEnvironment: [...]
InvalidEnvironment:
Traceback (most recent call last):
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 530, in wrap_scorer
    evaluator.score(cie)
  File "/visualizer/videos.py", line 12, in score
    logdir = cie.get_completed_step_evaluation_file('step1-simulation', 'episodes')
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 398, in get_completed_step_evaluation_file
    return get_completed_step_evaluation_file(self.root, step_name, basename)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 427, in get_completed_step_evaluation_file
    available = get_completed_step_evaluation_files(root, step_name)
  File "/visualizer/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 421, in get_completed_step_evaluation_files
    raise InvalidEnvironment(msg)
InvalidEnvironment: Could not find dir /previous-steps/step1-simulation/challenge-evaluation-output
Hidden. If you are the author, please login using the top-right link or use the dashboard.
464165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationfailedno-0:00:34
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 580, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 575, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 107, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 52, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 74, in _failsafe_observe
    raise Exception("Can't connect to the gym-duckietown-server")
Exception: Can't connect to the gym-duckietown-server


Hidden. If you are the author, please login using the top-right link or use the dashboard.
311165Andrea CensiRandom executionaido1_LF1_r3-v3step3-videossuccessno-2:00:35Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
310165Andrea CensiRandom executionaido1_LF1_r3-v3step2-scoringsuccessno-2:00:15Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median4.899999999999992


other stats
episodes
details{"ep000": {"nsteps": 257, "reward": -5.935388254848806, "good_angle": 6.296167709086408, "survival_time": 8.566666666666647, "traveled_tiles": 3, "valid_direction": 3.2333333333333227}, "ep001": {"nsteps": 206, "reward": -5.920791627916064, "good_angle": 23.061219503310294, "survival_time": 6.866666666666652, "traveled_tiles": 2, "valid_direction": 3.966666666666653}, "ep002": {"nsteps": 147, "reward": -9.38501587449288, "good_angle": 26.454510536122378, "survival_time": 4.899999999999992, "traveled_tiles": 2, "valid_direction": 3.966666666666659}, "ep003": {"nsteps": 110, "reward": -9.560454756290314, "good_angle": 1.177928792743366, "survival_time": 3.6666666666666634, "traveled_tiles": 1, "valid_direction": 2.1333333333333284}, "ep004": {"nsteps": 82, "reward": -12.843059412499027, "good_angle": 0.7175396934921026, "survival_time": 2.7333333333333334, "traveled_tiles": 1, "valid_direction": 1.7333333333333334}}
good_angle_max26.454510536122378
good_angle_mean11.541473246950908
good_angle_median6.296167709086408
good_angle_min0.7175396934921026
reward_max-5.920791627916064
reward_mean-8.728941985209417
reward_median-9.38501587449288
reward_min-12.843059412499027
survival_time_max8.566666666666647
survival_time_mean5.346666666666658
survival_time_min2.7333333333333334
traveled_tiles_max3
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max3.966666666666659
valid_direction_mean3.0066666666666593
valid_direction_median3.2333333333333227
valid_direction_min1.7333333333333334
309165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:01:04Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
308165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:02:13Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
307165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:01:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
306165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:01:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
305165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:02:23Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
304165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno-2:01:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
298165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationabortedno-2:00:10
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/duckietown_challenges-3.0.36-py2.7.egg/duckietown_challenges/runner.py", line 378, in run
    pruned = client.networks.prune()
  File "/usr/local/lib/python2.7/dist-packages/docker/models/networks.py", line 214, in prune
    return self.client.api.prune_networks(filters=filters)
  File "/usr/local/lib/python2.7/dist-packages/docker/utils/decorators.py", line 34, in wrapper
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/docker/api/network.py", line 174, in prune_networks
    return self._result(self._post(url, params=params), True)
  File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 235, in _result
    self._raise_for_status(response)
  File "/usr/local/lib/python2.7/dist-packages/docker/api/client.py", line 231, in _raise_for_status
    raise create_api_error_from_http_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/docker/errors.py", line 31, in create_api_error_from_http_exception
    raise cls(e, response=response, explanation=explanation)
APIError: 409 Client Error: Conflict ("a prune operation is already running")
Hidden. If you are the author, please login using the top-right link or use the dashboard.
297165Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationabortedno-2:01:06Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1