AI Driving Olympics Home Challenges Submissions Jobs

Evaluator 296

evaluator296
ownerAndrea Censi
machineidsc-rudolf
processrudolf-7
versionDC:3.1.54;DCR:3.2.14;DTS:3.0.30
first heard
last heard
statusinactive
# evaluating
# success28
# timeout
# failed2
# error1
# aborted3
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
PI Camera
# Duckiebots
Map 3x3 avaiable
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5944799Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step3-videosabortedno2960:00:54(hidden)
other stats
videos1
5940786martatintoreAMOD18-AIDO not that random executionaido1_LF1_r3-v3step3-videossuccessno2960:00:46(hidden)
other stats
videos1
5933786martatintoreAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationsuccessno2960:01:58(hidden)
other stats
simulation-passed1
5929799Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step4-vizabortedno2960:01:09(hidden)
other stats
videos1
5922799Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationabortedno2960:02:32(hidden)
other stats
simulation-passed1
5915798CpPIRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2960:00:19(hidden)
survival_time_median8.999999999999979


other stats
episodes
details{"ep000": {"nsteps": 171, "reward": -7.729742120232499, "good_angle": 25.07383215378148, "survival_time": 5.6999999999999895, "traveled_tiles": 2, "valid_direction": 3.9333333333333207}, "ep001": {"nsteps": 382, "reward": -4.457744235767744, "good_angle": 24.218758506416364, "survival_time": 12.7333333333333, "traveled_tiles": 3, "valid_direction": 4.066666666666652}, "ep002": {"nsteps": 338, "reward": -3.039140198254471, "good_angle": 0.1268074882377727, "survival_time": 11.266666666666636, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 258, "reward": -6.04335063233856, "good_angle": 27.571145449058, "survival_time": 8.59999999999998, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 270, "reward": -5.542087158836701, "good_angle": 4.78818581978914, "survival_time": 8.999999999999979, "traveled_tiles": 3, "valid_direction": 2.9999999999999902}}
good_angle_max27.571145449058
good_angle_mean16.355745883456553
good_angle_median24.218758506416364
good_angle_min0.1268074882377727
reward_max-3.039140198254471
reward_mean-5.362412869085995
reward_median-5.542087158836701
reward_min-7.729742120232499
survival_time_max12.7333333333333
survival_time_mean9.459999999999976
survival_time_min5.6999999999999895
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max4.066666666666652
valid_direction_mean2.9866666666666566
valid_direction_median3.93333333333332
valid_direction_min0
5913797heyt0nySAIC MOSCOW MMLaido1_LF1_r3-v3step1-simulationfailedno2960:02:57
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 118, in run
    solve(params, cis)
  File "solution.py", line 83, in solve
    observation, reward, done, info = env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 332, in step
    return self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/workspace/wrappers.py", line 86, in step
    ob, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/workspace/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator


(hidden)
5911795Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizsuccessno2960:01:13(hidden)
other stats
videos1
5906794ManishAMOD18-AIDOaido1_LF1_r3-v3step3-videossuccessno2960:00:55(hidden)
other stats
videos1
5898793CpPISolution templateaido1_luck-v3step1successyes2960:00:42(hidden)
score140.29349400585619


5888792EdwardRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2960:01:42(hidden)
other stats
simulation-passed1
5884789miksazROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2960:01:26(hidden)
other stats
videos1
5876789miksazROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2960:07:29(hidden)
other stats
simulation-passed1
5874785fmilanoAMOD18-AIDO not that random executionaido1_LF1_r3-v3step3-videossuccessno2960:00:34(hidden)
other stats
videos1
5871788zxcvfd13502Random executionaido1_LF1_r3-v3step4-vizsuccessno2960:01:08(hidden)
other stats
videos1
5863784haliangRandom executionaido1_LF1_r3-v3step3-videossuccessno2960:00:49(hidden)
other stats
videos1
5857782zjdongAMOD18-AIDO not that random executionaido1_LF1_r3-v3step3-videossuccessno2960:00:50(hidden)
other stats
videos1
5848782zjdongAMOD18-AIDO not that random executionaido1_LF1_r3-v3step2-scoringsuccessno2960:02:07(hidden)
survival_time_median9.033333333333312


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.712571316957474, "good_angle": 25.023844936044405, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 376, "reward": -3.592209410199777, "good_angle": 23.219706056138055, "survival_time": 12.5333333333333, "traveled_tiles": 3, "valid_direction": 3.9333333333333194}, "ep002": {"nsteps": 389, "reward": -2.583105791011343, "good_angle": 0.11764789041300484, "survival_time": 12.966666666666631, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 259, "reward": -6.194280704936466, "good_angle": 25.345004752583304, "survival_time": 8.633333333333313, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 271, "reward": -5.501812648533318, "good_angle": 4.538982516695632, "survival_time": 9.033333333333312, "traveled_tiles": 3, "valid_direction": 2.966666666666657}}
good_angle_max25.345004752583304
good_angle_mean15.649037230374878
good_angle_median23.219706056138055
good_angle_min0.11764789041300484
reward_max-2.583105791011343
reward_mean-5.116795974327675
reward_median-5.501812648533318
reward_min-7.712571316957474
survival_time_max12.966666666666631
survival_time_mean9.76666666666664
survival_time_min5.666666666666656
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.933333333333321
valid_direction_mean2.953333333333324
valid_direction_median3.9333333333333194
valid_direction_min0
5845781amuniAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2960:01:02(hidden)
other stats
videos1
5838783licAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationerrorno2960:00:52
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 488, in wrap_evaluator
    evaluator.score(cie)
  File "eval.py", line 96, in score
    raise dc.InvalidEvaluator(msg)
InvalidEvaluator: Gym exited with code 2
(hidden)
5830780AmaurXAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2960:00:50(hidden)
other stats
videos1
5823780AmaurXAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationsuccessno2960:01:37(hidden)
other stats
simulation-passed1
5811778morraaAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationsuccessno2960:01:28(hidden)
other stats
simulation-passed1
5808775frtimAMOD18-AIDO not tht random executionaido1_LF1_r3-v3step3-videossuccessno2960:01:11(hidden)
other stats
videos1
5802777yanberAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationsuccessno2960:01:23(hidden)
other stats
simulation-passed1
5800772tomaszfAMOD18-AIDO not that random executionaido1_LF1_r3-v3step3-videossuccessno2960:00:40(hidden)
other stats
videos1
5788775frtimAMOD18-AIDO not tht random executionaido1_LF1_r3-v3step1-simulationsuccessno2960:02:29(hidden)
other stats
simulation-passed1
5781771mpicquetAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationfailedno2960:00:37
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 64, in run
    solve(gym_environment, cis)  # let's try to solve the challenge, exciting ah?
  File "solution.py", line 39, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 105, in step
    assert len(action) == 2
AssertionError


(hidden)
5778770sischaefAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2960:01:00(hidden)
other stats
videos1
5771768Dzenan LapandicRandom executionaido1_LF1_r3-v3step3-videossuccessno2960:00:44(hidden)
other stats
videos1
5767768Dzenan LapandicRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2960:01:58(hidden)
other stats
simulation-passed1
5761765Andrea CensiRandom executionaido1_LF1_r3-v3step4-vizsuccessno2960:01:51(hidden)
other stats
videos1
5751766Andrea CensiRandom executionaido1_LFV_r1-v3step3-videossuccessno2960:00:53(hidden)
other stats
videos1
5743765Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2960:02:05(hidden)
other stats
simulation-passed1