AI Driving Olympics Home Challenges Submissions Jobs

Evaluator 293

evaluator293
ownerAndrea Censi
machinenutonomy-P50
processnutonomy-P50-2210
versionDC:3.1.54;DCR:3.2.14;DTS:3.0.30
first heard
last heard
statusinactive
# evaluating
# success16
# timeout
# failed1
# error1
# aborted1
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5937786martatintoreAMOD18-AIDO not that random executionaido1_LF1_r3-v3step3-videossuccessno2930:05:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5928799Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step4-vizabortedno2930:02:00Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5919798CpPIRandom executionaido1_LF1_r3-v3step3-videossuccessno2930:02:37Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5900792EdwardRandom executionaido1_LF1_r3-v3step4-vizsuccessno2930:00:56Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5890792EdwardRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2930:03:30Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
5880789miksazROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2930:02:14Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 147, "reward": -7.483778158215438, "good_angle": 11.5501667838303, "survival_time": 4.899999999999992, "traveled_tiles": 3, "valid_direction": 2.866666666666658}, "ep001": {"nsteps": 500, "reward": -0.32564113597408867, "good_angle": 18.040583723581097, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 3.1666666666666607}, "ep002": {"nsteps": 500, "reward": -0.3706761087477207, "good_angle": 0.04195710814195461, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -0.3631334312947947, "good_angle": 18.964130352810113, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 3.09999999999999}, "ep004": {"nsteps": 500, "reward": -0.7984821920012182, "good_angle": 2.844816453757069, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.266666666666653}}
good_angle_max18.964130352810113
good_angle_mean10.288330884424106
good_angle_median11.5501667838303
good_angle_min0.04195710814195461
reward_max-0.32564113597408867
reward_mean-1.8683422052466525
reward_median-0.3706761087477207
reward_min-7.483778158215438
survival_time_max16.666666666666654
survival_time_mean14.313333333333322
survival_time_min4.899999999999992
traveled_tiles_max5
traveled_tiles_mean4.2
traveled_tiles_median4
traveled_tiles_min3
valid_direction_max6.266666666666653
valid_direction_mean3.079999999999992
valid_direction_median3.09999999999999
valid_direction_min0
5877790heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationfailedno2930:02:39
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 118, in run
    solve(params, cis)
  File "solution.py", line 83, in solve
    observation, reward, done, info = env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 332, in step
    return self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/workspace/wrappers.py", line 86, in step
    ob, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/workspace/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator


Hidden. If you are the author, please login using the top-right link or use the dashboard.
5872785fmilanoAMOD18-AIDO not that random executionaido1_LF1_r3-v3step2-scoringsuccessno2930:01:45Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median1.500000000000001


other stats
episodes
details{"ep000": {"nsteps": 45, "reward": -22.71031572982255, "good_angle": 1.396690490222933, "survival_time": 1.500000000000001, "traveled_tiles": 1, "valid_direction": 1.2000000000000013}, "ep001": {"nsteps": 44, "reward": -23.13684787942012, "good_angle": 1.3536491238787145, "survival_time": 1.466666666666668, "traveled_tiles": 1, "valid_direction": 1.200000000000001}, "ep002": {"nsteps": 37, "reward": -27.19666234525298, "good_angle": 0.934874607755796, "survival_time": 1.2333333333333338, "traveled_tiles": 2, "valid_direction": 0.9666666666666672}, "ep003": {"nsteps": 57, "reward": -18.826057127325615, "good_angle": 1.97953109777104, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.566666666666669}, "ep004": {"nsteps": 74, "reward": -15.535065579253274, "good_angle": 2.326161475490153, "survival_time": 2.4666666666666677, "traveled_tiles": 2, "valid_direction": 2.133333333333334}}
good_angle_max2.326161475490153
good_angle_mean1.5981813590237273
good_angle_median1.396690490222933
good_angle_min0.934874607755796
reward_max-15.535065579253274
reward_mean-21.480989732214905
reward_median-22.71031572982255
reward_min-27.19666234525298
survival_time_max2.4666666666666677
survival_time_mean1.7133333333333347
survival_time_min1.2333333333333338
traveled_tiles_max2
traveled_tiles_mean1.6
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.133333333333334
valid_direction_mean1.4133333333333349
valid_direction_median1.2000000000000013
valid_direction_min0.9666666666666672
5861782zjdongAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2930:02:14Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5844781amuniAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2930:04:08Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5835783licAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationerrorno2930:00:52
InvalidEvaluator: Tr [...]
InvalidEvaluator:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 488, in wrap_evaluator
    evaluator.score(cie)
  File "eval.py", line 96, in score
    raise dc.InvalidEvaluator(msg)
InvalidEvaluator: Gym exited with code 2
Hidden. If you are the author, please login using the top-right link or use the dashboard.
5831780AmaurXAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2930:02:15Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5819778morraaAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2930:03:04Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5810778morraaAMOD18-AIDO not that random executionaido1_LF1_r3-v3step1-simulationsuccessno2930:01:58Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
5799775frtimAMOD18-AIDO not tht random executionaido1_LF1_r3-v3step2-scoringsuccessno2930:02:33Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.2629533237218855, "good_angle": 13.558177147634575, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.93333333333332}, "ep001": {"nsteps": 500, "reward": -1.1507397584803405, "good_angle": 30.37251736114832, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 13.33333333333332}, "ep002": {"nsteps": 500, "reward": -1.5188683025373613, "good_angle": 56.82600488254528, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 14.866666666666648}, "ep003": {"nsteps": 500, "reward": -1.2894317615032196, "good_angle": 13.520643297797143, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 12.866666666666651}, "ep004": {"nsteps": 500, "reward": -1.3244163292050362, "good_angle": 13.522749450401015, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 12.899999999999986}}
good_angle_max56.82600488254528
good_angle_mean25.560018427905263
good_angle_median13.558177147634575
good_angle_min13.520643297797143
reward_max-1.1507397584803405
reward_mean-1.3092818950895686
reward_median-1.2894317615032196
reward_min-1.5188683025373613
survival_time_max16.666666666666654
survival_time_mean16.666666666666654
survival_time_min16.666666666666654
traveled_tiles_max2
traveled_tiles_mean1.4
traveled_tiles_median1
traveled_tiles_min1
valid_direction_max14.866666666666648
valid_direction_mean13.379999999999988
valid_direction_median12.93333333333332
valid_direction_min12.866666666666651
5787773AmaurXAMOD18-AIDO not that random executionaido1_LF1_r3-v3step2-scoringsuccessno2930:02:35Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median4.166666666666662


other stats
episodes
details{"ep000": {"nsteps": 125, "reward": -10.275045363903043, "good_angle": 19.028327950707173, "survival_time": 4.166666666666662, "traveled_tiles": 2, "valid_direction": 3.0999999999999943}, "ep001": {"nsteps": 94, "reward": -10.949103168954954, "good_angle": 0.6273082138140861, "survival_time": 3.133333333333332, "traveled_tiles": 1, "valid_direction": 1.733333333333331}, "ep002": {"nsteps": 179, "reward": -6.57558033108814, "good_angle": 0.8388841699109951, "survival_time": 5.966666666666655, "traveled_tiles": 3, "valid_direction": 3.0333333333333226}, "ep003": {"nsteps": 185, "reward": -7.416382503509522, "good_angle": 19.876774516690535, "survival_time": 6.1666666666666545, "traveled_tiles": 3, "valid_direction": 2.766666666666657}, "ep004": {"nsteps": 66, "reward": -17.69188007441434, "good_angle": 0.1784253969051379, "survival_time": 2.200000000000002, "traveled_tiles": 2, "valid_direction": 0.4666666666666665}}
good_angle_max19.876774516690535
good_angle_mean8.109944049605584
good_angle_median0.8388841699109951
good_angle_min0.1784253969051379
reward_max-6.57558033108814
reward_mean-10.581598288374
reward_median-10.275045363903043
reward_min-17.69188007441434
survival_time_max6.1666666666666545
survival_time_mean4.326666666666661
survival_time_min2.200000000000002
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max3.0999999999999943
valid_direction_mean2.2199999999999944
valid_direction_median2.766666666666657
valid_direction_min0.4666666666666665
5779770sischaefAMOD18-AIDO not that random executionaido1_LF1_r3-v3step4-vizsuccessno2930:03:23Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5756766Andrea CensiRandom executionaido1_LFV_r1-v3step4-vizsuccessno2930:01:34Hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
5747766Andrea CensiRandom executionaido1_LFV_r1-v3step2-scoringsuccessno2930:02:02Hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median5.666666666666656


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.793226854240193, "good_angle": 24.514739373338095, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 180, "reward": -6.071764395137628, "good_angle": 0.009215519679207373, "survival_time": 5.9999999999999885, "traveled_tiles": 2, "valid_direction": 0}, "ep002": {"nsteps": 112, "reward": -10.107387881293628, "good_angle": 4.629534678140666, "survival_time": 3.73333333333333, "traveled_tiles": 1, "valid_direction": 2.9666666666666632}, "ep003": {"nsteps": 41, "reward": -26.19490526071409, "good_angle": 0.04891935475235413, "survival_time": 1.3666666666666676, "traveled_tiles": 1, "valid_direction": 0}, "ep004": {"nsteps": 500, "reward": -1.094580393433571, "good_angle": 0.09763253251803804, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 0}}
good_angle_max24.514739373338095
good_angle_mean5.860008291685672
good_angle_median0.09763253251803804
good_angle_min0.009215519679207373
reward_max-1.094580393433571
reward_mean-10.25237295696382
reward_median-7.793226854240193
reward_min-26.19490526071409
survival_time_max16.666666666666654
survival_time_mean6.6866666666666585
survival_time_min1.3666666666666676
traveled_tiles_max5
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max3.933333333333321
valid_direction_mean1.3799999999999968
valid_direction_median0
valid_direction_min0