AI Driving Olympics Home Challenges Submissions Jobs

Evaluator 241

evaluator241
ownerAndrea Censi
machineidsc-rudolf
processrudolf-9
versionDC:3.1.52;DCR:3.2.12;DTS:3.0.30
first heard
last heard
statusinactive
# evaluating
# success217
# timeout174
# failed26
# error15
# aborted37
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
PI Camera
# Duckiebots
Map 3x3 avaiable
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
5739764miksazRL solutionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median9.299999999999978


other stats
episodes
details{"ep000": {"nsteps": 151, "reward": -7.049707989082348, "good_angle": 3.2308304545972866, "survival_time": 5.033333333333325, "traveled_tiles": 1, "valid_direction": 4.199999999999992}, "ep001": {"nsteps": 279, "reward": -3.891245903510694, "good_angle": 0.9299167932474692, "survival_time": 9.299999999999978, "traveled_tiles": 2, "valid_direction": 2.599999999999998}, "ep002": {"nsteps": 500, "reward": -0.31183041554270313, "good_angle": 0.42334523255448425, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 0.7333333333333307}, "ep003": {"nsteps": 203, "reward": -5.425280580441054, "good_angle": 2.0918994328631215, "survival_time": 6.766666666666652, "traveled_tiles": 2, "valid_direction": 5.499999999999988}, "ep004": {"nsteps": 500, "reward": -1.5710352215059102, "good_angle": 42.97100593665618, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 11.499999999999991}}
good_angle_max42.97100593665618
good_angle_mean9.929399569983708
good_angle_median2.0918994328631215
good_angle_min0.42334523255448425
reward_max-0.31183041554270313
reward_mean-3.6498200220165415
reward_median-3.891245903510694
reward_min-7.049707989082348
survival_time_max16.666666666666654
survival_time_mean10.886666666666653
survival_time_min5.033333333333325
traveled_tiles_max6
traveled_tiles_mean2.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max11.499999999999991
valid_direction_mean4.90666666666666
valid_direction_median4.199999999999992
valid_direction_min0.7333333333333307
5737763Bhairav MehtaTuned lane controller - ETHZ baseline extensionaido1_LF1_r3-v3step1-simulationabortedno2410:12:05
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 477, in wrap_evaluator
    cie.wait_for_solution()
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 270, in wait_for_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Time out: Timeout of 600 while waiting for /challenge-solution-output/output-solution.yaml.
(hidden)
5730762miksazRL solutionaido1_LF1_r3-v3step2-scoringsuccessno2410:02:19(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.4633594030737877, "good_angle": 8.964284849678757, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 15.83333333333332}, "ep001": {"nsteps": 500, "reward": -0.6293033194085583, "good_angle": 7.011532880445017, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 6.866666666666683}, "ep002": {"nsteps": 500, "reward": -0.20836991069852956, "good_angle": 0.3530570460101898, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -0.02938091635890305, "good_angle": 3.4537989437343484, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 14.76666666666666}, "ep004": {"nsteps": 500, "reward": -1.1880763600829296, "good_angle": 18.29885897621506, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 9.33333333333334}}
good_angle_max18.29885897621506
good_angle_mean7.616306539216675
good_angle_median7.011532880445017
good_angle_min0.3530570460101898
reward_max-0.02938091635890305
reward_mean-0.5036979819245416
reward_median-0.4633594030737877
reward_min-1.1880763600829296
survival_time_max16.666666666666654
survival_time_mean16.666666666666654
survival_time_min16.666666666666654
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max15.83333333333332
valid_direction_mean9.36
valid_direction_median9.33333333333334
valid_direction_min0
5725761Maxim KuzminROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:02:10(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 149, "reward": -7.392234276105094, "good_angle": 13.035006652002206, "survival_time": 4.966666666666659, "traveled_tiles": 3, "valid_direction": 2.933333333333324}, "ep001": {"nsteps": 500, "reward": -0.3217624191492796, "good_angle": 18.06820610150098, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 2.933333333333323}, "ep002": {"nsteps": 500, "reward": -0.38393971337378024, "good_angle": 0.04169037891178736, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 266, "reward": -4.495429577824092, "good_angle": 12.341967398129391, "survival_time": 8.866666666666646, "traveled_tiles": 4, "valid_direction": 2.866666666666658}, "ep004": {"nsteps": 500, "reward": -0.7817404213696718, "good_angle": 2.8608257095229974, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.066666666666652}}
good_angle_max18.06820610150098
good_angle_mean9.269539248013473
good_angle_median12.341967398129391
good_angle_min0.04169037891178736
reward_max-0.3217624191492796
reward_mean-2.6750212815643835
reward_median-0.7817404213696718
reward_min-7.392234276105094
survival_time_max16.666666666666654
survival_time_mean12.766666666666657
survival_time_min4.966666666666659
traveled_tiles_max5
traveled_tiles_mean4
traveled_tiles_median4
traveled_tiles_min3
valid_direction_max6.066666666666652
valid_direction_mean2.959999999999991
valid_direction_median2.933333333333323
valid_direction_min0
5721760yjz093Solution templateaido1_luck-v3step1successyes2410:00:47(hidden)
score190.27056076258516


5719759gunshiBaseline solution using imitation learning from logsaido1_LF1_r3-v3step4-vizsuccessno2410:01:56(hidden)
other stats
videos1
5716759gunshiBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationsuccessno2410:04:43(hidden)
other stats
simulation-passed1
5710757miksazRL solutionaido1_LF1_r3-v3step4-vizsuccessno2410:01:40(hidden)
other stats
videos1
5688754miksazPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationfailedno2410:03:40
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 100, in run
    solve(params, cis)
  File "solution.py", line 65, in solve
    observation, reward, done, info = env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 332, in step
    return self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 332, in step
    return self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 304, in step
    observation, reward, done, info = self.env.step(action)
  File "/opt/conda/lib/python2.7/site-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/workspace/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/workspace/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator


(hidden)
5679751placailleROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:06:16(hidden)
other stats
simulation-passed1
5676748VincentMaiROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:01:29(hidden)
other stats
videos1
5671748VincentMaiROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.5439484689310193, "good_angle": 28.83683186987422, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 7.166666666666678}, "ep001": {"nsteps": 436, "reward": -2.757260437761092, "good_angle": 15.102608798343253, "survival_time": 14.533333333333292, "traveled_tiles": 4, "valid_direction": 3.233333333333322}, "ep002": {"nsteps": 500, "reward": -0.4009358189553022, "good_angle": 0.07103457008622621, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0.06666666666666665}, "ep003": {"nsteps": 500, "reward": -0.36101953714492263, "good_angle": 18.935654578659303, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 3.3333333333333224}, "ep004": {"nsteps": 84, "reward": -14.466945367200037, "good_angle": 0.7720088784914125, "survival_time": 2.8, "traveled_tiles": 2, "valid_direction": 1.6333333333333329}}
good_angle_max28.83683186987422
good_angle_mean12.743627739090885
good_angle_median15.102608798343253
good_angle_min0.07103457008622621
reward_max-0.36101953714492263
reward_mean-3.706021925998474
reward_median-0.5439484689310193
reward_min-14.466945367200037
survival_time_max16.666666666666654
survival_time_mean13.46666666666665
survival_time_min2.8
traveled_tiles_max5
traveled_tiles_mean4
traveled_tiles_median4
traveled_tiles_min2
valid_direction_max7.166666666666678
valid_direction_mean3.086666666666664
valid_direction_median3.233333333333322
valid_direction_min0.06666666666666665
5621737heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.7947607793244533, "good_angle": 50.871895001177585, "survival_time": 16.666666666666654, "traveled_tiles": 7, "valid_direction": 8.79999999999997}, "ep001": {"nsteps": 500, "reward": -0.791930663626641, "good_angle": 36.79047601703185, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 7.233333333333344}, "ep002": {"nsteps": 500, "reward": -1.007000924691558, "good_angle": 1.2174371064857936, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 2.0999999999999925}, "ep003": {"nsteps": 500, "reward": -0.8188124289009721, "good_angle": 42.098294342947696, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 6.466666666666679}, "ep004": {"nsteps": 500, "reward": -1.3624944336237386, "good_angle": 5.402375667365257, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 6.700000000000013}}
good_angle_max50.871895001177585
good_angle_mean27.27609562700163
good_angle_median36.79047601703185
good_angle_min1.2174371064857936
reward_max-0.791930663626641
reward_mean-0.9549998460334724
reward_median-0.8188124289009721
reward_min-1.3624944336237386
survival_time_max16.666666666666654
survival_time_mean16.666666666666654
survival_time_min16.666666666666654
traveled_tiles_max7
traveled_tiles_mean6.2
traveled_tiles_median6
traveled_tiles_min6
valid_direction_max8.79999999999997
valid_direction_mean6.26
valid_direction_median6.700000000000013
valid_direction_min2.0999999999999925
5598730heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:02:14(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.05452753536724049, "good_angle": 24.23535417432326, "survival_time": 16.666666666666654, "traveled_tiles": 7, "valid_direction": 4.833333333333322}, "ep001": {"nsteps": 500, "reward": -0.12353763392486145, "good_angle": 15.52345061305662, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 2.7999999999999954}, "ep002": {"nsteps": 175, "reward": -6.045018788062568, "good_angle": 0.0924096889573637, "survival_time": 5.833333333333322, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 294, "reward": -5.256518963851085, "good_angle": 25.865919918500797, "survival_time": 9.799999999999976, "traveled_tiles": 3, "valid_direction": 5.933333333333313}, "ep004": {"nsteps": 500, "reward": -0.7373143969518132, "good_angle": 1.845400832835972, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 5.499999999999981}}
good_angle_max25.865919918500797
good_angle_mean13.512507045534804
good_angle_median15.52345061305662
good_angle_min0.0924096889573637
reward_max-0.05452753536724049
reward_mean-2.4433834636315135
reward_median-0.7373143969518132
reward_min-6.045018788062568
survival_time_max16.666666666666654
survival_time_mean13.126666666666653
survival_time_min5.833333333333322
traveled_tiles_max7
traveled_tiles_mean5
traveled_tiles_median5
traveled_tiles_min3
valid_direction_max5.933333333333313
valid_direction_mean3.813333333333323
valid_direction_median4.833333333333322
valid_direction_min0
5594729heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizsuccessno2410:01:13(hidden)
other stats
videos1
5585727heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:15(hidden)
survival_time_median4.066666666666662


other stats
episodes
details{"ep000": {"nsteps": 122, "reward": -9.584048272156323, "good_angle": 4.15024539593361, "survival_time": 4.066666666666662, "traveled_tiles": 1, "valid_direction": 3.2999999999999954}, "ep001": {"nsteps": 125, "reward": -9.326586581707, "good_angle": 4.253873169300722, "survival_time": 4.166666666666662, "traveled_tiles": 1, "valid_direction": 3.366666666666662}, "ep002": {"nsteps": 150, "reward": -7.492702591568231, "good_angle": 5.0505519054772465, "survival_time": 4.999999999999992, "traveled_tiles": 2, "valid_direction": 4.0999999999999925}, "ep003": {"nsteps": 97, "reward": -12.444948281209491, "good_angle": 2.6592831813146716, "survival_time": 3.2333333333333316, "traveled_tiles": 2, "valid_direction": 2.5666666666666647}, "ep004": {"nsteps": 70, "reward": -16.90499336378915, "good_angle": 1.0227292718412762, "survival_time": 2.333333333333335, "traveled_tiles": 2, "valid_direction": 1.666666666666668}}
good_angle_max5.0505519054772465
good_angle_mean3.427336584773505
good_angle_median4.15024539593361
good_angle_min1.0227292718412762
reward_max-7.492702591568231
reward_mean-11.15065581808604
reward_median-9.584048272156323
reward_min-16.90499336378915
survival_time_max4.999999999999992
survival_time_mean3.7599999999999967
survival_time_min2.333333333333335
traveled_tiles_max2
traveled_tiles_mean1.6
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max4.0999999999999925
valid_direction_mean2.9999999999999964
valid_direction_median3.2999999999999954
valid_direction_min1.666666666666668
5574725JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:21(hidden)
other stats
videos1
5572725JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.4406520579289646, "good_angle": 40.96600144619954, "survival_time": 16.666666666666654, "traveled_tiles": 16, "valid_direction": 6.766666666666651}, "ep001": {"nsteps": 500, "reward": -1.199175238309428, "good_angle": 40.123178141166946, "survival_time": 16.666666666666654, "traveled_tiles": 15, "valid_direction": 5.9666666666666455}, "ep002": {"nsteps": 500, "reward": -0.1594446971627185, "good_angle": 1.63454841808679, "survival_time": 16.666666666666654, "traveled_tiles": 15, "valid_direction": 2.9999999999999893}, "ep003": {"nsteps": 274, "reward": -5.511251442569451, "good_angle": 31.406782331891332, "survival_time": 9.133333333333312, "traveled_tiles": 10, "valid_direction": 4.099999999999991}, "ep004": {"nsteps": 88, "reward": -13.374755779302426, "good_angle": 1.7719252236731329, "survival_time": 2.9333333333333327, "traveled_tiles": 5, "valid_direction": 1.1333333333333315}}
good_angle_max40.96600144619954
good_angle_mean23.18048711220355
good_angle_median31.406782331891332
good_angle_min1.63454841808679
reward_max-0.1594446971627185
reward_mean-4.337055843054598
reward_median-1.4406520579289646
reward_min-13.374755779302426
survival_time_max16.666666666666654
survival_time_mean12.41333333333332
survival_time_min2.9333333333333327
traveled_tiles_max16
traveled_tiles_mean12.2
traveled_tiles_median15
traveled_tiles_min5
valid_direction_max6.766666666666651
valid_direction_mean4.193333333333322
valid_direction_median4.099999999999991
valid_direction_min1.1333333333333315
5562723BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:00:49(hidden)
other stats
videos1
5557722BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step3-videossuccessno2410:01:15(hidden)
other stats
videos1
5548722BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:04:03(hidden)
other stats
simulation-passed1
5546721BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:01:42(hidden)
other stats
videos1
5540721BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.3583180518115114, "good_angle": 35.388577433801856, "survival_time": 16.666666666666654, "traveled_tiles": 13, "valid_direction": 6.066666666666652}, "ep001": {"nsteps": 500, "reward": -0.32232150391326286, "good_angle": 27.701117487878584, "survival_time": 16.666666666666654, "traveled_tiles": 13, "valid_direction": 5.33333333333335}, "ep002": {"nsteps": 500, "reward": -0.6438964181095361, "good_angle": 1.3346811401222445, "survival_time": 16.666666666666654, "traveled_tiles": 17, "valid_direction": 2.5999999999999917}, "ep003": {"nsteps": 409, "reward": -2.8165878027633133, "good_angle": 23.786906038949652, "survival_time": 13.633333333333296, "traveled_tiles": 11, "valid_direction": 6.233333333333315}, "ep004": {"nsteps": 500, "reward": -0.7099452675133944, "good_angle": 1.1987815010916103, "survival_time": 16.666666666666654, "traveled_tiles": 16, "valid_direction": 2.8333333333333277}}
good_angle_max35.388577433801856
good_angle_mean17.88201272036879
good_angle_median23.786906038949652
good_angle_min1.1987815010916103
reward_max-0.32232150391326286
reward_mean-0.9702138088222036
reward_median-0.6438964181095361
reward_min-2.8165878027633133
survival_time_max16.666666666666654
survival_time_mean16.05999999999998
survival_time_min13.633333333333296
traveled_tiles_max17
traveled_tiles_mean14
traveled_tiles_median13
traveled_tiles_min11
valid_direction_max6.233333333333315
valid_direction_mean4.613333333333327
valid_direction_median5.33333333333335
valid_direction_min2.5999999999999917
5533719BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:05:29(hidden)
other stats
simulation-passed1
5528717miksazPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationsuccessno2410:03:19(hidden)
other stats
simulation-passed1
5522715MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step3-videoserrorno2410:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job5522-8799', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job5522-8799', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
5509713BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.6631416728068144, "good_angle": 25.37730008367105, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 6.0666666666666496}, "ep001": {"nsteps": 500, "reward": -0.34981133888754995, "good_angle": 25.89019170460868, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 4.933333333333317}, "ep002": {"nsteps": 500, "reward": -0.625916432082653, "good_angle": 1.154775913319328, "survival_time": 16.666666666666654, "traveled_tiles": 7, "valid_direction": 3.533333333333321}, "ep003": {"nsteps": 500, "reward": -0.3946926848629955, "good_angle": 36.599325058878726, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 7.366666666666677}, "ep004": {"nsteps": 500, "reward": -0.7433039758391679, "good_angle": 2.6250036309808085, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.366666666666648}}
good_angle_max36.599325058878726
good_angle_mean18.329319278291717
good_angle_median25.37730008367105
good_angle_min1.154775913319328
reward_max-0.34981133888754995
reward_mean-0.5553732208958362
reward_median-0.625916432082653
reward_min-0.7433039758391679
survival_time_max16.666666666666654
survival_time_mean16.666666666666654
survival_time_min16.666666666666654
traveled_tiles_max7
traveled_tiles_mean5.4
traveled_tiles_median5
traveled_tiles_min4
valid_direction_max7.366666666666677
valid_direction_mean5.653333333333322
valid_direction_median6.0666666666666496
valid_direction_min3.533333333333321
5498712Liam PaullROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:01:33(hidden)
other stats
videos1
5451702BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:07:08(hidden)
other stats
simulation-passed1
5448700BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step3-videossuccessno2410:01:55(hidden)
other stats
videos1
5442700BenjaminMy modified ROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:07:05(hidden)
other stats
simulation-passed1
5433699MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:08:19(hidden)
other stats
simulation-passed1
5429696MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step3-videossuccessno2410:02:13(hidden)
other stats
videos1
5426696MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:08:29(hidden)
other stats
simulation-passed1
5424695BenjaminROS-based Lane Followingaido1_LF1_r3-v3step3-videossuccessno2410:03:33(hidden)
other stats
videos1
5420694BenjaminROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailedno2410:03:47
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "./solution.py", line 93, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: Giving up to connect to the gym duckietown server at host: evaluator

(hidden)
5415693BenjaminROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median4.399999999999994


other stats
episodes
details{"ep000": {"nsteps": 72, "reward": -14.567843713073266, "good_angle": 5.409950047682946, "survival_time": 2.4000000000000012, "traveled_tiles": 3, "valid_direction": 1.3666666666666685}, "ep001": {"nsteps": 210, "reward": -5.208157076970452, "good_angle": 6.8180968465147345, "survival_time": 6.999999999999985, "traveled_tiles": 4, "valid_direction": 1.4333333333333282}, "ep002": {"nsteps": 46, "reward": -21.826466938905902, "good_angle": 0.8028761321397663, "survival_time": 1.5333333333333348, "traveled_tiles": 2, "valid_direction": 1.3000000000000014}, "ep003": {"nsteps": 132, "reward": -8.285709033296866, "good_angle": 6.550680495557027, "survival_time": 4.399999999999994, "traveled_tiles": 4, "valid_direction": 1.6666666666666616}, "ep004": {"nsteps": 500, "reward": -0.7081116912816651, "good_angle": 1.6195188128960034, "survival_time": 16.666666666666654, "traveled_tiles": 8, "valid_direction": 3.533333333333323}}
good_angle_max6.8180968465147345
good_angle_mean4.240224466958096
good_angle_median5.409950047682946
good_angle_min0.8028761321397663
reward_max-0.7081116912816651
reward_mean-10.11925769070563
reward_median-8.285709033296866
reward_min-21.826466938905902
survival_time_max16.666666666666654
survival_time_mean6.399999999999993
survival_time_min1.5333333333333348
traveled_tiles_max8
traveled_tiles_mean4.2
traveled_tiles_median4
traveled_tiles_min2
valid_direction_max3.533333333333323
valid_direction_mean1.8599999999999963
valid_direction_median1.4333333333333282
valid_direction_min1.3000000000000014
5411692MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.5231579206675815, "good_angle": 28.464478016032334, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.333333333333348}, "ep001": {"nsteps": 500, "reward": -0.31609272902086377, "good_angle": 17.538302246765024, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 2.8999999999999897}, "ep002": {"nsteps": 500, "reward": -0.38211745707690714, "good_angle": 0.04292090082765797, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 271, "reward": -4.448645181651485, "good_angle": 11.543214536801488, "survival_time": 9.033333333333312, "traveled_tiles": 4, "valid_direction": 2.866666666666658}, "ep004": {"nsteps": 500, "reward": -0.7898046460151672, "good_angle": 2.789892196969417, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.333333333333318}}
good_angle_max28.464478016032334
good_angle_mean12.075761579479185
good_angle_median11.543214536801488
good_angle_min0.04292090082765797
reward_max-0.31609272902086377
reward_mean-1.291963586886401
reward_median-0.5231579206675815
reward_min-4.448645181651485
survival_time_max16.666666666666654
survival_time_mean15.139999999999985
survival_time_min9.033333333333312
traveled_tiles_max5
traveled_tiles_mean4.4
traveled_tiles_median4
traveled_tiles_min4
valid_direction_max6.333333333333348
valid_direction_mean3.6866666666666625
valid_direction_median2.8999999999999897
valid_direction_min0
5406691MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 147, "reward": -7.480147649439013, "good_angle": 12.936741904799534, "survival_time": 4.899999999999992, "traveled_tiles": 3, "valid_direction": 2.966666666666658}, "ep001": {"nsteps": 417, "reward": -2.8512310617686, "good_angle": 13.097591619358909, "survival_time": 13.89999999999996, "traveled_tiles": 4, "valid_direction": 2.8666666666666565}, "ep002": {"nsteps": 500, "reward": -0.37255497513711455, "good_angle": 0.03398209534597587, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -0.3874121592715382, "good_angle": 19.730310644349633, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 3.3333333333333224}, "ep004": {"nsteps": 500, "reward": -0.7757100465288386, "good_angle": 2.8003055291338157, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.066666666666652}}
good_angle_max19.730310644349633
good_angle_mean9.719786358597572
good_angle_median12.936741904799534
good_angle_min0.03398209534597587
reward_max-0.37255497513711455
reward_mean-2.373411178429021
reward_median-0.7757100465288386
reward_min-7.480147649439013
survival_time_max16.666666666666654
survival_time_mean13.759999999999986
survival_time_min4.899999999999992
traveled_tiles_max5
traveled_tiles_mean4.2
traveled_tiles_median4
traveled_tiles_min3
valid_direction_max6.066666666666652
valid_direction_mean3.0466666666666575
valid_direction_median2.966666666666658
valid_direction_min0
5403690yun chenRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:31(hidden)
other stats
videos1
5394688MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median14.09999999999996


other stats
episodes
details{"ep000": {"nsteps": 148, "reward": -7.431404875541055, "good_angle": 12.738676669549523, "survival_time": 4.933333333333326, "traveled_tiles": 3, "valid_direction": 2.966666666666658}, "ep001": {"nsteps": 423, "reward": -2.8331109985497824, "good_angle": 12.362033404302332, "survival_time": 14.09999999999996, "traveled_tiles": 4, "valid_direction": 3.0333333333333226}, "ep002": {"nsteps": 500, "reward": -0.3844907549321652, "good_angle": 0.036203257846110753, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -0.3662934238284361, "good_angle": 19.312327801366663, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 3.3666666666666565}, "ep004": {"nsteps": 80, "reward": -15.13905012011528, "good_angle": 0.6538558751059325, "survival_time": 2.666666666666667, "traveled_tiles": 2, "valid_direction": 1.2666666666666653}}
good_angle_max19.312327801366663
good_angle_mean9.020619401634113
good_angle_median12.362033404302332
good_angle_min0.036203257846110753
reward_max-0.3662934238284361
reward_mean-5.230870034593344
reward_median-2.8331109985497824
reward_min-15.13905012011528
survival_time_max16.666666666666654
survival_time_mean11.00666666666665
survival_time_min2.666666666666667
traveled_tiles_max5
traveled_tiles_mean3.6
traveled_tiles_median4
traveled_tiles_min2
valid_direction_max3.3666666666666565
valid_direction_mean2.1266666666666607
valid_direction_median2.966666666666658
valid_direction_min0
5380684Maxim KuzminROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:01:28(hidden)
other stats
videos1
5378684Maxim KuzminROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 150, "reward": -7.327975000838439, "good_angle": 14.525063232991249, "survival_time": 4.999999999999992, "traveled_tiles": 3, "valid_direction": 2.966666666666657}, "ep001": {"nsteps": 418, "reward": -2.8526243432402683, "good_angle": 11.157167456008327, "survival_time": 13.933333333333294, "traveled_tiles": 4, "valid_direction": 2.6999999999999904}, "ep002": {"nsteps": 500, "reward": -0.3866348545998335, "good_angle": 0.061309959778751186, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0.033333333333333326}, "ep003": {"nsteps": 500, "reward": -0.3716373074473813, "good_angle": 18.897023400467955, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 3.1999999999999895}, "ep004": {"nsteps": 500, "reward": -0.7995721662753494, "good_angle": 2.9464125568008797, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.566666666666653}}
good_angle_max18.897023400467955
good_angle_mean9.517395321209433
good_angle_median11.157167456008327
good_angle_min0.061309959778751186
reward_max-0.3716373074473813
reward_mean-2.347688734480254
reward_median-0.7995721662753494
reward_min-7.327975000838439
survival_time_max16.666666666666654
survival_time_mean13.786666666666653
survival_time_min4.999999999999992
traveled_tiles_max5
traveled_tiles_mean4.2
traveled_tiles_median4
traveled_tiles_min3
valid_direction_max6.566666666666653
valid_direction_mean3.0933333333333244
valid_direction_median2.966666666666657
valid_direction_min0.033333333333333326
5368682MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:02:09(hidden)
other stats
videos1
5364681Maxim KuzminTemplate for ROS Submissionaido1_LF1_r3-v3step4-vizsuccessno2410:00:58(hidden)
other stats
videos1
5356681Maxim KuzminTemplate for ROS Submissionaido1_LF1_r3-v3step1-simulationsuccessno2410:04:28(hidden)
other stats
simulation-passed1
5353680PolyProgrammistRandom executionaido1_LF1_r3-v3step3-videossuccessno2410:00:43(hidden)
other stats
videos1
5347679MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:02:23(hidden)
other stats
videos1
5339676gunshiROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:01:28(hidden)
other stats
videos1
5337676gunshiROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.5388972914866171, "good_angle": 29.057884183825465, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 7.500000000000011}, "ep001": {"nsteps": 500, "reward": -0.31558736668806525, "good_angle": 18.137565781950062, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 2.9999999999999893}, "ep002": {"nsteps": 500, "reward": -0.3998104407936335, "good_angle": 0.05126862267456619, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -0.3674015710903332, "good_angle": 18.531900752126656, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 3.3666666666666556}, "ep004": {"nsteps": 82, "reward": -14.834476252881492, "good_angle": 0.6466970649111259, "survival_time": 2.7333333333333334, "traveled_tiles": 2, "valid_direction": 1.1666666666666647}}
good_angle_max29.057884183825465
good_angle_mean13.285063281097576
good_angle_median18.137565781950062
good_angle_min0.05126862267456619
reward_max-0.31558736668806525
reward_mean-3.291234584588028
reward_median-0.3998104407936335
reward_min-14.834476252881492
survival_time_max16.666666666666654
survival_time_mean13.87999999999999
survival_time_min2.7333333333333334
traveled_tiles_max5
traveled_tiles_mean4
traveled_tiles_median4
traveled_tiles_min2
valid_direction_max7.500000000000011
valid_direction_mean3.006666666666664
valid_direction_median2.9999999999999893
valid_direction_min0
5302667JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median15.23333333333329


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.089489060146967, "good_angle": 42.50883035362384, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 10.866666666666632}, "ep001": {"nsteps": 457, "reward": -3.328711111269712, "good_angle": 23.74858426614005, "survival_time": 15.23333333333329, "traveled_tiles": 5, "valid_direction": 6.433333333333311}, "ep002": {"nsteps": 382, "reward": -3.758374079262243, "good_angle": 1.6653612812436744, "survival_time": 12.7333333333333, "traveled_tiles": 5, "valid_direction": 2.5666666666666575}, "ep003": {"nsteps": 500, "reward": -1.0516083194464445, "good_angle": 41.83166610917556, "survival_time": 16.666666666666654, "traveled_tiles": 7, "valid_direction": 6.1000000000000085}, "ep004": {"nsteps": 178, "reward": -7.865400701426388, "good_angle": 0.9720002224876048, "survival_time": 5.933333333333322, "traveled_tiles": 3, "valid_direction": 1.4666666666666617}}
good_angle_max42.50883035362384
good_angle_mean22.145288446534146
good_angle_median23.74858426614005
good_angle_min0.9720002224876048
reward_max-1.0516083194464445
reward_mean-3.4187166543103507
reward_median-3.328711111269712
reward_min-7.865400701426388
survival_time_max16.666666666666654
survival_time_mean13.446666666666644
survival_time_min5.933333333333322
traveled_tiles_max7
traveled_tiles_mean5.2
traveled_tiles_median5
traveled_tiles_min3
valid_direction_max10.866666666666632
valid_direction_mean5.486666666666655
valid_direction_median6.1000000000000085
valid_direction_min1.4666666666666617
5297665aleksandarpetrTuned lane controller - ETHZ baseline extensionaido1_LF1_r3-v3step1-simulationfailedno2410:10:21
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 477, in wrap_evaluator
    cie.wait_for_solution()
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 270, in wait_for_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Time out: Timeout of 600 while waiting for /challenge-solution-output/output-solution.yaml.
(hidden)
5294664ManishTuned lane controller - ETHZ baseline extensionaido1_LF1_r3-v3step1-simulationfailedno2410:10:31
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 477, in wrap_evaluator
    cie.wait_for_solution()
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 270, in wait_for_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Time out: Timeout of 600 while waiting for /challenge-solution-output/output-solution.yaml.
(hidden)
5287661gunshiTemplate for ROS Submissionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median3.6999999999999966


other stats
episodes
details{"ep000": {"nsteps": 124, "reward": -10.013951965397403, "good_angle": 19.37086350788007, "survival_time": 4.133333333333328, "traveled_tiles": 2, "valid_direction": 2.8333333333333277}, "ep001": {"nsteps": 111, "reward": -9.270556113794909, "good_angle": 0.4804694140713148, "survival_time": 3.6999999999999966, "traveled_tiles": 2, "valid_direction": 1.6999999999999968}, "ep002": {"nsteps": 73, "reward": -13.83623775277105, "good_angle": 0.659371664540585, "survival_time": 2.4333333333333345, "traveled_tiles": 2, "valid_direction": 1.566666666666668}, "ep003": {"nsteps": 166, "reward": -8.230933695672507, "good_angle": 13.576030423600743, "survival_time": 5.5333333333333234, "traveled_tiles": 3, "valid_direction": 2.333333333333325}, "ep004": {"nsteps": 79, "reward": -15.269296552561508, "good_angle": 0.1811754366367032, "survival_time": 2.6333333333333337, "traveled_tiles": 2, "valid_direction": 0.6999999999999988}}
good_angle_max19.37086350788007
good_angle_mean6.853582089345885
good_angle_median0.659371664540585
good_angle_min0.1811754366367032
reward_max-8.230933695672507
reward_mean-11.324195216039474
reward_median-10.013951965397403
reward_min-15.269296552561508
survival_time_max5.5333333333333234
survival_time_mean3.686666666666663
survival_time_min2.4333333333333345
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max2.8333333333333277
valid_direction_mean1.826666666666663
valid_direction_median1.6999999999999968
valid_direction_min0.6999999999999988
5282660iban_alexPyTorch templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:15(hidden)
survival_time_median5.566666666666657


other stats
episodes
details{"ep000": {"nsteps": 140, "reward": -9.273192444017957, "good_angle": 15.722703005021485, "survival_time": 4.66666666666666, "traveled_tiles": 2, "valid_direction": 3.0333333333333248}, "ep001": {"nsteps": 226, "reward": -4.634353206410543, "good_angle": 0.4022384868181178, "survival_time": 7.533333333333316, "traveled_tiles": 2, "valid_direction": 0.033333333333333215}, "ep002": {"nsteps": 167, "reward": -6.033847034742225, "good_angle": 0.24878993700028496, "survival_time": 5.566666666666657, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 150, "reward": -8.837737381458282, "good_angle": 8.154583031456879, "survival_time": 4.999999999999992, "traveled_tiles": 3, "valid_direction": 1.4333333333333282}, "ep004": {"nsteps": 299, "reward": -4.601150532369891, "good_angle": 3.3704399545984214, "survival_time": 9.966666666666642, "traveled_tiles": 3, "valid_direction": 4.099999999999986}}
good_angle_max15.722703005021485
good_angle_mean5.579750882979036
good_angle_median3.3704399545984214
good_angle_min0.24878993700028496
reward_max-4.601150532369891
reward_mean-6.676056119799779
reward_median-6.033847034742225
reward_min-9.273192444017957
survival_time_max9.966666666666642
survival_time_mean6.546666666666654
survival_time_min4.66666666666666
traveled_tiles_max3
traveled_tiles_mean2.6
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max4.099999999999986
valid_direction_mean1.7199999999999946
valid_direction_median1.4333333333333282
valid_direction_min0
5274658aleksandarpetrTuned lane controller - ETHZ baseline extensionaido1_LF1_r3-v3step1-simulationfailedno2410:10:21
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 477, in wrap_evaluator
    cie.wait_for_solution()
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 270, in wait_for_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Time out: Timeout of 600 while waiting for /challenge-solution-output/output-solution.yaml.
(hidden)
5267656aleksandarpetrTuned lane controller - ETHZ baseline extensionaido1_LF1_r3-v3step1-simulationfailedno2410:10:20
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 477, in wrap_evaluator
    cie.wait_for_solution()
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 270, in wait_for_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Time out: Timeout of 600 while waiting for /challenge-solution-output/output-solution.yaml.
(hidden)
5265655miksazDDPGaido1_LF1_r3-v3step3-videossuccessno2410:00:51(hidden)
other stats
videos1
5259653iban_alexTemplate for ROS Submissionaido1_LF1_r3-v3step4-vizsuccessno2410:00:53(hidden)
other stats
videos1
5235646Julian ZillyROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:06:16(hidden)
other stats
simulation-passed1
5229644heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:01:31(hidden)
other stats
videos1
5223642miksazRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:07(hidden)
other stats
videos1
5219642miksazRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median8.966666666666645


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.706741852269453, "good_angle": 24.64479871236659, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.899999999999987}, "ep001": {"nsteps": 375, "reward": -3.6739715505838393, "good_angle": 22.83957644280516, "survival_time": 12.499999999999966, "traveled_tiles": 3, "valid_direction": 3.9333333333333194}, "ep002": {"nsteps": 500, "reward": -0.14389567598188296, "good_angle": 0.3288594622363396, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 0.9000000000000323}, "ep003": {"nsteps": 258, "reward": -6.0509295953336615, "good_angle": 26.069345910823735, "survival_time": 8.59999999999998, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 269, "reward": -5.610776582374463, "good_angle": 5.127024492581995, "survival_time": 8.966666666666645, "traveled_tiles": 3, "valid_direction": 2.9999999999999902}}
good_angle_max26.069345910823735
good_angle_mean15.801921004162764
good_angle_median22.83957644280516
good_angle_min0.3288594622363396
reward_max-0.14389567598188296
reward_mean-4.6372630513086595
reward_median-5.610776582374463
reward_min-7.706741852269453
survival_time_max16.666666666666654
survival_time_mean10.47999999999998
survival_time_min5.666666666666656
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.93333333333332
valid_direction_mean3.13333333333333
valid_direction_median3.899999999999987
valid_direction_min0.9000000000000323
5196636JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:02:03(hidden)
other stats
videos1
5193636JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:02:21(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.1122087126707192, "good_angle": 50.40463661992644, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 12.199999999999962}, "ep001": {"nsteps": 500, "reward": -0.9444563968166476, "good_angle": 24.82625101482094, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 7.266666666666677}, "ep002": {"nsteps": 429, "reward": -3.4185708913179265, "good_angle": 1.6738886928328292, "survival_time": 14.29999999999996, "traveled_tiles": 5, "valid_direction": 2.5999999999999908}, "ep003": {"nsteps": 500, "reward": -1.1791939079463485, "good_angle": 36.93715410045572, "survival_time": 16.666666666666654, "traveled_tiles": 6, "valid_direction": 5.43333333333335}, "ep004": {"nsteps": 187, "reward": -7.59629884633151, "good_angle": 0.9873761327044304, "survival_time": 6.233333333333321, "traveled_tiles": 3, "valid_direction": 1.699999999999994}}
good_angle_max50.40463661992644
good_angle_mean22.96586131214807
good_angle_median24.82625101482094
good_angle_min0.9873761327044304
reward_max-0.9444563968166476
reward_mean-2.8501457510166306
reward_median-1.1791939079463485
reward_min-7.59629884633151
survival_time_max16.666666666666654
survival_time_mean14.10666666666665
survival_time_min6.233333333333321
traveled_tiles_max6
traveled_tiles_mean5
traveled_tiles_median5
traveled_tiles_min3
valid_direction_max12.199999999999962
valid_direction_mean5.8399999999999945
valid_direction_median5.43333333333335
valid_direction_min1.699999999999994
5186635JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:01:52(hidden)
other stats
videos1
5181633tnd_duck18Solution templateaido1_luck-v3step1successyes2410:00:46(hidden)
score128.888039220678465


5176628heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:03:31(hidden)
other stats
videos1
5164628heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:00(hidden)
5162629lavoiemsPyTorch templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:03(hidden)
5158629lavoiemsPyTorch templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:00(hidden)
5154629lavoiemsPyTorch templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:01(hidden)
5151629lavoiemsPyTorch templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:03(hidden)
5150629lavoiemsPyTorch templateaido1_LF1_r3-v3step1-simulationsuccessno2410:02:59(hidden)
other stats
simulation-passed1
5148595heyt0nyPyTorch templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:02(hidden)
5141595heyt0nyPyTorch templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:02(hidden)
5137627heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:00:43(hidden)
other stats
videos1
5131605DavidRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:45(hidden)
other stats
videos1
5124605DavidRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median9.033333333333312


other stats
episodes
details{"ep000": {"nsteps": 169, "reward": -7.824909472253901, "good_angle": 24.243147217990867, "survival_time": 5.633333333333323, "traveled_tiles": 2, "valid_direction": 3.899999999999988}, "ep001": {"nsteps": 378, "reward": -3.5262110224634253, "good_angle": 23.31296872293371, "survival_time": 12.599999999999966, "traveled_tiles": 3, "valid_direction": 3.966666666666653}, "ep002": {"nsteps": 402, "reward": -2.644075601845887, "good_angle": 0.11324115025088108, "survival_time": 13.399999999999965, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 259, "reward": -6.0955510742415795, "good_angle": 26.09740699496414, "survival_time": 8.633333333333313, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 271, "reward": -5.456060972174705, "good_angle": 4.593241289748081, "survival_time": 9.033333333333312, "traveled_tiles": 3, "valid_direction": 2.9999999999999902}}
good_angle_max26.09740699496414
good_angle_mean15.672001075177535
good_angle_median23.31296872293371
good_angle_min0.11324115025088108
reward_max-2.644075601845887
reward_mean-5.1093616285959
reward_median-5.456060972174705
reward_min-7.824909472253901
survival_time_max13.399999999999965
survival_time_mean9.859999999999976
survival_time_min5.633333333333323
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.966666666666653
valid_direction_mean2.95999999999999
valid_direction_median3.899999999999988
valid_direction_min0
5118595heyt0nyPyTorch templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:03(hidden)
5111606lavoiemsPyTorch templateaido1_LF1_r3-v3step4-vizsuccessno2410:01:01(hidden)
other stats
videos1
5104606lavoiemsPyTorch templateaido1_LF1_r3-v3step3-videossuccessno2410:01:29(hidden)
other stats
videos1
5098624DavidPyTorch templateaido1_LF1_r3-v3step4-vizsuccessno2410:01:05(hidden)
other stats
videos1
5091624DavidPyTorch templateaido1_LF1_r3-v3step3-videossuccessno2410:01:22(hidden)
other stats
videos1
5088624DavidPyTorch templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:18(hidden)
survival_time_median5.933333333333322


other stats
episodes
details{"ep000": {"nsteps": 142, "reward": -9.917901731712718, "good_angle": 16.846760959914825, "survival_time": 4.733333333333326, "traveled_tiles": 2, "valid_direction": 3.5666666666666593}, "ep001": {"nsteps": 178, "reward": -7.297310151410906, "good_angle": 0.5916927943423778, "survival_time": 5.933333333333322, "traveled_tiles": 2, "valid_direction": 2.299999999999992}, "ep002": {"nsteps": 187, "reward": -5.444137409613293, "good_angle": 0.3367102143986073, "survival_time": 6.233333333333321, "traveled_tiles": 3, "valid_direction": 0.5666666666666647}, "ep003": {"nsteps": 278, "reward": -5.44840099442777, "good_angle": 28.24790521094908, "survival_time": 9.266666666666644, "traveled_tiles": 3, "valid_direction": 4.199999999999986}, "ep004": {"nsteps": 59, "reward": -19.60689396373296, "good_angle": 0.1995172026463348, "survival_time": 1.9666666666666697, "traveled_tiles": 2, "valid_direction": 0.766666666666669}}
good_angle_max28.24790521094908
good_angle_mean9.244517276450244
good_angle_median0.5916927943423778
good_angle_min0.1995172026463348
reward_max-5.444137409613293
reward_mean-9.54292885017953
reward_median-7.297310151410906
reward_min-19.60689396373296
survival_time_max9.266666666666644
survival_time_mean5.626666666666656
survival_time_min1.9666666666666697
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max4.199999999999986
valid_direction_mean2.279999999999994
valid_direction_median2.299999999999992
valid_direction_min0.5666666666666647
5077624DavidPyTorch templateaido1_LF1_r3-v3step1-simulationsuccessno2410:03:48(hidden)
other stats
simulation-passed1
5073608DavidRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2410:03:06(hidden)
other stats
simulation-passed1
5067611DavidRandom executionaido1_LF1_r3-v3step4-vizerrorno2410:00:11
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job5067-1513', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job5067-1513', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
5061613lmandrilePyTorch templateaido1_LF1_r3-v3step3-videossuccessno2410:01:18(hidden)
other stats
videos1
5056611DavidRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2410:02:58(hidden)
other stats
simulation-passed1
5052618yun chenSolution templateaido1_luck-v3step1successyes2410:00:21(hidden)
score141.20631332428603


5051616lmandrilePyTorch templateaido1_LF1_r3-v3step4-vizsuccessno2410:01:01(hidden)
other stats
videos1
5049616lmandrilePyTorch templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median5.866666666666656


other stats
episodes
details{"ep000": {"nsteps": 142, "reward": -9.355348875824834, "good_angle": 16.15872564054106, "survival_time": 4.733333333333326, "traveled_tiles": 2, "valid_direction": 3.166666666666658}, "ep001": {"nsteps": 176, "reward": -7.429431302303618, "good_angle": 0.5355208355709057, "survival_time": 5.866666666666656, "traveled_tiles": 2, "valid_direction": 1.1333333333333293}, "ep002": {"nsteps": 191, "reward": -5.358579120837659, "good_angle": 0.23348835226556688, "survival_time": 6.366666666666654, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 181, "reward": -7.646736396610407, "good_angle": 10.739359937801254, "survival_time": 6.033333333333322, "traveled_tiles": 3, "valid_direction": 1.9333333333333265}, "ep004": {"nsteps": 62, "reward": -18.755157839867376, "good_angle": 0.17664296003277655, "survival_time": 2.066666666666669, "traveled_tiles": 2, "valid_direction": 0.5000000000000009}}
good_angle_max16.15872564054106
good_angle_mean5.568747545242313
good_angle_median0.5355208355709057
good_angle_min0.17664296003277655
reward_max-5.358579120837659
reward_mean-9.709050707088778
reward_median-7.646736396610407
reward_min-18.755157839867376
survival_time_max6.366666666666654
survival_time_mean5.013333333333326
survival_time_min2.066666666666669
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max3.166666666666658
valid_direction_mean1.346666666666663
valid_direction_median1.1333333333333293
valid_direction_min0
5043600heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationtimeoutno2410:31:15(hidden)
5040603heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationtimeoutno2410:31:19(hidden)
5034609DavidRandom executionaido1_LF1_r3-v3step1-simulationtimeoutno2410:31:00(hidden)
5033614lavoiemsPyTorch templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:31:01(hidden)
5031599heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationtimeoutno2410:31:02(hidden)
5023610ayushsoniSolution templateaido1_luck-v3step1timeoutyes2410:31:00(hidden)
5018602heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationtimeoutno2410:36:31(hidden)
5015605DavidRandom executionaido1_LF1_r3-v3step1-simulationtimeoutno2410:36:35(hidden)
4996601heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationtimeoutno2410:38:14(hidden)
4969595heyt0nyPyTorch templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:31:04(hidden)
4965597heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationtimeoutno2410:36:36(hidden)
4964594heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationtimeoutno2410:41:11(hidden)
4956488JonJP pipelineaido1_LF1_r3-v3step1-simulationtimeoutno2410:33:04(hidden)
4946488JonJP pipelineaido1_LF1_r3-v3step1-simulationtimeoutno2410:32:37(hidden)
493855Andrea CensiSolution templateaido1_luck-v3step1successyes2410:01:16(hidden)
score143.94530292619362


493715Andrea Censinullaido1_log_processing-v3logstep1abortedyes2410:00:02
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 303, in go_
    challenge_step_name, project, do_pull)
  File "/project/src/duckietown_challenges_runner/runner.py", line 351, in run_single
    prepare_dir(wd, aws_config, steps2artefacts)
  File "/project/src/duckietown_challenges_runner/runner.py", line 551, in prepare_dir
    os.makedirs(wd)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 150, in makedirs
    makedirs(head, mode)
  File "/usr/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18'
(hidden)
4928488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno241----
4917488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:04(hidden)
4915590Allen OuTensorflow templateaido1_LF1_r3-v3step4-vizsuccessno2410:01:07(hidden)
other stats
videos1
4911590Allen OuTensorflow templateaido1_LF1_r3-v3step1-simulationsuccessno2410:02:13(hidden)
other stats
simulation-passed1
4907589Allen OuRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:13(hidden)
other stats
videos1
4905589Allen OuRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median8.999999999999979


other stats
episodes
details{"ep000": {"nsteps": 171, "reward": -7.8863654373682035, "good_angle": 23.908231293454904, "survival_time": 5.6999999999999895, "traveled_tiles": 2, "valid_direction": 3.9333333333333207}, "ep001": {"nsteps": 376, "reward": -3.855074338773464, "good_angle": 24.270587087612824, "survival_time": 12.5333333333333, "traveled_tiles": 3, "valid_direction": 3.899999999999986}, "ep002": {"nsteps": 500, "reward": 0.03386699096229859, "good_angle": 0.5247332069054883, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 1.466666666666697}, "ep003": {"nsteps": 258, "reward": -6.128380675648534, "good_angle": 25.682366412276533, "survival_time": 8.59999999999998, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 270, "reward": -5.633314865970708, "good_angle": 5.408137036986907, "survival_time": 8.999999999999979, "traveled_tiles": 3, "valid_direction": 2.9999999999999902}}
good_angle_max25.682366412276533
good_angle_mean15.958811007447329
good_angle_median23.908231293454904
good_angle_min0.5247332069054883
reward_max0.03386699096229859
reward_mean-4.693853665359722
reward_median-5.633314865970708
reward_min-7.8863654373682035
survival_time_max16.666666666666654
survival_time_mean10.49999999999998
survival_time_min5.6999999999999895
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.9333333333333207
valid_direction_mean3.2466666666666626
valid_direction_median3.899999999999986
valid_direction_min1.466666666666697
4898488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:00(hidden)
4895488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:08(hidden)
4892488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:00(hidden)
4891488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:05(hidden)
4881584JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:06:20(hidden)
other stats
simulation-passed1
4871583JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:06:25(hidden)
other stats
simulation-passed1
4864581JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:03:22(hidden)
other stats
simulation-passed1
4863580JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:00:51(hidden)
other stats
videos1
4858580JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median4.699999999999993


other stats
episodes
details{"ep000": {"nsteps": 222, "reward": -6.359649095180872, "good_angle": 27.388063439098673, "survival_time": 7.399999999999984, "traveled_tiles": 3, "valid_direction": 4.899999999999988}, "ep001": {"nsteps": 141, "reward": -9.073139651024595, "good_angle": 0.6233174032214033, "survival_time": 4.699999999999993, "traveled_tiles": 2, "valid_direction": 2.099999999999997}, "ep002": {"nsteps": 173, "reward": -7.460346334630009, "good_angle": 0.7461143637402835, "survival_time": 5.766666666666656, "traveled_tiles": 3, "valid_direction": 2.766666666666662}, "ep003": {"nsteps": 104, "reward": -11.815939071086737, "good_angle": 0.5588586146605803, "survival_time": 3.466666666666664, "traveled_tiles": 2, "valid_direction": 1.3000000000000005}, "ep004": {"nsteps": 83, "reward": -14.738914880407862, "good_angle": 0.24759771750360615, "survival_time": 2.7666666666666666, "traveled_tiles": 2, "valid_direction": 0.36666666666666536}}
good_angle_max27.388063439098673
good_angle_mean5.912790307644909
good_angle_median0.6233174032214033
good_angle_min0.24759771750360615
reward_max-6.359649095180872
reward_mean-9.889597806466016
reward_median-9.073139651024595
reward_min-14.738914880407862
survival_time_max7.399999999999984
survival_time_mean4.819999999999992
survival_time_min2.7666666666666666
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max4.899999999999988
valid_direction_mean2.286666666666662
valid_direction_median2.099999999999997
valid_direction_min0.36666666666666536
4848579JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:03(hidden)
other stats
videos1
4843578JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:05(hidden)
other stats
videos1
4839578JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:03:49(hidden)
other stats
simulation-passed1
4837577JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:00:53(hidden)
other stats
videos1
4831576JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:01:01(hidden)
other stats
videos1
4827575JonJP pipelineaido1_LF1_r3-v3step1-simulationfailedno2410:00:31
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 63, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: local variable 'omega' referenced before assignment

(hidden)
4825488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:05(hidden)
4821574trimcaoPyTorch templateaido1_LF1_r3-v3step4-vizsuccessno2410:00:54(hidden)
other stats
videos1
4817574trimcaoPyTorch templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median5.433333333333324


other stats
episodes
details{"ep000": {"nsteps": 139, "reward": -9.22843984562716, "good_angle": 15.887726283857342, "survival_time": 4.633333333333327, "traveled_tiles": 2, "valid_direction": 3.0666666666666584}, "ep001": {"nsteps": 163, "reward": -8.051187865207531, "good_angle": 0.6003773000910735, "survival_time": 5.433333333333324, "traveled_tiles": 2, "valid_direction": 1.933333333333329}, "ep002": {"nsteps": 165, "reward": -6.13072710690683, "good_angle": 0.2581357457693412, "survival_time": 5.49999999999999, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 192, "reward": -7.511248488600056, "good_angle": 13.813209884311137, "survival_time": 6.399999999999987, "traveled_tiles": 3, "valid_direction": 2.299999999999992}, "ep004": {"nsteps": 76, "reward": -15.740712852854475, "good_angle": 0.17373617892537446, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 0.16666666666666652}}
good_angle_max15.887726283857342
good_angle_mean6.146637078590854
good_angle_median0.6003773000910735
good_angle_min0.17373617892537446
reward_max-6.13072710690683
reward_mean-9.33246323183921
reward_median-8.051187865207531
reward_min-15.740712852854475
survival_time_max6.399999999999987
survival_time_mean4.899999999999993
survival_time_min2.533333333333334
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max3.0666666666666584
valid_direction_mean1.4933333333333292
valid_direction_median1.933333333333329
valid_direction_min0
4813573JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:00:57(hidden)
other stats
videos1
4809571JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:17(hidden)
other stats
videos1
4804571JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median11.633333333333304


other stats
episodes
details{"ep000": {"nsteps": 358, "reward": -3.6225417243697966, "good_angle": 18.469789239470646, "survival_time": 11.933333333333302, "traveled_tiles": 3, "valid_direction": 8.96666666666664}, "ep001": {"nsteps": 349, "reward": -3.673431348834317, "good_angle": 16.676676850768253, "survival_time": 11.633333333333304, "traveled_tiles": 4, "valid_direction": 5.1999999999999815}, "ep002": {"nsteps": 273, "reward": -4.708979325662384, "good_angle": 1.2563777701214542, "survival_time": 9.099999999999978, "traveled_tiles": 4, "valid_direction": 1.8333333333333277}, "ep003": {"nsteps": 500, "reward": -0.9746185102467424, "good_angle": 37.49237582246197, "survival_time": 16.666666666666654, "traveled_tiles": 7, "valid_direction": 6.866666666666678}, "ep004": {"nsteps": 117, "reward": -10.833399183729776, "good_angle": 0.7198292815837966, "survival_time": 3.899999999999996, "traveled_tiles": 2, "valid_direction": 1.2666666666666622}}
good_angle_max37.49237582246197
good_angle_mean14.923009792881226
good_angle_median16.676676850768253
good_angle_min0.7198292815837966
reward_max-0.9746185102467424
reward_mean-4.762594018568603
reward_median-3.673431348834317
reward_min-10.833399183729776
survival_time_max16.666666666666654
survival_time_mean10.646666666666649
survival_time_min3.899999999999996
traveled_tiles_max7
traveled_tiles_mean4
traveled_tiles_median4
traveled_tiles_min2
valid_direction_max8.96666666666664
valid_direction_mean4.826666666666658
valid_direction_median5.1999999999999815
valid_direction_min1.2666666666666622
4792570JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:04:58(hidden)
other stats
simulation-passed1
4787569JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:05:23(hidden)
other stats
simulation-passed1
4783567JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median8.666666666666647


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.7467835637026001, "good_angle": 33.34727872016288, "survival_time": 16.666666666666654, "traveled_tiles": 9, "valid_direction": 8.066666666666647}, "ep001": {"nsteps": 500, "reward": -0.869724905455485, "good_angle": 24.315377554516807, "survival_time": 16.666666666666654, "traveled_tiles": 8, "valid_direction": 6.566666666666644}, "ep002": {"nsteps": 260, "reward": -4.892326775794992, "good_angle": 0.9184990893135082, "survival_time": 8.666666666666647, "traveled_tiles": 5, "valid_direction": 1.5333333333333288}, "ep003": {"nsteps": 86, "reward": -12.874912440083747, "good_angle": 1.6130375737417066, "survival_time": 2.8666666666666663, "traveled_tiles": 3, "valid_direction": 0.8666666666666636}, "ep004": {"nsteps": 29, "reward": -37.013473157224986, "good_angle": 0.2462435577903238, "survival_time": 0.9666666666666666, "traveled_tiles": 2, "valid_direction": 0.4333333333333333}}
good_angle_max33.34727872016288
good_angle_mean12.088087299105045
good_angle_median1.6130375737417066
good_angle_min0.2462435577903238
reward_max-0.7467835637026001
reward_mean-11.27944416845236
reward_median-4.892326775794992
reward_min-37.013473157224986
survival_time_max16.666666666666654
survival_time_mean9.16666666666666
survival_time_min0.9666666666666666
traveled_tiles_max9
traveled_tiles_mean5.4
traveled_tiles_median5
traveled_tiles_min2
valid_direction_max8.066666666666647
valid_direction_mean3.493333333333323
valid_direction_median1.5333333333333288
valid_direction_min0.4333333333333333
4777565JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:04:28(hidden)
other stats
simulation-passed1
4776488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:04(hidden)
4772564JonJP pipelineaido1_LF1_r3-v3step1-simulationfailedno2410:00:56
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 63, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: expected non-empty vector for x

(hidden)
4769562JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:15(hidden)
other stats
videos1
4766562JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:01:30(hidden)
other stats
videos1
4762562JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:21(hidden)
survival_time_median10.599999999999971


other stats
episodes
details{"ep000": {"nsteps": 243, "reward": -5.052297092548767, "good_angle": 15.339197912912562, "survival_time": 8.099999999999982, "traveled_tiles": 4, "valid_direction": 5.699999999999986}, "ep001": {"nsteps": 372, "reward": -4.232470288071581, "good_angle": 23.508310868718105, "survival_time": 12.399999999999968, "traveled_tiles": 5, "valid_direction": 5.199999999999982}, "ep002": {"nsteps": 318, "reward": -4.36347883288403, "good_angle": 1.4644345761069255, "survival_time": 10.599999999999971, "traveled_tiles": 6, "valid_direction": 2.4333333333333247}, "ep003": {"nsteps": 500, "reward": -0.9941675311401488, "good_angle": 37.827818538112474, "survival_time": 16.666666666666654, "traveled_tiles": 7, "valid_direction": 6.400000000000009}, "ep004": {"nsteps": 152, "reward": -8.926124608830401, "good_angle": 0.8918140733947912, "survival_time": 5.066666666666658, "traveled_tiles": 3, "valid_direction": 1.6333333333333275}}
good_angle_max37.827818538112474
good_angle_mean15.80631519384897
good_angle_median15.339197912912562
good_angle_min0.8918140733947912
reward_max-0.9941675311401488
reward_mean-4.713707670694985
reward_median-4.36347883288403
reward_min-8.926124608830401
survival_time_max16.666666666666654
survival_time_mean10.566666666666649
survival_time_min5.066666666666658
traveled_tiles_max7
traveled_tiles_mean5
traveled_tiles_median5
traveled_tiles_min3
valid_direction_max6.400000000000009
valid_direction_mean4.273333333333326
valid_direction_median5.199999999999982
valid_direction_min1.6333333333333275
4760488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:04(hidden)
4754561JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:01:28(hidden)
other stats
videos1
4747560Dzenan LapandicBaseline solution using imitation learning from logsaido1_LFV_r1-v3step3-videossuccessno2410:01:22(hidden)
other stats
videos1
4739556JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:09(hidden)
other stats
videos1
4714488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:03(hidden)
4703488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:03(hidden)
4698524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videostimeoutno2410:31:02(hidden)
4691551JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.45223559687659143, "good_angle": 32.104666536889944, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 14.66666666666666}, "ep001": {"nsteps": 500, "reward": -2.1534141018390653, "good_angle": 115.73651774118628, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 12.933333333333325}, "ep002": {"nsteps": 500, "reward": -0.8519684868138283, "good_angle": 23.74105689643963, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 12.06666666666666}, "ep003": {"nsteps": 500, "reward": -1.6816085438728332, "good_angle": 0.31941645860210116, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 0.13333333333333375}, "ep004": {"nsteps": 500, "reward": -2.860552188873291, "good_angle": 0.38688295777628984, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 0.1333333333333334}}
good_angle_max115.73651774118628
good_angle_mean34.45770811817885
good_angle_median23.74105689643963
good_angle_min0.31941645860210116
reward_max-0.45223559687659143
reward_mean-1.599955783655122
reward_median-1.6816085438728332
reward_min-2.860552188873291
survival_time_max16.666666666666654
survival_time_mean16.666666666666654
survival_time_min16.666666666666654
traveled_tiles_max4
traveled_tiles_mean2.8
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max14.66666666666666
valid_direction_mean7.986666666666662
valid_direction_median12.06666666666666
valid_direction_min0.1333333333333334
4689550JonJP pipelineaido1_LF1_r3-v3step1-simulationfailedno2410:07:13
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 63, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: Giving up to connect to the gym duckietown server at host: evaluator

(hidden)
4688549JonJP pipelineaido1_LF1_r3-v3step1-simulationfailedno2410:00:25
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 63, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: 'module' object has no attribute 'np'

(hidden)
4686488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:01(hidden)
4681524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:00:48(hidden)
other stats
videos1
4678548JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:00:51(hidden)
other stats
videos1
4675548JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:00:49(hidden)
other stats
videos1
4673548JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:02:32(hidden)
other stats
simulation-passed1
4670546JonJP pipelineaido1_LF1_r3-v3step1-simulationfailedno2410:00:28
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 63, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: unbound method step() must be called with step_JP instance as first argument (got ndarray instance instead)

(hidden)
4669545yun chenSolution templateaido1_luck-v3step1successyes2410:00:45(hidden)
score133.04355719048442


4667488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:06(hidden)
4664488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:03(hidden)
4661524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:00:52(hidden)
other stats
videos1
4657488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:01(hidden)
4653524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:18(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4650488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:01(hidden)
4648488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:00(hidden)
4644524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:19(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4640488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:01(hidden)
4638524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:19(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4636543JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:00:53(hidden)
other stats
videos1
4631542JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:00:35(hidden)
other stats
videos1
4623488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:02(hidden)
4619539JonRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median8.966666666666645


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.778510023916469, "good_angle": 24.533829092157166, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 377, "reward": -3.550914846182818, "good_angle": 22.993722241015856, "survival_time": 12.566666666666633, "traveled_tiles": 3, "valid_direction": 3.9333333333333194}, "ep002": {"nsteps": 500, "reward": 0.15113034510482976, "good_angle": 0.4603662689427866, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 1.2000000000000313}, "ep003": {"nsteps": 259, "reward": -5.995273528412042, "good_angle": 26.860868523571614, "survival_time": 8.633333333333313, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 269, "reward": -5.528992282946812, "good_angle": 4.518499269846921, "survival_time": 8.966666666666645, "traveled_tiles": 3, "valid_direction": 3.033333333333323}}
good_angle_max26.860868523571614
good_angle_mean15.87345707910687
good_angle_median22.993722241015856
good_angle_min0.4603662689427866
reward_max0.15113034510482976
reward_mean-4.5405120672706625
reward_median-5.528992282946812
reward_min-7.778510023916469
survival_time_max16.666666666666654
survival_time_mean10.49999999999998
survival_time_min5.666666666666656
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.933333333333321
valid_direction_mean3.206666666666663
valid_direction_median3.9333333333333194
valid_direction_min1.2000000000000313
4617540JonJP pipelineaido1_LF1_r3-v3step1-simulationfailedno2410:00:28
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 63, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: local variable 'a' referenced before assignment

(hidden)
4613524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:00(hidden)
4611538JonRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:16(hidden)
other stats
videos1
4593524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:00(hidden)
4589536BenjaminPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:15(hidden)
survival_time_median2.966666666666666


other stats
episodes
details{"ep000": {"nsteps": 89, "reward": -13.127272469274114, "good_angle": 10.665048509791976, "survival_time": 2.966666666666666, "traveled_tiles": 2, "valid_direction": 1.966666666666666}, "ep001": {"nsteps": 88, "reward": -12.823042251847006, "good_angle": 0.661973213908129, "survival_time": 2.9333333333333327, "traveled_tiles": 2, "valid_direction": 1.933333333333333}, "ep002": {"nsteps": 101, "reward": -11.170705346630353, "good_angle": 0.9513547538946444, "survival_time": 3.3666666666666645, "traveled_tiles": 2, "valid_direction": 2.3999999999999977}, "ep003": {"nsteps": 145, "reward": -9.278553971750982, "good_angle": 3.1949558105588975, "survival_time": 4.833333333333326, "traveled_tiles": 3, "valid_direction": 0.36666666666666536}, "ep004": {"nsteps": 50, "reward": -22.65142310142517, "good_angle": 0.2515818940641918, "survival_time": 1.6666666666666683, "traveled_tiles": 2, "valid_direction": 1.100000000000002}}
good_angle_max10.665048509791976
good_angle_mean3.1449828364435675
good_angle_median0.9513547538946444
good_angle_min0.2515818940641918
reward_max-9.278553971750982
reward_mean-13.810199428185523
reward_median-12.823042251847006
reward_min-22.65142310142517
survival_time_max4.833333333333326
survival_time_mean3.153333333333331
survival_time_min1.6666666666666683
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max2.3999999999999977
valid_direction_mean1.5533333333333328
valid_direction_median1.933333333333333
valid_direction_min0.36666666666666536
4584488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:04(hidden)
4580535heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationfailedno2410:00:32
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 102, in run
    solve(params, cis)
  File "solution.py", line 59, in solve
    observation = env.reset()
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 335, in reset
    return self.env.reset()
  File "/opt/conda/lib/python2.7/site-packages/gym/core.py", line 308, in reset
    observation = self.env.reset(**kwargs)
  File "/workspace/wrappers.py", line 89, in reset
    self.append_frame(ob)
  File "/workspace/wrappers.py", line 104, in append_frame
    ob = self.convert_img_to_gray(ob)
  File "/workspace/wrappers.py", line 108, in convert_img_to_gray
    if self.env.distortion:
AttributeError: 'TimeLimit' object has no attribute 'distortion'


(hidden)
4577524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:20(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4570533heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:02:25(hidden)
other stats
videos1
4564534heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizsuccessno2410:00:42(hidden)
other stats
videos1
4562524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:21(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4560534heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:18(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4556531heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizerrorno2410:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job4556-8437', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job4556-8437', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
4551531heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationsuccessno2410:02:47(hidden)
other stats
simulation-passed1
4549488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:00(hidden)
4544528Dzenan LapandicBaseline solution using imitation learning from logsaido1_LFV_r1-v3step3-videossuccessno2410:00:57(hidden)
other stats
videos1
4539524heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:18(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4537530JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:01:20(hidden)
other stats
videos1
4532527heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizsuccessno2410:00:42(hidden)
other stats
videos1
4531488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:08(hidden)
4530488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:04(hidden)
4528525heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizerrorno2410:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job4528-361', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job4528-361', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
4524527heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videoserrorno2410:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job4524-7365', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job4524-7365', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
4521527heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median3.0999999999999988


other stats
episodes
details{"ep000": {"nsteps": 93, "reward": -11.707524042937065, "good_angle": 4.558573615649922, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.333333333333332}, "ep001": {"nsteps": 95, "reward": -12.149568855135064, "good_angle": 2.2168865970853284, "survival_time": 3.166666666666665, "traveled_tiles": 1, "valid_direction": 2.366666666666666}, "ep002": {"nsteps": 109, "reward": -10.434394388844114, "good_angle": 2.993975240420685, "survival_time": 3.63333333333333, "traveled_tiles": 2, "valid_direction": 2.6999999999999966}, "ep003": {"nsteps": 76, "reward": -15.223712928985297, "good_angle": 1.3084261872204903, "survival_time": 2.533333333333334, "traveled_tiles": 2, "valid_direction": 1.8666666666666671}, "ep004": {"nsteps": 57, "reward": -20.1426948706309, "good_angle": 0.5654761377833007, "survival_time": 1.9000000000000024, "traveled_tiles": 2, "valid_direction": 1.2333333333333358}}
good_angle_max4.558573615649922
good_angle_mean2.3286675556319456
good_angle_median2.2168865970853284
good_angle_min0.5654761377833007
reward_max-10.434394388844114
reward_mean-13.931579017306486
reward_median-12.149568855135064
reward_min-20.1426948706309
survival_time_max3.63333333333333
survival_time_mean2.8666666666666663
survival_time_min1.9000000000000024
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.6999999999999966
valid_direction_mean2.0999999999999996
valid_direction_median2.333333333333332
valid_direction_min1.2333333333333358
4512525heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationsuccessno2410:03:11(hidden)
other stats
simulation-passed1
4503523heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizsuccessno2410:01:43(hidden)
other stats
videos1
4473488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:00(hidden)
4470516heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizsuccessno2410:00:43(hidden)
other stats
videos1
4460514JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:32(hidden)
other stats
videos1
4455512JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:35(hidden)
other stats
videos1
4451513Dzenan LapandicBaseline solution using imitation learning from logsaido1_LF1_r3-v3step3-videossuccessno2410:00:58(hidden)
other stats
videos1
4445508JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:40(hidden)
other stats
videos1
4442510Dzenan LapandicBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationfailedno2410:00:55
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 120, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: Giving up to connect to the gym duckietown server at host: evaluator

(hidden)
4439509Dzenan LapandicBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationfailedno2410:00:39
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 120, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: can't multiply sequence by non-int of type 'float'

(hidden)
4436488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:01(hidden)
4426488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:02(hidden)
4420504JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:41(hidden)
other stats
videos1
4416504JonJP pipelineaido1_LF1_r3-v3step3-videoserrorno2410:00:11
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job4416-8746', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job4416-8746', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
4400488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:00(hidden)
4393501Manfred DiazTensorflow templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median7.499999999999983


other stats
episodes
details{"ep000": {"nsteps": 95, "reward": -11.11132962695372, "good_angle": 3.481079425999562, "survival_time": 3.166666666666665, "traveled_tiles": 3, "valid_direction": 2.666666666666665}, "ep001": {"nsteps": 281, "reward": -4.521031430077298, "good_angle": 15.84928678080855, "survival_time": 9.366666666666644, "traveled_tiles": 4, "valid_direction": 3.4999999999999885}, "ep002": {"nsteps": 500, "reward": -0.16403283863254184, "good_angle": 0.24367834358166565, "survival_time": 16.666666666666654, "traveled_tiles": 7, "valid_direction": 0.2333333333333325}, "ep003": {"nsteps": 216, "reward": -6.629146578135313, "good_angle": 14.501328060144532, "survival_time": 7.199999999999984, "traveled_tiles": 3, "valid_direction": 3.599999999999987}, "ep004": {"nsteps": 225, "reward": -4.7080057672977755, "good_angle": 1.5709037293728847, "survival_time": 7.499999999999983, "traveled_tiles": 3, "valid_direction": 3.499999999999996}}
good_angle_max15.84928678080855
good_angle_mean7.129255267981439
good_angle_median3.481079425999562
good_angle_min0.24367834358166565
reward_max-0.16403283863254184
reward_mean-5.426709248219329
reward_median-4.7080057672977755
reward_min-11.11132962695372
survival_time_max16.666666666666654
survival_time_mean8.779999999999987
survival_time_min3.166666666666665
traveled_tiles_max7
traveled_tiles_mean4
traveled_tiles_median3
traveled_tiles_min3
valid_direction_max3.599999999999987
valid_direction_mean2.699999999999994
valid_direction_median3.4999999999999885
valid_direction_min0.2333333333333325
4391501Manfred DiazTensorflow templateaido1_LF1_r3-v3step1-simulationsuccessno2410:02:27(hidden)
other stats
simulation-passed1
4386488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:01(hidden)
4380498heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:00:35(hidden)
other stats
videos1
4376497heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationfailedno2410:00:32
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 590, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 93, in run
    solve(params, cis)
  File "solution.py", line 32, in solve
    env = FrameStackWrappper(env=env, k=3)
  File "/workspace/wrappers.py", line 76, in __init__
    self.undistort = UndistortWrapper(env)
  File "/workspace/wrappers.py", line 185, in __init__
    self.env.unwrapped.undistort = True
AttributeError: 'UndistortWrapper' object has no attribute 'env'


(hidden)
4372495heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:00:40(hidden)
other stats
videos1
4368488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:05(hidden)
4367488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:01(hidden)
4364494heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step3-videossuccessno2410:00:40(hidden)
other stats
videos1
4360488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:04(hidden)
4358488JonJP pipelineaido1_LF1_r3-v3step4-vizabortedno2410:31:03(hidden)
4356493heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step4-vizsuccessno2410:00:43(hidden)
other stats
videos1
4354493heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median3.0666666666666655


other stats
episodes
details{"ep000": {"nsteps": 92, "reward": -13.48861142474672, "good_angle": 10.741238477608729, "survival_time": 3.0666666666666655, "traveled_tiles": 2, "valid_direction": 2.1999999999999993}, "ep001": {"nsteps": 94, "reward": -12.171223379830096, "good_angle": 0.509084796303537, "survival_time": 3.133333333333332, "traveled_tiles": 2, "valid_direction": 1.5333333333333303}, "ep002": {"nsteps": 110, "reward": -10.20598264472051, "good_angle": 0.6540528014792222, "survival_time": 3.6666666666666634, "traveled_tiles": 3, "valid_direction": 1.7999999999999945}, "ep003": {"nsteps": 73, "reward": -15.73796364052655, "good_angle": 0.31774501506334774, "survival_time": 2.4333333333333345, "traveled_tiles": 2, "valid_direction": 1.1000000000000003}, "ep004": {"nsteps": 53, "reward": -21.41955620387815, "good_angle": 0.12778965102043954, "survival_time": 1.7666666666666688, "traveled_tiles": 2, "valid_direction": 0.4333333333333347}}
good_angle_max10.741238477608729
good_angle_mean2.4699821482950552
good_angle_median0.509084796303537
good_angle_min0.12778965102043954
reward_max-10.20598264472051
reward_mean-14.604667458740405
reward_median-13.48861142474672
reward_min-21.41955620387815
survival_time_max3.6666666666666634
survival_time_mean2.813333333333333
survival_time_min1.7666666666666688
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max2.1999999999999993
valid_direction_mean1.4133333333333318
valid_direction_median1.5333333333333303
valid_direction_min0.4333333333333347
4335490MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step4-viztimeoutno2410:31:01(hidden)
4328488JonJP pipelineaido1_LF1_r3-v3step3-videosabortedno2410:31:00(hidden)
4327490MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step4-viztimeoutno2410:31:00(hidden)
4326490MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step3-videossuccessno2410:02:03(hidden)
other stats
videos1
4325490MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:21(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.5327242352018365, "good_angle": 28.651288114844927, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 5.933333333333348}, "ep001": {"nsteps": 500, "reward": -0.3152015329403803, "good_angle": 18.008227041415477, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 3.1666666666666554}, "ep002": {"nsteps": 500, "reward": -0.3838280936330557, "good_angle": 0.04016557764447126, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -0.3823062506136484, "good_angle": 20.09321848402581, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 3.266666666666656}, "ep004": {"nsteps": 91, "reward": -13.493354661124092, "good_angle": 0.878727362866366, "survival_time": 3.0333333333333323, "traveled_tiles": 2, "valid_direction": 1.3333333333333304}}
good_angle_max28.651288114844927
good_angle_mean13.53432531615941
good_angle_median18.008227041415477
good_angle_min0.04016557764447126
reward_max-0.3152015329403803
reward_mean-3.021482954702603
reward_median-0.3838280936330557
reward_min-13.493354661124092
survival_time_max16.666666666666654
survival_time_mean13.939999999999989
survival_time_min3.0333333333333323
traveled_tiles_max5
traveled_tiles_mean4
traveled_tiles_median4
traveled_tiles_min2
valid_direction_max5.933333333333348
valid_direction_mean2.739999999999998
valid_direction_median3.1666666666666554
valid_direction_min0
4324488JonJP pipelineaido1_LF1_r3-v3step3-videosabortedno2410:31:01(hidden)
4323490MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:03(hidden)
4313488JonJP pipelineaido1_LF1_r3-v3step3-videosabortedno2410:31:16(hidden)
4291485JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 387, "reward": -3.798729584715991, "good_angle": 28.747428507460334, "survival_time": 12.899999999999965, "traveled_tiles": 3, "valid_direction": 9.966666666666637}, "ep001": {"nsteps": 500, "reward": -1.0573182745575904, "good_angle": 2.160446237165721, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 2.6666666666666936}, "ep002": {"nsteps": 500, "reward": -0.6233571230620146, "good_angle": 8.913384487303238, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 4.700000000000019}, "ep003": {"nsteps": 500, "reward": -1.1608060921104626, "good_angle": 1.0175718546566106, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 1.9333333333333265}, "ep004": {"nsteps": 500, "reward": -1.1846791566777974, "good_angle": 1.0843032316371852, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 1.5999999999999943}}
good_angle_max28.747428507460334
good_angle_mean8.384626863644618
good_angle_median2.160446237165721
good_angle_min1.0175718546566106
reward_max-0.6233571230620146
reward_mean-1.564978046224771
reward_median-1.1608060921104626
reward_min-3.798729584715991
survival_time_max16.666666666666654
survival_time_mean15.913333333333318
survival_time_min12.899999999999965
traveled_tiles_max4
traveled_tiles_mean2.8
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max9.966666666666637
valid_direction_mean4.173333333333334
valid_direction_median2.6666666666666936
valid_direction_min1.5999999999999943
4275455orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:03(hidden)
4269455orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:01(hidden)
4263479JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:38(hidden)
other stats
videos1
4260479JonJP pipelineaido1_LF1_r3-v3step1-simulationsuccessno2410:06:19(hidden)
other stats
simulation-passed1
4256455orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:18(hidden)
survival_time_median1.3000000000000007


other stats
episodes
details{"ep000": {"nsteps": 39, "reward": -26.71801254535333, "good_angle": 1.288231067747685, "survival_time": 1.3000000000000007, "traveled_tiles": 1, "valid_direction": 1.133333333333334}, "ep001": {"nsteps": 40, "reward": -26.004260089062154, "good_angle": 1.293420470438038, "survival_time": 1.333333333333334, "traveled_tiles": 1, "valid_direction": 1.1666666666666674}, "ep002": {"nsteps": 500, "reward": -0.6970270132733276, "good_angle": 25.551554082811936, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 14.033333333333315}, "ep003": {"nsteps": 28, "reward": -37.51613992878369, "good_angle": 0.989852945856892, "survival_time": 0.9333333333333332, "traveled_tiles": 2, "valid_direction": 0.7999999999999998}, "ep004": {"nsteps": 19, "reward": -55.15117726827923, "good_angle": 0.3782988700801687, "survival_time": 0.6333333333333333, "traveled_tiles": 2, "valid_direction": 0.49999999999999994}}
good_angle_max25.551554082811936
good_angle_mean5.900271487386944
good_angle_median1.288231067747685
good_angle_min0.3782988700801687
reward_max-0.6970270132733276
reward_mean-29.21732336895035
reward_median-26.71801254535333
reward_min-55.15117726827923
survival_time_max16.666666666666654
survival_time_mean4.173333333333331
survival_time_min0.6333333333333333
traveled_tiles_max2
traveled_tiles_mean1.6
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max14.033333333333315
valid_direction_mean3.5266666666666637
valid_direction_median1.133333333333334
valid_direction_min0.49999999999999994
4255478JonJP pipelineaido1_LF1_r3-v3step4-vizsuccessno2410:01:35(hidden)
other stats
videos1
4252478JonJP pipelineaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -3.88935456764698, "good_angle": 107.92879702473356, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 16.266666666666655}, "ep001": {"nsteps": 500, "reward": -2.7104420055150986, "good_angle": 4.969803507369229, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 16.266666666666655}, "ep002": {"nsteps": 471, "reward": -4.892908734437513, "good_angle": 5.097164217983522, "survival_time": 15.699999999999957, "traveled_tiles": 3, "valid_direction": 11.166666666666636}, "ep003": {"nsteps": 500, "reward": -2.9333713345527648, "good_angle": 9.101736004840609, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 16.199999999999985}, "ep004": {"nsteps": 500, "reward": -2.3411339049339293, "good_angle": 1.1657804343981717, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 0}}
good_angle_max107.92879702473356
good_angle_mean25.652656237865017
good_angle_median5.097164217983522
good_angle_min1.1657804343981717
reward_max-2.3411339049339293
reward_mean-3.3534421094172573
reward_median-2.9333713345527648
reward_min-4.892908734437513
survival_time_max16.666666666666654
survival_time_mean16.473333333333315
survival_time_min15.699999999999957
traveled_tiles_max3
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max16.266666666666655
valid_direction_mean11.979999999999986
valid_direction_median16.199999999999985
valid_direction_min0
4248476JonJP pipelineaido1_LF1_r3-v3step4-vizerrorno2410:00:16
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job4248-6777', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job4248-6777', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
4244476JonJP pipelineaido1_LF1_r3-v3step3-videossuccessno2410:00:46(hidden)
other stats
videos1
4223454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step4-viztimeoutno2410:31:04(hidden)
4218455orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:00(hidden)
4216448JonRandom executionaido1_LF1_r3-v3step4-viztimeoutno2410:31:00(hidden)
4209454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step3-videossuccessno2410:00:45(hidden)
other stats
videos1
4202468sodabetaPyTorch templateaido1_LF1_r3-v3step3-videossuccessno2410:00:41(hidden)
other stats
videos1
4198448JonRandom executionaido1_LF1_r3-v3step4-viztimeoutno2410:31:04(hidden)
4194454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step3-videossuccessno2410:00:44(hidden)
other stats
videos1
4187468sodabetaPyTorch templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median6.099999999999988


other stats
episodes
details{"ep000": {"nsteps": 188, "reward": -7.2790766222045775, "good_angle": 30.50878834936682, "survival_time": 6.266666666666654, "traveled_tiles": 2, "valid_direction": 4.633333333333319}, "ep001": {"nsteps": 147, "reward": -8.680411295825932, "good_angle": 0.663842989600597, "survival_time": 4.899999999999992, "traveled_tiles": 2, "valid_direction": 2.8333333333333246}, "ep002": {"nsteps": 183, "reward": -5.517767048685155, "good_angle": 0.2684690486587212, "survival_time": 6.099999999999988, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 240, "reward": -6.538190926114718, "good_angle": 24.485943347751952, "survival_time": 7.999999999999981, "traveled_tiles": 3, "valid_direction": 3.566666666666654}, "ep004": {"nsteps": 65, "reward": -18.041522733981793, "good_angle": 0.20704407955792573, "survival_time": 2.1666666666666687, "traveled_tiles": 2, "valid_direction": 0.9666666666666692}}
good_angle_max30.50878834936682
good_angle_mean11.226817562987204
good_angle_median0.663842989600597
good_angle_min0.20704407955792573
reward_max-5.517767048685155
reward_mean-9.211393725362436
reward_median-7.2790766222045775
reward_min-18.041522733981793
survival_time_max7.999999999999981
survival_time_mean5.486666666666657
survival_time_min2.1666666666666687
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max4.633333333333319
valid_direction_mean2.3999999999999932
valid_direction_median2.8333333333333246
valid_direction_min0
4181466sodabetaRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:13(hidden)
other stats
videos1
4179466sodabetaRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median8.999999999999979


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.787392634503981, "good_angle": 24.50121165713266, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 380, "reward": -3.1892436623175233, "good_angle": 19.47182270537363, "survival_time": 12.666666666666632, "traveled_tiles": 3, "valid_direction": 3.966666666666653}, "ep002": {"nsteps": 500, "reward": 0.1488148927950533, "good_angle": 0.7449046813869539, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 1.6000000000000298}, "ep003": {"nsteps": 259, "reward": -6.1682174353065635, "good_angle": 26.38992744286716, "survival_time": 8.633333333333313, "traveled_tiles": 3, "valid_direction": 3.9666666666666535}, "ep004": {"nsteps": 270, "reward": -5.433456733843519, "good_angle": 4.070757089988011, "survival_time": 8.999999999999979, "traveled_tiles": 3, "valid_direction": 3.133333333333323}}
good_angle_max26.38992744286716
good_angle_mean15.035724715349682
good_angle_median19.47182270537363
good_angle_min0.7449046813869539
reward_max0.1488148927950533
reward_mean-4.485899114635306
reward_median-5.433456733843519
reward_min-7.787392634503981
survival_time_max16.666666666666654
survival_time_mean10.526666666666646
survival_time_min5.666666666666656
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.9666666666666535
valid_direction_mean3.319999999999996
valid_direction_median3.933333333333321
valid_direction_min1.6000000000000298
4178454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:02(hidden)
4168448JonRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:09(hidden)
other stats
videos1
4165465JonROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:01:16(hidden)
other stats
videos1
4162466sodabetaRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2410:02:09(hidden)
other stats
simulation-passed1
4158454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:04(hidden)
4154455orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:01(hidden)
4145464orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationsuccessno2410:01:27(hidden)
other stats
simulation-passed1
4140462JonJP pipelineaido1_LF1_r3-v3step1-simulationerrorno2410:00:31
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 526, in run_one
    cr = read_challenge_results(wd)
  File "/project/src/duckietown-challenges/src/duckietown_challenges/challenge_results.py", line 63, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission462/step1-simulation-rudolf-9-job4140/challenge-results/challenge_results.yaml' does not exist.
(hidden)
4139460MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step4-vizsuccessno2410:01:33(hidden)
other stats
videos1
4134448JonRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:11(hidden)
other stats
videos1
4123460MandanaSamieiROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:08:51(hidden)
other stats
simulation-passed1
4122454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step3-videostimeoutno2410:31:04(hidden)
4119454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:19(hidden)
survival_time_median1.3000000000000007


other stats
episodes
details{"ep000": {"nsteps": 39, "reward": -26.71766674862458, "good_angle": 1.287772796146936, "survival_time": 1.3000000000000007, "traveled_tiles": 1, "valid_direction": 1.133333333333334}, "ep001": {"nsteps": 40, "reward": -26.004260089062154, "good_angle": 1.293420470438038, "survival_time": 1.333333333333334, "traveled_tiles": 1, "valid_direction": 1.1666666666666674}, "ep002": {"nsteps": 500, "reward": -0.6970270132733276, "good_angle": 25.551554082811936, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 14.033333333333315}, "ep003": {"nsteps": 28, "reward": -37.51613992878369, "good_angle": 0.989852945856892, "survival_time": 0.9333333333333332, "traveled_tiles": 2, "valid_direction": 0.7999999999999998}, "ep004": {"nsteps": 19, "reward": -55.15117726827923, "good_angle": 0.3782988700801687, "survival_time": 0.6333333333333333, "traveled_tiles": 2, "valid_direction": 0.49999999999999994}}
good_angle_max25.551554082811936
good_angle_mean5.900179833066795
good_angle_median1.287772796146936
good_angle_min0.3782988700801687
reward_max-0.6970270132733276
reward_mean-29.217254209604597
reward_median-26.71766674862458
reward_min-55.15117726827923
survival_time_max16.666666666666654
survival_time_mean4.173333333333331
survival_time_min0.6333333333333333
traveled_tiles_max2
traveled_tiles_mean1.6
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max14.033333333333315
valid_direction_mean3.5266666666666637
valid_direction_median1.133333333333334
valid_direction_min0.49999999999999994
4116455orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:01(hidden)
4111457JonRandom executionaido1_LF1_r3-v3step1-simulationfailedno2410:00:20
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 59, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: No module named cv2

(hidden)
4107448JonRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:18(hidden)
survival_time_median9.066666666666643


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.691971688761431, "good_angle": 24.224734073352742, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 378, "reward": -3.188493261832581, "good_angle": 21.61928711516943, "survival_time": 12.599999999999966, "traveled_tiles": 3, "valid_direction": 3.999999999999986}, "ep002": {"nsteps": 320, "reward": -3.099984949549969, "good_angle": 0.14839830863200518, "survival_time": 10.66666666666664, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 258, "reward": -6.346006216004837, "good_angle": 26.050083784140835, "survival_time": 8.59999999999998, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 272, "reward": -5.463044718527646, "good_angle": 4.40837184930493, "survival_time": 9.066666666666643, "traveled_tiles": 3, "valid_direction": 2.9999999999999902}}
good_angle_max26.050083784140835
good_angle_mean15.290175026119988
good_angle_median21.61928711516943
good_angle_min0.14839830863200518
reward_max-3.099984949549969
reward_mean-5.157900166935294
reward_median-5.463044718527646
reward_min-7.691971688761431
survival_time_max12.599999999999966
survival_time_mean9.31999999999998
survival_time_min5.666666666666656
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.999999999999986
valid_direction_mean2.9733333333333234
valid_direction_median3.93333333333332
valid_direction_min0
4101455orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:19(hidden)
survival_time_median1.3000000000000007


other stats
episodes
details{"ep000": {"nsteps": 39, "reward": -26.71801254535333, "good_angle": 1.288231067747685, "survival_time": 1.3000000000000007, "traveled_tiles": 1, "valid_direction": 1.133333333333334}, "ep001": {"nsteps": 40, "reward": -26.004260089062154, "good_angle": 1.293420470438038, "survival_time": 1.333333333333334, "traveled_tiles": 1, "valid_direction": 1.1666666666666674}, "ep002": {"nsteps": 500, "reward": -0.6970270132733276, "good_angle": 25.551554082811936, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 14.033333333333315}, "ep003": {"nsteps": 28, "reward": -37.51613992878369, "good_angle": 0.989852945856892, "survival_time": 0.9333333333333332, "traveled_tiles": 2, "valid_direction": 0.7999999999999998}, "ep004": {"nsteps": 19, "reward": -55.15117726827923, "good_angle": 0.3782988700801687, "survival_time": 0.6333333333333333, "traveled_tiles": 2, "valid_direction": 0.49999999999999994}}
good_angle_max25.551554082811936
good_angle_mean5.900271487386944
good_angle_median1.288231067747685
good_angle_min0.3782988700801687
reward_max-0.6970270132733276
reward_mean-29.21732336895035
reward_median-26.71801254535333
reward_min-55.15117726827923
survival_time_max16.666666666666654
survival_time_mean4.173333333333331
survival_time_min0.6333333333333333
traveled_tiles_max2
traveled_tiles_mean1.6
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max14.033333333333315
valid_direction_mean3.5266666666666637
valid_direction_median1.133333333333334
valid_direction_min0.49999999999999994
4097456hvrigazovROS-based Lane Followingaido1_LF1_r3-v3step3-videossuccessno2410:01:42(hidden)
other stats
videos1
4095452JonRandom executionaido1_LF1_r3-v3step1-simulationfailedno2410:00:19
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "solution.py", line 59, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: No module named cv2

(hidden)
4084454orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:31:04(hidden)
4048438BenjaminROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:11:48(hidden)
other stats
simulation-passed1
4044436orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median1.3000000000000007


other stats
episodes
details{"ep000": {"nsteps": 39, "reward": -26.66025702655315, "good_angle": 1.2893309412281913, "survival_time": 1.3000000000000007, "traveled_tiles": 1, "valid_direction": 1.133333333333334}, "ep001": {"nsteps": 40, "reward": -26.004260089062154, "good_angle": 1.293420470438038, "survival_time": 1.333333333333334, "traveled_tiles": 1, "valid_direction": 1.1666666666666674}, "ep002": {"nsteps": 500, "reward": -0.6970270132733276, "good_angle": 25.551554082811936, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 14.033333333333315}, "ep003": {"nsteps": 28, "reward": -37.51613992878369, "good_angle": 0.989852945856892, "survival_time": 0.9333333333333332, "traveled_tiles": 2, "valid_direction": 0.7999999999999998}, "ep004": {"nsteps": 19, "reward": -55.15117726827923, "good_angle": 0.3782988700801687, "survival_time": 0.6333333333333333, "traveled_tiles": 2, "valid_direction": 0.49999999999999994}}
good_angle_max25.551554082811936
good_angle_mean5.900491462083045
good_angle_median1.2893309412281913
good_angle_min0.3782988700801687
reward_max-0.6970270132733276
reward_mean-29.20577226519031
reward_median-26.66025702655315
reward_min-55.15117726827923
survival_time_max16.666666666666654
survival_time_mean4.173333333333331
survival_time_min0.6333333333333333
traveled_tiles_max2
traveled_tiles_mean1.6
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max14.033333333333315
valid_direction_mean3.5266666666666637
valid_direction_median1.133333333333334
valid_direction_min0.49999999999999994
4038432Bhairav MehtaTemplate for ROS Submissionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median6.733333333333319


other stats
episodes
details{"ep000": {"nsteps": 148, "reward": -7.8658295266504865, "good_angle": 14.092658021645844, "survival_time": 4.933333333333326, "traveled_tiles": 3, "valid_direction": 4.299999999999992}, "ep001": {"nsteps": 263, "reward": -5.0282359027700805, "good_angle": 16.169656728411926, "survival_time": 8.766666666666646, "traveled_tiles": 3, "valid_direction": 4.2999999999999865}, "ep002": {"nsteps": 267, "reward": -4.513132370502622, "good_angle": 1.059585117086331, "survival_time": 8.899999999999979, "traveled_tiles": 4, "valid_direction": 3.1999999999999913}, "ep003": {"nsteps": 99, "reward": -10.643398757304318, "good_angle": 0.9178286381359037, "survival_time": 3.299999999999998, "traveled_tiles": 2, "valid_direction": 2.6999999999999984}, "ep004": {"nsteps": 202, "reward": -7.107296272715794, "good_angle": 6.122154665319772, "survival_time": 6.733333333333319, "traveled_tiles": 3, "valid_direction": 3.133333333333322}}
good_angle_max16.169656728411926
good_angle_mean7.672376634119955
good_angle_median6.122154665319772
good_angle_min0.9178286381359037
reward_max-4.513132370502622
reward_mean-7.03157856598866
reward_median-7.107296272715794
reward_min-10.643398757304318
survival_time_max8.899999999999979
survival_time_mean6.526666666666654
survival_time_min3.299999999999998
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max4.299999999999992
valid_direction_mean3.526666666666658
valid_direction_median3.1999999999999913
valid_direction_min2.6999999999999984
4036434BenjaminPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrorno2410:00:33
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 526, in run_one
    cr = read_challenge_results(wd)
  File "/project/src/duckietown-challenges/src/duckietown_challenges/challenge_results.py", line 63, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission434/step1-simulation-rudolf-9-job4036/challenge-results/challenge_results.yaml' does not exist.
(hidden)
4033431BenjaminPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:31:02(hidden)
4031429orlando.m.bol@gmail.comPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrorno2410:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job4031-569', 'pull']:

   >  Command '['docker-compose', '-p', 'job4031-569', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
4030428SashkasemRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:18(hidden)
other stats
videos1
4013423MandanaSamieiPyTorch templateaido1_LF1_r3-v3step3-videossuccessno2410:00:46(hidden)
other stats
videos1
4011423MandanaSamieiPyTorch templateaido1_LF1_r3-v3step1-simulationsuccessno2410:02:03(hidden)
other stats
simulation-passed1
4010422jahanviRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:11(hidden)
other stats
videos1
4007422jahanviRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:17(hidden)
survival_time_median8.966666666666645


other stats
episodes
details{"ep000": {"nsteps": 171, "reward": -7.639713048934936, "good_angle": 25.19146169910629, "survival_time": 5.6999999999999895, "traveled_tiles": 2, "valid_direction": 3.9333333333333207}, "ep001": {"nsteps": 375, "reward": -3.8937896972497303, "good_angle": 24.622449211170416, "survival_time": 12.499999999999966, "traveled_tiles": 3, "valid_direction": 3.899999999999986}, "ep002": {"nsteps": 500, "reward": 0.02070584638358559, "good_angle": 0.41528803023102856, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 1.1333333333333648}, "ep003": {"nsteps": 258, "reward": -5.905082889305529, "good_angle": 26.07364517252711, "survival_time": 8.59999999999998, "traveled_tiles": 3, "valid_direction": 3.899999999999987}, "ep004": {"nsteps": 269, "reward": -5.511848372260981, "good_angle": 4.860136898949597, "survival_time": 8.966666666666645, "traveled_tiles": 3, "valid_direction": 2.966666666666657}}
good_angle_max26.07364517252711
good_angle_mean16.232596202396884
good_angle_median24.622449211170416
good_angle_min0.41528803023102856
reward_max0.02070584638358559
reward_mean-4.585945632273519
reward_median-5.511848372260981
reward_min-7.639713048934936
survival_time_max16.666666666666654
survival_time_mean10.486666666666649
survival_time_min5.6999999999999895
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.9333333333333207
valid_direction_mean3.1666666666666634
valid_direction_median3.899999999999986
valid_direction_min1.1333333333333648
4003420ManishSolution templateaido1_luck-v3step1successyes2410:00:39(hidden)
score145.76719983336335


4000419happyduckiePyTorch templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median5.2333333333333245


other stats
episodes
details{"ep000": {"nsteps": 148, "reward": -9.671616262680775, "good_angle": 19.40268859391938, "survival_time": 4.933333333333326, "traveled_tiles": 2, "valid_direction": 3.8333333333333255}, "ep001": {"nsteps": 246, "reward": -4.31970249087175, "good_angle": 0.3785123304269607, "survival_time": 8.199999999999982, "traveled_tiles": 2, "valid_direction": 0}, "ep002": {"nsteps": 189, "reward": -5.382517373467279, "good_angle": 0.28755759132123193, "survival_time": 6.299999999999987, "traveled_tiles": 3, "valid_direction": 0.13333333333333286}, "ep003": {"nsteps": 157, "reward": -8.512830208820901, "good_angle": 8.316643904153919, "survival_time": 5.2333333333333245, "traveled_tiles": 3, "valid_direction": 1.2666666666666622}, "ep004": {"nsteps": 68, "reward": -17.351234295788935, "good_angle": 0.1810773026668893, "survival_time": 2.2666666666666684, "traveled_tiles": 2, "valid_direction": 0.26666666666666683}}
good_angle_max19.40268859391938
good_angle_mean5.713295944497675
good_angle_median0.3785123304269607
good_angle_min0.1810773026668893
reward_max-4.31970249087175
reward_mean-9.04758012632593
reward_median-8.512830208820901
reward_min-17.351234295788935
survival_time_max8.199999999999982
survival_time_mean5.386666666666658
survival_time_min2.2666666666666684
traveled_tiles_max3
traveled_tiles_mean2.4
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max3.8333333333333255
valid_direction_mean1.0999999999999974
valid_direction_median0.26666666666666683
valid_direction_min0
3998418happyduckieSolution templateaido1_luck-v3step1successyes2410:00:45(hidden)
score132.36060311350303


3986414Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step4-vizerrorno2410:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job3986-2054', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job3986-2054', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
(hidden)
3984415Bhairav MehtaTemplate for ROS Submissionaido1_LF1_r3-v3step4-vizsuccessno2410:00:51(hidden)
other stats
videos1
3980414Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.5340267053380375, "good_angle": 28.648187511217987, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.900000000000011}, "ep001": {"nsteps": 413, "reward": -2.8753469691817033, "good_angle": 13.230943548885904, "survival_time": 13.766666666666628, "traveled_tiles": 4, "valid_direction": 2.633333333333324}, "ep002": {"nsteps": 500, "reward": -0.3948863276541233, "good_angle": 0.05126689695642577, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 257, "reward": -4.6175787197371605, "good_angle": 7.588529457346171, "survival_time": 8.566666666666647, "traveled_tiles": 4, "valid_direction": 2.799999999999992}, "ep004": {"nsteps": 500, "reward": -0.7729631737247109, "good_angle": 2.694737039029585, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 6.266666666666661}}
good_angle_max28.648187511217987
good_angle_mean10.442732890687214
good_angle_median7.588529457346171
good_angle_min0.05126689695642577
reward_max-0.3948863276541233
reward_mean-1.838960379127147
reward_median-0.7729631737247109
reward_min-4.6175787197371605
survival_time_max16.666666666666654
survival_time_mean14.466666666666647
survival_time_min8.566666666666647
traveled_tiles_max5
traveled_tiles_mean4.4
traveled_tiles_median4
traveled_tiles_min4
valid_direction_max6.900000000000011
valid_direction_mean3.7199999999999975
valid_direction_median2.799999999999992
valid_direction_min0
3975413Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step3-videossuccessno2410:01:21(hidden)
other stats
videos1
3973413Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationsuccessno2410:05:24(hidden)
other stats
simulation-passed1
3948405Liam PaullROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailedno2410:00:45
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "./solution.py", line 89, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: error loading <rosparam> tag: 
	'param' attribute must be set for non-dictionary values
XML is <rosparam command="load" file="$(find duckietown)/config/$(arg config)/line_detector/$(arg node_name)/$(arg param_file_name).yaml"/>

(hidden)
3943402SashkasemSolution templateaido1_luck-v3step1successyes2410:00:43(hidden)
score117.66853804395717


3942401PolyProgrammistROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailedno2410:00:37
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "./solution.py", line 89, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: error loading <rosparam> tag: 
	'param' attribute must be set for non-dictionary values
XML is <rosparam command="load" file="$(find duckietown)/config/$(arg config)/line_detector/$(arg node_name)/$(arg param_file_name).yaml"/>

(hidden)
3935398Maxim KuzminRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2410:02:02(hidden)
other stats
simulation-passed1
3932397PolyProgrammistRandom executionaido1_LF1_r3-v3step3-videossuccessno2410:00:57(hidden)
other stats
videos1
3927396anna.tsalapovasimple submitionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 117, "reward": -9.991474496503162, "good_angle": 8.866560177579744, "survival_time": 3.899999999999996, "traveled_tiles": 3, "valid_direction": 1.4666666666666617}, "ep001": {"nsteps": 500, "reward": -0.3888421014547348, "good_angle": 3.638173429102398, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 0.4}, "ep002": {"nsteps": 500, "reward": 0.23706295147072523, "good_angle": 0.1326869096702976, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 500, "reward": -2.333500215291977, "good_angle": 121.97850228305688, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 14.566666666666652}, "ep004": {"nsteps": 500, "reward": -2.341332432746887, "good_angle": 0.016734103965512847, "survival_time": 16.666666666666654, "traveled_tiles": 2, "valid_direction": 0}}
good_angle_max121.97850228305688
good_angle_mean26.92653138067497
good_angle_median3.638173429102398
good_angle_min0.016734103965512847
reward_max0.23706295147072523
reward_mean-2.9636172589052068
reward_median-2.333500215291977
reward_min-9.991474496503162
survival_time_max16.666666666666654
survival_time_mean14.113333333333324
survival_time_min3.899999999999996
traveled_tiles_max3
traveled_tiles_mean2.8
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max14.566666666666652
valid_direction_mean3.286666666666662
valid_direction_median0.4
valid_direction_min0
3920395Maxim KuzminRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median9.033333333333312


other stats
episodes
details{"ep000": {"nsteps": 170, "reward": -7.777325500460232, "good_angle": 23.73206443704702, "survival_time": 5.666666666666656, "traveled_tiles": 2, "valid_direction": 3.933333333333321}, "ep001": {"nsteps": 379, "reward": -3.903377290608701, "good_angle": 25.225543794159297, "survival_time": 12.6333333333333, "traveled_tiles": 3, "valid_direction": 3.966666666666653}, "ep002": {"nsteps": 304, "reward": -3.299739558917185, "good_angle": 0.1450889818403155, "survival_time": 10.133333333333308, "traveled_tiles": 4, "valid_direction": 0}, "ep003": {"nsteps": 258, "reward": -6.069876175056132, "good_angle": 25.91763516453875, "survival_time": 8.59999999999998, "traveled_tiles": 3, "valid_direction": 3.899999999999987}, "ep004": {"nsteps": 271, "reward": -5.515748887463745, "good_angle": 4.9426335303917845, "survival_time": 9.033333333333312, "traveled_tiles": 3, "valid_direction": 2.9999999999999902}}
good_angle_max25.91763516453875
good_angle_mean15.992593181595437
good_angle_median23.73206443704702
good_angle_min0.1450889818403155
reward_max-3.299739558917185
reward_mean-5.313213482501199
reward_median-5.515748887463745
reward_min-7.777325500460232
survival_time_max12.6333333333333
survival_time_mean9.213333333333312
survival_time_min5.666666666666656
traveled_tiles_max4
traveled_tiles_mean3
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.966666666666653
valid_direction_mean2.95999999999999
valid_direction_median3.899999999999987
valid_direction_min0
3918394Maxim KuzminRandom executionaido1_LF1_r3-v3step4-vizsuccessno2410:01:10(hidden)
other stats
videos1
3914394Maxim KuzminRandom executionaido1_LF1_r3-v3step2-scoringsuccessno2410:00:16(hidden)
survival_time_median9.666666666666645


other stats
episodes
details{"ep000": {"nsteps": 169, "reward": -7.831943151866191, "good_angle": 23.8618270463908, "survival_time": 5.633333333333323, "traveled_tiles": 2, "valid_direction": 3.899999999999988}, "ep001": {"nsteps": 377, "reward": -4.132220233150756, "good_angle": 24.795338011008287, "survival_time": 12.566666666666633, "traveled_tiles": 3, "valid_direction": 3.9333333333333194}, "ep002": {"nsteps": 500, "reward": -0.10804257408017293, "good_angle": 0.4144173593921496, "survival_time": 16.666666666666654, "traveled_tiles": 5, "valid_direction": 1.2000000000000313}, "ep003": {"nsteps": 259, "reward": -6.331310865961907, "good_angle": 25.597793455877788, "survival_time": 8.633333333333313, "traveled_tiles": 3, "valid_direction": 3.93333333333332}, "ep004": {"nsteps": 290, "reward": -5.483258781668827, "good_angle": 7.096534809600988, "survival_time": 9.666666666666645, "traveled_tiles": 5, "valid_direction": 3.8333333333333206}}
good_angle_max25.597793455877788
good_angle_mean16.353182136454002
good_angle_median23.8618270463908
good_angle_min0.4144173593921496
reward_max-0.10804257408017293
reward_mean-4.77735512134557
reward_median-5.483258781668827
reward_min-7.831943151866191
survival_time_max16.666666666666654
survival_time_mean10.633333333333312
survival_time_min5.633333333333323
traveled_tiles_max5
traveled_tiles_mean3.6
traveled_tiles_median3
traveled_tiles_min2
valid_direction_max3.93333333333332
valid_direction_mean3.359999999999996
valid_direction_median3.899999999999988
valid_direction_min1.2000000000000313
3911393Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailedno2410:00:36
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "./solution.py", line 89, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: error loading <rosparam> tag: 
	'param' attribute must be set for non-dictionary values
XML is <rosparam command="load" file="$(find duckietown)/config/$(arg config)/line_detector/$(arg node_name)/$(arg param_file_name).yaml"/>

(hidden)
3908392Liam PaullPyTorch templateaido1_LF1_r3-v3step3-videossuccessno2410:00:44(hidden)
other stats
videos1
3904389Liam PaullRandom executionaido1_LF1_r3-v3step3-videossuccessno2410:00:48(hidden)
other stats
videos1
3900389Liam PaullRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2410:01:59(hidden)
other stats
simulation-passed1
3896386Bhairav MehtaPyTorch DDPG baselineaido1_LF1_r3-v3step3-videossuccessno2410:00:40(hidden)
other stats
videos1
3891385Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step3-videossuccessno2410:00:37(hidden)
other stats
videos1
3889385Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationsuccessno2410:01:34(hidden)
other stats
simulation-passed1
3887384ivanthaSolution templateaido1_luck-v3step1successyes2410:00:43(hidden)
score117.785209806993436


3877383Andrea CensiRandom executionaido1_LF1_r3-v3step4-viztimeoutno2410:11:02(hidden)
3873383Andrea CensiRandom executionaido1_LF1_r3-v3step1-simulationsuccessno2410:02:02(hidden)
other stats
simulation-passed1
3872382Andrea CensiTensorflow templateaido1_LFV_r1-v3step2-scoringtimeoutno2410:11:14(hidden)
3867382Andrea CensiTensorflow templateaido1_LFV_r1-v3step1-simulationsuccessno2410:02:00(hidden)
other stats
simulation-passed1
3860376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step4-viztimeoutno2410:11:17(hidden)
3858379ivanthaSolution templateaido1_luck-v3step1successyes2410:00:19(hidden)
score110.08482019199016


3857372lavoiemsTensorflow templateaido1_LF1_r3-v3step4-viztimeoutno2410:11:29(hidden)
3855372lavoiemsTensorflow templateaido1_LF1_r3-v3step3-videossuccessno2410:00:54(hidden)
other stats
videos1
3854376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step4-viztimeoutno2410:11:03(hidden)
3844376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step4-viztimeoutno2410:11:12(hidden)
3840380ivanthaSolution templateaido1_luck-v3step1timeoutyes2410:11:09(hidden)
3832379ivanthaSolution templateaido1_luck-v3step1timeoutyes2410:11:05(hidden)
3825375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailedno2410:00:27
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Env Duckietown-Lf-Lfv-Navv-Silent-v1 not found (valid versions include ['Duckietown-Lf-Lfv-Navv-Silent-v0'])
(hidden)
3820372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationsuccessno2410:02:39(hidden)
other stats
simulation-passed1
3819374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationerrorno2410:00:25
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 526, in run_one
    cr = read_challenge_results(wd)
  File "/project/src/duckietown-challenges/src/duckietown_challenges/challenge_results.py", line 63, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission374/step1-simulation-rudolf-9-job3819/challenge-results/challenge_results.yaml' does not exist.
(hidden)
3818376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step4-viztimeoutno2410:11:04(hidden)
3816376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step3-videossuccessno2410:00:32(hidden)
other stats
videos1
3814376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:15(hidden)
survival_time_median3.0333333333333323


other stats
episodes
details{"ep000": {"nsteps": 74, "reward": -15.144965386873968, "good_angle": 7.830836087374783, "survival_time": 2.4666666666666677, "traveled_tiles": 2, "valid_direction": 1.500000000000001}, "ep001": {"nsteps": 91, "reward": -12.447482296398707, "good_angle": 0.623453309942581, "survival_time": 3.0333333333333323, "traveled_tiles": 2, "valid_direction": 1.9999999999999991}, "ep002": {"nsteps": 111, "reward": -10.540010888834257, "good_angle": 1.2075460157833389, "survival_time": 3.6999999999999966, "traveled_tiles": 2, "valid_direction": 2.7999999999999967}, "ep003": {"nsteps": 142, "reward": -9.408231461551829, "good_angle": 2.71803237621097, "survival_time": 4.733333333333326, "traveled_tiles": 3, "valid_direction": 0.3999999999999989}, "ep004": {"nsteps": 42, "reward": -26.426816662152607, "good_angle": 0.2275503934080461, "survival_time": 1.400000000000001, "traveled_tiles": 2, "valid_direction": 0.900000000000001}}
good_angle_max7.830836087374783
good_angle_mean2.521483636543944
good_angle_median1.2075460157833389
good_angle_min0.2275503934080461
reward_max-9.408231461551829
reward_mean-14.793501339162273
reward_median-12.447482296398707
reward_min-26.426816662152607
survival_time_max4.733333333333326
survival_time_mean3.0666666666666647
survival_time_min1.400000000000001
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_median2
traveled_tiles_min2
valid_direction_max2.7999999999999967
valid_direction_mean1.5199999999999994
valid_direction_median1.500000000000001
valid_direction_min0.3999999999999989
3811376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationsuccessno2410:01:23(hidden)
other stats
simulation-passed1
3809378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationfailedno2410:00:35
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 585, in wrap_solution
    solution.run(cis)
  File "./solution.py", line 89, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: error loading <rosparam> tag: 
	'param' attribute must be set for non-dictionary values
XML is <rosparam command="load" file="$(find duckietown)/config/$(arg config)/line_detector/$(arg node_name)/$(arg param_file_name).yaml"/>

(hidden)
3807375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:05(hidden)
3805372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:04(hidden)
3803374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:01(hidden)
3802376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:02(hidden)
3800376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:01(hidden)
3798378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3796375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:16(hidden)
3794372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3792374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3790376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3788378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:16(hidden)
3786375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:14(hidden)
3784372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:06(hidden)
3782374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:13(hidden)
3780376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:16(hidden)
3778378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3776375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3772372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:06(hidden)
3771374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3766376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3763378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:16(hidden)
3759375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3757375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3755372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3751374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:21(hidden)
3749374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3747376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3745376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3743378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:26(hidden)
3741378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3740375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3737372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3733374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3730374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3729376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:21(hidden)
3726376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3721375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:11(hidden)
3717372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:11(hidden)
3715374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:11(hidden)
3713376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:11(hidden)
3709378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:08(hidden)
3706375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:07(hidden)
3701372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:10(hidden)
3697374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:10(hidden)
3696376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:09(hidden)
3692378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:07(hidden)
3690375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:09(hidden)
3687372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:09(hidden)
3686372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:05(hidden)
3683374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:09(hidden)
3679376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3676378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:08(hidden)
3673375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:27(hidden)
3672375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:09(hidden)
3670372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:08(hidden)
3668374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:06(hidden)
3666376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:06(hidden)
3664378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:06(hidden)
3663375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:23(hidden)
3660372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3658374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:14(hidden)
3655376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:20(hidden)
3652378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:21(hidden)
3649375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:16(hidden)
3644374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:23(hidden)
3643374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:26(hidden)
3641376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:20(hidden)
3639378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3637375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3635372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3633374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:20(hidden)
3630376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:20(hidden)
3629376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:24(hidden)
3627378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:12(hidden)
3623375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:13(hidden)
3620375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:09(hidden)
3616374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:12(hidden)
3601376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:13(hidden)
3588374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:14(hidden)
3583378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:20(hidden)
3577375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3566376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:08(hidden)
3564378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3561375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3556372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:23(hidden)
3554374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3549376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:16(hidden)
3544378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3540375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3539372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3534374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:23(hidden)
3529376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3525378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3522375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:24(hidden)
3518372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:27(hidden)
3515374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3509376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3504375Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3501372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3495374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3472378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:17(hidden)
3451378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:21(hidden)
3433378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:13(hidden)
3419377anna.tsalapovasimple submissionaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:25(hidden)
3409378Bhairav MehtaROS-based Lane Followingaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:40(hidden)
3393377anna.tsalapovasimple submissionaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:26(hidden)
3374377anna.tsalapovasimple submissionaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:42(hidden)
3354376Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:23(hidden)
3304372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:25(hidden)
3254372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:22(hidden)
3223374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:31(hidden)
3220372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:26(hidden)
3216374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:15(hidden)
3215373lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3210372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:25(hidden)
3207373lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:20(hidden)
3206374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:23(hidden)
3201372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:09(hidden)
3199373lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:19(hidden)
3198374lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:22(hidden)
3193372lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:18(hidden)
3182373lavoiemsTensorflow templateaido1_LF1_r3-v3step1-simulationtimeoutno2410:11:16(hidden)
3166369Bhairav MehtaTemplate for ROS Submissionaido1_LF1_r3-v3step3-videossuccessno2410:01:12(hidden)
other stats
videos1
3162370MandanaSamieiPyTorch DDPG templateaido1_LF1_r2-v3step2-scoringsuccessyes2410:00:29(hidden)
traveled_tiles_median2
survival_time_mean3.2933333333333294
good_angle_mean5.273807540445858
valid_direction_mean4.173333333333322


other stats
episodes
details{"ep000": {"nsteps": 139, "reward": -7.565178305818532, "good_angle": 2.9537029902021565, "survival_time": 4.633333333333327, "traveled_tiles": 2, "valid_direction": 3.833333333333327}, "ep001": {"nsteps": 132, "reward": -8.31976330325459, "good_angle": 3.3420249625431753, "survival_time": 4.399999999999994, "traveled_tiles": 1, "valid_direction": 3.199999999999994}, "ep002": {"nsteps": 266, "reward": -4.626618917087002, "good_angle": 14.881676847466942, "survival_time": 1.166666666666667, "traveled_tiles": 3, "valid_direction": 6.966666666666649}, "ep003": {"nsteps": 500, "reward": -1.7608802081346513, "good_angle": 2.912241784465008, "survival_time": 1.1333333333333335, "traveled_tiles": 5, "valid_direction": 4.166666666666652}, "ep004": {"nsteps": 154, "reward": -8.994127320004749, "good_angle": 2.2793911175520125, "survival_time": 5.133333333333325, "traveled_tiles": 1, "valid_direction": 2.6999999999999904}}
good_angle_max14.881676847466942
good_angle_median2.9537029902021565
good_angle_min2.2793911175520125
reward_max-1.7608802081346513
reward_mean-6.2533136108599034
reward_median-7.565178305818532
reward_min-8.994127320004749
survival_time_max5.133333333333325
survival_time_median4.399999999999994
survival_time_min1.1333333333333335
traveled_tiles_max5
traveled_tiles_mean2.4
traveled_tiles_min1
valid_direction_max6.966666666666649
valid_direction_median3.833333333333327
valid_direction_min2.6999999999999904
3160370MandanaSamieiPyTorch DDPG templateaido1_LF1_r2-v3step1-simulationsuccessyes2410:01:47(hidden)
other stats
simulation-passed1
3144366MandanaSamieiPyTorch DDPG templateaido1_LF1_r3-v3step1-simulationfailedno2410:00:40
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 87, in run
    solve(params, cis)
  File "solution.py", line 60, in solve
    action = model.predict(observation)
  File "/workspace/model.py", line 176, in predict
    return self.actor(state).cpu().data.numpy().flatten()
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/workspace/model.py", line 72, in forward
    x = self.lr(self.lin1(x))
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/linear.py", line 55, in forward
    return F.linear(input, self.weight, self.bias)
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/functional.py", line 1024, in linear
    return torch.addmm(bias, input, weight.t())
RuntimeError: size mismatch, m1: [1 x 127872], m2: [4032 x 512] at /tmp/pip-req-build-vRRdPa/aten/src/TH/generic/THTensorMath.cpp:2070


(hidden)
3142364pravishsainathPyTorch templateaido1_LF1_r3-v3step2-scoringtimeoutno2410:11:14(hidden)
3138273heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrorno2410:00:25
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 526, in run_one
    cr = read_challenge_results(wd)
  File "/project/src/duckietown-challenges/src/duckietown_challenges/challenge_results.py", line 63, in read_challenge_results
    raise Exception(msg)
Exception: File u'/tmp/duckietown/DT18/evaluator/executions/aido1_LF1_r3-v3/submission273/step1-simulation-rudolf-9-job3138/challenge-results/challenge_results.yaml' does not exist.
(hidden)
3136362heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationfailedno2410:00:21
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 93, in run
    solve(params, cis)
  File "solution.py", line 28, in solve
    env = gym.make(params['env'])
  File "/opt/conda/lib/python2.7/site-packages/gym/envs/registration.py", line 167, in make
    return registry.make(id)
  File "/opt/conda/lib/python2.7/site-packages/gym/envs/registration.py", line 118, in make
    spec = self.spec(id)
  File "/opt/conda/lib/python2.7/site-packages/gym/envs/registration.py", line 151, in spec
    raise error.DeprecatedEnv('Env {} not found (valid versions include {})'.format(id, matching_envs))
DeprecatedEnv: Env Duckietown-Lf-Lfv-Navv-Silent-v1 not found (valid versions include ['Duckietown-Lf-Lfv-Navv-Silent-v0'])


(hidden)
3095280heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrorno2410:10:28
Timeout: Waited 602 [...]
Timeout:

Waited 602.23418808 for container to finish. Giving up. 
(hidden)
3079286heyt0nyAI DL RL MML XXXL 2k18 yooaido1_LF1_r3-v3step1-simulationerrorno2410:10:27
Timeout: Waited 602 [...]
Timeout:

Waited 602.517543793 for container to finish. Giving up. 
(hidden)
3076346Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step4-vizsuccessno2410:00:42(hidden)
other stats
videos1
3073346Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step3-videossuccessno2410:00:51(hidden)
other stats
videos1
3071346Bhairav MehtaPyTorch DDPG templateaido1_LF1_r3-v3step2-scoringsuccessno2410:00:15(hidden)
survival_time_median2.7


other stats
episodes
details{"ep000": {"nsteps": 81, "reward": -13.405025780568888, "good_angle": 6.262775494763301, "survival_time": 2.7, "traveled_tiles": 2, "valid_direction": 1.866666666666667}, "ep001": {"nsteps": 84, "reward": -13.452968209981918, "good_angle": 1.2460545710891588, "survival_time": 2.8, "traveled_tiles": 1, "valid_direction": 1.9333333333333331}, "ep002": {"nsteps": 93, "reward": -11.952256054006597, "good_angle": 1.648655928746135, "survival_time": 3.0999999999999988, "traveled_tiles": 2, "valid_direction": 2.133333333333332}, "ep003": {"nsteps": 69, "reward": -16.535080417342808, "good_angle": 0.784520177016416, "survival_time": 2.3000000000000016, "traveled_tiles": 2, "valid_direction": 1.5666666666666684}, "ep004": {"nsteps": 52, "reward": -21.821165781754715, "good_angle": 0.3697920026022718, "survival_time": 1.7333333333333354, "traveled_tiles": 2, "valid_direction": 1.0000000000000022}}
good_angle_max6.262775494763301
good_angle_mean2.0623596348434567
good_angle_median1.2460545710891588
good_angle_min0.3697920026022718
reward_max-11.952256054006597
reward_mean-15.433299248730986
reward_median-13.452968209981918
reward_min-21.821165781754715
survival_time_max3.0999999999999988
survival_time_mean2.526666666666667
survival_time_min1.7333333333333354
traveled_tiles_max2
traveled_tiles_mean1.8
traveled_tiles_median2
traveled_tiles_min1
valid_direction_max2.133333333333332
valid_direction_mean1.7000000000000004
valid_direction_median1.866666666666667
valid_direction_min1.0000000000000022
3070350Manfred DiazTensorflow templateaido1_LF1_r3-v3step4-viztimeoutno2410:11:44(hidden)
3066350Manfred DiazTensorflow templateaido1_LF1_r3-v3step3-videossuccessno2410:00:47(hidden)
other stats
videos1
3056315Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationfailedno2410:02:35
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 88, in run
    raise InvalidSubmission(str(e))
InvalidSubmission: Can't connect to the gym-duckietown-server

(hidden)