Duckietown Challenges Home Challenges Submissions

Evaluator 222

ID222
evaluatoridsc-rudolf-11218
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processidsc-rudolf-11218
versionDC:3.1.50;DCR:3.2.10;DTS:3.0.29
first heard
last heard
statusinactive
# evaluating
# success6 2822
# timeout
# failed3 2818
# error2 2814
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
2855335Andrea Censi 🇨🇭Tensorflow templateaido1_LF1_r3-v3step1-simulationfailednoidsc-rudolf-112180:00:44
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 86, in run
    solve(params, cis)  # let's try to solve the challenge,
  File "solution.py", line 52, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2854334Andrea Censi 🇨🇭Tensorflow templateaido1_LFV_r1-v3step1-simulationfailednoidsc-rudolf-112180:00:43
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/notebooks/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 86, in run
    solve(params, cis)  # let's try to solve the challenge,
  File "solution.py", line 52, in solve
    observation, reward, done, info = env.step(action)
  File "/usr/local/lib/python2.7/dist-packages/gym/wrappers/time_limit.py", line 31, in step
    observation, reward, done, info = self.env.step(action)
  File "/notebooks/src/gym-duckietown-agent/gym_duckietown_agent/envs/simplesimagent_env.py", line 109, in step
    obs, rew, done, misc = self.sim.step(action, with_observation=True)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 55, in step
    return self._failsafe_observe(msg)
  File "/notebooks/src/duckietown-slimremote/duckietown_slimremote/pc/robot.py", line 86, in _failsafe_observe
    raise Exception(msg)
Exception: Giving up to connect to the gym duckietown server at host: evaluator
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2850331Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step4-vizsuccessnoidsc-rudolf-112180:01:08
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2848331Andrea Censi 🇨🇭Random executionaido1_LF1_r3-v3step3-videossuccessnoidsc-rudolf-112180:00:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2844332Andrea Censi 🇨🇭Random executionaido1_LFV_r1-v3step3-videossuccessnoidsc-rudolf-112180:00:44
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
videos1
No reset possible
2839330Benjamin Ramtoula 🇨🇦PyTorch DDPG templateaido1_LF1_r3-v3step1-simulationerrornoidsc-rudolf-112180:00:10
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job2839-7283', 'up', '-d']:

   >  Command '['docker-compose', '-p', 'job2839-7283', 'up', '-d']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2833325Wei Gao 🇸🇬Random executionaido1_LF1_r2-v3step2-scoringsuccessyesidsc-rudolf-112180:00:31
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median2
survival_time_mean16.613333333333316
good_angle_mean17.497665611971577
valid_direction_mean13.71999999999998


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.6445460510677659, "good_angle": 19.505871755226224, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 15.733333333333317}, "ep001": {"nsteps": 500, "reward": -1.8768870373666289, "good_angle": 24.17052341370135, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 13.733333333333324}, "ep002": {"nsteps": 500, "reward": -1.862653324007988, "good_angle": 18.74208932980178, "survival_time": 16.599999999999984, "traveled_tiles": 2, "valid_direction": 11.7333333333333}, "ep003": {"nsteps": 500, "reward": -1.051497200846672, "good_angle": 8.75556162218461, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 12.299999999999974}, "ep004": {"nsteps": 500, "reward": -1.0257753971219064, "good_angle": 16.314281938943935, "survival_time": 16.599999999999984, "traveled_tiles": 2, "valid_direction": 15.099999999999978}}
good_angle_max24.17052341370135
good_angle_median18.74208932980178
good_angle_min8.75556162218461
reward_max-0.6445460510677659
reward_mean-1.2922718020821922
reward_median-1.051497200846672
reward_min-1.8768870373666289
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min16.599999999999984
traveled_tiles_max3
traveled_tiles_mean2.2
traveled_tiles_min1
valid_direction_max15.733333333333317
valid_direction_median13.733333333333324
valid_direction_min11.7333333333333
No reset possible
2829324Manfred DiazTensorflow templateaido1_LF1_r3-v3step1-simulationsuccessnoidsc-rudolf-112180:02:37
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
2822224Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step2-scoringsuccessnoidsc-rudolf-112180:00:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median16.666666666666654


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -1.2812926945090295, "good_angle": 60.62226718882041, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 7.133333333333309}, "ep001": {"nsteps": 500, "reward": -0.39393230858445166, "good_angle": 18.195943465006614, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 1.933333333333362}, "ep002": {"nsteps": 364, "reward": -2.7482636302229366, "good_angle": 0.7961713109202434, "survival_time": 12.1333333333333, "traveled_tiles": 3, "valid_direction": 1.5999999999999943}, "ep003": {"nsteps": 500, "reward": -1.1700545222759249, "good_angle": 58.375203150184426, "survival_time": 16.666666666666654, "traveled_tiles": 4, "valid_direction": 7.066666666666666}, "ep004": {"nsteps": 500, "reward": -1.5698140780329704, "good_angle": 12.501617076206289, "survival_time": 16.666666666666654, "traveled_tiles": 3, "valid_direction": 6.866666666666678}}
good_angle_max60.62226718882041
good_angle_mean30.098240438227595
good_angle_median18.195943465006614
good_angle_min0.7961713109202434
reward_max-0.39393230858445166
reward_mean-1.4326714467250627
reward_median-1.2812926945090295
reward_min-2.7482636302229366
survival_time_max16.666666666666654
survival_time_mean15.759999999999986
survival_time_min12.1333333333333
traveled_tiles_max4
traveled_tiles_mean3.2
traveled_tiles_median3
traveled_tiles_min3
valid_direction_max7.133333333333309
valid_direction_mean4.920000000000002
valid_direction_median6.866666666666678
valid_direction_min1.5999999999999943
No reset possible
2818320Andrea Censi 🇨🇭PyTorch DDPG templateaido1_LF1_r3-v3step1-simulationfailednoidsc-rudolf-112180:00:34
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 586, in wrap_solution
    raise InvalidSubmission(msg)
InvalidSubmission: Uncaught exception in solution:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 581, in wrap_solution
    solution.run(cis)
  File "solution.py", line 87, in run
    solve(params, cis)
  File "solution.py", line 60, in solve
    action = model.predict(observation)
  File "/workspace/model.py", line 176, in predict
    return self.actor(state).cpu().data.numpy().flatten()
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/workspace/model.py", line 72, in forward
    x = self.lr(self.lin1(x))
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/module.py", line 477, in __call__
    result = self.forward(*input, **kwargs)
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/modules/linear.py", line 55, in forward
    return F.linear(input, self.weight, self.bias)
  File "/opt/conda/lib/python2.7/site-packages/torch/nn/functional.py", line 1024, in linear
    return torch.addmm(bias, input, weight.t())
RuntimeError: size mismatch, m1: [1 x 127872], m2: [4032 x 512] at /tmp/pip-req-build-vRRdPa/aten/src/TH/generic/THTensorMath.cpp:2070
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
2814149Julian ZillyBaseline solution using imitation learning from logsaido1_LF1_r3-v3step1-simulationerrornoidsc-rudolf-112180:00:09
Error while running [...]
Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job2814-7063', 'pull']:

   >  Command '['docker-compose', '-p', 'job2814-7063', 'pull']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible