Duckietown Challenges Home Challenges Submissions

Evaluator 82

ID82
evaluatorrudolf-3
ownerAndrea Censi 🇨🇭
machineidsc-rudolf
processrudolf-3
versionDC:3.1.41;DCR:3.2.3;DTS:3.0.24
first heard
last heard
statusinactive
# evaluating
# success6 445
# timeout
# failed2 380
# error1 424
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
456262Vadim Volodin 🇷🇺Simple based on center pixelaido1_LF1_r2-v3step2-scoringsuccessyesrudolf-30:00:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean1.4466666666666677
good_angle_mean33.66180062299222
valid_direction_mean4.846666666666651


other stats
episodes
details{"ep000": {"nsteps": 43, "reward": -24.1210139712622, "good_angle": 0.0001490357416107672, "survival_time": 1.4333333333333345, "traveled_tiles": 2, "valid_direction": 0}, "ep001": {"nsteps": 500, "reward": -0.779671716362238, "good_angle": 59.90173853263349, "survival_time": 1.3666666666666676, "traveled_tiles": 6, "valid_direction": 10.099999999999966}, "ep002": {"nsteps": 500, "reward": -1.5157463335990906, "good_angle": 0.565799216449206, "survival_time": 1.466666666666668, "traveled_tiles": 3, "valid_direction": 0.6999999999999998}, "ep003": {"nsteps": 23, "reward": -45.41889384518499, "good_angle": 0.0007966565707015758, "survival_time": 0.7666666666666666, "traveled_tiles": 1, "valid_direction": 0}, "ep004": {"nsteps": 500, "reward": -3.354815784573555, "good_angle": 107.84051967356608, "survival_time": 2.200000000000002, "traveled_tiles": 5, "valid_direction": 13.433333333333287}}
good_angle_max107.84051967356608
good_angle_median0.565799216449206
good_angle_min0.0001490357416107672
reward_max-0.779671716362238
reward_mean-15.038028330196417
reward_median-3.354815784573555
reward_min-45.41889384518499
survival_time_max2.200000000000002
survival_time_median1.4333333333333345
survival_time_min0.7666666666666666
traveled_tiles_max6
traveled_tiles_mean3.4
traveled_tiles_min1
valid_direction_max13.433333333333287
valid_direction_median0.6999999999999998
valid_direction_min0
No reset possible
454261David AbrahamRandom executionaido1_LF1_r2-v3step2-scoringsuccessyesrudolf-30:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median2
survival_time_mean2.373333333333332
good_angle_mean3.2448348216917005
valid_direction_mean0.7733333333333331


other stats
episodes
details{"ep000": {"nsteps": 60, "reward": -18.55895467996597, "good_angle": 8.631361811097454, "survival_time": 2.0000000000000027, "traveled_tiles": 2, "valid_direction": 1.4000000000000028}, "ep001": {"nsteps": 125, "reward": -8.8210487947464, "good_angle": 5.922127375530279, "survival_time": 4.166666666666662, "traveled_tiles": 3, "valid_direction": 1.399999999999995}, "ep002": {"nsteps": 40, "reward": -26.14327304959297, "good_angle": 1.60895172094825, "survival_time": 1.333333333333334, "traveled_tiles": 1, "valid_direction": 1.066666666666667}, "ep003": {"nsteps": 15, "reward": -68.38100338776907, "good_angle": 0.017532853174180282, "survival_time": 0.49999999999999994, "traveled_tiles": 1, "valid_direction": 0}, "ep004": {"nsteps": 116, "reward": -9.494371339903566, "good_angle": 0.04420034770833667, "survival_time": 3.8666666666666623, "traveled_tiles": 4, "valid_direction": 0}}
good_angle_max8.631361811097454
good_angle_median1.60895172094825
good_angle_min0.017532853174180282
reward_max-8.8210487947464
reward_mean-26.279730250395595
reward_median-18.55895467996597
reward_min-68.38100338776907
survival_time_max4.166666666666662
survival_time_median2.0000000000000027
survival_time_min0.49999999999999994
traveled_tiles_max4
traveled_tiles_mean2.2
traveled_tiles_min1
valid_direction_max1.4000000000000028
valid_direction_median1.066666666666667
valid_direction_min0
No reset possible
452260Vadim Volodin 🇷🇺Simple based on center pixelaido1_LF1_r2-v3step1-simulationfailedyesrudolf-30:00:23
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/workspace/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 475, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
451259David AbrahamRandom executionaido1_LF1_r2-v3step2-scoringsuccessyesrudolf-30:00:24
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median2
survival_time_mean2.373333333333332
good_angle_mean3.2448348216917005
valid_direction_mean0.7733333333333331


other stats
episodes
details{"ep000": {"nsteps": 60, "reward": -18.55895467996597, "good_angle": 8.631361811097454, "survival_time": 2.0000000000000027, "traveled_tiles": 2, "valid_direction": 1.4000000000000028}, "ep001": {"nsteps": 125, "reward": -8.8210487947464, "good_angle": 5.922127375530279, "survival_time": 4.166666666666662, "traveled_tiles": 3, "valid_direction": 1.399999999999995}, "ep002": {"nsteps": 40, "reward": -26.14327304959297, "good_angle": 1.60895172094825, "survival_time": 1.333333333333334, "traveled_tiles": 1, "valid_direction": 1.066666666666667}, "ep003": {"nsteps": 15, "reward": -68.38100338776907, "good_angle": 0.017532853174180282, "survival_time": 0.49999999999999994, "traveled_tiles": 1, "valid_direction": 0}, "ep004": {"nsteps": 116, "reward": -9.494371339903566, "good_angle": 0.04420034770833667, "survival_time": 3.8666666666666623, "traveled_tiles": 4, "valid_direction": 0}}
good_angle_max8.631361811097454
good_angle_median1.60895172094825
good_angle_min0.017532853174180282
reward_max-8.8210487947464
reward_mean-26.279730250395595
reward_median-18.55895467996597
reward_min-68.38100338776907
survival_time_max4.166666666666662
survival_time_median2.0000000000000027
survival_time_min0.49999999999999994
traveled_tiles_max4
traveled_tiles_mean2.2
traveled_tiles_min1
valid_direction_max1.4000000000000028
valid_direction_median1.066666666666667
valid_direction_min0
No reset possible
449258Vadim Volodin 🇷🇺Simple based on center pixelaido1_LF1_r2-v3step2-scoringsuccessyesrudolf-30:01:01
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median2
survival_time_mean10.273333333333325
good_angle_mean22.69422169851428
valid_direction_mean2.886666666666657


other stats
episodes
details{"ep000": {"nsteps": 500, "reward": -0.6508665301799774, "good_angle": 0.001770686549137422, "survival_time": 16.666666666666654, "traveled_tiles": 1, "valid_direction": 0}, "ep001": {"nsteps": 500, "reward": -2.0878472118377687, "good_angle": 0.5146141849170722, "survival_time": 16.599999999999984, "traveled_tiles": 2, "valid_direction": 0}, "ep002": {"nsteps": 500, "reward": -1.4231658036708832, "good_angle": 0.19190295995786152, "survival_time": 16.599999999999984, "traveled_tiles": 3, "valid_direction": 0}, "ep003": {"nsteps": 25, "reward": -41.962430267333986, "good_angle": 0.0007642423451666636, "survival_time": 0.7666666666666666, "traveled_tiles": 2, "valid_direction": 0}, "ep004": {"nsteps": 500, "reward": -5.137258462488651, "good_angle": 112.76205641880216, "survival_time": 0.7333333333333333, "traveled_tiles": 5, "valid_direction": 14.433333333333284}}
good_angle_max112.76205641880216
good_angle_median0.19190295995786152
good_angle_min0.0007642423451666636
reward_max-0.6508665301799774
reward_mean-10.252313655102252
reward_median-2.0878472118377687
reward_min-41.962430267333986
survival_time_max16.666666666666654
survival_time_median16.599999999999984
survival_time_min0.7333333333333333
traveled_tiles_max5
traveled_tiles_mean2.6
traveled_tiles_min1
valid_direction_max14.433333333333284
valid_direction_median0
valid_direction_min0
No reset possible
446257Vadim Volodin 🇷🇺Simple based on center pixelaido1_LF1_r2-v3step1-simulationsuccessyesrudolf-30:00:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
other stats
simulation-passed1
No reset possible
445256Mandana Samiei 🇨🇦Tensorflow templateaido1_LF1_r2-v3step2-scoringsuccessyesrudolf-30:00:26
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
traveled_tiles_median3
survival_time_mean3.099999999999996
good_angle_mean0.8540867043033711
valid_direction_mean1.0933333333333328


other stats
episodes
details{"ep000": {"nsteps": 45, "reward": -22.730665498682193, "good_angle": 2.464011645649803, "survival_time": 1.500000000000001, "traveled_tiles": 3, "valid_direction": 1.400000000000001}, "ep001": {"nsteps": 108, "reward": -9.473568658999822, "good_angle": 0.21552867377610185, "survival_time": 3.599999999999997, "traveled_tiles": 2, "valid_direction": 0.5333333333333314}, "ep002": {"nsteps": 237, "reward": -4.432748170956362, "good_angle": 0.44670148280839816, "survival_time": 7.899999999999982, "traveled_tiles": 5, "valid_direction": 0.6666666666666666}, "ep003": {"nsteps": 39, "reward": -27.63335389510179, "good_angle": 0.3410915951631881, "survival_time": 1.3000000000000007, "traveled_tiles": 1, "valid_direction": 0.9333333333333336}, "ep004": {"nsteps": 112, "reward": -9.770550391495428, "good_angle": 0.8031001241193636, "survival_time": 1.2000000000000004, "traveled_tiles": 5, "valid_direction": 1.9333333333333316}}
good_angle_max2.464011645649803
good_angle_median0.44670148280839816
good_angle_min0.21552867377610185
reward_max-4.432748170956362
reward_mean-14.808177323047122
reward_median-9.770550391495428
reward_min-27.63335389510179
survival_time_max7.899999999999982
survival_time_median1.500000000000001
survival_time_min1.2000000000000004
traveled_tiles_max5
traveled_tiles_mean3.2
traveled_tiles_min1
valid_direction_max1.9333333333333316
valid_direction_median0.9333333333333336
valid_direction_min0.5333333333333314
No reset possible
424243Benjamin Ramtoula 🇨🇦Template for ROS Submissionaido1_LF1_r2-v3step1-simulationerroryesrudolf-30:02:23
Uncaught exception w [...]
Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 412, in run
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 560, in run_docker
    raise DockerComposeFail(msg)
DockerComposeFail: Could not run ['docker-compose', '-p', 'job424-5232', 'up', '--abort-on-container-exit']:

   >  Command '['docker-compose', '-p', 'job424-5232', 'up', '--abort-on-container-exit']' returned non-zero exit status 1

 docker-compose stdout  | None
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
380215Andrea Censi 🇨🇭Java templateaido1_amod_efficiency_r1-v3step1-simulationfailednorudolf-30:00:30
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/amod/target/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 486, in wrap_evaluator
    raise InvalidSubmission(out[SPECIAL_INVALID_SUBMISSION])
InvalidSubmission: Invalid solution:
Uncaught exception in solution:
Traceback (most recent call last):
  File "/aidamod/target/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 485, in wrap_solution
    solution.run(cis)
  File "/project/solution.py", line 25, in run
    subprocess.check_call(cmd, cwd=cwd, stdout=sys.stdout, stderr=sys.stderr)
  File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
CalledProcessError: Command '['java', '-Xmx10000m', '-cp', 'aidamod-1.4.7.jar', 'aidamod.demo.AidoGuest', 'aido-host']' returned non-zero exit status 1

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible