Duckietown Challenges Home Challenges Submissions

Evaluator 891

ID891
evaluatorip-172-31-40-253-32125
ownerI don't have one 😀
machineip-172-31-40-253
processip-172-31-40-253-32125
versiond-c:4.0.25;d-c-r:4.0.31;d-s:4.0.21
first heard
last heard
statusinactive
# evaluating
# success1 21653
# timeout1 21674
# failed
# error
# aborted
# host-error1 21672
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
216742762Bhairav Mehtachallenge-aido_LF-template-ros - Template solution using ROSaido2-LFVI-sim-testingstep1-simulationtimeoutnoip-172-31-40-253-321251:05:23
I can see how the jo [...]
I can see how the job 21674 is timeout because passed 3923 seconds and the timeout is 3600.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
216722799Rami Al-Naim 🇷🇺challenge-aido_LF-baseline-duckietownaido2-LFVI-sim-testingstep1-simulationhost-errornoip-172-31-40-253-321250:00:14
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 484, in get_cr
    submission_id=submission_id, timeout_sec=timeout_sec)
  File "/usr/lib/python3.6/contextlib.py", line 88, in __exit__
    next(self.gen)
  File "/project/src/duckietown_challenges_runner/runner.py", line 343, in setup_logging
    convert(f_stdout)
  File "/project/src/duckietown_challenges_runner/runner.py", line 332, in convert
    data = open(log).read().strip()
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFVI-sim-testing/submission2799/step1-simulation-ip-172-31-40-253-32125-job21672/logs/challenges-runner/stdout.log'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
216533220Amaury Camus 🇨🇭challenge-aido_LF-template-randomaido2-LFVI-sim-testingstep1-simulationsuccessnoip-172-31-40-253-321250:29:58
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.6537162971975778
survival_time_median2.5999999999999988
deviation-center-line_median0.14884708769804722
in-drivable-lane_median0.549999999999998


other stats
agent_compute-ego_max0.16781344780555138
agent_compute-ego_mean0.14688355789649263
agent_compute-ego_median0.1462394893169403
agent_compute-ego_min0.12939976740487014
deviation-center-line_max0.5141120095458062
deviation-center-line_mean0.1769000227005324
deviation-center-line_min0.06216029860587694
deviation-heading_max1.9929437250081896
deviation-heading_mean0.692977823395608
deviation-heading_median0.4928440244444201
deviation-heading_min0.24234723632033853
driven_any_max3.5343244022418343
driven_any_mean1.3470385992234175
driven_any_median1.001633421251194
driven_any_min0.3412551960104244
driven_lanedir_consec_max2.494308053441959
driven_lanedir_consec_mean0.881391085527715
driven_lanedir_consec_min0.3199180625488627
driven_lanedir_max2.494308053441959
driven_lanedir_mean0.8962528018383226
driven_lanedir_median0.7206363324756615
driven_lanedir_min0.3199180625488627
in-drivable-lane_max3.549999999999991
in-drivable-lane_mean0.839999999999998
in-drivable-lane_min0
per-episodes
details{"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.1768752985362765, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.05080753942079182, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.12939976740487014, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.07467711726321449, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.03036651732046393, "sim_compute_performance-ego": 0.05323112765444985, "sim_compute_robot_state-ego": 0.0622326724136932, "sim_compute_robot_state-npc0": 0.08242515672611285, "sim_compute_robot_state-npc1": 0.04983117610593386, "sim_compute_robot_state-npc2": 0.05529931828945498, "sim_compute_robot_state-npc3": 0.060190985474405406}, "udem1-1-0": {"driven_any": 0.47798123795353353, "sim_physics": 0.16460884468896048, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.05702662467956543, "in-drivable-lane": 0, "agent_compute-ego": 0.136674165725708, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.08325252362659999, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.03609880379268101, "sim_compute_performance-ego": 0.0631153838975089, "sim_compute_robot_state-ego": 0.06627359560557775, "sim_compute_robot_state-npc0": 0.0662711773599897, "sim_compute_robot_state-npc1": 0.06416175195149013, "sim_compute_robot_state-npc2": 0.06222374950136457, "sim_compute_robot_state-npc3": 0.06102472543716431}, "udem1-2-0": {"driven_any": 2.1864003300000694, "sim_physics": 0.14909810232884677, "survival_time": 5.14999999999999, "driven_lanedir": 1.826231945363564, "sim_render-ego": 0.05002612512088516, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.13402331222608252, "deviation-heading": 0.8115272458314897, "set_robot_commands": 0.08179477812017052, "deviation-center-line": 0.1927316041609401, "driven_lanedir_consec": 1.826231945363564, "sim_compute_sim_state": 0.033629776204673986, "sim_compute_performance-ego": 0.05687677281574138, "sim_compute_robot_state-ego": 0.06116690219027325, "sim_compute_robot_state-npc0": 0.05746145155823346, "sim_compute_robot_state-npc1": 0.05615798246513293, "sim_compute_robot_state-npc2": 0.05596827071847268, "sim_compute_robot_state-npc3": 0.05723110448966906}, "udem1-3-0": {"driven_any": 0.9410513078477724, "sim_physics": 0.2049365391333898, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.06264586746692657, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.1462394893169403, "deviation-heading": 0.35154027506548424, "set_robot_commands": 0.09234122435251872, "deviation-center-line": 0.08975118546699407, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.03754214942455292, "sim_compute_performance-ego": 0.06319484611352284, "sim_compute_robot_state-ego": 0.0696956863005956, "sim_compute_robot_state-npc0": 0.06776386002699535, "sim_compute_robot_state-npc1": 0.07050059735774994, "sim_compute_robot_state-npc2": 0.06413554648558299, "sim_compute_robot_state-npc3": 0.06668285528818767}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.20104416211446127, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.06183298428853353, "in-drivable-lane": 0, "agent_compute-ego": 0.1458124773842948, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.08839483488173712, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.03650072642735073, "sim_compute_performance-ego": 0.06391383352733794, "sim_compute_robot_state-ego": 0.06472625051225935, "sim_compute_robot_state-npc0": 0.06198883056640625, "sim_compute_robot_state-npc1": 0.06756244386945452, "sim_compute_robot_state-npc2": 0.06621554919651576, "sim_compute_robot_state-npc3": 0.06405455725533622}, "udem1-5-0": {"driven_any": 3.5343244022418343, "sim_physics": 0.2073414368543796, "survival_time": 8.349999999999984, "driven_lanedir": 2.494308053441959, "sim_render-ego": 0.06347436819247856, "in-drivable-lane": 2, "agent_compute-ego": 0.1548478946000516, "deviation-heading": 1.9929437250081896, "set_robot_commands": 0.09704539019190622, "deviation-center-line": 0.5141120095458062, "driven_lanedir_consec": 2.494308053441959, "sim_compute_sim_state": 0.04000169645526452, "sim_compute_performance-ego": 0.07087023672229516, "sim_compute_robot_state-ego": 0.07957053041743661, "sim_compute_robot_state-npc0": 0.07140984078367313, "sim_compute_robot_state-npc1": 0.07200689230136528, "sim_compute_robot_state-npc2": 0.07148936694253705, "sim_compute_robot_state-npc3": 0.07179338917760791}, "udem1-6-0": {"driven_any": 2.176040550989443, "sim_physics": 0.19478244457430052, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.06149461894359404, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.1486973091236596, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.08765566233292367, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.0409413425667772, "sim_compute_performance-ego": 0.06624000743754858, "sim_compute_robot_state-ego": 0.06866096061410255, "sim_compute_robot_state-npc0": 0.06432475627047345, "sim_compute_robot_state-npc1": 0.06643827910562164, "sim_compute_robot_state-npc2": 0.06618649519762947, "sim_compute_robot_state-npc3": 0.06540533871326631}, "udem1-7-0": {"driven_any": 1.0050236108346422, "sim_physics": 0.20859968185424804, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684156, "sim_render-ego": 0.05888418197631836, "in-drivable-lane": 0, "agent_compute-ego": 0.14561088562011718, "deviation-heading": 0.9227446658975216, "set_robot_commands": 0.08856624126434326, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.035811843872070315, "sim_compute_performance-ego": 0.06361225605010987, "sim_compute_robot_state-ego": 0.0708263635635376, "sim_compute_robot_state-npc0": 0.06220446109771729, "sim_compute_robot_state-npc1": 0.06591622829437256, "sim_compute_robot_state-npc2": 0.0712980079650879, "sim_compute_robot_state-npc3": 0.06688118934631347}, "udem1-8-0": {"driven_any": 0.9970416603019576, "sim_physics": 0.21769689138119036, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.0671058388856741, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.164464166531196, "deviation-heading": 0.6817155913229235, "set_robot_commands": 0.10380698167360744, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.042894684351407565, "sim_compute_performance-ego": 0.07514602404374343, "sim_compute_robot_state-ego": 0.08323511710533729, "sim_compute_robot_state-npc0": 0.07294236238186176, "sim_compute_robot_state-npc1": 0.07620129676965567, "sim_compute_robot_state-npc2": 0.07541237886135395, "sim_compute_robot_state-npc3": 0.07486619857641366}, "udem1-9-0": {"driven_any": 0.4374998710472299, "sim_physics": 0.22146127774165228, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.058483463067274824, "in-drivable-lane": 0, "agent_compute-ego": 0.14519636447613057, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.09510488693530744, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.0370102387208205, "sim_compute_performance-ego": 0.06593792255108173, "sim_compute_robot_state-ego": 0.06627689875089206, "sim_compute_robot_state-npc0": 0.06925139060387245, "sim_compute_robot_state-npc1": 0.06869645302112286, "sim_compute_robot_state-npc2": 0.06760042447310227, "sim_compute_robot_state-npc3": 0.07082473314725436}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.20182536948810925, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.06200620261105624, "in-drivable-lane": 0, "agent_compute-ego": 0.14789298447695645, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.08109032024036754, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.040906971151178535, "sim_compute_performance-ego": 0.06309703263369473, "sim_compute_robot_state-ego": 0.07059977271340111, "sim_compute_robot_state-npc0": 0.06157065521587025, "sim_compute_robot_state-npc1": 0.0641443512656472, "sim_compute_robot_state-npc2": 0.0671960332176902, "sim_compute_robot_state-npc3": 0.07527886737476695}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.19374521255493163, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756615, "sim_render-ego": 0.05949097156524658, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.14782638549804689, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.08911380767822266, "deviation-center-line": 0.16567670185555422, "driven_lanedir_consec": 0.7185075497455284, "sim_compute_sim_state": 0.03825984001159668, "sim_compute_performance-ego": 0.065500807762146, "sim_compute_robot_state-ego": 0.07134492874145508, "sim_compute_robot_state-npc0": 0.06797968864440918, "sim_compute_robot_state-npc1": 0.06736161708831787, "sim_compute_robot_state-npc2": 0.06751606464385987, "sim_compute_robot_state-npc3": 0.06373690605163575}, "udem1-12-0": {"driven_any": 1.964687870253437, "sim_physics": 0.1899296298940131, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.05826095824546002, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.14170155373025448, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.08881115152480755, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.03862387322364969, "sim_compute_performance-ego": 0.0641814926837353, "sim_compute_robot_state-ego": 0.07090866058430773, "sim_compute_robot_state-npc0": 0.0641264687193201, "sim_compute_robot_state-npc1": 0.06611758850990458, "sim_compute_robot_state-npc2": 0.06418082815535525, "sim_compute_robot_state-npc3": 0.06699493083548039}, "udem1-13-0": {"driven_any": 1.001633421251194, "sim_physics": 0.2277573713889489, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.06601889316852276, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.16781344780555138, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.1026833699299739, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.04262584447860718, "sim_compute_performance-ego": 0.07624106682263888, "sim_compute_robot_state-ego": 0.08484506148558396, "sim_compute_robot_state-npc0": 0.07454374661812416, "sim_compute_robot_state-npc1": 0.07167724462655875, "sim_compute_robot_state-npc2": 0.07315832376480103, "sim_compute_robot_state-npc3": 0.07514810103636521}, "udem1-14-0": {"driven_any": 2.218769782727203, "sim_physics": 0.1946402572450184, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433656, "sim_render-ego": 0.0618524051847912, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.14705316452752976, "deviation-heading": 1.7163026916317128, "set_robot_commands": 0.09182580539158412, "deviation-center-line": 0.24126933160503777, "driven_lanedir_consec": 0.6077880672015845, "sim_compute_sim_state": 0.03875269435700916, "sim_compute_performance-ego": 0.06687833695184617, "sim_compute_robot_state-ego": 0.06960142226446242, "sim_compute_robot_state-npc0": 0.06573177746364049, "sim_compute_robot_state-npc1": 0.06815770467122396, "sim_compute_robot_state-npc2": 0.06879421869913736, "sim_compute_robot_state-npc3": 0.06991132554553804}}
set_robot_commands_max0.10380698167360744
set_robot_commands_mean0.08974427302715231
set_robot_commands_median0.08881115152480755
set_robot_commands_min0.07467711726321449
sim_compute_performance-ego_max0.07624106682263888
sim_compute_performance-ego_mean0.06520247651116005
sim_compute_performance-ego_median0.0641814926837353
sim_compute_performance-ego_min0.05323112765444985
sim_compute_robot_state-ego_max0.08484506148558396
sim_compute_robot_state-ego_mean0.07066432155086104
sim_compute_robot_state-ego_median0.0696956863005956
sim_compute_robot_state-ego_min0.06116690219027325
sim_compute_robot_state-npc0_max0.08242515672611285
sim_compute_robot_state-npc0_mean0.06733304160244666
sim_compute_robot_state-npc0_median0.0662711773599897
sim_compute_robot_state-npc0_min0.05746145155823346
sim_compute_robot_state-npc1_max0.07620129676965567
sim_compute_robot_state-npc1_mean0.06632877382690346
sim_compute_robot_state-npc1_median0.06736161708831787
sim_compute_robot_state-npc1_min0.04983117610593386
sim_compute_robot_state-npc2_max0.07541237886135395
sim_compute_robot_state-npc2_mean0.06644497174079635
sim_compute_robot_state-npc2_median0.0671960332176902
sim_compute_robot_state-npc2_min0.05529931828945498
sim_compute_robot_state-npc3_max0.07527886737476695
sim_compute_robot_state-npc3_mean0.06733501384996031
sim_compute_robot_state-npc3_median0.06688118934631347
sim_compute_robot_state-npc3_min0.05723110448966906
sim_compute_sim_state_max0.042894684351407565
sim_compute_sim_state_mean0.03799780015720692
sim_compute_sim_state_median0.03825984001159668
sim_compute_sim_state_min0.03036651732046393
sim_physics_max0.2277573713889489
sim_physics_mean0.19695616798524845
sim_physics_median0.20104416211446127
sim_physics_min0.14909810232884677
sim_render-ego_max0.0671058388856741
sim_render-ego_mean0.05996073618780795
sim_render-ego_median0.06149461894359404
sim_render-ego_min0.05002612512088516
simulation-passed1
survival_time_max8.349999999999984
survival_time_mean3.3333333333333286
survival_time_min1.0500000000000005
No reset possible