Evaluator 1586

ID	1586
evaluator	`ip-172-31-45-34-10501`
owner	I don't have one 😀
machine	ip-172-31-45-34
process	ip-172-31-45-34-10501
version	d-c:5.0.7;d-c-r:5.0.10;d-s:5.0.3
first heard	2019-10-17 02:44:09+00:00
last heard	2019-11-08 10:54:15+00:00
status	inactive
# evaluating
# success	22 `24455`
# timeout
# failed	5 `24423`
# error	14 `24351`
# aborted
# host-error	7 `24552`
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job ID

submission

user

user label

challenge

step

status

up to date

evaluator

date started

date completed

duration

message

25121

4394

Jeick Hincapie Barrera 🇨🇴

challenge-aido_LF-template-ros - Template solution using ROS

aido3-LFVI-sim-testing

step1-simulation

success

ip-172-31-45-34-10501

2019-11-07 14:23:45+00:00

2019-11-07 14:38:30+00:00

0:14:45

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.2875407694053156
survival_time_median	2.849999999999998
deviation-center-line_median	0.11031194850197353
in-drivable-lane_median	0.9499999999999966

other stats

agent_compute-ego_max	0.02321464927108199
agent_compute-ego_mean	0.016415493508537746
agent_compute-ego_median	0.015277101880028137
agent_compute-ego_min	0.01460445445516835
deviation-center-line_max	0.2701922695523504
deviation-center-line_mean	0.13746050030700233
deviation-center-line_min	0.07113819587235681
deviation-heading_max	1.4854398920276664
deviation-heading_mean	0.7236995121312618
deviation-heading_median	0.6953408166028195
deviation-heading_min	0.31186465737091673
driven_any_max	1.934434121703833
driven_any_mean	0.8067578066341482
driven_any_median	0.8072381933001381
driven_any_min	0.2091258588042385
driven_lanedir_consec_max	0.7222251056746308
driven_lanedir_consec_mean	0.396283186751621
driven_lanedir_consec_min	0.1917983424483971
driven_lanedir_max	0.7287863177121016
driven_lanedir_mean	0.397564077689832
driven_lanedir_median	0.2875407694053156
driven_lanedir_min	0.1917983424483971
in-drivable-lane_max	4.04999999999999
in-drivable-lane_mean	1.2133333333333305
in-drivable-lane_min	0
per-episodes	details {"udem1-0-0": {"driven_any": 1.093753441791868, "sim_physics": 0.148813121290092, "survival_time": 4.149999999999993, "driven_lanedir": 0.4702323960536803, "sim_render-ego": 0.009582938918148178, "in-drivable-lane": 1.6499999999999957, "agent_compute-ego": 0.019589929695588995, "deviation-heading": 1.2111022380750738, "set_robot_commands": 0.007223861763276249, "deviation-center-line": 0.2487133429854821, "driven_lanedir_consec": 0.4702323960536803, "sim_compute_sim_state": 0.003858221582619541, "sim_compute_performance-ego": 0.005449418562004365, "sim_compute_robot_state-ego": 0.006756839981998305, "sim_compute_robot_state-npc0": 0.006659668612192912, "sim_compute_robot_state-npc1": 0.00638021618486887, "sim_compute_robot_state-npc2": 0.006379779562892684, "sim_compute_robot_state-npc3": 0.006155585668173181}, "udem1-1-0": {"driven_any": 0.29795781475632777, "sim_physics": 0.12208587151986582, "survival_time": 1.3500000000000003, "driven_lanedir": 0.2312609677194346, "sim_render-ego": 0.008382320404052734, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.02321464927108199, "deviation-heading": 0.6953408166028195, "set_robot_commands": 0.006662333453143084, "deviation-center-line": 0.10073232857747254, "driven_lanedir_consec": 0.2312609677194346, "sim_compute_sim_state": 0.0032602945963541665, "sim_compute_performance-ego": 0.004500450911345305, "sim_compute_robot_state-ego": 0.00573703094764992, "sim_compute_robot_state-npc0": 0.005769394062183521, "sim_compute_robot_state-npc1": 0.005420985045256438, "sim_compute_robot_state-npc2": 0.005419422079015661, "sim_compute_robot_state-npc3": 0.005397814291494864}, "udem1-2-0": {"driven_any": 1.0809477890804953, "sim_physics": 0.15122781961392134, "survival_time": 3.899999999999994, "driven_lanedir": 0.6758329617605024, "sim_render-ego": 0.009601189539982725, "in-drivable-lane": 1.099999999999996, "agent_compute-ego": 0.020882789905254655, "deviation-heading": 1.0268575719271245, "set_robot_commands": 0.007127926899836614, "deviation-center-line": 0.2701922695523504, "driven_lanedir_consec": 0.6735785385729686, "sim_compute_sim_state": 0.003917853037516276, "sim_compute_performance-ego": 0.005348492891360552, "sim_compute_robot_state-ego": 0.006462904123159556, "sim_compute_robot_state-npc0": 0.006519980919666779, "sim_compute_robot_state-npc1": 0.006363596671666854, "sim_compute_robot_state-npc2": 0.006144520563957019, "sim_compute_robot_state-npc3": 0.0060775493964170795}, "udem1-3-0": {"driven_any": 0.8072381933001381, "sim_physics": 0.11891710967348332, "survival_time": 2.849999999999998, "driven_lanedir": 0.47292588175405326, "sim_render-ego": 0.007786110827797337, "in-drivable-lane": 0.7999999999999976, "agent_compute-ego": 0.0158538818359375, "deviation-heading": 0.7978002362162196, "set_robot_commands": 0.005310171528866417, "deviation-center-line": 0.13623952041905, "driven_lanedir_consec": 0.47292588175405326, "sim_compute_sim_state": 0.003043977837813528, "sim_compute_performance-ego": 0.004294546026932566, "sim_compute_robot_state-ego": 0.005315709532352916, "sim_compute_robot_state-npc0": 0.00516082529436078, "sim_compute_robot_state-npc1": 0.004874522225898609, "sim_compute_robot_state-npc2": 0.004898326438769959, "sim_compute_robot_state-npc3": 0.004931717588190446}, "udem1-4-0": {"driven_any": 0.31118732113900155, "sim_physics": 0.10706536666206692, "survival_time": 1.1500000000000004, "driven_lanedir": 0.2875407694053156, "sim_render-ego": 0.007550322491189708, "in-drivable-lane": 0, "agent_compute-ego": 0.01460445445516835, "deviation-heading": 0.39989969254594865, "set_robot_commands": 0.0051971933116083556, "deviation-center-line": 0.07144385614371519, "driven_lanedir_consec": 0.2875407694053156, "sim_compute_sim_state": 0.0029835597328517747, "sim_compute_performance-ego": 0.0040922579557999324, "sim_compute_robot_state-ego": 0.00509799045065175, "sim_compute_robot_state-npc0": 0.005072261976159137, "sim_compute_robot_state-npc1": 0.004836559295654297, "sim_compute_robot_state-npc2": 0.004795551300048828, "sim_compute_robot_state-npc3": 0.004769729531329611}, "udem1-5-0": {"driven_any": 1.3021233152606908, "sim_physics": 0.1134319024927476, "survival_time": 5.09999999999999, "driven_lanedir": 0.20473092327823528, "sim_render-ego": 0.007152781767003676, "in-drivable-lane": 4.04999999999999, "agent_compute-ego": 0.01482703639011757, "deviation-heading": 0.394574000309935, "set_robot_commands": 0.004892795693640616, "deviation-center-line": 0.10111471927467218, "driven_lanedir_consec": 0.20473092327823528, "sim_compute_sim_state": 0.0028148398679845475, "sim_compute_performance-ego": 0.003953195085712508, "sim_compute_robot_state-ego": 0.005022883415222168, "sim_compute_robot_state-npc0": 0.004824322812697466, "sim_compute_robot_state-npc1": 0.004581435053956275, "sim_compute_robot_state-npc2": 0.0045553843180338545, "sim_compute_robot_state-npc3": 0.004538690342622644}, "udem1-6-0": {"driven_any": 1.934434121703833, "sim_physics": 0.110510858467647, "survival_time": 6.999999999999983, "driven_lanedir": 0.6746993221948854, "sim_render-ego": 0.007171400955745152, "in-drivable-lane": 3.899999999999988, "agent_compute-ego": 0.015123663629804338, "deviation-heading": 1.4854398920276664, "set_robot_commands": 0.004939820085253034, "deviation-center-line": 0.2528639324079792, "driven_lanedir_consec": 0.6740101897237718, "sim_compute_sim_state": 0.002907764911651611, "sim_compute_performance-ego": 0.003974027293069022, "sim_compute_robot_state-ego": 0.005032052312578474, "sim_compute_robot_state-npc0": 0.004923207419259208, "sim_compute_robot_state-npc1": 0.0046527538980756485, "sim_compute_robot_state-npc2": 0.004638111591339112, "sim_compute_robot_state-npc3": 0.004610240459442139}, "udem1-7-0": {"driven_any": 1.0625977382418332, "sim_physics": 0.11749808102437896, "survival_time": 3.649999999999995, "driven_lanedir": 0.7287863177121016, "sim_render-ego": 0.007262354027735044, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.014884390243112224, "deviation-heading": 0.7849150213204692, "set_robot_commands": 0.004846497757794106, "deviation-center-line": 0.1387309750370881, "driven_lanedir_consec": 0.7222251056746308, "sim_compute_sim_state": 0.002883659650201667, "sim_compute_performance-ego": 0.003957777807157333, "sim_compute_robot_state-ego": 0.005019135671119167, "sim_compute_robot_state-npc0": 0.00488016376756642, "sim_compute_robot_state-npc1": 0.004600315877836044, "sim_compute_robot_state-npc2": 0.004578319314407975, "sim_compute_robot_state-npc3": 0.004579514673311417}, "udem1-8-0": {"driven_any": 1.1602870725462513, "sim_physics": 0.10489815334941066, "survival_time": 4.299999999999993, "driven_lanedir": 0.3591309051781777, "sim_render-ego": 0.007174059402110965, "in-drivable-lane": 2.399999999999996, "agent_compute-ego": 0.014733602834302324, "deviation-heading": 1.2197696860883536, "set_robot_commands": 0.004962563514709473, "deviation-center-line": 0.15216623639979054, "driven_lanedir_consec": 0.3591309051781777, "sim_compute_sim_state": 0.0028524620588435685, "sim_compute_performance-ego": 0.003958377727242403, "sim_compute_robot_state-ego": 0.0050235709478688795, "sim_compute_robot_state-npc0": 0.004915897236313931, "sim_compute_robot_state-npc1": 0.004612517911334371, "sim_compute_robot_state-npc2": 0.004585338193316793, "sim_compute_robot_state-npc3": 0.004587974659232206}, "udem1-9-0": {"driven_any": 0.2911508182793437, "sim_physics": 0.14023685455322266, "survival_time": 1.3000000000000005, "driven_lanedir": 0.2409218388122456, "sim_render-ego": 0.0076141174022968, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.016139103816105768, "deviation-heading": 0.5613432181706873, "set_robot_commands": 0.004988716198847844, "deviation-center-line": 0.0760477654868993, "driven_lanedir_consec": 0.2409218388122456, "sim_compute_sim_state": 0.0029527865923368013, "sim_compute_performance-ego": 0.004083468363835261, "sim_compute_robot_state-ego": 0.00512687059549185, "sim_compute_robot_state-npc0": 0.0050582243846013, "sim_compute_robot_state-npc1": 0.004786573923551119, "sim_compute_robot_state-npc2": 0.004793937389667218, "sim_compute_robot_state-npc3": 0.004786051236666166}, "udem1-10-0": {"driven_any": 1.0214919961604745, "sim_physics": 0.10039145151774088, "survival_time": 3.7499999999999942, "driven_lanedir": 0.7026537170655107, "sim_render-ego": 0.007250900268554687, "in-drivable-lane": 0.9499999999999966, "agent_compute-ego": 0.015233771006266276, "deviation-heading": 0.7506445024068278, "set_robot_commands": 0.004966052373250326, "deviation-center-line": 0.11031194850197353, "driven_lanedir_consec": 0.6929451206884647, "sim_compute_sim_state": 0.002925866444905599, "sim_compute_performance-ego": 0.003989931742350261, "sim_compute_robot_state-ego": 0.005069106419881185, "sim_compute_robot_state-npc0": 0.004937744140625, "sim_compute_robot_state-npc1": 0.00464198112487793, "sim_compute_robot_state-npc2": 0.00461204210917155, "sim_compute_robot_state-npc3": 0.004604899088541667}, "udem1-11-0": {"driven_any": 0.2091258588042385, "sim_physics": 0.10328850292024158, "survival_time": 1.0500000000000005, "driven_lanedir": 0.1917983424483971, "sim_render-ego": 0.007213535762968517, "in-drivable-lane": 0, "agent_compute-ego": 0.015277101880028137, "deviation-heading": 0.3455718910535376, "set_robot_commands": 0.0048445747012183776, "deviation-center-line": 0.10333747877015348, "driven_lanedir_consec": 0.1917983424483971, "sim_compute_sim_state": 0.002787885211762928, "sim_compute_performance-ego": 0.003930750347319103, "sim_compute_robot_state-ego": 0.004971776689801898, "sim_compute_robot_state-npc0": 0.004793246587117513, "sim_compute_robot_state-npc1": 0.004564024153209868, "sim_compute_robot_state-npc2": 0.004548220407395135, "sim_compute_robot_state-npc3": 0.004582722981770833}, "udem1-12-0": {"driven_any": 0.5918906368569804, "sim_physics": 0.1130267908406812, "survival_time": 2.1500000000000004, "driven_lanedir": 0.19652140216443992, "sim_render-ego": 0.007347650306169377, "in-drivable-lane": 1.15, "agent_compute-ego": 0.01495952938878259, "deviation-heading": 0.32606788824321575, "set_robot_commands": 0.004943498345308526, "deviation-center-line": 0.10664063848740413, "driven_lanedir_consec": 0.19652140216443992, "sim_compute_sim_state": 0.002941564072010129, "sim_compute_performance-ego": 0.0040570248005002045, "sim_compute_robot_state-ego": 0.005091317864351495, "sim_compute_robot_state-npc0": 0.004949431086695472, "sim_compute_robot_state-npc1": 0.004723704138467478, "sim_compute_robot_state-npc2": 0.0046920277351556825, "sim_compute_robot_state-npc3": 0.004722678384115529}, "udem1-13-0": {"driven_any": 0.2815655086390766, "sim_physics": 0.1073780996458871, "survival_time": 1.4000000000000006, "driven_lanedir": 0.27004610417980546, "sim_render-ego": 0.007415098803383964, "in-drivable-lane": 0, "agent_compute-ego": 0.015319083418164934, "deviation-heading": 0.31186465737091673, "set_robot_commands": 0.005012708050864083, "deviation-center-line": 0.12223429668864684, "driven_lanedir_consec": 0.27004610417980546, "sim_compute_sim_state": 0.002856935773577009, "sim_compute_performance-ego": 0.004025987216404506, "sim_compute_robot_state-ego": 0.005088244165693011, "sim_compute_robot_state-npc0": 0.004922126020703997, "sim_compute_robot_state-npc1": 0.004676835877554757, "sim_compute_robot_state-npc2": 0.004662351948874337, "sim_compute_robot_state-npc3": 0.0046558209827968055}, "udem1-14-0": {"driven_any": 0.6556154729516691, "sim_physics": 0.13866802757861568, "survival_time": 2.549999999999999, "driven_lanedir": 0.25637931562069505, "sim_render-ego": 0.007163211411120845, "in-drivable-lane": 1.2999999999999985, "agent_compute-ego": 0.015589414858350569, "deviation-heading": 0.5443013696101296, "set_robot_commands": 0.004831351485906863, "deviation-center-line": 0.07113819587235681, "driven_lanedir_consec": 0.25637931562069505, "sim_compute_sim_state": 0.0028334505417767692, "sim_compute_performance-ego": 0.003976784500421262, "sim_compute_robot_state-ego": 0.005002779119154986, "sim_compute_robot_state-npc0": 0.0048832612879136025, "sim_compute_robot_state-npc1": 0.0046059758055443855, "sim_compute_robot_state-npc2": 0.004581227022058824, "sim_compute_robot_state-npc3": 0.004560550053914388}}
set_robot_commands_max	0.007223861763276249
set_robot_commands_mean	0.005383337677568265
set_robot_commands_median	0.004966052373250326
set_robot_commands_min	0.004831351485906863
sim_compute_performance-ego_max	0.005449418562004365
sim_compute_performance-ego_mean	0.0042394994154303055
sim_compute_performance-ego_median	0.004025987216404506
sim_compute_performance-ego_min	0.003930750347319103
sim_compute_robot_state-ego_max	0.006756839981998305
sim_compute_robot_state-ego_mean	0.005321214149131703
sim_compute_robot_state-ego_median	0.005088244165693011
sim_compute_robot_state-ego_min	0.004971776689801898
sim_compute_robot_state-npc0_max	0.006659668612192912
sim_compute_robot_state-npc0_mean	0.005217983707203803
sim_compute_robot_state-npc0_median	0.004937744140625
sim_compute_robot_state-npc0_min	0.004793246587117513
sim_compute_robot_state-npc1_max	0.00638021618486887
sim_compute_robot_state-npc1_mean	0.004954799812516863
sim_compute_robot_state-npc1_median	0.004676835877554757
sim_compute_robot_state-npc1_min	0.004564024153209868
sim_compute_robot_state-npc2_max	0.006379779562892684
sim_compute_robot_state-npc2_mean	0.0049256373316069755
sim_compute_robot_state-npc2_median	0.004662351948874337
sim_compute_robot_state-npc2_min	0.004548220407395135
sim_compute_robot_state-npc3_max	0.006155585668173181
sim_compute_robot_state-npc3_mean	0.004904102622534598
sim_compute_robot_state-npc3_median	0.0046558209827968055
sim_compute_robot_state-npc3_min	0.004538690342622644
sim_compute_sim_state_max	0.003917853037516276
sim_compute_sim_state_mean	0.0030547414608137276
sim_compute_sim_state_median	0.002925866444905599
sim_compute_sim_state_min	0.002787885211762928
sim_physics_max	0.15122781961392134
sim_physics_mean	0.1198292007433335
sim_physics_median	0.1134319024927476
sim_physics_min	0.10039145151774088
sim_render-ego_max	0.009601189539982725
sim_render-ego_mean	0.00771119948588398
sim_render-ego_median	0.007347650306169377
sim_render-ego_min	0.007152781767003676
simulation-passed	1
survival_time_max	6.999999999999983
survival_time_mean	3.043333333333329
survival_time_min	1.0500000000000005

No reset possible

25105

4385

Soroush Saryazdi 🇨🇦

bluejay

aido3-LFVI-sim-validation

step1-simulation

failed

ip-172-31-45-34-10501

2019-11-06 18:40:46+00:00

2019-11-06 18:46:51+00:00

0:06:05

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 296, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 239, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/agent-out after 258 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25070

4368

Jeick Hincapie Barrera 🇨🇴

challenge-aido_LF-template-tensorflow

aido3-LFV-sim-testing

step1-simulation

success

ip-172-31-45-34-10501

2019-11-05 15:16:04+00:00

2019-11-05 15:30:32+00:00

0:14:28

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.2287096756476532
survival_time_median	1.5500000000000007
deviation-center-line_median	0.09119771829043843
in-drivable-lane_median	0.04999999999999993

other stats

agent_compute-ego_max	0.033750454584757485
agent_compute-ego_mean	0.018246947301065613
agent_compute-ego_median	0.015553259080456149
agent_compute-ego_min	0.014695481820539995
deviation-center-line_max	0.2599992089214533
deviation-center-line_mean	0.10635622109572546
deviation-center-line_min	0.05837064533251564
deviation-heading_max	1.7207292079338334
deviation-heading_mean	0.6510128669081571
deviation-heading_median	0.5291329283059608
deviation-heading_min	0.24626917008312296
driven_any_max	2.38651403579366
driven_any_mean	0.48916051613335537
driven_any_median	0.2845395962701251
driven_any_min	0.16014162186451372
driven_lanedir_consec_max	1.294346851754225
driven_lanedir_consec_mean	0.3101899241762751
driven_lanedir_consec_min	0.11681077371457738
driven_lanedir_max	1.294346851754225
driven_lanedir_mean	0.3101899241762751
driven_lanedir_median	0.2287096756476532
driven_lanedir_min	0.11681077371457738
in-drivable-lane_max	4.35000000000002
in-drivable-lane_mean	0.45000000000000107
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.20857643999428416, "sim_physics": 0.25046794755118235, "survival_time": 1.0500000000000005, "driven_lanedir": 0.11681077371457738, "sim_render-ego": 0.013317028681437174, "in-drivable-lane": 0, "agent_compute-ego": 0.033750454584757485, "deviation-heading": 0.6725004849310052, "set_robot_commands": 0.010430506297520228, "deviation-center-line": 0.07318184940311374, "driven_lanedir_consec": 0.11681077371457738, "sim_compute_sim_state": 0.0057492029099237354, "sim_compute_performance-ego": 0.00931042716616676, "sim_compute_robot_state-ego": 0.010128793262300036, "sim_compute_robot_state-npc0": 0.010377986090523856, "sim_compute_robot_state-npc1": 0.009173870086669922, "sim_compute_robot_state-npc2": 0.008593547911871047, "sim_compute_robot_state-npc3": 0.008639176686604818}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 2.38651403579366, "sim_physics": 0.2197084284540433, "survival_time": 10.050000000000008, "driven_lanedir": 1.294346851754225, "sim_render-ego": 0.01224797637901496, "in-drivable-lane": 4.35000000000002, "agent_compute-ego": 0.02761428273139308, "deviation-heading": 1.1708894069740508, "set_robot_commands": 0.010330787345544616, "deviation-center-line": 0.24138226282774125, "driven_lanedir_consec": 1.294346851754225, "sim_compute_sim_state": 0.005125271147163353, "sim_compute_performance-ego": 0.0074879933352494125, "sim_compute_robot_state-ego": 0.009412867512869004, "sim_compute_robot_state-npc0": 0.009067855664153598, "sim_compute_robot_state-npc1": 0.008584024894296827, "sim_compute_robot_state-npc2": 0.008624654504197153, "sim_compute_robot_state-npc3": 0.008296754229721145}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.24251378316913091, "sim_physics": 0.1724625587463379, "survival_time": 1.0000000000000002, "driven_lanedir": 0.20098342815594217, "sim_render-ego": 0.010830187797546386, "in-drivable-lane": 0.04999999999999993, "agent_compute-ego": 0.022352445125579833, "deviation-heading": 0.24626917008312296, "set_robot_commands": 0.00790245532989502, "deviation-center-line": 0.08072983923012816, "driven_lanedir_consec": 0.20098342815594217, "sim_compute_sim_state": 0.003978919982910156, "sim_compute_performance-ego": 0.00576092004776001, "sim_compute_robot_state-ego": 0.007613980770111084, "sim_compute_robot_state-npc0": 0.007787966728210449, "sim_compute_robot_state-npc1": 0.0068991661071777345, "sim_compute_robot_state-npc2": 0.007094871997833252, "sim_compute_robot_state-npc3": 0.0068159341812133786}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.24816738013108297, "sim_physics": 0.13006354570388795, "survival_time": 1.0000000000000002, "driven_lanedir": 0.20230021160664524, "sim_render-ego": 0.008011603355407714, "in-drivable-lane": 0.04999999999999993, "agent_compute-ego": 0.017023515701293946, "deviation-heading": 0.2669838064053396, "set_robot_commands": 0.0059342503547668455, "deviation-center-line": 0.07945828950777138, "driven_lanedir_consec": 0.20230021160664524, "sim_compute_sim_state": 0.003076469898223877, "sim_compute_performance-ego": 0.004376685619354248, "sim_compute_robot_state-ego": 0.005517065525054932, "sim_compute_robot_state-npc0": 0.005404114723205566, "sim_compute_robot_state-npc1": 0.005101537704467774, "sim_compute_robot_state-npc2": 0.005159199237823486, "sim_compute_robot_state-npc3": 0.00521383285522461}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.7156520776137921, "sim_physics": 0.1145061189478094, "survival_time": 2.7499999999999982, "driven_lanedir": 0.28213603037274404, "sim_render-ego": 0.008941377292979847, "in-drivable-lane": 0.8499999999999972, "agent_compute-ego": 0.019722474705089223, "deviation-heading": 1.135354829906112, "set_robot_commands": 0.007117059014060281, "deviation-center-line": 0.1032073562545081, "driven_lanedir_consec": 0.28213603037274404, "sim_compute_sim_state": 0.003480711850253018, "sim_compute_performance-ego": 0.004936387322165749, "sim_compute_robot_state-ego": 0.006159279563210227, "sim_compute_robot_state-npc0": 0.00628856745633212, "sim_compute_robot_state-npc1": 0.00599291974847967, "sim_compute_robot_state-npc2": 0.006068819219415838, "sim_compute_robot_state-npc3": 0.006089440259066495}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.2926408448044228, "sim_physics": 0.09184840417677356, "survival_time": 1.5500000000000007, "driven_lanedir": 0.251221468587147, "sim_render-ego": 0.007290932439988659, "in-drivable-lane": 0, "agent_compute-ego": 0.01526239610487415, "deviation-heading": 0.6301657520880055, "set_robot_commands": 0.004912914768342049, "deviation-center-line": 0.09461792825355968, "driven_lanedir_consec": 0.251221468587147, "sim_compute_sim_state": 0.002802810361308436, "sim_compute_performance-ego": 0.003927446180774319, "sim_compute_robot_state-ego": 0.004989516350530809, "sim_compute_robot_state-npc0": 0.004806687754969443, "sim_compute_robot_state-npc1": 0.004578136628673923, "sim_compute_robot_state-npc2": 0.0045621395111083984, "sim_compute_robot_state-npc3": 0.004531491187310988}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 0.16014162186451372, "sim_physics": 0.1034382371341481, "survival_time": 0.8500000000000002, "driven_lanedir": 0.12154608566318116, "sim_render-ego": 0.007141491946052103, "in-drivable-lane": 0, "agent_compute-ego": 0.015246335197897518, "deviation-heading": 0.3600523027706, "set_robot_commands": 0.0048115113202263325, "deviation-center-line": 0.06199485152278269, "driven_lanedir_consec": 0.12154608566318116, "sim_compute_sim_state": 0.0027140589321360867, "sim_compute_performance-ego": 0.003923233817605411, "sim_compute_robot_state-ego": 0.0049014371984145225, "sim_compute_robot_state-npc0": 0.004724418415742762, "sim_compute_robot_state-npc1": 0.004564299302942613, "sim_compute_robot_state-npc2": 0.0045080044690300435, "sim_compute_robot_state-npc3": 0.0045086075277889475}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.2790283810664501, "sim_physics": 0.11341198285420737, "survival_time": 1.6500000000000008, "driven_lanedir": 0.2287096756476532, "sim_render-ego": 0.007159926674582742, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.015267935666170988, "deviation-heading": 0.5019479240854331, "set_robot_commands": 0.004813916755445076, "deviation-center-line": 0.09636168576521029, "driven_lanedir_consec": 0.2287096756476532, "sim_compute_sim_state": 0.0027656699671889796, "sim_compute_performance-ego": 0.003905267426461885, "sim_compute_robot_state-ego": 0.005070101131092419, "sim_compute_robot_state-npc0": 0.004728916919592655, "sim_compute_robot_state-npc1": 0.004551302302967419, "sim_compute_robot_state-npc2": 0.004526615142822266, "sim_compute_robot_state-npc3": 0.004502708261663263}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.2845395962701251, "sim_physics": 0.11303543161462856, "survival_time": 1.3500000000000003, "driven_lanedir": 0.2546467109262034, "sim_render-ego": 0.007283175433123553, "in-drivable-lane": 0, "agent_compute-ego": 0.015610889152244286, "deviation-heading": 0.4602120667478826, "set_robot_commands": 0.004962099923027886, "deviation-center-line": 0.0677613792124371, "driven_lanedir_consec": 0.2546467109262034, "sim_compute_sim_state": 0.0027738765433982567, "sim_compute_performance-ego": 0.003925279334739402, "sim_compute_robot_state-ego": 0.004998816384209527, "sim_compute_robot_state-npc0": 0.004731549157036675, "sim_compute_robot_state-npc1": 0.004598794160065828, "sim_compute_robot_state-npc2": 0.004568294242576316, "sim_compute_robot_state-npc3": 0.0045850806766086155}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.42626873228274453, "sim_physics": 0.11710225280962493, "survival_time": 1.900000000000001, "driven_lanedir": 0.3966125744666775, "sim_render-ego": 0.007173801723279451, "in-drivable-lane": 0, "agent_compute-ego": 0.01520831961380808, "deviation-heading": 0.5291329283059608, "set_robot_commands": 0.004885899393182052, "deviation-center-line": 0.09119771829043843, "driven_lanedir_consec": 0.3966125744666775, "sim_compute_sim_state": 0.002777319205434698, "sim_compute_performance-ego": 0.003929571101540013, "sim_compute_robot_state-ego": 0.00499666364569413, "sim_compute_robot_state-npc0": 0.00475889758059853, "sim_compute_robot_state-npc1": 0.004564341745878521, "sim_compute_robot_state-npc2": 0.0045603890168039425, "sim_compute_robot_state-npc3": 0.004529231473019249}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 0.7801754986830883, "sim_physics": 0.1003307474070582, "survival_time": 4.3499999999999925, "driven_lanedir": 0.5558047883070034, "sim_render-ego": 0.008754406852283698, "in-drivable-lane": 0.34999999999999876, "agent_compute-ego": 0.015614254721279804, "deviation-heading": 1.7207292079338334, "set_robot_commands": 0.004906226848733836, "deviation-center-line": 0.2599992089214533, "driven_lanedir_consec": 0.5558047883070034, "sim_compute_sim_state": 0.0027896069932257994, "sim_compute_performance-ego": 0.003934114828877066, "sim_compute_robot_state-ego": 0.0050038030777854485, "sim_compute_robot_state-npc0": 0.004837954181364213, "sim_compute_robot_state-npc1": 0.004553594808468874, "sim_compute_robot_state-npc2": 0.004534066408530049, "sim_compute_robot_state-npc3": 0.0045429153003911866}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.19978318139470813, "sim_physics": 0.09020307931033048, "survival_time": 1.1000000000000003, "driven_lanedir": 0.15937101577113655, "sim_render-ego": 0.007263010198419744, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.014695481820539995, "deviation-heading": 0.37694198172736815, "set_robot_commands": 0.00489607724276456, "deviation-center-line": 0.09456351301541426, "driven_lanedir_consec": 0.15937101577113655, "sim_compute_sim_state": 0.0027626969597556376, "sim_compute_performance-ego": 0.003925355997952548, "sim_compute_robot_state-ego": 0.0050201416015625, "sim_compute_robot_state-npc0": 0.004775534976612438, "sim_compute_robot_state-npc1": 0.0045500993728637695, "sim_compute_robot_state-npc2": 0.004540627652948553, "sim_compute_robot_state-npc3": 0.004546859047629617}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.33608812291491225, "sim_physics": 0.10582965420138452, "survival_time": 1.5500000000000007, "driven_lanedir": 0.25286269153460983, "sim_render-ego": 0.007172799879504788, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.015553259080456149, "deviation-heading": 0.7045181252637903, "set_robot_commands": 0.004814524804392169, "deviation-center-line": 0.1051665529543208, "driven_lanedir_consec": 0.25286269153460983, "sim_compute_sim_state": 0.0027383142902005103, "sim_compute_performance-ego": 0.0038824619785431894, "sim_compute_robot_state-ego": 0.004913314696281187, "sim_compute_robot_state-npc0": 0.004740445844588741, "sim_compute_robot_state-npc1": 0.004513463666362147, "sim_compute_robot_state-npc2": 0.004491429175100019, "sim_compute_robot_state-npc3": 0.004482438487391318}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.22380169927097507, "sim_physics": 0.11573832685297186, "survival_time": 1.1000000000000003, "driven_lanedir": 0.21215938536078172, "sim_render-ego": 0.007291945544156161, "in-drivable-lane": 0, "agent_compute-ego": 0.015322067520835182, "deviation-heading": 0.255566377191523, "set_robot_commands": 0.004977486350319602, "deviation-center-line": 0.08735023594448697, "driven_lanedir_consec": 0.21215938536078172, "sim_compute_sim_state": 0.00275642221624201, "sim_compute_performance-ego": 0.003906293348832564, "sim_compute_robot_state-ego": 0.005017367276278409, "sim_compute_robot_state-npc0": 0.004734136841513894, "sim_compute_robot_state-npc1": 0.004562778906388717, "sim_compute_robot_state-npc2": 0.004552765326066451, "sim_compute_robot_state-npc3": 0.004538763653148304}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 0.55351634674644, "sim_physics": 0.11269031167030334, "survival_time": 2.000000000000001, "driven_lanedir": 0.12333717077560014, "sim_render-ego": 0.009914743900299072, "in-drivable-lane": 0.9500000000000006, "agent_compute-ego": 0.015460097789764404, "deviation-heading": 0.7339286392083293, "set_robot_commands": 0.004786962270736694, "deviation-center-line": 0.05837064533251564, "driven_lanedir_consec": 0.12333717077560014, "sim_compute_sim_state": 0.002778351306915283, "sim_compute_performance-ego": 0.003860288858413696, "sim_compute_robot_state-ego": 0.004943162202835083, "sim_compute_robot_state-npc0": 0.004704487323760986, "sim_compute_robot_state-npc1": 0.0044865250587463375, "sim_compute_robot_state-npc2": 0.004470717906951904, "sim_compute_robot_state-npc3": 0.004471051692962647}}
set_robot_commands_max	0.010430506297520228
set_robot_commands_mean	0.006032178534597149
set_robot_commands_median	0.004912914768342049
set_robot_commands_min	0.004786962270736694
sim_compute_performance-ego_max	0.00931042716616676
sim_compute_performance-ego_mean	0.004732781757629085
sim_compute_performance-ego_median	0.003927446180774319
sim_compute_performance-ego_min	0.003860288858413696
sim_compute_robot_state-ego_max	0.010128793262300036
sim_compute_robot_state-ego_mean	0.005912420679881955
sim_compute_robot_state-ego_median	0.005017367276278409
sim_compute_robot_state-ego_min	0.0049014371984145225
sim_compute_robot_state-npc0_max	0.010377986090523856
sim_compute_robot_state-npc0_mean	0.005764634643880395
sim_compute_robot_state-npc0_median	0.004775534976612438
sim_compute_robot_state-npc0_min	0.004704487323760986
sim_compute_robot_state-npc1_max	0.009173870086669922
sim_compute_robot_state-npc1_mean	0.005418323632963337
sim_compute_robot_state-npc1_median	0.004564341745878521
sim_compute_robot_state-npc1_min	0.0044865250587463375
sim_compute_robot_state-npc2_max	0.008624654504197153
sim_compute_robot_state-npc2_mean	0.005390409448205249
sim_compute_robot_state-npc2_median	0.0045603890168039425
sim_compute_robot_state-npc2_min	0.004470717906951904
sim_compute_robot_state-npc3_max	0.008639176686604818
sim_compute_robot_state-npc3_mean	0.005352952367982972
sim_compute_robot_state-npc3_median	0.0045429153003911866
sim_compute_robot_state-npc3_min	0.004471051692962647
sim_compute_sim_state_max	0.0057492029099237354
sim_compute_sim_state_mean	0.0032713135042853224
sim_compute_sim_state_median	0.002778351306915283
sim_compute_sim_state_min	0.0027140589321360867
sim_physics_max	0.25046794755118235
sim_physics_mean	0.13005580182897944
sim_physics_median	0.11341198285420737
sim_physics_min	0.09020307931033048
sim_render-ego_max	0.013317028681437174
sim_render-ego_mean	0.008652960539871737
sim_render-ego_median	0.007291945544156161
sim_render-ego_min	0.007141491946052103
simulation-passed	1
survival_time_max	10.050000000000008
survival_time_mean	2.2166666666666677
survival_time_min	0.8500000000000002

No reset possible

25020

4360

Soroush Saryazdi 🇨🇦

bluejay

aido3-LFV-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-11-04 15:17:13+00:00

2019-11-04 15:20:43+00:00

0:03:30

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

25012

4350

Manfred Diaz

challenge-aido_LF-template-tensorflow

aido3-LF-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-11-04 14:18:38+00:00

2019-11-04 14:25:22+00:00

0:06:44

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.22341993779676628
survival_time_median	1.6000000000000008
deviation-center-line_median	0.08567511776671757
in-drivable-lane_median	0

other stats

agent_compute-ego_max	0.03139075365933505
agent_compute-ego_mean	0.025535231716255692
agent_compute-ego_median	0.025169163215451124
agent_compute-ego_min	0.021247833967208862
deviation-center-line_max	0.13499902918816015
deviation-center-line_mean	0.09230697115372026
deviation-center-line_min	0.07007657509843666
deviation-heading_max	0.8149425722891988
deviation-heading_mean	0.6656805059249713
deviation-heading_median	0.7045896187715504
deviation-heading_min	0.37317069746191234
driven_any_max	0.657372899748944
driven_any_mean	0.4277944366876418
driven_any_median	0.41823997506333327
driven_any_min	0.22937850716605743
driven_lanedir_consec_max	0.4837737701696917
driven_lanedir_consec_mean	0.28573362220731513
driven_lanedir_consec_min	0.1349573117344951
driven_lanedir_max	0.4837737701696917
driven_lanedir_mean	0.28573362220731513
driven_lanedir_median	0.22341993779676628
driven_lanedir_min	0.1349573117344951
in-drivable-lane_max	1.149999999999998
in-drivable-lane_mean	0.2299999999999996
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.22937850716605743, "sim_physics": 0.12966309894214978, "survival_time": 1.1000000000000003, "driven_lanedir": 0.1349573117344951, "sim_render-ego": 0.012446013363924894, "in-drivable-lane": 0, "agent_compute-ego": 0.03139075365933505, "deviation-heading": 0.6791108394163036, "set_robot_commands": 0.010821992700750176, "deviation-center-line": 0.07007657509843666, "driven_lanedir_consec": 0.1349573117344951, "sim_compute_sim_state": 0.005638556046919389, "sim_compute_performance-ego": 0.007523352449590509, "sim_compute_robot_state-ego": 0.009493600238453258}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.5690428160671005, "sim_physics": 0.09573161892774627, "survival_time": 2.0500000000000007, "driven_lanedir": 0.4837737701696917, "sim_render-ego": 0.01226881073742378, "in-drivable-lane": 0, "agent_compute-ego": 0.025169163215451124, "deviation-heading": 0.8149425722891988, "set_robot_commands": 0.009190960628230398, "deviation-center-line": 0.13499902918816015, "driven_lanedir_consec": 0.4837737701696917, "sim_compute_sim_state": 0.004908433774622475, "sim_compute_performance-ego": 0.006627495695904988, "sim_compute_robot_state-ego": 0.008646348627602182}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.657372899748944, "sim_physics": 0.10235176896149258, "survival_time": 2.6499999999999986, "driven_lanedir": 0.22341993779676628, "sim_render-ego": 0.012213329099259288, "in-drivable-lane": 1.149999999999998, "agent_compute-ego": 0.026321685539101653, "deviation-heading": 0.7565888016858918, "set_robot_commands": 0.009112848425811192, "deviation-center-line": 0.09760422676436505, "driven_lanedir_consec": 0.22341993779676628, "sim_compute_sim_state": 0.00510894127611844, "sim_compute_performance-ego": 0.007182337203115787, "sim_compute_robot_state-ego": 0.009537782309190284}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.2649379853927738, "sim_physics": 0.0869823385168005, "survival_time": 1.3500000000000003, "driven_lanedir": 0.1919500727611716, "sim_render-ego": 0.010473171869913736, "in-drivable-lane": 0, "agent_compute-ego": 0.02354672220018175, "deviation-heading": 0.7045896187715504, "set_robot_commands": 0.007908697481508608, "deviation-center-line": 0.07317990695092194, "driven_lanedir_consec": 0.1919500727611716, "sim_compute_sim_state": 0.004058263919971607, "sim_compute_performance-ego": 0.006152991895322446, "sim_compute_robot_state-ego": 0.007602656329119647}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.41823997506333327, "sim_physics": 0.0926901176571846, "survival_time": 1.6000000000000008, "driven_lanedir": 0.3945670185744507, "sim_render-ego": 0.009066753089427948, "in-drivable-lane": 0, "agent_compute-ego": 0.021247833967208862, "deviation-heading": 0.37317069746191234, "set_robot_commands": 0.007197573781013489, "deviation-center-line": 0.08567511776671757, "driven_lanedir_consec": 0.3945670185744507, "sim_compute_sim_state": 0.0036972910165786743, "sim_compute_performance-ego": 0.005087628960609436, "sim_compute_robot_state-ego": 0.006460957229137421}}
set_robot_commands_max	0.010821992700750176
set_robot_commands_mean	0.008846414603462773
set_robot_commands_median	0.009112848425811192
set_robot_commands_min	0.007197573781013489
sim_compute_performance-ego_max	0.007523352449590509
sim_compute_performance-ego_mean	0.006514761240908632
sim_compute_performance-ego_median	0.006627495695904988
sim_compute_performance-ego_min	0.005087628960609436
sim_compute_robot_state-ego_max	0.009537782309190284
sim_compute_robot_state-ego_mean	0.008348268946700557
sim_compute_robot_state-ego_median	0.008646348627602182
sim_compute_robot_state-ego_min	0.006460957229137421
sim_compute_sim_state_max	0.005638556046919389
sim_compute_sim_state_mean	0.004682297206842117
sim_compute_sim_state_median	0.004908433774622475
sim_compute_sim_state_min	0.0036972910165786743
sim_physics_max	0.12966309894214978
sim_physics_mean	0.10148378860107474
sim_physics_median	0.09573161892774627
sim_physics_min	0.0869823385168005
sim_render-ego_max	0.012446013363924894
sim_render-ego_mean	0.011293615631989927
sim_render-ego_median	0.012213329099259288
sim_render-ego_min	0.009066753089427948
simulation-passed	1
survival_time_max	2.6499999999999986
survival_time_mean	1.7500000000000004
survival_time_min	1.1000000000000003

No reset possible

25004

4125

Andrea Censi 🇨🇭

rotation

aido3-LF-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-11-04 09:34:45+00:00

2019-11-04 10:01:01+00:00

0:26:16

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24995

4138

Andrea Censi 🇨🇭

rotation

aido3-LFV-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-11-04 09:12:35+00:00

2019-11-04 09:34:37+00:00

0:22:02

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.00031446221011421915
survival_time_median	14.950000000000076
deviation-center-line_median	0.4891048530415207
in-drivable-lane_median	7.600000000000043

other stats

agent_compute-ego_max	0.023967736562093098
agent_compute-ego_mean	0.022820947806040447
agent_compute-ego_median	0.022748308181762697
agent_compute-ego_min	0.02179041862487793
deviation-center-line_max	0.6113242374220981
deviation-center-line_mean	0.422546032853888
deviation-center-line_min	0.020737127615471695
deviation-heading_max	5.830135897709593
deviation-heading_mean	5.6845705507076865
deviation-heading_median	5.657649106857694
deviation-heading_min	5.536662996854859
driven_any_max	0.026478672834058543
driven_any_mean	0.02486423959225546
driven_any_median	0.025930195541210208
driven_any_min	0.022165452661382513
driven_lanedir_consec_max	0.0021599934447711355
driven_lanedir_consec_mean	-0.0004859282560538025
driven_lanedir_consec_min	-0.003645956811333395
driven_lanedir_max	0.0021599934447711355
driven_lanedir_mean	-0.0004859282560538025
driven_lanedir_median	0.00031446221011421915
driven_lanedir_min	-0.003645956811333395
in-drivable-lane_max	7.650000000000043
in-drivable-lane_mean	7.600000000000042
in-drivable-lane_min	7.550000000000044
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.026478672834058543, "sim_physics": 0.1804728142420451, "survival_time": 14.950000000000076, "driven_lanedir": 0.0021599934447711355, "sim_render-ego": 0.011607162952423096, "in-drivable-lane": 7.600000000000043, "agent_compute-ego": 0.02179041862487793, "deviation-heading": 5.657649106857694, "set_robot_commands": 0.00936142921447754, "deviation-center-line": 0.5610133203215761, "driven_lanedir_consec": 0.0021599934447711355, "sim_compute_sim_state": 0.004734939734141032, "sim_compute_performance-ego": 0.006868552366892496, "sim_compute_robot_state-ego": 0.008809340794881186, "sim_compute_robot_state-npc0": 0.00831129789352417, "sim_compute_robot_state-npc1": 0.00792142391204834, "sim_compute_robot_state-npc2": 0.008042296568552654, "sim_compute_robot_state-npc3": 0.007912616729736328}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.025930195541210208, "sim_physics": 0.18195761601130167, "survival_time": 14.950000000000076, "driven_lanedir": -0.003645956811333395, "sim_render-ego": 0.012245713869730631, "in-drivable-lane": 7.650000000000043, "agent_compute-ego": 0.023967736562093098, "deviation-heading": 5.795316370396739, "set_robot_commands": 0.01031174659729004, "deviation-center-line": 0.6113242374220981, "driven_lanedir_consec": -0.003645956811333395, "sim_compute_sim_state": 0.005174225966135661, "sim_compute_performance-ego": 0.007391034762064616, "sim_compute_robot_state-ego": 0.009189902146657308, "sim_compute_robot_state-npc0": 0.00911286989847819, "sim_compute_robot_state-npc1": 0.008773910204569498, "sim_compute_robot_state-npc2": 0.008677070140838622, "sim_compute_robot_state-npc3": 0.008837711016337077}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.023804404672443435, "sim_physics": 0.18116616249084472, "survival_time": 14.950000000000076, "driven_lanedir": 0.0017527010091673567, "sim_render-ego": 0.012173512776692708, "in-drivable-lane": 7.550000000000044, "agent_compute-ego": 0.022748308181762697, "deviation-heading": 5.603088381719547, "set_robot_commands": 0.009987014134724937, "deviation-center-line": 0.4891048530415207, "driven_lanedir_consec": 0.0017527010091673567, "sim_compute_sim_state": 0.004912153879801432, "sim_compute_performance-ego": 0.0072832036018371585, "sim_compute_robot_state-ego": 0.009153636296590169, "sim_compute_robot_state-npc0": 0.008698039849599203, "sim_compute_robot_state-npc1": 0.008571637471516928, "sim_compute_robot_state-npc2": 0.008459407488505046, "sim_compute_robot_state-npc3": 0.008478194872538248}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.022165452661382513, "sim_physics": 0.1829378112157186, "survival_time": 14.950000000000076, "driven_lanedir": -0.003010841132988329, "sim_render-ego": 0.0122657577196757, "in-drivable-lane": 7.600000000000044, "agent_compute-ego": 0.023110771973927816, "deviation-heading": 5.536662996854859, "set_robot_commands": 0.009739327430725095, "deviation-center-line": 0.4305506258687733, "driven_lanedir_consec": -0.003010841132988329, "sim_compute_sim_state": 0.005003759860992432, "sim_compute_performance-ego": 0.007353773117065429, "sim_compute_robot_state-ego": 0.009188183943430584, "sim_compute_robot_state-npc0": 0.008784504731496175, "sim_compute_robot_state-npc1": 0.008652139504750569, "sim_compute_robot_state-npc2": 0.008581496079762777, "sim_compute_robot_state-npc3": 0.008429788748423258}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.02594247225218261, "sim_physics": 0.17201183954874674, "survival_time": 14.950000000000076, "driven_lanedir": 0.00031446221011421915, "sim_render-ego": 0.011790324052174886, "in-drivable-lane": 7.600000000000042, "agent_compute-ego": 0.02248750368754069, "deviation-heading": 5.830135897709593, "set_robot_commands": 0.009383902549743651, "deviation-center-line": 0.020737127615471695, "driven_lanedir_consec": 0.00031446221011421915, "sim_compute_sim_state": 0.00482311487197876, "sim_compute_performance-ego": 0.007110257148742676, "sim_compute_robot_state-ego": 0.008805302778879802, "sim_compute_robot_state-npc0": 0.008612000942230224, "sim_compute_robot_state-npc1": 0.008286322752634684, "sim_compute_robot_state-npc2": 0.00839143991470337, "sim_compute_robot_state-npc3": 0.008126718997955322}}
set_robot_commands_max	0.01031174659729004
set_robot_commands_mean	0.009756683985392253
set_robot_commands_median	0.009739327430725095
set_robot_commands_min	0.00936142921447754
sim_compute_performance-ego_max	0.007391034762064616
sim_compute_performance-ego_mean	0.007201364199320476
sim_compute_performance-ego_median	0.0072832036018371585
sim_compute_performance-ego_min	0.006868552366892496
sim_compute_robot_state-ego_max	0.009189902146657308
sim_compute_robot_state-ego_mean	0.00902927319208781
sim_compute_robot_state-ego_median	0.009153636296590169
sim_compute_robot_state-ego_min	0.008805302778879802
sim_compute_robot_state-npc0_max	0.00911286989847819
sim_compute_robot_state-npc0_mean	0.008703742663065592
sim_compute_robot_state-npc0_median	0.008698039849599203
sim_compute_robot_state-npc0_min	0.00831129789352417
sim_compute_robot_state-npc1_max	0.008773910204569498
sim_compute_robot_state-npc1_mean	0.008441086769104005
sim_compute_robot_state-npc1_median	0.008571637471516928
sim_compute_robot_state-npc1_min	0.00792142391204834
sim_compute_robot_state-npc2_max	0.008677070140838622
sim_compute_robot_state-npc2_mean	0.008430342038472493
sim_compute_robot_state-npc2_median	0.008459407488505046
sim_compute_robot_state-npc2_min	0.008042296568552654
sim_compute_robot_state-npc3_max	0.008837711016337077
sim_compute_robot_state-npc3_mean	0.008357006072998047
sim_compute_robot_state-npc3_median	0.008429788748423258
sim_compute_robot_state-npc3_min	0.007912616729736328
sim_compute_sim_state_max	0.005174225966135661
sim_compute_sim_state_mean	0.004929638862609864
sim_compute_sim_state_median	0.004912153879801432
sim_compute_sim_state_min	0.004734939734141032
sim_physics_max	0.1829378112157186
sim_physics_mean	0.17970924870173138
sim_physics_median	0.18116616249084472
sim_physics_min	0.17201183954874674
sim_render-ego_max	0.0122657577196757
sim_render-ego_mean	0.012016494274139405
sim_render-ego_median	0.012173512776692708
sim_render-ego_min	0.011607162952423096
simulation-passed	1
survival_time_max	14.950000000000076
survival_time_mean	14.950000000000076
survival_time_min	14.950000000000076

No reset possible

24988

4162

Tanli Chou 🇹🇼

challenge-aido_LF-template-random

aido3-LF-sim-testing

step1-simulation

success

ip-172-31-45-34-10501

2019-11-04 09:00:44+00:00

2019-11-04 09:12:20+00:00

0:11:36

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.5723310224856362
survival_time_median	2.549999999999999
deviation-center-line_median	0.10099507669212504
in-drivable-lane_median	0.4499999999999984

other stats

agent_compute-ego_max	0.024115347394756244
agent_compute-ego_mean	0.0227148307835675
agent_compute-ego_median	0.02305838411504572
agent_compute-ego_min	0.02010225296020508
deviation-center-line_max	0.28386357003337986
deviation-center-line_mean	0.12786165368054708
deviation-center-line_min	0.07112529346797256
deviation-heading_max	0.5289022616823588
deviation-heading_mean	0.3391682685652288
deviation-heading_median	0.3292838510353387
deviation-heading_min	0.19401296908406307
driven_any_max	2.1706243860917107
driven_any_mean	0.975949037827615
driven_any_median	1.0117660515775695
driven_any_min	0.23626560261560617
driven_lanedir_consec_max	1.919400421063861
driven_lanedir_consec_mean	0.6896171461819268
driven_lanedir_consec_min	0.2264965947033466
driven_lanedir_max	1.919400421063861
driven_lanedir_mean	0.6896171461819268
driven_lanedir_median	0.5723310224856362
driven_lanedir_min	0.2264965947033466
in-drivable-lane_max	1.7999999999999936
in-drivable-lane_mean	0.5899999999999982
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.0243289631429704, "sim_physics": 0.1037035827283506, "survival_time": 2.6999999999999984, "driven_lanedir": 0.5723310224856362, "sim_render-ego": 0.01266204869305646, "in-drivable-lane": 0.9999999999999976, "agent_compute-ego": 0.021998802820841473, "deviation-heading": 0.19401296908406307, "set_robot_commands": 0.008761737081739638, "deviation-center-line": 0.15907928712407876, "driven_lanedir_consec": 0.5723310224856362, "sim_compute_sim_state": 0.004556792753714102, "sim_compute_performance-ego": 0.007463777506792987, "sim_compute_robot_state-ego": 0.00886565226095694}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9933867744747804, "sim_physics": 0.1367930150499531, "survival_time": 2.549999999999999, "driven_lanedir": 0.8010406009604816, "sim_render-ego": 0.012944815205592736, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.023315878475413603, "deviation-heading": 0.5289022616823588, "set_robot_commands": 0.010478814442952476, "deviation-center-line": 0.13005513204193506, "driven_lanedir_consec": 0.8010406009604816, "sim_compute_sim_state": 0.0053435026430616196, "sim_compute_performance-ego": 0.007507988050872204, "sim_compute_robot_state-ego": 0.010298775691612094}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.23626560261560617, "sim_physics": 0.114122657214894, "survival_time": 0.8500000000000002, "driven_lanedir": 0.2264965947033466, "sim_render-ego": 0.01298827283522662, "in-drivable-lane": 0, "agent_compute-ego": 0.02330111054813161, "deviation-heading": 0.23048898560984332, "set_robot_commands": 0.00985107702367446, "deviation-center-line": 0.0788839760023766, "driven_lanedir_consec": 0.2264965947033466, "sim_compute_sim_state": 0.0052823880139519185, "sim_compute_performance-ego": 0.007449079962337718, "sim_compute_robot_state-ego": 0.009705501444199504}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4346971317320432, "sim_physics": 0.10416437149047852, "survival_time": 1.2500000000000004, "driven_lanedir": 0.40985475638935265, "sim_render-ego": 0.012360696792602538, "in-drivable-lane": 0, "agent_compute-ego": 0.02010225296020508, "deviation-heading": 0.4138862827398061, "set_robot_commands": 0.009341259002685547, "deviation-center-line": 0.08421324241865966, "driven_lanedir_consec": 0.40985475638935265, "sim_compute_sim_state": 0.005097417831420898, "sim_compute_performance-ego": 0.006764755249023437, "sim_compute_robot_state-ego": 0.00869668960571289}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6182144577009443, "sim_physics": 0.1140526035163976, "survival_time": 3.949999999999994, "driven_lanedir": 0.7890999076769956, "sim_render-ego": 0.012670076346095604, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.023465778254255464, "deviation-heading": 0.23606825392187256, "set_robot_commands": 0.010112400296368175, "deviation-center-line": 0.2072381278624381, "driven_lanedir_consec": 0.7890999076769956, "sim_compute_sim_state": 0.005251253707499444, "sim_compute_performance-ego": 0.00769670100151738, "sim_compute_robot_state-ego": 0.0092546366438081}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.003851495683299, "sim_physics": 0.12360480366920938, "survival_time": 2.4499999999999993, "driven_lanedir": 0.7531703123735949, "sim_render-ego": 0.012757155360007773, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.023420796102407028, "deviation-heading": 0.4596437246381204, "set_robot_commands": 0.009608711515154158, "deviation-center-line": 0.08968283443958694, "driven_lanedir_consec": 0.7531703123735949, "sim_compute_sim_state": 0.00540110529685507, "sim_compute_performance-ego": 0.008038671649232203, "sim_compute_robot_state-ego": 0.010005712509155272}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.0542898001504386, "sim_physics": 0.12180161050387792, "survival_time": 2.799999999999998, "driven_lanedir": 0.5389398655375865, "sim_render-ego": 0.012049402509416851, "in-drivable-lane": 1.1499999999999972, "agent_compute-ego": 0.02328670024871826, "deviation-heading": 0.4109275195546752, "set_robot_commands": 0.009860302720751082, "deviation-center-line": 0.10786249131209884, "driven_lanedir_consec": 0.5389398655375865, "sim_compute_sim_state": 0.005108548062188285, "sim_compute_performance-ego": 0.007513787065233503, "sim_compute_robot_state-ego": 0.008944375174386161}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.3164475681403097, "sim_physics": 0.13061481430417016, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3093025166124813, "sim_render-ego": 0.02317312785557338, "in-drivable-lane": 0, "agent_compute-ego": 0.022842509405953546, "deviation-heading": 0.2242052814634115, "set_robot_commands": 0.01070321173894973, "deviation-center-line": 0.09758615464743424, "driven_lanedir_consec": 0.3093025166124813, "sim_compute_sim_state": 0.005435285114106678, "sim_compute_performance-ego": 0.007429951713198707, "sim_compute_robot_state-ego": 0.009606100264049712}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.2018528489510794, "sim_physics": 0.1062735177702823, "survival_time": 2.9499999999999975, "driven_lanedir": 0.6922532395203, "sim_render-ego": 0.012037289344658286, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.022124496556944765, "deviation-heading": 0.3292838510353387, "set_robot_commands": 0.009080078642247087, "deviation-center-line": 0.07112529346797256, "driven_lanedir_consec": 0.6922532395203, "sim_compute_sim_state": 0.004549976122581353, "sim_compute_performance-ego": 0.006963426783933478, "sim_compute_robot_state-ego": 0.008868463968826552}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 1.043548155325732, "sim_physics": 0.11187624931335448, "survival_time": 2.7499999999999982, "driven_lanedir": 0.5204405548974873, "sim_render-ego": 0.01591447483409535, "in-drivable-lane": 1.1499999999999977, "agent_compute-ego": 0.02305838411504572, "deviation-heading": 0.33296447766287096, "set_robot_commands": 0.009284331581809304, "deviation-center-line": 0.0950267943453044, "driven_lanedir_consec": 0.5204405548974873, "sim_compute_sim_state": 0.005254520069469105, "sim_compute_performance-ego": 0.007181271639737216, "sim_compute_robot_state-ego": 0.00922308401627974}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5954548672193285, "sim_physics": 0.13813339889823617, "survival_time": 3.8499999999999943, "driven_lanedir": 1.1042463557848785, "sim_render-ego": 0.014892339706420898, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.023760049374072584, "deviation-heading": 0.35668107106767694, "set_robot_commands": 0.010332280939275568, "deviation-center-line": 0.2251391443608223, "driven_lanedir_consec": 1.1042463557848785, "sim_compute_sim_state": 0.00533557247805905, "sim_compute_performance-ego": 0.00782439306184843, "sim_compute_robot_state-ego": 0.009494809361247272}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.49030923107489305, "sim_physics": 0.11205720049994332, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4826393189617785, "sim_render-ego": 0.012192470686776298, "in-drivable-lane": 0, "agent_compute-ego": 0.021276371819632395, "deviation-heading": 0.23322336656472228, "set_robot_commands": 0.00874424832207816, "deviation-center-line": 0.10099507669212504, "driven_lanedir_consec": 0.4826393189617785, "sim_compute_sim_state": 0.004642580236707415, "sim_compute_performance-ego": 0.007057045187268939, "sim_compute_robot_state-ego": 0.009130818503243585}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.0117660515775695, "sim_physics": 0.1129777384739296, "survival_time": 2.549999999999999, "driven_lanedir": 0.791717039833255, "sim_render-ego": 0.013251173729990042, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.024115347394756244, "deviation-heading": 0.320449356593834, "set_robot_commands": 0.010674934761196958, "deviation-center-line": 0.1017166571540705, "driven_lanedir_consec": 0.791717039833255, "sim_compute_sim_state": 0.005593800077251359, "sim_compute_performance-ego": 0.007714729683071959, "sim_compute_robot_state-ego": 0.009699017393822763}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.44419823353351984, "sim_physics": 0.14800058470831978, "survival_time": 1.3500000000000003, "driven_lanedir": 0.4333246859278659, "sim_render-ego": 0.012230917259498878, "in-drivable-lane": 0, "agent_compute-ego": 0.02247778574625651, "deviation-heading": 0.2988926743481924, "set_robot_commands": 0.010015275743272567, "deviation-center-line": 0.08545702330592322, "driven_lanedir_consec": 0.4333246859278659, "sim_compute_sim_state": 0.00515266701027199, "sim_compute_performance-ego": 0.007782459259033203, "sim_compute_robot_state-ego": 0.009152050371523257}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1706243860917107, "sim_physics": 0.10243413495082482, "survival_time": 5.09999999999999, "driven_lanedir": 1.919400421063861, "sim_render-ego": 0.01154776647979138, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.02217619793087828, "deviation-heading": 0.5178939525116448, "set_robot_commands": 0.008806137477650362, "deviation-center-line": 0.28386357003337986, "driven_lanedir_consec": 1.919400421063861, "sim_compute_sim_state": 0.0047922812256158565, "sim_compute_performance-ego": 0.006624149341209262, "sim_compute_robot_state-ego": 0.008662170054865819}}
set_robot_commands_max	0.01070321173894973
set_robot_commands_mean	0.009710320085987018
set_robot_commands_median	0.00985107702367446
set_robot_commands_min	0.00874424832207816
sim_compute_performance-ego_max	0.008038671649232203
sim_compute_performance-ego_mean	0.007400812476954041
sim_compute_performance-ego_median	0.007463777506792987
sim_compute_performance-ego_min	0.006624149341209262
sim_compute_robot_state-ego_max	0.010298775691612094
sim_compute_robot_state-ego_mean	0.009307190484245978
sim_compute_robot_state-ego_median	0.00922308401627974
sim_compute_robot_state-ego_min	0.008662170054865819
sim_compute_sim_state_max	0.005593800077251359
sim_compute_sim_state_mean	0.005119846042850276
sim_compute_sim_state_median	0.005251253707499444
sim_compute_sim_state_min	0.004549976122581353
sim_physics_max	0.14800058470831978
sim_physics_mean	0.11870735220614811
sim_physics_median	0.1140526035163976
sim_physics_min	0.10243413495082482
sim_render-ego_max	0.02317312785557338
sim_render-ego_mean	0.013578135175920209
sim_render-ego_median	0.012670076346095604
sim_render-ego_min	0.01154776647979138
simulation-passed	1
survival_time_max	5.09999999999999
survival_time_mean	2.503333333333331
survival_time_min	0.8500000000000002

No reset possible

24957

4189

Liam Paull 🇨🇦

challenge-aido_LF-template-ros - Template solution using ROS

aido3-LFV-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-11-04 08:50:34+00:00

2019-11-04 09:00:23+00:00

0:09:49

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24941

4209

Bhairav Mehta

challenge-aido_LF-template-pytorch

aido3-LFV-sim-testing

step1-simulation

failed

ip-172-31-45-34-10501

2019-11-04 08:45:50+00:00

2019-11-04 08:50:19+00:00

0:04:29

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 298, in run_episode
    agent.write_topic_and_expect_zero('observations', recv.data.observations)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 105, in write_topic_and_expect_zero
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 296, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "agent".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24932

4217

Manfred Diaz

Baseline-IL-logs-tensorflow

aido3-LFV-sim-testing

step1-simulation

failed

ip-172-31-45-34-10501

2019-11-04 08:44:09+00:00

2019-11-04 08:45:17+00:00

0:01:08

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[{{node prefix/ConvNet/fc_layer_2/bias/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 58, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/fc_layer_2/bias/read', defined at:
               ||   File "imitation_agent.py", line 134, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 130, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 69, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 41, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/fc_layer_2/bias
               || 	 [[node prefix/ConvNet/fc_layer_2/bias/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24925

4226

Manfred Diaz

Baseline-IL-logs-tensorflow

aido3-LFV-sim-validation

step1-simulation

failed

ip-172-31-45-34-10501

2019-11-04 08:41:17+00:00

2019-11-04 08:43:51+00:00

0:02:34

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[{{node prefix/ConvNet/conv_layer_1/kernel/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 60, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/conv_layer_1/kernel/read', defined at:
               ||   File "imitation_agent.py", line 136, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 132, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 43, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24916

4233

Andrea Censi 🇨🇭

challenge-aido_LF-template-pytorch

aido3-LFV-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-11-04 08:39:00+00:00

2019-11-04 08:41:04+00:00

0:02:04

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24890

4258

Anastasiya Nikolskaya 🇷🇺

beta

aido3-LF-sim-testing

step1-simulation

success

ip-172-31-45-34-10501

2019-11-04 08:00:52+00:00

2019-11-04 08:38:51+00:00

0:37:59

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.9414348951223483
survival_time_median	13.850000000000062
deviation-center-line_median	0.5210157195478661
in-drivable-lane_median	1.0499999999999965

other stats

agent_compute-ego_max	0.10572948975440784
agent_compute-ego_mean	0.08294299648219879
agent_compute-ego_median	0.08175672910600433
agent_compute-ego_min	0.06225946545600891
deviation-center-line_max	1.1988037396268565
deviation-center-line_mean	0.5459128938268757
deviation-center-line_min	0.2946557050966541
deviation-heading_max	4.906433370232041
deviation-heading_mean	1.993311413541535
deviation-heading_median	1.4687706151417026
deviation-heading_min	0.6181031539696691
driven_any_max	1.9674365211673128
driven_any_mean	1.3648835850709202
driven_any_median	1.6078521541188833
driven_any_min	0.4435118715788304
driven_lanedir_consec_max	1.9457286419646989
driven_lanedir_consec_mean	1.0233326782988024
driven_lanedir_consec_min	0.437215022336316
driven_lanedir_max	1.9457286419646989
driven_lanedir_mean	1.1478670066126275
driven_lanedir_median	1.077835565390577
driven_lanedir_min	0.437215022336316
in-drivable-lane_max	4.0500000000000576
in-drivable-lane_mean	1.3233333333333377
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.1188310972879152, "sim_physics": 0.10357274421273846, "survival_time": 7.299999999999982, "driven_lanedir": 0.6145378392256813, "sim_render-ego": 0.011694610935367953, "in-drivable-lane": 2.499999999999991, "agent_compute-ego": 0.08409466972089794, "deviation-heading": 1.4687706151417026, "set_robot_commands": 0.009508781237144992, "deviation-center-line": 0.3082133081032693, "driven_lanedir_consec": 0.6088364969973994, "sim_compute_sim_state": 0.0051442858290998905, "sim_compute_performance-ego": 0.007419205691716443, "sim_compute_robot_state-ego": 0.009074975366461766}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.9988512045570016, "sim_physics": 0.07755326553129814, "survival_time": 7.099999999999983, "driven_lanedir": 0.7231836933067752, "sim_render-ego": 0.009262669254356709, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.06567489597159372, "deviation-heading": 1.1608405566115472, "set_robot_commands": 0.00719875349125392, "deviation-center-line": 0.2946557050966541, "driven_lanedir_consec": 0.7231836933067752, "sim_compute_sim_state": 0.003980018723178917, "sim_compute_performance-ego": 0.00544831954257589, "sim_compute_robot_state-ego": 0.006838560104370117}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.4435118715788304, "sim_physics": 0.06125668883323669, "survival_time": 3.999999999999994, "driven_lanedir": 0.437215022336316, "sim_render-ego": 0.007776862382888794, "in-drivable-lane": 0, "agent_compute-ego": 0.06225946545600891, "deviation-heading": 0.6181031539696691, "set_robot_commands": 0.005627378821372986, "deviation-center-line": 0.45972643159828, "driven_lanedir_consec": 0.437215022336316, "sim_compute_sim_state": 0.003318995237350464, "sim_compute_performance-ego": 0.004456830024719238, "sim_compute_robot_state-ego": 0.00551328957080841}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6362771260367033, "sim_physics": 0.11958129839463667, "survival_time": 14.300000000000068, "driven_lanedir": 1.0237834052295067, "sim_render-ego": 0.013209180398420856, "in-drivable-lane": 4.0500000000000576, "agent_compute-ego": 0.0928397987272356, "deviation-heading": 2.385491513564685, "set_robot_commands": 0.010183301838961515, "deviation-center-line": 0.6871200200649433, "driven_lanedir_consec": 1.0237834052295067, "sim_compute_sim_state": 0.0055036819898165185, "sim_compute_performance-ego": 0.007807238952263252, "sim_compute_robot_state-ego": 0.009704478137143008}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.7042948904352808, "sim_physics": 0.0944025921046949, "survival_time": 13.850000000000062, "driven_lanedir": 1.353485740513531, "sim_render-ego": 0.011396927954057494, "in-drivable-lane": 2.10000000000003, "agent_compute-ego": 0.0753157026931267, "deviation-heading": 2.8989786709061276, "set_robot_commands": 0.008732573650373878, "deviation-center-line": 0.5355670705610617, "driven_lanedir_consec": 1.2049404046838723, "sim_compute_sim_state": 0.00458942079371924, "sim_compute_performance-ego": 0.00675594333276852, "sim_compute_robot_state-ego": 0.008312407813777992}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 1.930865866528254, "sim_physics": 0.08830453793207804, "survival_time": 14.950000000000076, "driven_lanedir": 1.9026378385762752, "sim_render-ego": 0.010426994959513346, "in-drivable-lane": 0, "agent_compute-ego": 0.07168255726496378, "deviation-heading": 2.3633653215362886, "set_robot_commands": 0.008281928698221842, "deviation-center-line": 0.5210157195478661, "driven_lanedir_consec": 1.9026378385762752, "sim_compute_sim_state": 0.004404305617014567, "sim_compute_performance-ego": 0.006124861240386963, "sim_compute_robot_state-ego": 0.007659485340118408}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.9066523197758127, "sim_physics": 0.09093789418538412, "survival_time": 14.950000000000076, "driven_lanedir": 1.7026340403695088, "sim_render-ego": 0.010938161214192709, "in-drivable-lane": 1.2499999999999956, "agent_compute-ego": 0.07467645645141602, "deviation-heading": 2.987442253962837, "set_robot_commands": 0.00866166353225708, "deviation-center-line": 0.36586590193930374, "driven_lanedir_consec": 0.7623303727238553, "sim_compute_sim_state": 0.004424403508504232, "sim_compute_performance-ego": 0.006320865154266358, "sim_compute_robot_state-ego": 0.007974689801534016}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 1.6136365823680157, "sim_physics": 0.09054889917373656, "survival_time": 14.950000000000076, "driven_lanedir": 1.2197361429600546, "sim_render-ego": 0.01088167667388916, "in-drivable-lane": 4.0000000000000036, "agent_compute-ego": 0.07212323188781739, "deviation-heading": 3.9853492067232694, "set_robot_commands": 0.008625337282816569, "deviation-center-line": 1.1988037396268565, "driven_lanedir_consec": 1.2025025672673653, "sim_compute_sim_state": 0.004455859661102295, "sim_compute_performance-ego": 0.006266287167867025, "sim_compute_robot_state-ego": 0.007896016438802084}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 1.9349235292275004, "sim_physics": 0.129453866481781, "survival_time": 14.950000000000076, "driven_lanedir": 1.930658293271314, "sim_render-ego": 0.013639082113901774, "in-drivable-lane": 0, "agent_compute-ego": 0.09871077060699462, "deviation-heading": 0.8563836142659718, "set_robot_commands": 0.01111685276031494, "deviation-center-line": 0.590140483184857, "driven_lanedir_consec": 1.930658293271314, "sim_compute_sim_state": 0.005789640744527181, "sim_compute_performance-ego": 0.008312352498372396, "sim_compute_robot_state-ego": 0.0099157985051473}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.9039006376260048, "sim_physics": 0.124899671627925, "survival_time": 7.79999999999998, "driven_lanedir": 0.4879161976395219, "sim_render-ego": 0.013249111481201954, "in-drivable-lane": 2.7499999999999902, "agent_compute-ego": 0.10572948975440784, "deviation-heading": 1.2366238059779044, "set_robot_commands": 0.009962972922202868, "deviation-center-line": 0.2991529013719021, "driven_lanedir_consec": 0.4879161976395219, "sim_compute_sim_state": 0.005344953292455428, "sim_compute_performance-ego": 0.007817740623767559, "sim_compute_robot_state-ego": 0.010193295967884554}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.0949748410930047, "sim_physics": 0.10505314522388716, "survival_time": 9.55, "driven_lanedir": 1.077835565390577, "sim_render-ego": 0.012516283864126156, "in-drivable-lane": 0, "agent_compute-ego": 0.08175672910600433, "deviation-heading": 1.2457340251870632, "set_robot_commands": 0.009338626062682783, "deviation-center-line": 0.3867965559017575, "driven_lanedir_consec": 1.077835565390577, "sim_compute_sim_state": 0.005019880714216781, "sim_compute_performance-ego": 0.007318831239071192, "sim_compute_robot_state-ego": 0.008842076306567767}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 1.6078521541188833, "sim_physics": 0.11048418885739422, "survival_time": 14.350000000000067, "driven_lanedir": 1.3540205243071175, "sim_render-ego": 0.012721875818764291, "in-drivable-lane": 1.0499999999999965, "agent_compute-ego": 0.08495080346429805, "deviation-heading": 4.906433370232041, "set_robot_commands": 0.009763064700136615, "deviation-center-line": 0.9456130116531666, "driven_lanedir_consec": 0.597789520996023, "sim_compute_sim_state": 0.0051641273166244455, "sim_compute_performance-ego": 0.0075252612708753, "sim_compute_robot_state-ego": 0.009257990308754953}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 1.095549624763805, "sim_physics": 0.10135212233572297, "survival_time": 8.249999999999982, "driven_lanedir": 0.9414348951223483, "sim_render-ego": 0.01229540795990915, "in-drivable-lane": 0.8000000000000007, "agent_compute-ego": 0.07925714146007191, "deviation-heading": 1.2467005926582295, "set_robot_commands": 0.009246279976584696, "deviation-center-line": 0.5499063513523165, "driven_lanedir_consec": 0.9414348951223483, "sim_compute_sim_state": 0.005079065669666637, "sim_compute_performance-ego": 0.007452285651004676, "sim_compute_robot_state-ego": 0.00937777721520626}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.5156955094994762, "sim_physics": 0.13232710422613683, "survival_time": 3.899999999999994, "driven_lanedir": 0.5031972589761864, "sim_render-ego": 0.01376821750249618, "in-drivable-lane": 0, "agent_compute-ego": 0.104127138088911, "deviation-heading": 0.7627853007922785, "set_robot_commands": 0.010055306630256848, "deviation-center-line": 0.3362270256659414, "driven_lanedir_consec": 0.5031972589761864, "sim_compute_sim_state": 0.005313760195022974, "sim_compute_performance-ego": 0.008087436358133951, "sim_compute_robot_state-ego": 0.009710366909320535}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 1.9674365211673128, "sim_physics": 0.11384889920552572, "survival_time": 14.950000000000076, "driven_lanedir": 1.9457286419646989, "sim_render-ego": 0.012707364559173585, "in-drivable-lane": 0, "agent_compute-ego": 0.0909460965792338, "deviation-heading": 1.7766692015934171, "set_robot_commands": 0.0097990345954895, "deviation-center-line": 0.7098891817349615, "driven_lanedir_consec": 1.9457286419646989, "sim_compute_sim_state": 0.005175933043162028, "sim_compute_performance-ego": 0.007686052322387695, "sim_compute_robot_state-ego": 0.00961811065673828}}
set_robot_commands_max	0.01111685276031494
set_robot_commands_mean	0.009073457080004736
set_robot_commands_median	0.009338626062682783
set_robot_commands_min	0.005627378821372986
sim_compute_performance-ego_max	0.008312352498372396
sim_compute_performance-ego_mean	0.006986634071345098
sim_compute_performance-ego_median	0.007419205691716443
sim_compute_performance-ego_min	0.004456830024719238
sim_compute_robot_state-ego_max	0.010193295967884554
sim_compute_robot_state-ego_mean	0.008659287896175696
sim_compute_robot_state-ego_median	0.009074975366461766
sim_compute_robot_state-ego_min	0.00551328957080841
sim_compute_sim_state_max	0.005789640744527181
sim_compute_sim_state_mean	0.004847222155697439
sim_compute_sim_state_median	0.005079065669666637
sim_compute_sim_state_min	0.003318995237350464
sim_physics_max	0.13232710422613683
sim_physics_mean	0.1029051278884118
sim_physics_median	0.10357274421273846
sim_physics_min	0.06125668883323669
sim_render-ego_max	0.01376821750249618
sim_render-ego_mean	0.011765628471484008
sim_render-ego_median	0.01229540795990915
sim_render-ego_min	0.007776862382888794
simulation-passed	1
survival_time_max	14.950000000000076
survival_time_mean	11.010000000000032
survival_time_min	3.899999999999994

No reset possible

24865

4286

Liam Paull 🇨🇦

challenge-aido_LF-template-random

aido3-LFVI-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-11-04 07:47:30+00:00

2019-11-04 08:00:17+00:00

0:12:47

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24855

4301

Frank (Chude) Qian 🇨🇦

Frank test submission

aido3-LF-sim-testing

step1-simulation

success

ip-172-31-45-34-10501

2019-11-04 07:37:10+00:00

2019-11-04 07:47:01+00:00

0:09:51

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.6624558951504921
survival_time_median	2.000000000000001
deviation-center-line_median	0.08490920096961363
in-drivable-lane_median	0.1999999999999993

other stats

agent_compute-ego_max	0.02918168902397156
agent_compute-ego_mean	0.02524342952558075
agent_compute-ego_median	0.0251231590906779
agent_compute-ego_min	0.021551690450528772
deviation-center-line_max	0.2115306175656672
deviation-center-line_mean	0.1059365171201285
deviation-center-line_min	0.06902532930783761
deviation-heading_max	0.5529773614884596
deviation-heading_mean	0.31722659171317497
deviation-heading_median	0.3145505780820485
deviation-heading_min	0.17790279511896667
driven_any_max	2.1696546375441756
driven_any_mean	0.9422938839630692
driven_any_median	0.9996632207705836
driven_any_min	0.28362940600638153
driven_lanedir_consec_max	2.0004202552910058
driven_lanedir_consec_mean	0.7454689984048328
driven_lanedir_consec_min	0.275488859799796
driven_lanedir_max	2.0004202552910058
driven_lanedir_mean	0.7454689984048328
driven_lanedir_median	0.6624558951504921
driven_lanedir_min	0.275488859799796
in-drivable-lane_max	0.7500000000000002
in-drivable-lane_mean	0.27333333333333293
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9996632207705848, "sim_physics": 0.09687623381614684, "survival_time": 2.000000000000001, "driven_lanedir": 0.6856800730956207, "sim_render-ego": 0.011985737085342408, "in-drivable-lane": 0.4500000000000002, "agent_compute-ego": 0.026259207725524904, "deviation-heading": 0.3145505780820485, "set_robot_commands": 0.008946603536605835, "deviation-center-line": 0.08877415228107144, "driven_lanedir_consec": 0.6856800730956207, "sim_compute_sim_state": 0.004608058929443359, "sim_compute_performance-ego": 0.007262599468231201, "sim_compute_robot_state-ego": 0.00858154296875}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.0296612854712826, "sim_physics": 0.0782822632208103, "survival_time": 2.0500000000000007, "driven_lanedir": 0.8723011611746517, "sim_render-ego": 0.01435850887763791, "in-drivable-lane": 0.1499999999999997, "agent_compute-ego": 0.021551690450528772, "deviation-heading": 0.5268865240013682, "set_robot_commands": 0.008212903650795541, "deviation-center-line": 0.1481817317993273, "driven_lanedir_consec": 0.8723011611746517, "sim_compute_sim_state": 0.003968576105629526, "sim_compute_performance-ego": 0.005961842653228015, "sim_compute_robot_state-ego": 0.0074900010736977185}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.28362940600638153, "sim_physics": 0.10461592674255372, "survival_time": 0.8000000000000002, "driven_lanedir": 0.275488859799796, "sim_render-ego": 0.011725768446922302, "in-drivable-lane": 0, "agent_compute-ego": 0.023471549153327945, "deviation-heading": 0.19211543327251043, "set_robot_commands": 0.008359581232070923, "deviation-center-line": 0.07504719484739283, "driven_lanedir_consec": 0.275488859799796, "sim_compute_sim_state": 0.004677876830101013, "sim_compute_performance-ego": 0.007112026214599609, "sim_compute_robot_state-ego": 0.008417949080467224}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4605114688305756, "sim_physics": 0.13338642770593817, "survival_time": 1.1000000000000003, "driven_lanedir": 0.43876913939124096, "sim_render-ego": 0.012214584784074264, "in-drivable-lane": 0, "agent_compute-ego": 0.02760378880934282, "deviation-heading": 0.3393674233921455, "set_robot_commands": 0.009548393162814054, "deviation-center-line": 0.07142284970145178, "driven_lanedir_consec": 0.43876913939124096, "sim_compute_sim_state": 0.005291451107371937, "sim_compute_performance-ego": 0.006990476088090377, "sim_compute_robot_state-ego": 0.008638761260292747}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.5996546887300096, "sim_physics": 0.10807068745295206, "survival_time": 2.9999999999999973, "driven_lanedir": 1.1449045921439405, "sim_render-ego": 0.011456163724263509, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.0251231590906779, "deviation-heading": 0.17790279511896667, "set_robot_commands": 0.009527341524759928, "deviation-center-line": 0.2115306175656672, "driven_lanedir_consec": 1.1449045921439405, "sim_compute_sim_state": 0.004853800932566325, "sim_compute_performance-ego": 0.006825749079386393, "sim_compute_robot_state-ego": 0.008620889981587727}, "ETHZ_autolab_technical_track-5-0": {"driven_any": 0.4605114688305744, "sim_physics": 0.09388948570598256, "survival_time": 1.1000000000000003, "driven_lanedir": 0.43495812661747424, "sim_render-ego": 0.0101210204037753, "in-drivable-lane": 0, "agent_compute-ego": 0.02169266614046964, "deviation-heading": 0.3681646786246838, "set_robot_commands": 0.007291176102378152, "deviation-center-line": 0.07726482156623121, "driven_lanedir_consec": 0.43495812661747424, "sim_compute_sim_state": 0.004215392199429599, "sim_compute_performance-ego": 0.006006468426097523, "sim_compute_robot_state-ego": 0.00738138502294367}, "ETHZ_autolab_technical_track-6-0": {"driven_any": 1.02966128547129, "sim_physics": 0.09670416901751262, "survival_time": 2.0500000000000007, "driven_lanedir": 0.5601770218273892, "sim_render-ego": 0.010971458946786276, "in-drivable-lane": 0.7500000000000002, "agent_compute-ego": 0.022965012527093653, "deviation-heading": 0.32355444237015635, "set_robot_commands": 0.00883337346518912, "deviation-center-line": 0.08490920096961363, "driven_lanedir_consec": 0.5601770218273892, "sim_compute_sim_state": 0.004649953144352611, "sim_compute_performance-ego": 0.006619116155112662, "sim_compute_robot_state-ego": 0.008183054807709485}, "ETHZ_autolab_technical_track-7-0": {"driven_any": 0.31273308909619457, "sim_physics": 0.1073744717766257, "survival_time": 0.8500000000000002, "driven_lanedir": 0.3050503874991314, "sim_render-ego": 0.0116119805504294, "in-drivable-lane": 0, "agent_compute-ego": 0.026534571367151597, "deviation-heading": 0.18880886527581872, "set_robot_commands": 0.009651646894567153, "deviation-center-line": 0.07728062202497794, "driven_lanedir_consec": 0.3050503874991314, "sim_compute_sim_state": 0.0048223523532643035, "sim_compute_performance-ego": 0.006575289894552792, "sim_compute_robot_state-ego": 0.008772906135110295}, "ETHZ_autolab_technical_track-8-0": {"driven_any": 0.6997653880805514, "sim_physics": 0.12315812905629477, "survival_time": 1.5000000000000009, "driven_lanedir": 0.6932401426939361, "sim_render-ego": 0.012233614921569824, "in-drivable-lane": 0, "agent_compute-ego": 0.02695612112681071, "deviation-heading": 0.20499704380051056, "set_robot_commands": 0.010219550132751463, "deviation-center-line": 0.0893147083916865, "driven_lanedir_consec": 0.6932401426939361, "sim_compute_sim_state": 0.004812375704447428, "sim_compute_performance-ego": 0.006975881258646647, "sim_compute_robot_state-ego": 0.009376033147176109}, "ETHZ_autolab_technical_track-9-0": {"driven_any": 0.9996632207705836, "sim_physics": 0.11998682618141174, "survival_time": 2.000000000000001, "driven_lanedir": 0.6624558951504921, "sim_render-ego": 0.013795053958892824, "in-drivable-lane": 0.4500000000000002, "agent_compute-ego": 0.02817624807357788, "deviation-heading": 0.3376863277046913, "set_robot_commands": 0.009232807159423827, "deviation-center-line": 0.08436164061098934, "driven_lanedir_consec": 0.6624558951504921, "sim_compute_sim_state": 0.0057978034019470215, "sim_compute_performance-ego": 0.007998621463775635, "sim_compute_robot_state-ego": 0.009663695096969604}, "ETHZ_autolab_technical_track-10-0": {"driven_any": 1.5996546887300176, "sim_physics": 0.11310693820317588, "survival_time": 2.9999999999999973, "driven_lanedir": 1.2183806352793682, "sim_render-ego": 0.012013248602549235, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.02646256685256958, "deviation-heading": 0.28606515891922396, "set_robot_commands": 0.009582666556040446, "deviation-center-line": 0.11803705330676682, "driven_lanedir_consec": 1.2183806352793682, "sim_compute_sim_state": 0.005025990804036458, "sim_compute_performance-ego": 0.007139941056569417, "sim_compute_robot_state-ego": 0.0086126446723938}, "ETHZ_autolab_technical_track-11-0": {"driven_any": 0.999663220770584, "sim_physics": 0.12001340985298156, "survival_time": 2.000000000000001, "driven_lanedir": 0.6472595776031794, "sim_render-ego": 0.012721937894821168, "in-drivable-lane": 0.40000000000000013, "agent_compute-ego": 0.02918168902397156, "deviation-heading": 0.5529773614884596, "set_robot_commands": 0.010872113704681396, "deviation-center-line": 0.11678925740610405, "driven_lanedir_consec": 0.6472595776031794, "sim_compute_sim_state": 0.005472356081008911, "sim_compute_performance-ego": 0.007270610332489014, "sim_compute_robot_state-ego": 0.009288150072097778}, "ETHZ_autolab_technical_track-12-0": {"driven_any": 0.9996632207705816, "sim_physics": 0.1120836079120636, "survival_time": 2.000000000000001, "driven_lanedir": 0.7633687986487552, "sim_render-ego": 0.01218518018722534, "in-drivable-lane": 0.3500000000000001, "agent_compute-ego": 0.02484954595565796, "deviation-heading": 0.2265255398365549, "set_robot_commands": 0.009964579343795776, "deviation-center-line": 0.07499156567757533, "driven_lanedir_consec": 0.7633687986487552, "sim_compute_sim_state": 0.00523262619972229, "sim_compute_performance-ego": 0.006956273317337036, "sim_compute_robot_state-ego": 0.008747625350952148}, "ETHZ_autolab_technical_track-13-0": {"driven_any": 0.49031796957265295, "sim_physics": 0.11792631771253503, "survival_time": 1.1500000000000004, "driven_lanedir": 0.4795803098565079, "sim_render-ego": 0.011405706405639648, "in-drivable-lane": 0, "agent_compute-ego": 0.024913798207822052, "deviation-heading": 0.24111734124432463, "set_robot_commands": 0.01028479700503142, "deviation-center-line": 0.06902532930783761, "driven_lanedir_consec": 0.4795803098565079, "sim_compute_sim_state": 0.00553794529127038, "sim_compute_performance-ego": 0.007561683654785156, "sim_compute_robot_state-ego": 0.009728950002919071}, "ETHZ_autolab_technical_track-14-0": {"driven_any": 2.1696546375441756, "sim_physics": 0.0973854608173612, "survival_time": 3.949999999999994, "driven_lanedir": 2.0004202552910058, "sim_render-ego": 0.011008425603939009, "in-drivable-lane": 0.1999999999999993, "agent_compute-ego": 0.022909828379184385, "deviation-heading": 0.477679362566162, "set_robot_commands": 0.008081104181989839, "deviation-center-line": 0.2021170113452346, "driven_lanedir_consec": 2.0004202552910058, "sim_compute_sim_state": 0.004677368115775193, "sim_compute_performance-ego": 0.006882510607755637, "sim_compute_robot_state-ego": 0.008562779124779037}}
set_robot_commands_max	0.010872113704681396
set_robot_commands_mean	0.009240575843526326
set_robot_commands_median	0.009527341524759928
set_robot_commands_min	0.007291176102378152
sim_compute_performance-ego_max	0.007998621463775635
sim_compute_performance-ego_mean	0.0069426059780438075
sim_compute_performance-ego_median	0.006975881258646647
sim_compute_performance-ego_min	0.005961842653228015
sim_compute_robot_state-ego_max	0.009728950002919071
sim_compute_robot_state-ego_mean	0.008671091186523094
sim_compute_robot_state-ego_median	0.008620889981587727
sim_compute_robot_state-ego_min	0.00738138502294367
sim_compute_sim_state_max	0.0057978034019470215
sim_compute_sim_state_mean	0.004909595146691091
sim_compute_sim_state_median	0.0048223523532643035
sim_compute_sim_state_min	0.003968576105629526
sim_physics_max	0.13338642770593817
sim_physics_mean	0.10819069034495638
sim_physics_median	0.10807068745295206
sim_physics_min	0.0782822632208103
sim_render-ego_max	0.01435850887763791
sim_render-ego_mean	0.011987226026257895
sim_render-ego_median	0.011985737085342408
sim_render-ego_min	0.0101210204037753
simulation-passed	1
survival_time_max	3.949999999999994
survival_time_mean	1.9033333333333329
survival_time_min	0.8000000000000002

No reset possible

24848

4308

Gianmarco Bernasconi

Baseline-IL-logs-tensorflow

aido3-LF-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-11-04 07:23:25+00:00

2019-11-04 07:37:02+00:00

0:13:37

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	1.0718714901499822
survival_time_median	10.95000000000002
deviation-center-line_median	0.4772723564990606
in-drivable-lane_median	2.0000000000000284

other stats

agent_compute-ego_max	0.09232795599735144
agent_compute-ego_mean	0.08094267839916489
agent_compute-ego_median	0.08347843009042957
agent_compute-ego_min	0.07084373148476206
deviation-center-line_max	0.6406904942868517
deviation-center-line_mean	0.4938165212161748
deviation-center-line_min	0.31337710671129027
deviation-heading_max	3.8222884192685953
deviation-heading_mean	1.886257264033355
deviation-heading_median	1.6371023044300186
deviation-heading_min	1.0291983568469163
driven_any_max	1.9274734833117888
driven_any_mean	1.4728904662262727
driven_any_median	1.3955604499129293
driven_any_min	0.9401240481150034
driven_lanedir_consec_max	1.340127352921762
driven_lanedir_consec_mean	1.026068456766734
driven_lanedir_consec_min	0.5402661247909388
driven_lanedir_max	1.340127352921762
driven_lanedir_mean	1.0290906253644485
driven_lanedir_median	1.0725312381519267
driven_lanedir_min	0.5403233919782506
in-drivable-lane_max	7.450000000000096
in-drivable-lane_mean	2.630000000000027
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9401240481150034, "sim_physics": 0.10807773741808804, "survival_time": 6.5999999999999845, "driven_lanedir": 0.5403233919782506, "sim_render-ego": 0.012114604314168297, "in-drivable-lane": 2.199999999999992, "agent_compute-ego": 0.09232795599735144, "deviation-heading": 1.0291983568469163, "set_robot_commands": 0.009188877813743822, "deviation-center-line": 0.31337710671129027, "driven_lanedir_consec": 0.5402661247909388, "sim_compute_sim_state": 0.00508040189743042, "sim_compute_performance-ego": 0.007270751577435118, "sim_compute_robot_state-ego": 0.008758671355969978}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.3652604506430852, "sim_physics": 0.10007197018627707, "survival_time": 10.95000000000002, "driven_lanedir": 1.340127352921762, "sim_render-ego": 0.01205531211748515, "in-drivable-lane": 0, "agent_compute-ego": 0.08347843009042957, "deviation-heading": 1.6371023044300186, "set_robot_commands": 0.009612243469447303, "deviation-center-line": 0.5606664284435315, "driven_lanedir_consec": 1.340127352921762, "sim_compute_sim_state": 0.0050047053594023125, "sim_compute_performance-ego": 0.007147873917671099, "sim_compute_robot_state-ego": 0.009013061654077816}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.9274734833117888, "sim_physics": 0.10337613900502522, "survival_time": 14.950000000000076, "driven_lanedir": 0.98384977411352, "sim_render-ego": 0.011531301339467366, "in-drivable-lane": 7.450000000000096, "agent_compute-ego": 0.0855923859278361, "deviation-heading": 1.752652948828745, "set_robot_commands": 0.00928783098856608, "deviation-center-line": 0.4772723564990606, "driven_lanedir_consec": 0.98384977411352, "sim_compute_sim_state": 0.004889419078826904, "sim_compute_performance-ego": 0.006870504220326742, "sim_compute_robot_state-ego": 0.008729532559712727}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.7360338991485584, "sim_physics": 0.08517521619796753, "survival_time": 12.800000000000049, "driven_lanedir": 1.2086213696567825, "sim_render-ego": 0.010540620423853396, "in-drivable-lane": 2.0000000000000284, "agent_compute-ego": 0.07247088849544525, "deviation-heading": 3.8222884192685953, "set_robot_commands": 0.008179988712072372, "deviation-center-line": 0.6406904942868517, "driven_lanedir_consec": 1.1942275418574662, "sim_compute_sim_state": 0.004353993572294712, "sim_compute_performance-ego": 0.006287799216806889, "sim_compute_robot_state-ego": 0.00774653535336256}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.3955604499129293, "sim_physics": 0.06990003818418922, "survival_time": 10.25000000000001, "driven_lanedir": 1.0725312381519267, "sim_render-ego": 0.008711357814509694, "in-drivable-lane": 1.500000000000021, "agent_compute-ego": 0.07084373148476206, "deviation-heading": 1.1900442907925002, "set_robot_commands": 0.006596630375559737, "deviation-center-line": 0.47707622014014023, "driven_lanedir_consec": 1.0718714901499822, "sim_compute_sim_state": 0.003627040909557808, "sim_compute_performance-ego": 0.004990005493164062, "sim_compute_robot_state-ego": 0.006343105362682807}}
set_robot_commands_max	0.009612243469447303
set_robot_commands_mean	0.008573114271877864
set_robot_commands_median	0.009188877813743822
set_robot_commands_min	0.006596630375559737
sim_compute_performance-ego_max	0.007270751577435118
sim_compute_performance-ego_mean	0.0065133868850807825
sim_compute_performance-ego_median	0.006870504220326742
sim_compute_performance-ego_min	0.004990005493164062
sim_compute_robot_state-ego_max	0.009013061654077816
sim_compute_robot_state-ego_mean	0.008118181257161178
sim_compute_robot_state-ego_median	0.008729532559712727
sim_compute_robot_state-ego_min	0.006343105362682807
sim_compute_sim_state_max	0.00508040189743042
sim_compute_sim_state_mean	0.004591112163502431
sim_compute_sim_state_median	0.004889419078826904
sim_compute_sim_state_min	0.003627040909557808
sim_physics_max	0.10807773741808804
sim_physics_mean	0.09332022019830942
sim_physics_median	0.10007197018627707
sim_physics_min	0.06990003818418922
sim_render-ego_max	0.012114604314168297
sim_render-ego_mean	0.01099063920189678
sim_render-ego_median	0.011531301339467366
sim_render-ego_min	0.008711357814509694
simulation-passed	1
survival_time_max	14.950000000000076
survival_time_mean	11.110000000000028
survival_time_min	6.5999999999999845

No reset possible

24846

4309

Riza Velioglu 🇩🇪

Uni Bielefeld

aido3-LF-sim-validation

step1-simulation

error

ip-172-31-45-34-10501

2019-11-04 07:20:48+00:00

2019-11-04 07:23:16+00:00

0:02:28

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24830

4334

Soroush Saryazdi 🇨🇦

challenge-aido_LF-template-random

aido3-LFVI-sim-testing

step1-simulation

success

ip-172-31-45-34-10501

2019-11-04 07:00:20+00:00

2019-11-04 07:20:15+00:00

0:19:55

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.6537162971975778
survival_time_median	2.5999999999999988
deviation-center-line_median	0.14884708769804722
in-drivable-lane_median	0.549999999999998

other stats

agent_compute-ego_max	0.024222683906555176
agent_compute-ego_mean	0.0224738165636714
agent_compute-ego_median	0.02287678811156634
agent_compute-ego_min	0.019045352935791016
deviation-center-line_max	0.5141120095458045
deviation-center-line_mean	0.17690002270053232
deviation-center-line_min	0.06216029860587694
deviation-heading_max	1.992943725008188
deviation-heading_mean	0.6929778233956081
deviation-heading_median	0.4928440244444201
deviation-heading_min	0.24234723632033853
driven_any_max	3.534324402241835
driven_any_mean	1.3470385992234173
driven_any_median	1.001633421251193
driven_any_min	0.3412551960104244
driven_lanedir_consec_max	2.4943080534418787
driven_lanedir_consec_mean	0.8813910855277101
driven_lanedir_consec_min	0.3199180625488627
driven_lanedir_max	2.4943080534418787
driven_lanedir_mean	0.8962528018383177
driven_lanedir_median	0.7206363324756618
driven_lanedir_min	0.3199180625488627
in-drivable-lane_max	3.549999999999991
in-drivable-lane_mean	0.839999999999998
in-drivable-lane_min	0
per-episodes	details {"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.19762694081173668, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.011458728886857818, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.021755242649512955, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.009019724930389016, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.004752059526081326, "sim_compute_performance-ego": 0.006810007216055182, "sim_compute_robot_state-ego": 0.008279368847231322, "sim_compute_robot_state-npc0": 0.00823569901381867, "sim_compute_robot_state-npc1": 0.0077948358994496016, "sim_compute_robot_state-npc2": 0.008013317856607558, "sim_compute_robot_state-npc3": 0.00784560094905805}, "udem1-1-0": {"driven_any": 0.47798123795353264, "sim_physics": 0.19196991409574235, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.012453436851501465, "in-drivable-lane": 0, "agent_compute-ego": 0.023318137441362657, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.010062285831996374, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.005187613623482841, "sim_compute_performance-ego": 0.007041547979627337, "sim_compute_robot_state-ego": 0.009110357080187117, "sim_compute_robot_state-npc0": 0.008438689368111747, "sim_compute_robot_state-npc1": 0.00859224796295166, "sim_compute_robot_state-npc2": 0.008617520332336426, "sim_compute_robot_state-npc3": 0.008264218057904924}, "udem1-2-0": {"driven_any": 2.1864003300000703, "sim_physics": 0.20711002997981692, "survival_time": 5.14999999999999, "driven_lanedir": 1.8262319453635671, "sim_render-ego": 0.012333207917444914, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.02324943172121511, "deviation-heading": 0.8115272458314912, "set_robot_commands": 0.010194646502004086, "deviation-center-line": 0.19273160416094068, "driven_lanedir_consec": 1.8262319453635671, "sim_compute_sim_state": 0.004984274651240376, "sim_compute_performance-ego": 0.007099357623498417, "sim_compute_robot_state-ego": 0.009227500378506856, "sim_compute_robot_state-npc0": 0.008521621667065667, "sim_compute_robot_state-npc1": 0.008320993590123445, "sim_compute_robot_state-npc2": 0.00816810478284521, "sim_compute_robot_state-npc3": 0.008425758880318946}, "udem1-3-0": {"driven_any": 0.9410513078477728, "sim_physics": 0.20567854742209116, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.01209494968255361, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.022176310420036316, "deviation-heading": 0.35154027506548413, "set_robot_commands": 0.009756738940874737, "deviation-center-line": 0.08975118546699376, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.005200549960136414, "sim_compute_performance-ego": 0.006895681222279866, "sim_compute_robot_state-ego": 0.008614872892697653, "sim_compute_robot_state-npc0": 0.00851986805597941, "sim_compute_robot_state-npc1": 0.008842592438062033, "sim_compute_robot_state-npc2": 0.008846874038378397, "sim_compute_robot_state-npc3": 0.008714710672696432}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.2017881643204462, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.012775046484810966, "in-drivable-lane": 0, "agent_compute-ego": 0.02402428218296596, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.009628795442127046, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.004951817648751395, "sim_compute_performance-ego": 0.008449985867454893, "sim_compute_robot_state-ego": 0.01091569945925758, "sim_compute_robot_state-npc0": 0.007795265742710659, "sim_compute_robot_state-npc1": 0.008439631689162482, "sim_compute_robot_state-npc2": 0.007803258441743397, "sim_compute_robot_state-npc3": 0.007957515262422107}, "udem1-5-0": {"driven_any": 3.534324402241835, "sim_physics": 0.17206403738010428, "survival_time": 8.349999999999984, "driven_lanedir": 2.4943080534418787, "sim_render-ego": 0.010439040418156608, "in-drivable-lane": 2, "agent_compute-ego": 0.019045352935791016, "deviation-heading": 1.992943725008188, "set_robot_commands": 0.00800133465292925, "deviation-center-line": 0.5141120095458045, "driven_lanedir_consec": 2.4943080534418787, "sim_compute_sim_state": 0.004118903668340808, "sim_compute_performance-ego": 0.0059897871074562304, "sim_compute_robot_state-ego": 0.007583879425140198, "sim_compute_robot_state-npc0": 0.007276098171394028, "sim_compute_robot_state-npc1": 0.0069645336288178036, "sim_compute_robot_state-npc2": 0.007006165510166191, "sim_compute_robot_state-npc3": 0.007002493578516795}, "udem1-6-0": {"driven_any": 2.1760405509894407, "sim_physics": 0.2203291457833596, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.012560587484859723, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.02287678811156634, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.009373345421355903, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.004958055551769664, "sim_compute_performance-ego": 0.007478389925169714, "sim_compute_robot_state-ego": 0.009226778178539091, "sim_compute_robot_state-npc0": 0.008640657350854966, "sim_compute_robot_state-npc1": 0.00847616010499232, "sim_compute_robot_state-npc2": 0.008331106704415626, "sim_compute_robot_state-npc3": 0.008379459381103516}, "udem1-7-0": {"driven_any": 1.0050236108346402, "sim_physics": 0.27097134590148925, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684152, "sim_render-ego": 0.01291682243347168, "in-drivable-lane": 0, "agent_compute-ego": 0.024222683906555176, "deviation-heading": 0.9227446658975218, "set_robot_commands": 0.010607657432556152, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.005512666702270508, "sim_compute_performance-ego": 0.007475290298461914, "sim_compute_robot_state-ego": 0.009546022415161132, "sim_compute_robot_state-npc0": 0.009323930740356446, "sim_compute_robot_state-npc1": 0.008677043914794923, "sim_compute_robot_state-npc2": 0.008861594200134278, "sim_compute_robot_state-npc3": 0.008738751411437989}, "udem1-8-0": {"driven_any": 0.997041660301958, "sim_physics": 0.18518393773298997, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.012210983496445876, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.021789601215949424, "deviation-heading": 0.6817155913229234, "set_robot_commands": 0.009221099890195409, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.004765638938316932, "sim_compute_performance-ego": 0.006852984428405762, "sim_compute_robot_state-ego": 0.008830038400796743, "sim_compute_robot_state-npc0": 0.00854424788401677, "sim_compute_robot_state-npc1": 0.008112499347099891, "sim_compute_robot_state-npc2": 0.00842190247315627, "sim_compute_robot_state-npc3": 0.008095351549295278}, "udem1-9-0": {"driven_any": 0.43749987104723, "sim_physics": 0.257902218745305, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.012097890560443584, "in-drivable-lane": 0, "agent_compute-ego": 0.023106950979966383, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.009318186686589168, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.004729069196260893, "sim_compute_performance-ego": 0.006856753275944636, "sim_compute_robot_state-ego": 0.008495752628032979, "sim_compute_robot_state-npc0": 0.00819238332601694, "sim_compute_robot_state-npc1": 0.007940026429983286, "sim_compute_robot_state-npc2": 0.0080259946676401, "sim_compute_robot_state-npc3": 0.008050120793856107}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.1960717331279408, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.012594959952614525, "in-drivable-lane": 0, "agent_compute-ego": 0.02413888411088423, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.010208184068853205, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.004893595522100275, "sim_compute_performance-ego": 0.007206732576543634, "sim_compute_robot_state-ego": 0.009014920754866169, "sim_compute_robot_state-npc0": 0.008828975937583229, "sim_compute_robot_state-npc1": 0.00847834890538996, "sim_compute_robot_state-npc2": 0.008607560938054865, "sim_compute_robot_state-npc3": 0.008452892303466797}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.15299001216888428, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756618, "sim_render-ego": 0.011051344871520995, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.020510287284851075, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.008776664733886719, "deviation-center-line": 0.16567670185555416, "driven_lanedir_consec": 0.718507549745529, "sim_compute_sim_state": 0.004576263427734375, "sim_compute_performance-ego": 0.006292495727539062, "sim_compute_robot_state-ego": 0.008332924842834473, "sim_compute_robot_state-npc0": 0.007786026000976562, "sim_compute_robot_state-npc1": 0.00747502326965332, "sim_compute_robot_state-npc2": 0.007207083702087403, "sim_compute_robot_state-npc3": 0.007064504623413086}, "udem1-12-0": {"driven_any": 1.9646878702534365, "sim_physics": 0.2199573643664096, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.01208647261274622, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.023320677432608097, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.009561077077338036, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.005148854661495127, "sim_compute_performance-ego": 0.007125007345321331, "sim_compute_robot_state-ego": 0.009288016785966588, "sim_compute_robot_state-npc0": 0.009013708601606654, "sim_compute_robot_state-npc1": 0.008317759696473467, "sim_compute_robot_state-npc2": 0.00845996876980396, "sim_compute_robot_state-npc3": 0.008091130155198117}, "udem1-13-0": {"driven_any": 1.001633421251193, "sim_physics": 0.17326781382927528, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.012224266162285438, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.022531903707064115, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.008869143632742075, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.0046201302455021786, "sim_compute_performance-ego": 0.006807831617502065, "sim_compute_robot_state-ego": 0.008520479385669414, "sim_compute_robot_state-npc0": 0.008238292657412015, "sim_compute_robot_state-npc1": 0.007828203531411978, "sim_compute_robot_state-npc2": 0.007588579104496882, "sim_compute_robot_state-npc3": 0.007783344158759484}, "udem1-14-0": {"driven_any": 2.218769782727205, "sim_physics": 0.20406757309323265, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433687, "sim_render-ego": 0.01115325064886184, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.02104071435474214, "deviation-heading": 1.7163026916317137, "set_robot_commands": 0.0093313353402274, "deviation-center-line": 0.2412693316050378, "driven_lanedir_consec": 0.6077880672015876, "sim_compute_sim_state": 0.004596989495413644, "sim_compute_performance-ego": 0.006701271874564035, "sim_compute_robot_state-ego": 0.008602832612537203, "sim_compute_robot_state-npc0": 0.007908130827404204, "sim_compute_robot_state-npc1": 0.007806080863589332, "sim_compute_robot_state-npc2": 0.007882797150384813, "sim_compute_robot_state-npc3": 0.007743174689156669}}
set_robot_commands_max	0.010607657432556152
set_robot_commands_mean	0.009462014705604304
set_robot_commands_median	0.009373345421355903
set_robot_commands_min	0.00800133465292925
sim_compute_performance-ego_max	0.008449985867454893
sim_compute_performance-ego_mean	0.007005541605721606
sim_compute_performance-ego_median	0.006895681222279866
sim_compute_performance-ego_min	0.0059897871074562304
sim_compute_robot_state-ego_max	0.01091569945925758
sim_compute_robot_state-ego_mean	0.008905962939161634
sim_compute_robot_state-ego_median	0.008830038400796743
sim_compute_robot_state-ego_min	0.007583879425140198
sim_compute_robot_state-npc0_max	0.009323930740356446
sim_compute_robot_state-npc0_mean	0.008350906356353865
sim_compute_robot_state-npc0_median	0.008438689368111747
sim_compute_robot_state-npc0_min	0.007276098171394028
sim_compute_robot_state-npc1_max	0.008842592438062033
sim_compute_robot_state-npc1_mean	0.008137732084797032
sim_compute_robot_state-npc1_median	0.008317759696473467
sim_compute_robot_state-npc1_min	0.0069645336288178036
sim_compute_robot_state-npc2_max	0.008861594200134278
sim_compute_robot_state-npc2_mean	0.008122788578150093
sim_compute_robot_state-npc2_median	0.00816810478284521
sim_compute_robot_state-npc2_min	0.007006165510166191
sim_compute_robot_state-npc3_max	0.008738751411437989
sim_compute_robot_state-npc3_mean	0.008040601764440287
sim_compute_robot_state-npc3_median	0.008091130155198117
sim_compute_robot_state-npc3_min	0.007002493578516795
sim_compute_sim_state_max	0.005512666702270508
sim_compute_sim_state_mean	0.004866432187926451
sim_compute_sim_state_median	0.004893595522100275
sim_compute_sim_state_min	0.004118903668340808
sim_physics_max	0.27097134590148925
sim_physics_mean	0.20379858525058825
sim_physics_median	0.2017881643204462
sim_physics_min	0.15299001216888428
sim_render-ego_max	0.01291682243347168
sim_render-ego_mean	0.01203006589763835
sim_render-ego_median	0.012210983496445876
sim_render-ego_min	0.010439040418156608
simulation-passed	1
survival_time_max	8.349999999999984
survival_time_mean	3.3333333333333286
survival_time_min	1.0500000000000005

No reset possible

24826

4342

Soroush Saryazdi 🇨🇦

challenge-aido_LF-baseline-duckietown

aido3-LFV-sim-validation

step1-simulation

host-error

ip-172-31-45-34-10501

2019-11-04 06:58:38+00:00

2019-11-04 07:00:12+00:00

0:01:34

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 887, in run_one
    run_docker(wd, project, cmd)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1269, in run_docker
    subprocess.check_call(cmd0, cwd=cwd, stdout=tee_stdout, stderr=tee_stderr)
  File "/usr/lib/python3.6/subprocess.py", line 306, in check_call
    retcode = call(*popenargs, **kwargs)
  File "/usr/lib/python3.6/subprocess.py", line 287, in call
    with Popen(*popenargs, **kwargs) as p:
  File "/usr/lib/python3.6/subprocess.py", line 729, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.6/subprocess.py", line 1364, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido3-LFV-sim-validation/submission4342/step1-simulation-ip-172-31-45-34-10501-job24826': '/tmp/duckietown/DT18/evaluator/executions/aido3-LFV-sim-validation/submission4342/step1-simulation-ip-172-31-45-34-10501-job24826'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24820

4324

Gianmarco Bernasconi

baseline-IL-sim-tensorflow

aido3-LF-sim-validation

step1-simulation

error

ip-172-31-45-34-10501

2019-11-04 06:54:18+00:00

2019-11-04 06:58:12+00:00

0:03:54

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24805

4339

Soroush Saryazdi 🇨🇦

challenge-aido_LF-baseline-duckietown

aido3-LF-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-11-02 17:34:13+00:00

2019-11-02 17:46:21+00:00

0:12:08

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	1.031959539326358
survival_time_median	6.999999999999983
deviation-center-line_median	0.30813065597584094
in-drivable-lane_median	0.29999999999999893

other stats

agent_compute-ego_max	0.0597290672711878
agent_compute-ego_mean	0.05805425255794318
agent_compute-ego_median	0.05869487529629273
agent_compute-ego_min	0.05486702299737311
deviation-center-line_max	0.7613680476514193
deviation-center-line_mean	0.372160850332333
deviation-center-line_min	0.16951641575669354
deviation-heading_max	2.5933566245679356
deviation-heading_mean	1.6929375878992283
deviation-heading_median	1.754798433599611
deviation-heading_min	1.1111264813652466
driven_any_max	1.6579343168805438
driven_any_mean	0.9990825662731376
driven_any_median	1.0713887633341128
driven_any_min	0.5574578431728555
driven_lanedir_consec_max	1.2750950230896518
driven_lanedir_consec_mean	0.7901646760843628
driven_lanedir_consec_min	0.14158952519315818
driven_lanedir_max	1.3381079008161705
driven_lanedir_mean	0.8275615860286525
driven_lanedir_median	1.031959539326358
driven_lanedir_min	0.2655611971880877
in-drivable-lane_max	1.40000000000002
in-drivable-lane_mean	0.5700000000000035
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.5574578431728555, "sim_physics": 0.14154488699776785, "survival_time": 3.8499999999999943, "driven_lanedir": 0.4527803510355839, "sim_render-ego": 0.014929576353593306, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.0576204913003104, "deviation-heading": 1.1111264813652466, "set_robot_commands": 0.010990251194347034, "deviation-center-line": 0.16951641575669354, "driven_lanedir_consec": 0.4527803510355839, "sim_compute_sim_state": 0.005963539148305917, "sim_compute_performance-ego": 0.0092941816751059, "sim_compute_robot_state-ego": 0.011895504864779386}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 0.5649376153133937, "sim_physics": 0.1218559370412455, "survival_time": 3.8499999999999943, "driven_lanedir": 0.2655611971880877, "sim_render-ego": 0.014630283628191267, "in-drivable-lane": 1.049999999999999, "agent_compute-ego": 0.05486702299737311, "deviation-heading": 1.7660126575352704, "set_robot_commands": 0.009310558244779513, "deviation-center-line": 0.3197998969181359, "driven_lanedir_consec": 0.14158952519315818, "sim_compute_sim_state": 0.005367938574258383, "sim_compute_performance-ego": 0.007774229173536424, "sim_compute_robot_state-ego": 0.009910688771829976}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 1.6579343168805438, "sim_physics": 0.11799792840447224, "survival_time": 10.650000000000016, "driven_lanedir": 1.3381079008161705, "sim_render-ego": 0.014819888441775326, "in-drivable-lane": 1.40000000000002, "agent_compute-ego": 0.05869487529629273, "deviation-heading": 2.5933566245679356, "set_robot_commands": 0.010191122690836588, "deviation-center-line": 0.7613680476514193, "driven_lanedir_consec": 1.2750950230896518, "sim_compute_sim_state": 0.005696952622821073, "sim_compute_performance-ego": 0.009108188566467572, "sim_compute_robot_state-ego": 0.01089946428934733}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.0713887633341128, "sim_physics": 0.12264139141355242, "survival_time": 6.999999999999983, "driven_lanedir": 1.031959539326358, "sim_render-ego": 0.014622996534620012, "in-drivable-lane": 0, "agent_compute-ego": 0.059359805924551826, "deviation-heading": 1.2393937424280794, "set_robot_commands": 0.010466970716203962, "deviation-center-line": 0.30813065597584094, "driven_lanedir_consec": 1.031959539326358, "sim_compute_sim_state": 0.006004651955195835, "sim_compute_performance-ego": 0.008396082265036447, "sim_compute_robot_state-ego": 0.010751683371407646}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.1436942926647815, "sim_physics": 0.11484606154013, "survival_time": 7.449999999999981, "driven_lanedir": 1.0493989417770622, "sim_render-ego": 0.016730982185210158, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.0597290672711878, "deviation-heading": 1.754798433599611, "set_robot_commands": 0.010117249200808122, "deviation-center-line": 0.30198923535957517, "driven_lanedir_consec": 1.0493989417770622, "sim_compute_sim_state": 0.00588270481800873, "sim_compute_performance-ego": 0.008382910850064066, "sim_compute_robot_state-ego": 0.0108746426217508}}
set_robot_commands_max	0.010990251194347034
set_robot_commands_mean	0.010215230409395044
set_robot_commands_median	0.010191122690836588
set_robot_commands_min	0.009310558244779513
sim_compute_performance-ego_max	0.0092941816751059
sim_compute_performance-ego_mean	0.008591118506042083
sim_compute_performance-ego_median	0.008396082265036447
sim_compute_performance-ego_min	0.007774229173536424
sim_compute_robot_state-ego_max	0.011895504864779386
sim_compute_robot_state-ego_mean	0.010866396783823028
sim_compute_robot_state-ego_median	0.0108746426217508
sim_compute_robot_state-ego_min	0.009910688771829976
sim_compute_sim_state_max	0.006004651955195835
sim_compute_sim_state_mean	0.0057831574237179875
sim_compute_sim_state_median	0.00588270481800873
sim_compute_sim_state_min	0.005367938574258383
sim_physics_max	0.14154488699776785
sim_physics_mean	0.1237772410794336
sim_physics_median	0.1218559370412455
sim_physics_min	0.11484606154013
sim_render-ego_max	0.016730982185210158
sim_render-ego_mean	0.015146745428678016
sim_render-ego_median	0.014819888441775326
sim_render-ego_min	0.014622996534620012
simulation-passed	1
survival_time_max	10.650000000000016
survival_time_mean	6.559999999999993
survival_time_min	3.8499999999999943

No reset possible

24792

4329

Soroush Saryazdi 🇨🇦

challenge-aido_LF-template-random

aido3-LF-sim-testing

step1-simulation

host-error

ip-172-31-45-34-10501

2019-11-02 16:25:22+00:00

2019-11-02 16:26:30+00:00

0:01:08

Error while running [...]


stderr | Pulling scenario_maker ... 
stderr | Pulling solution       ... 
stderr | Pulling simulator      ... done
stderr | 
stderr | ERROR: for evaluator  Cannot overwrite digest sha256:accd9ed7dd51e3adcb9fe5b55e4f423eb0fb4460e779094dc72d480549db0fba
stderr | Cannot overwrite digest sha256:accd9ed7dd51e3adcb9fe5b55e4f423eb0fb4460e779094dc72d480549db0fba
stderr |

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24776

4321

Riza Velioglu 🇩🇪

UniBielefeld-test

aido3-LF-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-11-02 12:46:34+00:00

2019-11-02 12:53:09+00:00

0:06:35

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.13170534180199578
survival_time_median	4.099999999999993
deviation-center-line_median	0.2116662609487696
in-drivable-lane_median	1.2499999999999956

other stats

agent_compute-ego_max	0.018554609093890093
agent_compute-ego_mean	0.01648291352855372
agent_compute-ego_median	0.01607251458051728
agent_compute-ego_min	0.015746932794616492
deviation-center-line_max	0.5173798790069553
deviation-center-line_mean	0.2593619555195955
deviation-center-line_min	0.1255321323707093
deviation-heading_max	6.673956572392337
deviation-heading_mean	2.525845206836613
deviation-heading_median	1.770928428194367
deviation-heading_min	0.3453883924977256
driven_any_max	1.463527572461348
driven_any_mean	0.7066139423895453
driven_any_median	0.5193742504306614
driven_any_min	0.385086999231721
driven_lanedir_consec_max	0.4786031619330284
driven_lanedir_consec_mean	0.14799447761437864
driven_lanedir_consec_min	-0.3423378501180142
driven_lanedir_max	0.4786031619330284
driven_lanedir_mean	0.14799447761437864
driven_lanedir_median	0.13170534180199578
driven_lanedir_min	-0.3423378501180142
in-drivable-lane_max	6.450000000000037
in-drivable-lane_mean	2.2200000000000037
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.734803985675369, "sim_physics": 0.05044394051468613, "survival_time": 7.449999999999981, "driven_lanedir": 0.09498785682346568, "sim_render-ego": 0.007119092365239291, "in-drivable-lane": 3.2999999999999883, "agent_compute-ego": 0.018554609093890093, "deviation-heading": 2.4965414174300595, "set_robot_commands": 0.004611250538153936, "deviation-center-line": 0.2622101134547263, "driven_lanedir_consec": 0.09498785682346568, "sim_compute_sim_state": 0.0029765135489854236, "sim_compute_performance-ego": 0.00406336784362793, "sim_compute_robot_state-ego": 0.00497037452339326}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.463527572461348, "sim_physics": 0.04846849295059569, "survival_time": 14.650000000000071, "driven_lanedir": -0.3423378501180142, "sim_render-ego": 0.0069816747216234435, "in-drivable-lane": 6.450000000000037, "agent_compute-ego": 0.015746932794616492, "deviation-heading": 6.673956572392337, "set_robot_commands": 0.0044993272006715115, "deviation-center-line": 0.5173798790069553, "driven_lanedir_consec": -0.3423378501180142, "sim_compute_sim_state": 0.002917166049163089, "sim_compute_performance-ego": 0.0038066488077209263, "sim_compute_robot_state-ego": 0.004863297979986301}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.385086999231721, "sim_physics": 0.050910341739654544, "survival_time": 2.000000000000001, "driven_lanedir": 0.3770138776314176, "sim_render-ego": 0.007128018140792847, "in-drivable-lane": 0, "agent_compute-ego": 0.015878301858901978, "deviation-heading": 0.3453883924977256, "set_robot_commands": 0.00450446605682373, "deviation-center-line": 0.18002139181681712, "driven_lanedir_consec": 0.3770138776314176, "sim_compute_sim_state": 0.0029178738594055178, "sim_compute_performance-ego": 0.0038069546222686767, "sim_compute_robot_state-ego": 0.0048827588558197025}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.5193742504306614, "sim_physics": 0.04902559763764682, "survival_time": 3.649999999999995, "driven_lanedir": 0.4786031619330284, "sim_render-ego": 0.007197001208997753, "in-drivable-lane": 0.09999999999999964, "agent_compute-ego": 0.016162209314842746, "deviation-heading": 1.3424112236685797, "set_robot_commands": 0.004580657776087931, "deviation-center-line": 0.2116662609487696, "driven_lanedir_consec": 0.4786031619330284, "sim_compute_sim_state": 0.0029534607717435655, "sim_compute_performance-ego": 0.0038836688211519423, "sim_compute_robot_state-ego": 0.004891225736435145}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.43027690414862696, "sim_physics": 0.05002124425841541, "survival_time": 4.099999999999993, "driven_lanedir": 0.13170534180199578, "sim_render-ego": 0.007080877699503084, "in-drivable-lane": 1.2499999999999956, "agent_compute-ego": 0.01607251458051728, "deviation-heading": 1.770928428194367, "set_robot_commands": 0.004534567274698397, "deviation-center-line": 0.1255321323707093, "driven_lanedir_consec": 0.13170534180199578, "sim_compute_sim_state": 0.0029289722442626953, "sim_compute_performance-ego": 0.0037994559218243857, "sim_compute_robot_state-ego": 0.00487227556182117}}
set_robot_commands_max	0.004611250538153936
set_robot_commands_mean	0.004546053769287101
set_robot_commands_median	0.004534567274698397
set_robot_commands_min	0.0044993272006715115
sim_compute_performance-ego_max	0.00406336784362793
sim_compute_performance-ego_mean	0.003872019203318771
sim_compute_performance-ego_median	0.0038069546222686767
sim_compute_performance-ego_min	0.0037994559218243857
sim_compute_robot_state-ego_max	0.00497037452339326
sim_compute_robot_state-ego_mean	0.004895986531491116
sim_compute_robot_state-ego_median	0.0048827588558197025
sim_compute_robot_state-ego_min	0.004863297979986301
sim_compute_sim_state_max	0.0029765135489854236
sim_compute_sim_state_mean	0.002938797294712058
sim_compute_sim_state_median	0.0029289722442626953
sim_compute_sim_state_min	0.002917166049163089
sim_physics_max	0.050910341739654544
sim_physics_mean	0.04977392342019972
sim_physics_median	0.05002124425841541
sim_physics_min	0.04846849295059569
sim_render-ego_max	0.007197001208997753
sim_render-ego_mean	0.007101332827231284
sim_render-ego_median	0.007119092365239291
sim_render-ego_min	0.0069816747216234435
simulation-passed	1
survival_time_max	14.650000000000071
survival_time_mean	6.370000000000009
survival_time_min	2.000000000000001

No reset possible

24773

4314

Tanli Chou 🇹🇼

challenge-aido_LF-template-random

aido3-LF-sim-testing

step1-simulation

host-error

ip-172-31-45-34-10501

2019-11-02 06:49:33+00:00

2019-11-02 07:00:02+00:00

0:10:29

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 1569, in upload
    aws_object.load()
  File "/usr/local/lib/python3.6/dist-packages/boto3/resources/factory.py", line 505, in do_action
    response = action(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/boto3/resources/action.py", line 83, in __call__
    response = getattr(parent.meta.client, operation_name)(**params)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 661, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 641, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1301, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1579, in upload
    aws_object.upload_file(realfile, ExtraArgs={"ContentType": mime_type})
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 280, in object_upload_file
    ExtraArgs=ExtraArgs, Callback=Callback, Config=Config)
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 131, in upload_file
    extra_args=ExtraArgs, callback=Callback)
  File "/usr/local/lib/python3.6/dist-packages/boto3/s3/transfer.py", line 279, in upload_file
    future.result()
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 73, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 233, in result
    raise self._exception
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 255, in _main
    self._submit(transfer_future=transfer_future, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/upload.py", line 549, in _submit
    upload_input_manager.provide_transfer_size(transfer_future)
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/upload.py", line 237, in provide_transfer_size
    transfer_future.meta.call_args.fileobj))
  File "/usr/local/lib/python3.6/dist-packages/s3transfer/utils.py", line 228, in get_file_size
    return os.path.getsize(filename)
  File "/usr/lib/python3.6/genericpath.py", line 50, in getsize
    return os.stat(filename).st_size
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido3-LF-sim-testing/submission4314/step1-simulation-ip-172-31-45-34-10501-job24773/challenge-evaluation-output/episodes/ETHZ_autolab_technical_track-14-0/drawing.html'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24741

4302

Frank (Chude) Qian 🇨🇦

Frank test submission

aido3-LF-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-10-31 14:05:22+00:00

2019-10-31 14:10:43+00:00

0:05:21

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.6856800730956207
survival_time_median	2.000000000000001
deviation-center-line_median	0.08877415228107144
in-drivable-lane_median	0.1499999999999997

other stats

agent_compute-ego_max	0.030235499143600464
agent_compute-ego_mean	0.02765082296317186
agent_compute-ego_median	0.026950167446601683
agent_compute-ego_min	0.0250420610109965
deviation-center-line_max	0.2115306175656672
deviation-center-line_mean	0.11899130923898212
deviation-center-line_min	0.07142284970145178
deviation-heading_max	0.5268865240013682
deviation-heading_mean	0.3101645507734078
deviation-heading_median	0.3145505780820485
deviation-heading_min	0.17790279511896667
driven_any_max	1.5996546887300096
driven_any_mean	0.8746240139617669
driven_any_median	0.9996632207705848
driven_any_min	0.28362940600638153
driven_lanedir_consec_max	1.1449045921439405
driven_lanedir_consec_mean	0.68342876512105
driven_lanedir_consec_min	0.275488859799796
driven_lanedir_max	1.1449045921439405
driven_lanedir_mean	0.68342876512105
driven_lanedir_median	0.6856800730956207
driven_lanedir_min	0.275488859799796
in-drivable-lane_max	0.7499999999999973
in-drivable-lane_mean	0.26999999999999946
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9996632207705848, "sim_physics": 0.10662847161293028, "survival_time": 2.000000000000001, "driven_lanedir": 0.6856800730956207, "sim_render-ego": 0.011769020557403564, "in-drivable-lane": 0.4500000000000002, "agent_compute-ego": 0.02974121570587158, "deviation-heading": 0.3145505780820485, "set_robot_commands": 0.008663147687911987, "deviation-center-line": 0.08877415228107144, "driven_lanedir_consec": 0.6856800730956207, "sim_compute_sim_state": 0.0053188085556030275, "sim_compute_performance-ego": 0.006844544410705566, "sim_compute_robot_state-ego": 0.008881598711013794}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.0296612854712826, "sim_physics": 0.10517470429583294, "survival_time": 2.0500000000000007, "driven_lanedir": 0.8723011611746517, "sim_render-ego": 0.011662675113212772, "in-drivable-lane": 0.1499999999999997, "agent_compute-ego": 0.026950167446601683, "deviation-heading": 0.5268865240013682, "set_robot_commands": 0.008244351642887766, "deviation-center-line": 0.1481817317993273, "driven_lanedir_consec": 0.8723011611746517, "sim_compute_sim_state": 0.004827208635283679, "sim_compute_performance-ego": 0.006616121385155655, "sim_compute_robot_state-ego": 0.009034459183855755}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.28362940600638153, "sim_physics": 0.09964609146118164, "survival_time": 0.8000000000000002, "driven_lanedir": 0.275488859799796, "sim_render-ego": 0.013032093644142153, "in-drivable-lane": 0, "agent_compute-ego": 0.030235499143600464, "deviation-heading": 0.19211543327251043, "set_robot_commands": 0.010700300335884094, "deviation-center-line": 0.07504719484739283, "driven_lanedir_consec": 0.275488859799796, "sim_compute_sim_state": 0.005501419305801392, "sim_compute_performance-ego": 0.007144853472709656, "sim_compute_robot_state-ego": 0.009406477212905884}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.4605114688305756, "sim_physics": 0.10054212266748602, "survival_time": 1.1000000000000003, "driven_lanedir": 0.43876913939124096, "sim_render-ego": 0.012302745472301136, "in-drivable-lane": 0, "agent_compute-ego": 0.026285171508789062, "deviation-heading": 0.3393674233921455, "set_robot_commands": 0.009077386422590776, "deviation-center-line": 0.07142284970145178, "driven_lanedir_consec": 0.43876913939124096, "sim_compute_sim_state": 0.005255265669389205, "sim_compute_performance-ego": 0.007006829435175116, "sim_compute_robot_state-ego": 0.008888959884643555}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.5996546887300096, "sim_physics": 0.0957419474919637, "survival_time": 2.9999999999999973, "driven_lanedir": 1.1449045921439405, "sim_render-ego": 0.012523917357126872, "in-drivable-lane": 0.7499999999999973, "agent_compute-ego": 0.0250420610109965, "deviation-heading": 0.17790279511896667, "set_robot_commands": 0.008590062459309896, "deviation-center-line": 0.2115306175656672, "driven_lanedir_consec": 1.1449045921439405, "sim_compute_sim_state": 0.004934545358022054, "sim_compute_performance-ego": 0.007183361053466797, "sim_compute_robot_state-ego": 0.009134455521901449}}
set_robot_commands_max	0.010700300335884094
set_robot_commands_mean	0.009055049709716904
set_robot_commands_median	0.008663147687911987
set_robot_commands_min	0.008244351642887766
sim_compute_performance-ego_max	0.007183361053466797
sim_compute_performance-ego_mean	0.006959141951442559
sim_compute_performance-ego_median	0.007006829435175116
sim_compute_performance-ego_min	0.006616121385155655
sim_compute_robot_state-ego_max	0.009406477212905884
sim_compute_robot_state-ego_mean	0.009069190102864088
sim_compute_robot_state-ego_median	0.009034459183855755
sim_compute_robot_state-ego_min	0.008881598711013794
sim_compute_sim_state_max	0.005501419305801392
sim_compute_sim_state_mean	0.005167449504819872
sim_compute_sim_state_median	0.005255265669389205
sim_compute_sim_state_min	0.004827208635283679
sim_physics_max	0.10662847161293028
sim_physics_mean	0.10154666750587892
sim_physics_median	0.10054212266748602
sim_physics_min	0.0957419474919637
sim_render-ego_max	0.013032093644142153
sim_render-ego_mean	0.0122580904288373
sim_render-ego_median	0.012302745472301136
sim_render-ego_min	0.011662675113212772
simulation-passed	1
survival_time_max	2.9999999999999973
survival_time_mean	1.7899999999999998
survival_time_min	0.8000000000000002

No reset possible

24726

2700

Artem Ioselevskii

PredictorLast

aido2-PRED

step1-simulation

success

yes

ip-172-31-45-34-10501

2019-10-31 14:04:15+00:00

2019-10-31 14:04:47+00:00

0:00:32

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

error_L1	0.11224940678946654
error_L2	0.0196376817523911

No reset possible

24710

2895

Masaya Kambara 🇯🇵

PredictorLast

aido2-PRED

step1-simulation

success

yes

ip-172-31-45-34-10501

2019-10-31 14:03:19+00:00

2019-10-31 14:03:46+00:00

0:00:27

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

error_L1	0.11425248075742388
error_L2	0.0189835055000345

No reset possible

24702

3157

PredictorLast

aido2-PRED

step1-simulation

success

yes

ip-172-31-45-34-10501

2019-10-31 14:02:49+00:00

2019-10-31 14:03:05+00:00

0:00:16

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

error_L1	0.12505181373312838
error_L2	0.02476493615348624

No reset possible

24690

3175

Ian Li 🇹🇼

PredictorLast

aido2-PRED

step1-simulation

success

yes

ip-172-31-45-34-10501

2019-10-31 14:02:20+00:00

2019-10-31 14:02:35+00:00

0:00:15

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

error_L1	0.11006400075441804
error_L2	0.019170513947411625

No reset possible

24680

3257

Liu Sam 🇹🇼

PredictorLast

aido2-PRED

step1-simulation

host-error

yes

ip-172-31-45-34-10501

2019-10-31 14:02:00+00:00

2019-10-31 14:02:04+00:00

0:00:04

Error while running [...]


stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24673

3264

Liu Sam 🇹🇼

challenge-aido_LF-template-random

aido2-PRED

step1-simulation

host-error

yes

ip-172-31-45-34-10501

2019-10-31 14:01:38+00:00

2019-10-31 14:01:42+00:00

0:00:04

Error while running [...]


stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24665

3201

Masaya Kambara 🇯🇵

PredictorLast

aido2-PRED

step1-simulation

success

yes

ip-172-31-45-34-10501

2019-10-31 14:00:48+00:00

2019-10-31 14:01:24+00:00

0:00:36

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

error_L1	0.11588054133548784
error_L2	0.02001417256221306

No reset possible

24662

3265

Liu Sam 🇹🇼

challenge-aido_LF-template-random

aido2-PRED

step1-simulation

host-error

yes

ip-172-31-45-34-10501

2019-10-31 14:00:28+00:00

2019-10-31 14:00:34+00:00

0:00:06

Error while running [...]


stderr | Pulling solution   ... 
stderr | Pulling evaluation ... done
stderr | 
stderr | ERROR: for solution  pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr | pull access denied for coolcat647/aido-submissions, repository does not exist or may require 'docker login'
stderr |

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24552

4299

Liam Paull 🇨🇦

challenge-aido_LF-template-random

aido3-LFVI-sim-validation

step1-simulation

host-error

ip-172-31-45-34-10501

2019-10-31 13:51:21+00:00

2019-10-31 14:00:02+00:00

0:08:41

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 641, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1301, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1557, in upload
    copy_to_cache(realfile, sha256hex)
  File "/project/src/duckietown_challenges_runner/runner_cache.py", line 40, in copy_to_cache
    msg = "Copying %s to cache %s" % (friendly_size(os.stat(fn).st_size), have)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido3-LFVI-sim-validation/submission4299/step1-simulation-ip-172-31-45-34-10501-job24552/tmp/tmp1a_nph9q/episodes/udem1-2-0/drawing.html'

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24545

4291

Andrea Censi 🇨🇭

PredictorLast

aido2-PRED

step1-simulation

success

yes

ip-172-31-45-34-10501

2019-10-31 13:50:49+00:00

2019-10-31 13:51:05+00:00

0:00:16

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

error_L1	0.10452531843166044
error_L2	0.016869768710260354

No reset possible

24539

4290

Liam Paull 🇨🇦

PredictorLast

aido2-PRED

step1-simulation

success

yes

ip-172-31-45-34-10501

2019-10-31 13:50:16+00:00

2019-10-31 13:50:34+00:00

0:00:18

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

error_L1	0.11993025497652576
error_L2	0.02252576805969636

No reset possible

24529

4290

Liam Paull 🇨🇦

PredictorLast

aido2-PRED

step1-simulation

error

ip-172-31-45-34-10501

2019-10-31 13:40:29+00:00

2019-10-31 13:41:45+00:00

0:01:16

Unexpected exception [...]

Unexpected exception:
Traceback (most recent call last):
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 370, in deserialize_dataclass
    attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=expect_type)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 310, in ipce_to_object
    return deserialize_Dict(K, mj, global_symbols, encountered)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 428, in deserialize_Dict
    attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=V)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 282, in ipce_to_object
    return schema_to_type(schema, global_symbols, encountered)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 453, in schema_to_type
    res = schema_to_type_(schema0, global_symbols, encountered)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 555, in schema_to_type_
    assert False, schema  # pragma: no cover
AssertionError: {'title': 'Any'}

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 370, in deserialize_dataclass
    attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=expect_type)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 315, in ipce_to_object
    return deserialize_dataclass(K, mj, global_symbols, encountered)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 375, in deserialize_dataclass
    raise Exception(msg) from e
Exception: Cannot deserialize attribute inputs (expect: typing.Dict[str, type])
value: {'get_logs': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'get_state': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_state': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_config': {'$id': 'http://invalid.json-schema.org/SetConfig#', 'type': 'object', 'order': ['key', 'value'], 'title': 'SetConfig', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['key', 'value'], '__module__': 'zuper_nodes_wrapper.meta_protocol', 'properties': {'key': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'value': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'description': 'SetConfig(key: str, value: Any)', '__qualname__': 'SetConfig'}, 'describe_node': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_build': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_config': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_protocol': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}}

| Traceback (most recent call last):
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 370, in deserialize_dataclass
|     attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=expect_type)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 310, in ipce_to_object
|     return deserialize_Dict(K, mj, global_symbols, encountered)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 428, in deserialize_Dict
|     attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=V)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 282, in ipce_to_object
|     return schema_to_type(schema, global_symbols, encountered)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 453, in schema_to_type
|     res = schema_to_type_(schema0, global_symbols, encountered)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 555, in schema_to_type_
|     assert False, schema  # pragma: no cover
| AssertionError: {'title': 'Any'}
| 

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context
    yield cie
  File "experiment_manager.py", line 75, in <module>
    main(cie)
  File "experiment_manager.py", line 26, in main
    agent_ci._get_node_protocol()
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 79, in _get_node_protocol
    timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 92, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 211, in read_one
    data = ipce_to_object(msg[FIELD_DATA], {}, expect_type=klass)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 315, in ipce_to_object
    return deserialize_dataclass(K, mj, global_symbols, encountered)
  File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 375, in deserialize_dataclass
    raise Exception(msg) from e
Exception: Cannot deserialize attribute meta (expect: <class 'zuper_nodes.language.InteractionProtocol'>)
value: {'inputs': {'get_logs': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'get_state': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_state': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_config': {'$id': 'http://invalid.json-schema.org/SetConfig#', 'type': 'object', 'order': ['key', 'value'], 'title': 'SetConfig', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['key', 'value'], '__module__': 'zuper_nodes_wrapper.meta_protocol', 'properties': {'key': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'value': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'description': 'SetConfig(key: str, value: Any)', '__qualname__': 'SetConfig'}, 'describe_node': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_build': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_config': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_protocol': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'outputs': {'logs': {'type': 'array', 'items': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'title': 'List[str]', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'aborted': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'node_state': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_state_ack': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_config_ack': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_state_error': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'node_description': {'$id': 'http://invalid.json-schema.org/NodeDescription#', 'type': 'object', 'order': ['description'], 'title': 'NodeDescription', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['description'], '__module__': 'zuper_nodes_wrapper.meta_protocol', 'properties': {'description': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'description': 'NodeDescription(description: str)', '__qualname__': 'NodeDescription'}, 'set_config_error': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'build_description': {'$id': 'http://invalid.json-schema.org/BuildDescription#', 'type': 'object', 'order': [], 'title': 'BuildDescription', '$schema': 'http://json-schema.org/draft-07/schema#', '__module__': 'zuper_nodes_wrapper.meta_protocol', 'description': 'BuildDescription()', '__qualname__': 'BuildDescription'}, 'config_description': {'$id': 'http://invalid.json-schema.org/ConfigDescription#', 'type': 'object', 'order': ['config', 'current'], 'title': 'ConfigDescription', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['config', 'current'], '__module__': 'zuper_nodes_wrapper.meta_protocol', 'properties': {'config': {'$ref': 'http://json-schema.org/draft-07/schema#', 'title': 'type'}, 'current': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'description': 'ConfigDescription(config: type, current: Any)', '__qualname__': 'ConfigDescription'}, 'protocol_description': {'$id': 'http://invalid.json-schema.org/ProtocolDescription#', 'type': 'object', 'order': ['data', 'meta'], 'title': 'ProtocolDescription', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['data', 'meta'], '__module__': 'zuper_nodes_wrapper.meta_protocol', 'properties': {'data': {'$id': 'http://invalid.json-schema.org/InteractionProtocol#', 'type': 'object', 'order': ['description', 'inputs', 'outputs', 'language'], 'title': 'InteractionProtocol', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['description', 'inputs', 'language', 'outputs'], '__module__': 'zuper_nodes.language', 'properties': {'inputs': {'type': 'object', 'title': 'Dict[str,type]', '$schema': 'http://json-schema.org/draft-07/schema#', 'properties': {'$schema': {}}, 'additionalProperties': {'$ref': 'http://json-schema.org/draft-07/schema#', 'title': 'type'}}, 'outputs': {'type': 'object', 'title': 'Dict[str,type]', '$schema': 'http://json-schema.org/draft-07/schema#', 'properties': {'$schema': {}}, 'additionalProperties': {'$ref': 'http://json-schema.org/draft-07/schema#', 'title': 'type'}}, 'language': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'description': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'description': 'InteractionProtocol(description: str, inputs: Dict[str, type], outputs: Dict[str, type], language: str)', '__qualname__': 'InteractionProtocol'}, 'meta': {'$id': 'http://invalid.json-schema.org/InteractionProtocol#', 'type': 'object', 'order': ['description', 'inputs', 'outputs', 'language'], 'title': 'InteractionProtocol', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['description', 'inputs', 'language', 'outputs'], '__module__': 'zuper_nodes.language', 'properties': {'inputs': {'type': 'object', 'title': 'Dict[str,type]', '$schema': 'http://json-schema.org/draft-07/schema#', 'properties': {'$schema': {}}, 'additionalProperties': {'$ref': 'http://json-schema.org/draft-07/schema#', 'title': 'type'}}, 'outputs': {'type': 'object', 'title': 'Dict[str,type]', '$schema': 'http://json-schema.org/draft-07/schema#', 'properties': {'$schema': {}}, 'additionalProperties': {'$ref': 'http://json-schema.org/draft-07/schema#', 'title': 'type'}}, 'language': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'description': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'description': 'InteractionProtocol(description: str, inputs: Dict[str, type], outputs: Dict[str, type], language: str)', '__qualname__': 'InteractionProtocol'}}, 'description': 'ProtocolDescription(data: zuper_nodes.language.InteractionProtocol, meta: zuper_nodes.language.InteractionProtocol)', '__qualname__': 'ProtocolDescription'}}, 'language': '((in:describe_config ; out:config_description) | (in:set_config ; (out:set_config_ack | out:set_config_error)) | (in:describe_protocol ; out:protocol_description) | (in:describe_node ; out:node_description) | (in:describe_build ; out:build_description) | (in:get_state ; out:node_state) | (in:set_state ; (out:set_state_ack | out:set_state_error)) | (in:get_logs ; out:logs) | out:aborted)*', 'description': '\nBasic interaction protocol for nodes spoken by the node wrapper.\n\n    '}

| Traceback (most recent call last):
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 370, in deserialize_dataclass
|     attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=expect_type)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 310, in ipce_to_object
|     return deserialize_Dict(K, mj, global_symbols, encountered)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 428, in deserialize_Dict
|     attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=V)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 282, in ipce_to_object
|     return schema_to_type(schema, global_symbols, encountered)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 453, in schema_to_type
|     res = schema_to_type_(schema0, global_symbols, encountered)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 555, in schema_to_type_
|     assert False, schema  # pragma: no cover
| AssertionError: {'title': 'Any'}
| 
| The above exception was the direct cause of the following exception:
| 
| Traceback (most recent call last):
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 370, in deserialize_dataclass
|     attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=expect_type)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 315, in ipce_to_object
|     return deserialize_dataclass(K, mj, global_symbols, encountered)
|   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 375, in deserialize_dataclass
|     raise Exception(msg) from e
| Exception: Cannot deserialize attribute inputs (expect: typing.Dict[str, type])
| value: {'get_logs': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'get_state': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_state': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'set_config': {'$id': 'http://invalid.json-schema.org/SetConfig#', 'type': 'object', 'order': ['key', 'value'], 'title': 'SetConfig', '$schema': 'http://json-schema.org/draft-07/schema#', 'required': ['key', 'value'], '__module__': 'zuper_nodes_wrapper.meta_protocol', 'properties': {'key': {'type': 'string', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'value': {'title': 'Any', '$schema': 'http://json-schema.org/draft-07/schema#'}}, 'description': 'SetConfig(key: str, value: Any)', '__qualname__': 'SetConfig'}, 'describe_node': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_build': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_config': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}, 'describe_protocol': {'type': 'null', '$schema': 'http://json-schema.org/draft-07/schema#'}}
| 
| | Traceback (most recent call last):
| |   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 370, in deserialize_dataclass
| |     attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=expect_type)
| |   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 310, in ipce_to_object
| |     return deserialize_Dict(K, mj, global_symbols, encountered)
| |   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 428, in deserialize_Dict
| |     attrs[k] = ipce_to_object(v, global_symbols, encountered, expect_type=V)
| |   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 282, in ipce_to_object
| |     return schema_to_type(schema, global_symbols, encountered)
| |   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 453, in schema_to_type
| |     res = schema_to_type_(schema0, global_symbols, encountered)
| |   File "/project/src/zuper-utils/src/zuper_json/ipce.py", line 555, in schema_to_type_
| |     assert False, schema  # pragma: no cover
| | AssertionError: {'title': 'Any'}
| |
|

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24522

4286

Liam Paull 🇨🇦

challenge-aido_LF-template-random

aido3-LFVI-sim-testing

step1-simulation

success

ip-172-31-45-34-10501

2019-10-31 12:29:57+00:00

2019-10-31 12:44:32+00:00

0:14:35

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.6537162971975778
survival_time_median	2.5999999999999988
deviation-center-line_median	0.14884708769804722
in-drivable-lane_median	0.549999999999998

other stats

agent_compute-ego_max	0.019338360854557583
agent_compute-ego_mean	0.015088070039392018
agent_compute-ego_median	0.013900527587303748
agent_compute-ego_min	0.013299822807312012
deviation-center-line_max	0.5141120095458045
deviation-center-line_mean	0.17690002270053232
deviation-center-line_min	0.06216029860587694
deviation-heading_max	1.992943725008188
deviation-heading_mean	0.6929778233956081
deviation-heading_median	0.4928440244444201
deviation-heading_min	0.24234723632033853
driven_any_max	3.534324402241835
driven_any_mean	1.3470385992234173
driven_any_median	1.001633421251193
driven_any_min	0.3412551960104244
driven_lanedir_consec_max	2.4943080534418787
driven_lanedir_consec_mean	0.8813910855277101
driven_lanedir_consec_min	0.3199180625488627
driven_lanedir_max	2.4943080534418787
driven_lanedir_mean	0.8962528018383177
driven_lanedir_median	0.7206363324756618
driven_lanedir_min	0.3199180625488627
in-drivable-lane_max	3.549999999999991
in-drivable-lane_mean	0.839999999999998
in-drivable-lane_min	0
per-episodes	details {"udem1-0-0": {"driven_any": 1.5811782993597916, "sim_physics": 0.12454649466502515, "survival_time": 3.949999999999994, "driven_lanedir": 0.9217908232230632, "sim_render-ego": 0.009432062318053429, "in-drivable-lane": 1.3499999999999952, "agent_compute-ego": 0.01826599881618838, "deviation-heading": 0.5694288623982039, "set_robot_commands": 0.006226645240300819, "deviation-center-line": 0.2348196785686717, "driven_lanedir_consec": 0.9213701538661684, "sim_compute_sim_state": 0.003948024556606631, "sim_compute_performance-ego": 0.0054758289192296285, "sim_compute_robot_state-ego": 0.006701728965662703, "sim_compute_robot_state-npc0": 0.006222293346743041, "sim_compute_robot_state-npc1": 0.006120902073534229, "sim_compute_robot_state-npc2": 0.006062519701221321, "sim_compute_robot_state-npc3": 0.006020717983004413}, "udem1-1-0": {"driven_any": 0.47798123795353264, "sim_physics": 0.1174700345311846, "survival_time": 1.4000000000000006, "driven_lanedir": 0.4500753586658095, "sim_render-ego": 0.010061587606157576, "in-drivable-lane": 0, "agent_compute-ego": 0.019338360854557583, "deviation-heading": 0.46504862964833654, "set_robot_commands": 0.007075352328164237, "deviation-center-line": 0.09791133281082, "driven_lanedir_consec": 0.4500753586658095, "sim_compute_sim_state": 0.0035978725978306363, "sim_compute_performance-ego": 0.005270225661141532, "sim_compute_robot_state-ego": 0.006656442369733538, "sim_compute_robot_state-npc0": 0.006270638534000942, "sim_compute_robot_state-npc1": 0.0058710915701729914, "sim_compute_robot_state-npc2": 0.006253932203565325, "sim_compute_robot_state-npc3": 0.006035557815006801}, "udem1-2-0": {"driven_any": 2.1864003300000703, "sim_physics": 0.11961314284685746, "survival_time": 5.14999999999999, "driven_lanedir": 1.8262319453635671, "sim_render-ego": 0.009003060535319801, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.01634068165010619, "deviation-heading": 0.8115272458314912, "set_robot_commands": 0.005857541723158753, "deviation-center-line": 0.19273160416094068, "driven_lanedir_consec": 1.8262319453635671, "sim_compute_sim_state": 0.003472566604614258, "sim_compute_performance-ego": 0.0046966978647176505, "sim_compute_robot_state-ego": 0.0062476014627993685, "sim_compute_robot_state-npc0": 0.005764519126669874, "sim_compute_robot_state-npc1": 0.005572990306372781, "sim_compute_robot_state-npc2": 0.005593561431736622, "sim_compute_robot_state-npc3": 0.005517593865255708}, "udem1-3-0": {"driven_any": 0.9410513078477728, "sim_physics": 0.1305424024661382, "survival_time": 2.3999999999999995, "driven_lanedir": 0.493517638549009, "sim_render-ego": 0.008745138843854269, "in-drivable-lane": 0.8999999999999988, "agent_compute-ego": 0.016081695755322773, "deviation-heading": 0.35154027506548413, "set_robot_commands": 0.005804811914761861, "deviation-center-line": 0.08975118546699376, "driven_lanedir_consec": 0.493517638549009, "sim_compute_sim_state": 0.0035061885913213096, "sim_compute_performance-ego": 0.004975428183873494, "sim_compute_robot_state-ego": 0.006167277693748474, "sim_compute_robot_state-npc0": 0.005978415409723918, "sim_compute_robot_state-npc1": 0.005535115798314412, "sim_compute_robot_state-npc2": 0.005641733606656392, "sim_compute_robot_state-npc3": 0.005586211880048116}, "udem1-4-0": {"driven_any": 0.34749292376309954, "sim_physics": 0.15173400016058058, "survival_time": 1.0500000000000005, "driven_lanedir": 0.3325249708009643, "sim_render-ego": 0.00974357695806594, "in-drivable-lane": 0, "agent_compute-ego": 0.018409161340622676, "deviation-heading": 0.30044835691402494, "set_robot_commands": 0.006688674290974935, "deviation-center-line": 0.06216029860587694, "driven_lanedir_consec": 0.3325249708009643, "sim_compute_sim_state": 0.0038718382517496743, "sim_compute_performance-ego": 0.005396263939993722, "sim_compute_robot_state-ego": 0.007117646081107003, "sim_compute_robot_state-npc0": 0.006049769265311105, "sim_compute_robot_state-npc1": 0.006137348356701079, "sim_compute_robot_state-npc2": 0.005888700485229492, "sim_compute_robot_state-npc3": 0.0057726360502697175}, "udem1-5-0": {"driven_any": 3.534324402241835, "sim_physics": 0.1116500286285035, "survival_time": 8.349999999999984, "driven_lanedir": 2.4943080534418787, "sim_render-ego": 0.00816788216550907, "in-drivable-lane": 2, "agent_compute-ego": 0.01591879165101194, "deviation-heading": 1.992943725008188, "set_robot_commands": 0.005524425449485551, "deviation-center-line": 0.5141120095458045, "driven_lanedir_consec": 2.4943080534418787, "sim_compute_sim_state": 0.0033597532146705123, "sim_compute_performance-ego": 0.00457725553455467, "sim_compute_robot_state-ego": 0.005777554597683296, "sim_compute_robot_state-npc0": 0.005551934956076616, "sim_compute_robot_state-npc1": 0.0053864938770225666, "sim_compute_robot_state-npc2": 0.005347440342703265, "sim_compute_robot_state-npc3": 0.005278490260689559}, "udem1-6-0": {"driven_any": 2.1760405509894407, "sim_physics": 0.10139174600249357, "survival_time": 5.14999999999999, "driven_lanedir": 1.9748003483573369, "sim_render-ego": 0.007073154727232109, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.013414440803157473, "deviation-heading": 0.7877172510198454, "set_robot_commands": 0.004402077313765739, "deviation-center-line": 0.30661477465861414, "driven_lanedir_consec": 1.9748003483573369, "sim_compute_sim_state": 0.0028052839260656855, "sim_compute_performance-ego": 0.003707978331926957, "sim_compute_robot_state-ego": 0.0048315316728017865, "sim_compute_robot_state-npc0": 0.004500185401694288, "sim_compute_robot_state-npc1": 0.004374480941920605, "sim_compute_robot_state-npc2": 0.004377483164222495, "sim_compute_robot_state-npc3": 0.00436460856095101}, "udem1-7-0": {"driven_any": 1.0050236108346402, "sim_physics": 0.10915902137756348, "survival_time": 2.499999999999999, "driven_lanedir": 0.8701796426684152, "sim_render-ego": 0.007135825157165527, "in-drivable-lane": 0, "agent_compute-ego": 0.01364203929901123, "deviation-heading": 0.9227446658975218, "set_robot_commands": 0.004408988952636719, "deviation-center-line": 0.14884708769804722, "driven_lanedir_consec": 0.8546116490381102, "sim_compute_sim_state": 0.0028396081924438477, "sim_compute_performance-ego": 0.003739161491394043, "sim_compute_robot_state-ego": 0.004872641563415527, "sim_compute_robot_state-npc0": 0.004576287269592285, "sim_compute_robot_state-npc1": 0.004422135353088379, "sim_compute_robot_state-npc2": 0.004384245872497559, "sim_compute_robot_state-npc3": 0.004377188682556152}, "udem1-8-0": {"driven_any": 0.997041660301958, "sim_physics": 0.0914948949447045, "survival_time": 2.5999999999999988, "driven_lanedir": 0.8041152574540316, "sim_render-ego": 0.007050372087038481, "in-drivable-lane": 0.29999999999999893, "agent_compute-ego": 0.013385227093329797, "deviation-heading": 0.6817155913229234, "set_robot_commands": 0.004529709999377911, "deviation-center-line": 0.1898768080806288, "driven_lanedir_consec": 0.8041152574540316, "sim_compute_sim_state": 0.002785843152266282, "sim_compute_performance-ego": 0.0037230803416325496, "sim_compute_robot_state-ego": 0.004837590914506178, "sim_compute_robot_state-npc0": 0.004564042274768536, "sim_compute_robot_state-npc1": 0.004417148920205923, "sim_compute_robot_state-npc2": 0.004396956700545091, "sim_compute_robot_state-npc3": 0.004390542323772724}, "udem1-9-0": {"driven_any": 0.43749987104723, "sim_physics": 0.11725938320159912, "survival_time": 1.3000000000000005, "driven_lanedir": 0.4177527119659043, "sim_render-ego": 0.007066075618450458, "in-drivable-lane": 0, "agent_compute-ego": 0.013900527587303748, "deviation-heading": 0.3796137446285097, "set_robot_commands": 0.004216258342449482, "deviation-center-line": 0.08207813392324871, "driven_lanedir_consec": 0.4177527119659043, "sim_compute_sim_state": 0.0027267015897310697, "sim_compute_performance-ego": 0.003667336243849534, "sim_compute_robot_state-ego": 0.004764960362361028, "sim_compute_robot_state-npc0": 0.004486762560330904, "sim_compute_robot_state-npc1": 0.004292341379018931, "sim_compute_robot_state-npc2": 0.004277082589956431, "sim_compute_robot_state-npc3": 0.004359547908489521}, "udem1-10-0": {"driven_any": 0.3412551960104244, "sim_physics": 0.099037766456604, "survival_time": 1.1000000000000003, "driven_lanedir": 0.3199180625488627, "sim_render-ego": 0.00720890001817183, "in-drivable-lane": 0, "agent_compute-ego": 0.014066371050747956, "deviation-heading": 0.3796986873122644, "set_robot_commands": 0.004488706588745117, "deviation-center-line": 0.06276575300040536, "driven_lanedir_consec": 0.3199180625488627, "sim_compute_sim_state": 0.002780870957808061, "sim_compute_performance-ego": 0.003737698901783336, "sim_compute_robot_state-ego": 0.004872896454551003, "sim_compute_robot_state-npc0": 0.004602594809098678, "sim_compute_robot_state-npc1": 0.0044117624109441585, "sim_compute_robot_state-npc2": 0.0043812448328191585, "sim_compute_robot_state-npc3": 0.004371220415288752}, "udem1-11-0": {"driven_any": 0.9951985237696338, "sim_physics": 0.08897060871124268, "survival_time": 2.499999999999999, "driven_lanedir": 0.7206363324756618, "sim_render-ego": 0.006922039985656738, "in-drivable-lane": 0.549999999999998, "agent_compute-ego": 0.013445611000061036, "deviation-heading": 0.30074636349085415, "set_robot_commands": 0.004344930648803711, "deviation-center-line": 0.16567670185555416, "driven_lanedir_consec": 0.718507549745529, "sim_compute_sim_state": 0.002777590751647949, "sim_compute_performance-ego": 0.0036861610412597654, "sim_compute_robot_state-ego": 0.004778470993041992, "sim_compute_robot_state-npc0": 0.004521985054016114, "sim_compute_robot_state-npc1": 0.004341745376586914, "sim_compute_robot_state-npc2": 0.004313879013061524, "sim_compute_robot_state-npc3": 0.004311261177062989}, "udem1-12-0": {"driven_any": 1.9646878702534365, "sim_physics": 0.09559788856100528, "survival_time": 4.699999999999991, "driven_lanedir": 0.35162821871931205, "sim_render-ego": 0.006913593474854814, "in-drivable-lane": 3.549999999999991, "agent_compute-ego": 0.013299822807312012, "deviation-heading": 0.24234723632033853, "set_robot_commands": 0.004403050909651087, "deviation-center-line": 0.13126670634798973, "driven_lanedir_consec": 0.35162821871931205, "sim_compute_sim_state": 0.002771862009738354, "sim_compute_performance-ego": 0.003677877974002919, "sim_compute_robot_state-ego": 0.004784137644666306, "sim_compute_robot_state-npc0": 0.004502803721326463, "sim_compute_robot_state-npc1": 0.0043470783436552005, "sim_compute_robot_state-npc2": 0.0043321188460005095, "sim_compute_robot_state-npc3": 0.004335768679355053}, "udem1-13-0": {"driven_any": 1.001633421251193, "sim_physics": 0.09174925088882446, "survival_time": 2.5999999999999988, "driven_lanedir": 0.6537162971975778, "sim_render-ego": 0.007003761254824125, "in-drivable-lane": 0.5999999999999979, "agent_compute-ego": 0.01347889808508066, "deviation-heading": 0.4928440244444201, "set_robot_commands": 0.004381940915034368, "deviation-center-line": 0.13361893417935136, "driven_lanedir_consec": 0.6537162971975778, "sim_compute_sim_state": 0.0027479345981891337, "sim_compute_performance-ego": 0.0036930900353651782, "sim_compute_robot_state-ego": 0.004797756671905518, "sim_compute_robot_state-npc0": 0.004552868696359487, "sim_compute_robot_state-npc1": 0.004335522651672363, "sim_compute_robot_state-npc2": 0.004309356212615967, "sim_compute_robot_state-npc3": 0.004308530917534461}, "udem1-14-0": {"driven_any": 2.218769782727205, "sim_physics": 0.09446832338968912, "survival_time": 5.249999999999989, "driven_lanedir": 0.8125963661433687, "sim_render-ego": 0.006825011117117746, "in-drivable-lane": 2.499999999999993, "agent_compute-ego": 0.013333422797066823, "deviation-heading": 1.7163026916317137, "set_robot_commands": 0.004366876965477353, "deviation-center-line": 0.2412693316050378, "driven_lanedir_consec": 0.6077880672015876, "sim_compute_sim_state": 0.002741352717081706, "sim_compute_performance-ego": 0.003659012204124814, "sim_compute_robot_state-ego": 0.004759965624128069, "sim_compute_robot_state-npc0": 0.004496744700840541, "sim_compute_robot_state-npc1": 0.004312810443696522, "sim_compute_robot_state-npc2": 0.0042843500773111975, "sim_compute_robot_state-npc3": 0.004286834171840123}}
set_robot_commands_max	0.007075352328164237
set_robot_commands_mean	0.005114666105519176
set_robot_commands_median	0.004488706588745117
set_robot_commands_min	0.004216258342449482
sim_compute_performance-ego_max	0.0054758289192296285
sim_compute_performance-ego_mean	0.004245539777923319
sim_compute_performance-ego_median	0.003737698901783336
sim_compute_performance-ego_min	0.003659012204124814
sim_compute_robot_state-ego_max	0.007117646081107003
sim_compute_robot_state-ego_mean	0.005464546871474119
sim_compute_robot_state-ego_median	0.004872641563415527
sim_compute_robot_state-ego_min	0.004759965624128069
sim_compute_robot_state-npc0_max	0.006270638534000942
sim_compute_robot_state-npc0_mean	0.005109456341770185
sim_compute_robot_state-npc0_median	0.004576287269592285
sim_compute_robot_state-npc0_min	0.004486762560330904
sim_compute_robot_state-npc1_max	0.006137348356701079
sim_compute_robot_state-npc1_mean	0.004925264520193804
sim_compute_robot_state-npc1_median	0.004417148920205923
sim_compute_robot_state-npc1_min	0.004292341379018931
sim_compute_robot_state-npc2_max	0.006253932203565325
sim_compute_robot_state-npc2_mean	0.0049229736720094895
sim_compute_robot_state-npc2_median	0.004384245872497559
sim_compute_robot_state-npc2_min	0.004277082589956431
sim_compute_robot_state-npc3_max	0.006035557815006801
sim_compute_robot_state-npc3_mean	0.004887780712741673
sim_compute_robot_state-npc3_median	0.004377188682556152
sim_compute_robot_state-npc3_min	0.004286834171840123
sim_compute_sim_state_max	0.003948024556606631
sim_compute_sim_state_mean	0.003115552780784341
sim_compute_sim_state_median	0.0028052839260656855
sim_compute_sim_state_min	0.0027267015897310697
sim_physics_max	0.15173400016058058
sim_physics_mean	0.10964566578880104
sim_physics_median	0.10915902137756348
sim_physics_min	0.08897060871124268
sim_render-ego_max	0.010061587606157576
sim_render-ego_mean	0.00789013612449813
sim_render-ego_median	0.007135825157165527
sim_render-ego_min	0.006825011117117746
simulation-passed	1
survival_time_max	8.349999999999984
survival_time_mean	3.3333333333333286
survival_time_min	1.0500000000000005

No reset possible

24498

4265

Gianmarco Bernasconi

challenge-aido_LF-baseline-duckietown

aido3-LFV-sim-validation

step1-simulation

error

ip-172-31-45-34-10501

2019-10-29 17:55:01+00:00

2019-10-29 18:00:19+00:00

0:05:18

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24478

4254

Manfred Diaz

baseline-IL-sim-tensorflow

aido3-LF-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-10-25 17:03:46+00:00

2019-10-25 17:09:44+00:00

0:05:58

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.8643402612594112
survival_time_median	4.049999999999994
deviation-center-line_median	0.2292935727414086
in-drivable-lane_median	0.7999999999999972

other stats

agent_compute-ego_max	0.02156685560177534
agent_compute-ego_mean	0.020292826309206983
agent_compute-ego_median	0.020391063925660688
agent_compute-ego_min	0.01889581298828125
deviation-center-line_max	0.33028801588721224
deviation-center-line_mean	0.2277159300660386
deviation-center-line_min	0.09707790174193774
deviation-heading_max	1.3958061594947733
deviation-heading_mean	0.6703338074272118
deviation-heading_median	0.41820372319062216
deviation-heading_min	0.36813860997729714
driven_any_max	1.6538161472954858
driven_any_mean	1.1004153921340745
driven_any_median	1.0388103444448928
driven_any_min	0.200693405863396
driven_lanedir_consec_max	1.26538928748462
driven_lanedir_consec_mean	0.8210828692087336
driven_lanedir_consec_min	0.16934398354359148
driven_lanedir_max	1.2694856397988663
driven_lanedir_mean	0.8219021396715828
driven_lanedir_median	0.8643402612594112
driven_lanedir_min	0.16934398354359148
in-drivable-lane_max	1.7499999999999938
in-drivable-lane_mean	0.849999999999997
in-drivable-lane_min	0.05000000000000005
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.9968840767723192, "sim_physics": 0.06285648468213204, "survival_time": 3.899999999999994, "driven_lanedir": 0.6522929352320239, "sim_render-ego": 0.009705644387465255, "in-drivable-lane": 1.149999999999996, "agent_compute-ego": 0.02156685560177534, "deviation-heading": 0.41820372319062216, "set_robot_commands": 0.0061664550732343625, "deviation-center-line": 0.1801348331090155, "driven_lanedir_consec": 0.6522929352320239, "sim_compute_sim_state": 0.0038419381166115785, "sim_compute_performance-ego": 0.005547749690520458, "sim_compute_robot_state-ego": 0.007178291296347594}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.0388103444448928, "sim_physics": 0.0639974629437482, "survival_time": 4.049999999999994, "driven_lanedir": 0.8643402612594112, "sim_render-ego": 0.009447554011403778, "in-drivable-lane": 0.4999999999999982, "agent_compute-ego": 0.020391063925660688, "deviation-heading": 0.7710117946472442, "set_robot_commands": 0.0060189829932318795, "deviation-center-line": 0.2292935727414086, "driven_lanedir_consec": 0.8643402612594112, "sim_compute_sim_state": 0.003869769013958213, "sim_compute_performance-ego": 0.005349668455712589, "sim_compute_robot_state-ego": 0.006727704295405635}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.200693405863396, "sim_physics": 0.0618782156989688, "survival_time": 1.0500000000000005, "driven_lanedir": 0.16934398354359148, "sim_render-ego": 0.008766026723952521, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.019247543244134812, "deviation-heading": 0.36813860997729714, "set_robot_commands": 0.005778880346389044, "deviation-center-line": 0.09707790174193774, "driven_lanedir_consec": 0.16934398354359148, "sim_compute_sim_state": 0.0031988734290713354, "sim_compute_performance-ego": 0.004461969648088727, "sim_compute_robot_state-ego": 0.005664314542497907}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6118729862942784, "sim_physics": 0.06250332808885418, "survival_time": 6.099999999999986, "driven_lanedir": 1.2694856397988663, "sim_render-ego": 0.009746752801488658, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.02136285578618284, "deviation-heading": 1.3958061594947733, "set_robot_commands": 0.006547662078357133, "deviation-center-line": 0.33028801588721224, "driven_lanedir_consec": 1.26538928748462, "sim_compute_sim_state": 0.00395358781345555, "sim_compute_performance-ego": 0.005593899820671707, "sim_compute_robot_state-ego": 0.006941924329663887}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 1.6538161472954858, "sim_physics": 0.05696118354797363, "survival_time": 6.249999999999986, "driven_lanedir": 1.154047878524021, "sim_render-ego": 0.007987098693847656, "in-drivable-lane": 1.7499999999999938, "agent_compute-ego": 0.01889581298828125, "deviation-heading": 0.3985087498261223, "set_robot_commands": 0.0058980159759521485, "deviation-center-line": 0.30178532685061904, "driven_lanedir_consec": 1.154047878524021, "sim_compute_sim_state": 0.003394569396972656, "sim_compute_performance-ego": 0.004366767883300781, "sim_compute_robot_state-ego": 0.005586570739746094}}
set_robot_commands_max	0.006547662078357133
set_robot_commands_mean	0.006081999293432913
set_robot_commands_median	0.0060189829932318795
set_robot_commands_min	0.005778880346389044
sim_compute_performance-ego_max	0.005593899820671707
sim_compute_performance-ego_mean	0.0050640110996588526
sim_compute_performance-ego_median	0.005349668455712589
sim_compute_performance-ego_min	0.004366767883300781
sim_compute_robot_state-ego_max	0.007178291296347594
sim_compute_robot_state-ego_mean	0.0064197610407322235
sim_compute_robot_state-ego_median	0.006727704295405635
sim_compute_robot_state-ego_min	0.005586570739746094
sim_compute_sim_state_max	0.00395358781345555
sim_compute_sim_state_mean	0.0036517475540138664
sim_compute_sim_state_median	0.0038419381166115785
sim_compute_sim_state_min	0.0031988734290713354
sim_physics_max	0.0639974629437482
sim_physics_mean	0.06163933499233537
sim_physics_median	0.06250332808885418
sim_physics_min	0.05696118354797363
sim_render-ego_max	0.009746752801488658
sim_render-ego_mean	0.009130615323631576
sim_render-ego_median	0.009447554011403778
sim_render-ego_min	0.007987098693847656
simulation-passed	1
survival_time_max	6.249999999999986
survival_time_mean	4.269999999999992
survival_time_min	1.0500000000000005

No reset possible

24455

4249

Manfred Diaz

challenge-aido_LF-template-tensorflow

aido3-LF-sim-validation

step1-simulation

success

ip-172-31-45-34-10501

2019-10-25 05:19:37+00:00

2019-10-25 05:23:18+00:00

0:03:41

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	0.3836893529851255
survival_time_median	1.7000000000000008
deviation-center-line_median	0.0858253260721851
in-drivable-lane_median	0.05000000000000005

other stats

agent_compute-ego_max	0.017807653972080777
agent_compute-ego_mean	0.016184170329144515
agent_compute-ego_median	0.015813596108380484
agent_compute-ego_min	0.01553831602397718
deviation-center-line_max	0.36390839793757934
deviation-center-line_mean	0.1588780171996435
deviation-center-line_min	0.07030216825053653
deviation-heading_max	2.341707381808154
deviation-heading_mean	0.9609814573485826
deviation-heading_median	0.6693100312853487
deviation-heading_min	0.41021639088350054
driven_any_max	1.8171021522376647
driven_any_mean	0.6505805114676873
driven_any_median	0.450922994366908
driven_any_min	0.16548590166439342
driven_lanedir_consec_max	1.477053167522771
driven_lanedir_consec_mean	0.5259075202597365
driven_lanedir_consec_min	0.050542695865297915
driven_lanedir_max	1.4915574176913124
driven_lanedir_mean	0.5288083702934447
driven_lanedir_median	0.3836893529851255
driven_lanedir_min	0.050542695865297915
in-drivable-lane_max	0.4499999999999984
in-drivable-lane_mean	0.1199999999999997
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 0.16548590166439342, "sim_physics": 0.05753287814912342, "survival_time": 1.0500000000000005, "driven_lanedir": 0.050542695865297915, "sim_render-ego": 0.007279963720412482, "in-drivable-lane": 0, "agent_compute-ego": 0.017807653972080777, "deviation-heading": 0.8533383692765097, "set_robot_commands": 0.004479181198846726, "deviation-center-line": 0.07030216825053653, "driven_lanedir_consec": 0.050542695865297915, "sim_compute_sim_state": 0.0030317079453241256, "sim_compute_performance-ego": 0.004179579871041434, "sim_compute_robot_state-ego": 0.005015577588762555}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.8171021522376647, "sim_physics": 0.04735036983209498, "survival_time": 6.799999999999984, "driven_lanedir": 1.4915574176913124, "sim_render-ego": 0.00676997794824488, "in-drivable-lane": 0.4499999999999984, "agent_compute-ego": 0.01563309396014494, "deviation-heading": 2.341707381808154, "set_robot_commands": 0.0044726904700784125, "deviation-center-line": 0.36390839793757934, "driven_lanedir_consec": 1.477053167522771, "sim_compute_sim_state": 0.0027565395130830653, "sim_compute_performance-ego": 0.003691043923882877, "sim_compute_robot_state-ego": 0.004780343350242166}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 0.5478592140116708, "sim_physics": 0.04676494472905209, "survival_time": 1.900000000000001, "driven_lanedir": 0.5148594382810869, "sim_render-ego": 0.006934561227497302, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.01553831602397718, "deviation-heading": 0.41021639088350054, "set_robot_commands": 0.004356923856233296, "deviation-center-line": 0.1985141017139083, "driven_lanedir_consec": 0.5148594382810869, "sim_compute_sim_state": 0.0027895475688733554, "sim_compute_performance-ego": 0.00370394556145919, "sim_compute_robot_state-ego": 0.004819085723475406}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 0.2715322950577994, "sim_physics": 0.052039779149569, "survival_time": 1.3000000000000005, "driven_lanedir": 0.2033929466444011, "sim_render-ego": 0.007028121214646559, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.0161281915811392, "deviation-heading": 0.6693100312853487, "set_robot_commands": 0.004490558917705829, "deviation-center-line": 0.07584009202400846, "driven_lanedir_consec": 0.2033929466444011, "sim_compute_sim_state": 0.002839317688575158, "sim_compute_performance-ego": 0.0037206136263333834, "sim_compute_robot_state-ego": 0.004753470420837402}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 0.450922994366908, "sim_physics": 0.05028710645787856, "survival_time": 1.7000000000000008, "driven_lanedir": 0.3836893529851255, "sim_render-ego": 0.007035521899952608, "in-drivable-lane": 0.05000000000000005, "agent_compute-ego": 0.015813596108380484, "deviation-heading": 0.5303351134894, "set_robot_commands": 0.004541852894951315, "deviation-center-line": 0.0858253260721851, "driven_lanedir_consec": 0.3836893529851255, "sim_compute_sim_state": 0.002780339297126321, "sim_compute_performance-ego": 0.003780105534721823, "sim_compute_robot_state-ego": 0.004800291622386259}}
set_robot_commands_max	0.004541852894951315
set_robot_commands_mean	0.004468241467563115
set_robot_commands_median	0.004479181198846726
set_robot_commands_min	0.004356923856233296
sim_compute_performance-ego_max	0.004179579871041434
sim_compute_performance-ego_mean	0.0038150577034877418
sim_compute_performance-ego_median	0.0037206136263333834
sim_compute_performance-ego_min	0.003691043923882877
sim_compute_robot_state-ego_max	0.005015577588762555
sim_compute_robot_state-ego_mean	0.004833753741140758
sim_compute_robot_state-ego_median	0.004800291622386259
sim_compute_robot_state-ego_min	0.004753470420837402
sim_compute_sim_state_max	0.0030317079453241256
sim_compute_sim_state_mean	0.002839490402596405
sim_compute_sim_state_median	0.0027895475688733554
sim_compute_sim_state_min	0.0027565395130830653
sim_physics_max	0.05753287814912342
sim_physics_mean	0.05079501566354361
sim_physics_median	0.05028710645787856
sim_physics_min	0.04676494472905209
sim_render-ego_max	0.007279963720412482
sim_render-ego_mean	0.0070096292021507665
sim_render-ego_median	0.007028121214646559
sim_render-ego_min	0.00676997794824488
simulation-passed	1
survival_time_max	6.799999999999984
survival_time_mean	2.549999999999997
survival_time_min	1.0500000000000005

No reset possible

24449

4242

Manfred Diaz

Baseline-IL-logs-tensorflow

aido3-LFV-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-10-24 23:56:18+00:00

2019-10-25 00:00:31+00:00

0:04:13

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24429

4231

Andrea Censi 🇨🇭

challenge-aido_LF-template-pytorch

aido3-LF-sim-validation

step1-simulation

error

ip-172-31-45-34-10501

2019-10-24 17:13:38+00:00

2019-10-24 17:17:37+00:00

0:03:59

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24423

4225

Manfred Diaz

Baseline-IL-logs-tensorflow

aido3-LFV-sim-testing

step1-simulation

failed

ip-172-31-45-34-10501

2019-10-24 06:36:50+00:00

2019-10-24 06:37:46+00:00

0:00:56

InvalidSubmission: T [...]

InvalidSubmission:
Traceback (most recent call last):
  File "experiment_manager.py", line 299, in run_episode
    r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands')
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 96, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 177, in read_one
    nickname=self.nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 258, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.7/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 286, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted:

error in agent |Exception while handling a message on topic "get_commands".
               |
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1334, in _do_call
               ||     return fn(*args)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1319, in _run_fn
               ||     options, feed_dict, fetch_list, target_list, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1407, in _call_tf_sessionrun
               ||     run_metadata)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[{{node prefix/ConvNet/conv_layer_1/kernel/read}}]]
               ||
               || During handling of the above exception, another exception occurred:
               ||
               || Traceback (most recent call last):
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 60, in compute_action
               ||     x: observation
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 929, in run
               ||     run_metadata_ptr)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _run
               ||     feed_dict_tensor, options, run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1328, in _do_run
               ||     run_metadata)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/client/session.py", line 1348, in _do_call
               ||     raise type(e)(node_def, op, message)
               || tensorflow.python.framework.errors_impl.FailedPreconditionError: Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               || Caused by op 'prefix/ConvNet/conv_layer_1/kernel/read', defined at:
               ||   File "imitation_agent.py", line 136, in <module>
               ||     main()
               ||   File "imitation_agent.py", line 132, in main
               ||     wrap_direct(node=node, protocol=protocol)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/interface.py", line 22, in wrap_direct
               ||     run_loop(node, protocol, args)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 196, in run_loop
               ||     config=config)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 277, in loop
               ||     handle_message_node(parsed, receiver0, context0)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/wrapper.py", line 431, in handle_message_node
               ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
               ||   File "/usr/local/lib/python3.7/dist-packages/zuper_nodes_wrapper/utils.py", line 19, in call_if_fun_exists
               ||     f(**kwargs)
               ||   File "imitation_agent.py", line 71, in on_received_get_commands
               ||     pwm_left, pwm_right = self.compute_action(self.config.current_image)
               ||   File "imitation_agent.py", line 43, in compute_action
               ||     graph = load_graph(frozen_model_filename)
               ||   File "/workspace/graph_utils.py", line 14, in load_graph
               ||     tf.import_graph_def(graph_def, name="prefix")
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func
               ||     return func(*args, **kwargs)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 442, in import_graph_def
               ||     _ProcessNewOps(graph)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 235, in _ProcessNewOps
               ||     for new_op in graph._add_new_tf_operations(compute_devices=False):  # pylint: disable=protected-access
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in _add_new_tf_operations
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3433, in <listcomp>
               ||     for c_op in c_api_util.new_tf_operations(self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 3325, in _create_op_from_tf_operation
               ||     ret = Operation(c_op, self)
               ||   File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/ops.py", line 1801, in __init__
               ||     self._traceback = tf_stack.extract_stack()
               ||
               || FailedPreconditionError (see above for traceback): Attempting to use uninitialized value prefix/ConvNet/conv_layer_1/kernel
               || 	 [[node prefix/ConvNet/conv_layer_1/kernel/read (defined at /workspace/graph_utils.py:14) ]]
               ||
               ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/duckietown_challenges/cie_concrete.py", line 675, in scoring_context
    yield cie
  File "experiment_manager.py", line 444, in <module>
    wrap(cie)
  File "experiment_manager.py", line 432, in wrap
    main(cie, logdir, attempts)
  File "experiment_manager.py", line 136, in main
    physics_dt=config.physics_dt)
  File "experiment_manager.py", line 303, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24384

4189

Liam Paull 🇨🇦

challenge-aido_LF-template-ros - Template solution using ROS

aido3-LFV-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-10-23 19:51:59+00:00

2019-10-23 20:00:13+00:00

0:08:14

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24367

4181

Liam Paull 🇨🇦

challenge-aido_LF-baseline-duckietown

aido3-LFV-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-10-23 16:53:59+00:00

2019-10-23 16:57:14+00:00

0:03:15

The container "solut [...]

The container "solution" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

24351

4167

Tanli Chou 🇹🇼

challenge-aido_LF-template-random

aido3-LFVI-sim-testing

step1-simulation

error

ip-172-31-45-34-10501

2019-10-21 14:48:11+00:00

2019-10-21 15:00:10+00:00

0:11:59

The container "evalu [...]

The container "evaluator" exited with code 1.


Look at the logs for the container to know more about the error.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible