Duckietown Challenges Home Challenges Submissions

Evaluator 1880

ID1880
evaluatorreg12-cf110c084bc5-1
ownerI don't have one 😀
machinearchimede
processreg12-cf110c084bc5-1
versiond-c:6.1.5;d-c-r:6.0.29
first heard
last heard
statusinactive
# evaluating
# success3 33456
# timeout
# failed1 33652
# error30 33474
# aborted
# host-error2 33666
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
336729099Bea Baselines 🐤template-tensorflowaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-errornoreg12-cf110c084bc5-10:00:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 628, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LFV_multi-sim-validation/submission9099'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336669115Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simhost-errornoreg12-cf110c084bc5-10:00:45
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 673, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 1402, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 1650, in upload
    copy_to_cache(realfile, sha256hex)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner_cache.py", line 42, in copy_to_cache
    shutil.copy(fn, have)
  File "/usr/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 272, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/usr/lib/python3.8/shutil.py", line 163, in _fastcopy_sendfile
    raise err from None
  File "/usr/lib/python3.8/shutil.py", line 149, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LFV_multi-sim-validation/submission9115/LFVmultibodyv-sim-reg12-cf110c084bc5-1-job33666/docker-compose.original.yaml' -> '/tmp/duckietown/DT18/evaluator/cache/by-value/sha256hex/9163919d923d3df9aefc049809748c3264d8d2e829ceada2db43d5e9fb797c54'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336649055Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:57
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336526837Melisande Tengexercise_ros_templateaido5-LF-sim-validationLFv-simfailednoreg12-cf110c084bc5-10:04:06
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 216, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 299, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 337, in read_until_over
    raise TimeoutError(msg) from None
TimeoutError: Timeout while reading output of node "ego".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "experiment_manager.py", line 443, in run_episode
    r: MsgReceived = await loop.run_in_executor(executor, f)
  File "/usr/local/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 133, in write_topic_and_expect
    ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout)
  File "/usr/local/lib/python3.8/site-packages/zuper_nodes_wrapper/wrapper_outside.py", line 279, in read_one
    raise TimeoutError(msg) from e
TimeoutError: Timeout detected on /fifos/ego-out after 112 messages. Expected topic "commands".

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 213, in main
    length_s = await run_episode(
  File "experiment_manager.py", line 447, in run_episode
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336508903Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:50
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336478843Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:16
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336458851Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:04
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336438847Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:43
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336378735Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:44
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335938311Bea Baselines 🐤straightaido5-LF-sim-validationLFv-simsuccessnoreg12-cf110c084bc5-10:25:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9090606848991776
survival_time_median14.950000000000076
deviation-center-line_median0.43889021054934063
in-drivable-lane_median7.350000000000095


other stats
agent_compute-ego_max0.026447460651397706
agent_compute-ego_mean0.024873252823652964
agent_compute-ego_median0.024592941058309455
agent_compute-ego_min0.024035592873891194
complete-iteration_max0.2977997414271037
complete-iteration_mean0.2834755979955415
complete-iteration_median0.2859170091152191
complete-iteration_min0.2660157060623169
deviation-center-line_max0.9378023746981936
deviation-center-line_mean0.4748913029433971
deviation-center-line_min0.1706752999063784
deviation-heading_max4.524954561191712
deviation-heading_mean1.738939508159257
deviation-heading_median1.260174988196555
deviation-heading_min1.015682903311432
driven_any_max1.939990017341495
driven_any_mean1.9001610314666515
driven_any_median1.9399275851621989
driven_any_min1.7209216810089898
driven_lanedir_consec_max1.4957655800176883
driven_lanedir_consec_mean0.9157194983936154
driven_lanedir_consec_min0.6114560739771817
driven_lanedir_max1.4957655800176883
driven_lanedir_mean1.0049184479434985
driven_lanedir_median0.9396329585031704
driven_lanedir_min0.6114560739771817
get_state_dump_max0.017458664576212566
get_state_dump_mean0.016322653467309985
get_state_dump_median0.016092271010080973
get_state_dump_min0.015591518878936768
get_ui_image_max0.053417657321189346
get_ui_image_mean0.05177606805317292
get_ui_image_median0.05212386722851517
get_ui_image_min0.04965022325515747
in-drivable-lane_max9.150000000000093
in-drivable-lane_mean6.380000000000068
in-drivable-lane_min2.8499999999999934
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.933681663855542, "get_ui_image": 0.052621238231658936, "step_physics": 0.13771339257558188, "survival_time": 14.950000000000076, "driven_lanedir": 1.4957655800176883, "get_state_dump": 0.016152920722961425, "sim_render-ego": 0.006659911473592122, "in-drivable-lane": 2.900000000000041, "agent_compute-ego": 0.025081570148468017, "deviation-heading": 2.0832048690541876, "complete-iteration": 0.29043540398279827, "set_robot_commands": 0.006121911207834879, "deviation-center-line": 0.546791770464069, "driven_lanedir_consec": 1.4957655800176883, "sim_compute_sim_state": 0.02064635992050171, "sim_compute_performance-ego": 0.005708176294962565, "sim_compute_robot_state-ego": 0.009777150154113769, "sim_compute_robot_state-parked0": 0.009771174589792889}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 1.9399017340902232, "get_ui_image": 0.05218215942382813, "step_physics": 0.13976910432179768, "survival_time": 14.950000000000076, "driven_lanedir": 1.4830878649995356, "get_state_dump": 0.015902936458587646, "sim_render-ego": 0.006626376310984294, "in-drivable-lane": 2.8499999999999934, "agent_compute-ego": 0.0240962020556132, "deviation-heading": 2.591676990223147, "complete-iteration": 0.28914289077123007, "set_robot_commands": 0.006150180498758952, "deviation-center-line": 0.7105762881038086, "driven_lanedir_consec": 0.6585366651299052, "sim_compute_sim_state": 0.01913980642954509, "sim_compute_performance-ego": 0.005619115829467773, "sim_compute_robot_state-ego": 0.009850999514261882, "sim_compute_robot_state-parked0": 0.009619543552398682}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 1.9399534362341744, "get_ui_image": 0.05109099229176839, "step_physics": 0.13773441314697266, "survival_time": 14.950000000000076, "driven_lanedir": 0.7104957827553435, "get_state_dump": 0.01659658193588257, "sim_render-ego": 0.007010806401570638, "in-drivable-lane": 9.150000000000093, "agent_compute-ego": 0.02518358469009399, "deviation-heading": 1.1304458766216685, "complete-iteration": 0.2895978228251139, "set_robot_commands": 0.006610906124114991, "deviation-center-line": 0.36344742041206934, "driven_lanedir_consec": 0.7104957827553435, "sim_compute_sim_state": 0.01863038937250773, "sim_compute_performance-ego": 0.00587216854095459, "sim_compute_robot_state-ego": 0.010679805278778076, "sim_compute_robot_state-parked0": 0.009997459252675374}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 1.9399703054088389, "get_ui_image": 0.052960805892944336, "step_physics": 0.13907196362813315, "survival_time": 14.950000000000076, "driven_lanedir": 0.9382461600776024, "get_state_dump": 0.015591518878936768, "sim_render-ego": 0.006640291213989258, "in-drivable-lane": 7.450000000000096, "agent_compute-ego": 0.024035592873891194, "deviation-heading": 1.1225785928187535, "complete-iteration": 0.2884824546178182, "set_robot_commands": 0.006134403546651204, "deviation-center-line": 0.4106933969501067, "driven_lanedir_consec": 0.9382461600776024, "sim_compute_sim_state": 0.018752049605051675, "sim_compute_performance-ego": 0.005606693426767985, "sim_compute_robot_state-ego": 0.009820098082224529, "sim_compute_robot_state-parked0": 0.00968919277191162}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 1.7673914205993937, "get_ui_image": 0.053417657321189346, "step_physics": 0.12794990155286404, "survival_time": 13.65000000000006, "driven_lanedir": 0.6417866605496498, "get_state_dump": 0.015986225107213953, "sim_render-ego": 0.006868483819367685, "in-drivable-lane": 8.20000000000007, "agent_compute-ego": 0.02443877681271061, "deviation-heading": 1.0431327458152444, "complete-iteration": 0.2741021795587225, "set_robot_commands": 0.0063813029628096915, "deviation-center-line": 0.1706752999063784, "driven_lanedir_consec": 0.6417866605496498, "sim_compute_sim_state": 0.013856657258756868, "sim_compute_performance-ego": 0.005501305227314597, "sim_compute_robot_state-ego": 0.009653705380338452, "sim_compute_robot_state-parked0": 0.00986203256544176}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 1.7209216810089898, "get_ui_image": 0.05206557503320221, "step_physics": 0.13146565283151498, "survival_time": 13.300000000000054, "driven_lanedir": 0.6114560739771817, "get_state_dump": 0.015980978657428483, "sim_render-ego": 0.00665604709682608, "in-drivable-lane": 8.150000000000064, "agent_compute-ego": 0.02457364609366969, "deviation-heading": 1.015682903311432, "complete-iteration": 0.2771127734865461, "set_robot_commands": 0.006307367095373627, "deviation-center-line": 0.20857072800285553, "driven_lanedir_consec": 0.6114560739771817, "sim_compute_sim_state": 0.01423449444591551, "sim_compute_performance-ego": 0.0055938878453763805, "sim_compute_robot_state-ego": 0.010178873413487484, "sim_compute_robot_state-parked0": 0.009869131826816644}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 1.9399630110501775, "get_ui_image": 0.05173485040664673, "step_physics": 0.14867633660634358, "survival_time": 14.950000000000076, "driven_lanedir": 0.8798752097207525, "get_state_dump": 0.01603162129720052, "sim_render-ego": 0.006665163040161133, "in-drivable-lane": 7.800000000000093, "agent_compute-ego": 0.024612236022949215, "deviation-heading": 1.1870512201337824, "complete-iteration": 0.2977997414271037, "set_robot_commands": 0.006179940700531006, "deviation-center-line": 0.4115590914760112, "driven_lanedir_consec": 0.8798752097207525, "sim_compute_sim_state": 0.01857996304829915, "sim_compute_performance-ego": 0.00554440180460612, "sim_compute_robot_state-ego": 0.010072696208953857, "sim_compute_robot_state-parked0": 0.009514077504475912}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 1.939972658409679, "get_ui_image": 0.04965022325515747, "step_physics": 0.12973061084747314, "survival_time": 14.950000000000076, "driven_lanedir": 0.9410197569287384, "get_state_dump": 0.01694191058476766, "sim_render-ego": 0.007246038913726807, "in-drivable-lane": 7.250000000000093, "agent_compute-ego": 0.02585445562998454, "deviation-heading": 1.3332987562593277, "complete-iteration": 0.28335156361262004, "set_robot_commands": 0.006690505345662435, "deviation-center-line": 0.4662213296226701, "driven_lanedir_consec": 0.9410197569287384, "sim_compute_sim_state": 0.019793455600738526, "sim_compute_performance-ego": 0.006047269503275553, "sim_compute_robot_state-ego": 0.010846951802571614, "sim_compute_robot_state-parked0": 0.010352732340494792}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 1.939990017341495, "get_ui_image": 0.04973316351572672, "step_physics": 0.11154375791549682, "survival_time": 14.950000000000076, "driven_lanedir": 0.9992954384693876, "get_state_dump": 0.017458664576212566, "sim_render-ego": 0.007289435068766276, "in-drivable-lane": 6.950000000000098, "agent_compute-ego": 0.026447460651397706, "deviation-heading": 1.3573685661633146, "complete-iteration": 0.2660157060623169, "set_robot_commands": 0.006879388491312663, "deviation-center-line": 0.5225753297978097, "driven_lanedir_consec": 0.9992954384693876, "sim_compute_sim_state": 0.01812969446182251, "sim_compute_performance-ego": 0.00623290220896403, "sim_compute_robot_state-ego": 0.011228023370107016, "sim_compute_robot_state-parked0": 0.010869510968526204}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 1.9398643866680063, "get_ui_image": 0.052304015159606934, "step_physics": 0.12891005198160807, "survival_time": 14.950000000000076, "driven_lanedir": 1.348155951939102, "get_state_dump": 0.016583176453908284, "sim_render-ego": 0.006854215463002523, "in-drivable-lane": 3.100000000000044, "agent_compute-ego": 0.024409003257751465, "deviation-heading": 4.524954561191712, "complete-iteration": 0.27871544361114503, "set_robot_commands": 0.006340947945912679, "deviation-center-line": 0.9378023746981936, "driven_lanedir_consec": 1.280717656309906, "sim_compute_sim_state": 0.017269718647003173, "sim_compute_performance-ego": 0.005721133550008138, "sim_compute_robot_state-ego": 0.010318346818288167, "sim_compute_robot_state-parked0": 0.009819842179616293}}
set_robot_commands_max0.006879388491312663
set_robot_commands_mean0.006379685391896213
set_robot_commands_median0.006324157520643153
set_robot_commands_min0.006121911207834879
sim_compute_performance-ego_max0.00623290220896403
sim_compute_performance-ego_mean0.005744705423169774
sim_compute_performance-ego_median0.005663646062215169
sim_compute_performance-ego_min0.005501305227314597
sim_compute_robot_state-ego_max0.011228023370107016
sim_compute_robot_state-ego_mean0.010242665002312483
sim_compute_robot_state-ego_median0.010125784811220673
sim_compute_robot_state-ego_min0.009653705380338452
sim_compute_robot_state-parked0_max0.010869510968526204
sim_compute_robot_state-parked0_mean0.009936469755215015
sim_compute_robot_state-parked0_median0.009840937372529026
sim_compute_robot_state-parked0_min0.009514077504475912
sim_compute_sim_state_max0.02064635992050171
sim_compute_sim_state_mean0.017903258879014193
sim_compute_sim_state_median0.01860517621040344
sim_compute_sim_state_min0.013856657258756868
sim_render-ego_max0.007289435068766276
sim_render-ego_mean0.006851676880198681
sim_render-ego_median0.006759689251581828
sim_render-ego_min0.006626376310984294
simulation-passed1
step_physics_max0.14867633660634358
step_physics_mean0.13325651854077858
step_physics_median0.13458952270354843
step_physics_min0.11154375791549682
survival_time_max14.950000000000076
survival_time_mean14.655000000000072
survival_time_min13.300000000000054
No reset possible
335918259Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:17
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335878187Bea Baselines 🐤template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:16
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335848203Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:16
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335798123Bea Baselines 🐤template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:00
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335748147Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:04
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335718083Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:01
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335688091Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:18
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335658028Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:20
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335638035Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:14
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335627971Bea Baselines 🐤template-randomaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:06
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335607979Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:01
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335577915Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:15
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335157487Bea Baselines 🐤baseline-duckietownaido5-LF-sim-validationLFv-simsuccessnoreg12-cf110c084bc5-10:24:51
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.8745650551640242
survival_time_median14.950000000000076
deviation-center-line_median0.44697580273707493
in-drivable-lane_median6.800000000000072


other stats
agent_compute-ego_max0.028877816775429572
agent_compute-ego_mean0.027338434074128014
agent_compute-ego_median0.02723678946495056
agent_compute-ego_min0.02640428781509399
complete-iteration_max0.29530038833618166
complete-iteration_mean0.2842445326614491
complete-iteration_median0.2871713674068451
complete-iteration_min0.26947465280848226
deviation-center-line_max0.937134297074136
deviation-center-line_mean0.4791617666011934
deviation-center-line_min0.1883152205178527
deviation-heading_max2.785824926297592
deviation-heading_mean1.7011927403386118
deviation-heading_median1.6574618009090447
deviation-heading_min1.0188285210235903
driven_any_max1.9399799015736483
driven_any_mean1.8815869476956328
driven_any_median1.939968151738708
driven_any_min1.6412570351869302
driven_lanedir_consec_max1.240907363407739
driven_lanedir_consec_mean0.8927902496155156
driven_lanedir_consec_min0.61189209388601
driven_lanedir_max1.4912355684931695
driven_lanedir_mean0.921216692160544
driven_lanedir_median0.8814278479556906
driven_lanedir_min0.61189209388601
get_state_dump_max0.01887788290179657
get_state_dump_mean0.017731089344240582
get_state_dump_median0.017725929419199628
get_state_dump_min0.016606872081756593
get_ui_image_max0.052564573287963864
get_ui_image_mean0.05095114172355121
get_ui_image_median0.05074345827102661
get_ui_image_min0.04931914806365967
in-drivable-lane_max9.800000000000088
in-drivable-lane_mean6.8150000000000714
in-drivable-lane_min2.80000000000004
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.9336699665309252, "get_ui_image": 0.052564573287963864, "step_physics": 0.12944674650828045, "survival_time": 14.950000000000076, "driven_lanedir": 1.2299360390014609, "get_state_dump": 0.016606872081756593, "sim_render-ego": 0.007408782641092937, "in-drivable-lane": 4.450000000000063, "agent_compute-ego": 0.026541141668955485, "deviation-heading": 2.785824926297592, "complete-iteration": 0.2875490045547485, "set_robot_commands": 0.006638685862223308, "deviation-center-line": 0.6148793490769981, "driven_lanedir_consec": 1.2299360390014609, "sim_compute_sim_state": 0.021101977825164795, "sim_compute_performance-ego": 0.006126563549041748, "sim_compute_robot_state-ego": 0.010604445139567058, "sim_compute_robot_state-parked0": 0.01029534101486206}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 1.939921133996476, "get_ui_image": 0.05199186642964681, "step_physics": 0.13055991729100544, "survival_time": 14.950000000000076, "driven_lanedir": 1.4912355684931695, "get_state_dump": 0.01672150452931722, "sim_render-ego": 0.00717629591623942, "in-drivable-lane": 2.80000000000004, "agent_compute-ego": 0.02640428781509399, "deviation-heading": 2.568178371677761, "complete-iteration": 0.28824415763219197, "set_robot_commands": 0.006647392908732097, "deviation-center-line": 0.937134297074136, "driven_lanedir_consec": 1.240907363407739, "sim_compute_sim_state": 0.02150288184483846, "sim_compute_performance-ego": 0.006148858070373535, "sim_compute_robot_state-ego": 0.010413831869761149, "sim_compute_robot_state-parked0": 0.010462775230407714}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 1.9399797419338576, "get_ui_image": 0.05231675386428833, "step_physics": 0.13784092028935752, "survival_time": 14.950000000000076, "driven_lanedir": 0.6240963911501214, "get_state_dump": 0.017166769504547118, "sim_render-ego": 0.007580962181091309, "in-drivable-lane": 9.800000000000088, "agent_compute-ego": 0.026903332869211833, "deviation-heading": 1.0449472996386349, "complete-iteration": 0.29530038833618166, "set_robot_commands": 0.0067861660321553545, "deviation-center-line": 0.3198942984264824, "driven_lanedir_consec": 0.6240963911501214, "sim_compute_sim_state": 0.019026947021484376, "sim_compute_performance-ego": 0.006100153923034668, "sim_compute_robot_state-ego": 0.010919561386108398, "sim_compute_robot_state-parked0": 0.010443904399871829}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 1.9399767870530924, "get_ui_image": 0.05089849472045899, "step_physics": 0.1251549251874288, "survival_time": 14.950000000000076, "driven_lanedir": 0.6934375608634213, "get_state_dump": 0.017753353118896486, "sim_render-ego": 0.007583342393239339, "in-drivable-lane": 8.850000000000083, "agent_compute-ego": 0.027457240422566732, "deviation-heading": 1.8006921925352417, "complete-iteration": 0.2836643894513448, "set_robot_commands": 0.006696782112121582, "deviation-center-line": 0.3876624011382303, "driven_lanedir_consec": 0.6891915124328842, "sim_compute_sim_state": 0.01932031154632568, "sim_compute_performance-ego": 0.0065165861447652185, "sim_compute_robot_state-ego": 0.011256364186604818, "sim_compute_robot_state-parked0": 0.01080418348312378}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 1.6611732120219131, "get_ui_image": 0.050445236584555776, "step_physics": 0.10937065280365108, "survival_time": 12.850000000000048, "driven_lanedir": 0.61189209388601, "get_state_dump": 0.01887788290179657, "sim_render-ego": 0.008047773680334425, "in-drivable-lane": 7.700000000000058, "agent_compute-ego": 0.028877816775429572, "deviation-heading": 1.0269837935184505, "complete-iteration": 0.27045613504105503, "set_robot_commands": 0.007743558994990843, "deviation-center-line": 0.22108611832996403, "driven_lanedir_consec": 0.61189209388601, "sim_compute_sim_state": 0.016023534745093913, "sim_compute_performance-ego": 0.006781916674009093, "sim_compute_robot_state-ego": 0.012305993514302177, "sim_compute_robot_state-parked0": 0.011743500074987746}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 1.6412570351869302, "get_ui_image": 0.050053878093329, "step_physics": 0.11075303216618816, "survival_time": 12.700000000000044, "driven_lanedir": 0.6139672165924052, "get_state_dump": 0.01852196689665787, "sim_render-ego": 0.008117685167808232, "in-drivable-lane": 7.500000000000056, "agent_compute-ego": 0.02827239787484717, "deviation-heading": 1.0188285210235903, "complete-iteration": 0.26947465280848226, "set_robot_commands": 0.00751065269229919, "deviation-center-line": 0.1883152205178527, "driven_lanedir_consec": 0.6139672165924052, "sim_compute_sim_state": 0.01606450400014562, "sim_compute_performance-ego": 0.006665070225873332, "sim_compute_robot_state-ego": 0.011799087674598994, "sim_compute_robot_state-parked0": 0.011480469403304452}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 1.939971873724194, "get_ui_image": 0.04960667371749878, "step_physics": 0.1283096194267273, "survival_time": 14.950000000000076, "driven_lanedir": 1.06941813504796, "get_state_dump": 0.018392579555511476, "sim_render-ego": 0.007512040932973226, "in-drivable-lane": 6.050000000000086, "agent_compute-ego": 0.02760146697362264, "deviation-heading": 1.7672637251684566, "complete-iteration": 0.288284289042155, "set_robot_commands": 0.007215201854705811, "deviation-center-line": 0.5715700589641409, "driven_lanedir_consec": 1.059938597895164, "sim_compute_sim_state": 0.020294396877288817, "sim_compute_performance-ego": 0.00648717721303304, "sim_compute_robot_state-ego": 0.011635618209838869, "sim_compute_robot_state-parked0": 0.011008687019348144}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 1.9399753951820689, "get_ui_image": 0.04931914806365967, "step_physics": 0.1331538200378418, "survival_time": 14.950000000000076, "driven_lanedir": 1.0786431833092616, "get_state_dump": 0.017628442446390787, "sim_render-ego": 0.0076210546493530274, "in-drivable-lane": 6.100000000000087, "agent_compute-ego": 0.02701633850733439, "deviation-heading": 1.5476598766496328, "complete-iteration": 0.289560112953186, "set_robot_commands": 0.006850675741831461, "deviation-center-line": 0.5062892043359196, "driven_lanedir_consec": 1.0706858817345108, "sim_compute_sim_state": 0.019501514434814453, "sim_compute_performance-ego": 0.0063306268056233725, "sim_compute_robot_state-ego": 0.01123285134633382, "sim_compute_robot_state-parked0": 0.010687520503997804}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 1.939964429753222, "get_ui_image": 0.05058842182159424, "step_physics": 0.12481924533843994, "survival_time": 14.950000000000076, "driven_lanedir": 1.145097525906564, "get_state_dump": 0.01794301668802897, "sim_render-ego": 0.007720938523610433, "in-drivable-lane": 5.4500000000000774, "agent_compute-ego": 0.027481799920399983, "deviation-heading": 2.1240591550901, "complete-iteration": 0.28311846653620404, "set_robot_commands": 0.007162882486979167, "deviation-center-line": 0.6845112597191767, "driven_lanedir_consec": 1.1328441926997952, "sim_compute_sim_state": 0.01841085831324259, "sim_compute_performance-ego": 0.006494682629903157, "sim_compute_robot_state-ego": 0.011347219149271648, "sim_compute_robot_state-parked0": 0.010923160711924234}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 1.9399799015736483, "get_ui_image": 0.05172637065251668, "step_physics": 0.12917718648910523, "survival_time": 14.950000000000076, "driven_lanedir": 0.6544432073550652, "get_state_dump": 0.017698505719502766, "sim_render-ego": 0.00767945925394694, "in-drivable-lane": 9.450000000000088, "agent_compute-ego": 0.02682851791381836, "deviation-heading": 1.327489541786658, "complete-iteration": 0.28679373025894167, "set_robot_commands": 0.00694275697072347, "deviation-center-line": 0.36027545842903325, "driven_lanedir_consec": 0.6544432073550652, "sim_compute_sim_state": 0.017883315881093344, "sim_compute_performance-ego": 0.006380327542622884, "sim_compute_robot_state-ego": 0.01123362143834432, "sim_compute_robot_state-parked0": 0.011027928193410237}}
set_robot_commands_max0.007743558994990843
set_robot_commands_mean0.007019475565676227
set_robot_commands_median0.006896716356277466
set_robot_commands_min0.006638685862223308
sim_compute_performance-ego_max0.006781916674009093
sim_compute_performance-ego_mean0.006403196277828005
sim_compute_performance-ego_median0.0064337523778279625
sim_compute_performance-ego_min0.006100153923034668
sim_compute_robot_state-ego_max0.012305993514302177
sim_compute_robot_state-ego_mean0.011274859391473125
sim_compute_robot_state-ego_median0.011244992812474567
sim_compute_robot_state-ego_min0.010413831869761149
sim_compute_robot_state-parked0_max0.011743500074987746
sim_compute_robot_state-parked0_mean0.0108877470035238
sim_compute_robot_state-parked0_median0.010863672097524008
sim_compute_robot_state-parked0_min0.01029534101486206
sim_compute_sim_state_max0.02150288184483846
sim_compute_sim_state_mean0.01891302424894921
sim_compute_sim_state_median0.01917362928390503
sim_compute_sim_state_min0.016023534745093913
sim_render-ego_max0.008117685167808232
sim_render-ego_mean0.00764483353396893
sim_render-ego_median0.007602198521296183
sim_render-ego_min0.00717629591623942
simulation-passed1
step_physics_max0.13784092028935752
step_physics_mean0.12585860655380257
step_physics_median0.12874340295791625
step_physics_min0.10937065280365108
survival_time_max14.950000000000076
survival_time_mean14.515000000000072
survival_time_min12.700000000000044
No reset possible
335137479Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:49
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335117431Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:46
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335097435Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:04
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335077383Bea Baselines 🐤baseline-duckietownaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:43
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335057387Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:03
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335037319Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:00:45
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335017327Bea Baselines 🐤baseline-duckietownaido5-LF-sim-validationLFv-simerrornoreg12-cf110c084bc5-10:01:01
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334997251Bea Baselines 🐤template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:03
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334957275Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:34
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334897179Bea Baselines 🐤template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:18
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334787171Bea Baselines 🐤template-tensorflowaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:30
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334747163Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg12-cf110c084bc5-10:01:15
The container "solut [...]
The container "solution-ego0" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334566845Daniil Lisusexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg12-cf110c084bc5-10:24:16
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.6634424388576514
survival_time_median14.950000000000076
deviation-center-line_median0.3239534079817444
in-drivable-lane_median1.8750000000000109


other stats
agent_compute-ego_max0.02736552556355794
agent_compute-ego_mean0.026245057906333635
agent_compute-ego_median0.026280057390695784
agent_compute-ego_min0.0249635989849384
complete-iteration_max0.31929617661696214
complete-iteration_mean0.30335339632795233
complete-iteration_median0.30516672423545355
complete-iteration_min0.27963650512695315
deviation-center-line_max0.7571995093101491
deviation-center-line_mean0.4418246382780179
deviation-center-line_min0.21037266987450565
deviation-heading_max4.862235691897165
deviation-heading_mean2.2514386368496053
deviation-heading_median1.4513765729251342
deviation-heading_min0.6565084328562105
driven_any_max3.2581582236132167
driven_any_mean2.272450238029966
driven_any_median2.2181090865148203
driven_any_min1.522420859359339
driven_lanedir_consec_max2.517426708049721
driven_lanedir_consec_mean1.7480038471334236
driven_lanedir_consec_min1.1960860602753804
driven_lanedir_max2.517426708049721
driven_lanedir_mean1.7580976251325748
driven_lanedir_median1.6634424388576514
driven_lanedir_min1.205339879716095
get_state_dump_max0.019078445434570313
get_state_dump_mean0.01795432614547128
get_state_dump_median0.017976939280827837
get_state_dump_min0.017031665802001952
get_ui_image_max0.051622707843780515
get_ui_image_mean0.050364895407445154
get_ui_image_median0.050428831015918274
get_ui_image_min0.0495547350247701
in-drivable-lane_max4.25000000000006
in-drivable-lane_mean2.070000000000018
in-drivable-lane_min0.0
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.6064006261383987, "get_ui_image": 0.05057143449783325, "step_physics": 0.1465280548731486, "survival_time": 14.950000000000076, "driven_lanedir": 1.2970238402668917, "get_state_dump": 0.01800740321477254, "sim_render-ego": 0.00755476713180542, "in-drivable-lane": 1.9500000000000275, "agent_compute-ego": 0.027163334687550864, "deviation-heading": 3.3201019373709957, "complete-iteration": 0.3056198644638062, "set_robot_commands": 0.006873358090718588, "deviation-center-line": 0.5346783710705325, "driven_lanedir_consec": 1.1960860602753804, "sim_compute_sim_state": 0.020475498835245767, "sim_compute_performance-ego": 0.006270072460174561, "sim_compute_robot_state-ego": 0.011474831104278565, "sim_compute_robot_state-parked0": 0.010486424763997396}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 1.522420859359339, "get_ui_image": 0.05042531394958496, "step_physics": 0.12644690895080565, "survival_time": 6.249999999999986, "driven_lanedir": 1.205339879716095, "get_state_dump": 0.017031665802001952, "sim_render-ego": 0.007008323669433594, "in-drivable-lane": 0.8999999999999968, "agent_compute-ego": 0.02512420082092285, "deviation-heading": 0.6565084328562105, "complete-iteration": 0.27963650512695315, "set_robot_commands": 0.006514183044433594, "deviation-center-line": 0.21037266987450565, "driven_lanedir_consec": 1.205339879716095, "sim_compute_sim_state": 0.02034787940979004, "sim_compute_performance-ego": 0.005897640228271484, "sim_compute_robot_state-ego": 0.010555206298828125, "sim_compute_robot_state-parked0": 0.010080028533935546}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 2.8414262943207547, "get_ui_image": 0.05069143544588841, "step_physics": 0.14043594989539182, "survival_time": 12.050000000000036, "driven_lanedir": 1.951242783926841, "get_state_dump": 0.01772484344070878, "sim_render-ego": 0.007310636805300891, "in-drivable-lane": 3.0500000000000433, "agent_compute-ego": 0.02623149863911862, "deviation-heading": 1.280258349770251, "complete-iteration": 0.29700407249798916, "set_robot_commands": 0.006790929810140143, "deviation-center-line": 0.3021895722898456, "driven_lanedir_consec": 1.951242783926841, "sim_compute_sim_state": 0.02023404564600268, "sim_compute_performance-ego": 0.00619084013942861, "sim_compute_robot_state-ego": 0.011062021572065551, "sim_compute_robot_state-parked0": 0.01012053034612252}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 2.022631534137971, "get_ui_image": 0.0495547350247701, "step_physics": 0.15486740430196128, "survival_time": 14.950000000000076, "driven_lanedir": 1.4630224727782766, "get_state_dump": 0.019078445434570313, "sim_render-ego": 0.0073347036043802894, "in-drivable-lane": 4.25000000000006, "agent_compute-ego": 0.02638953685760498, "deviation-heading": 1.493998529139014, "complete-iteration": 0.3118005355199178, "set_robot_commands": 0.006394302845001221, "deviation-center-line": 0.34571724367364315, "driven_lanedir_consec": 1.4630224727782766, "sim_compute_sim_state": 0.02001519521077474, "sim_compute_performance-ego": 0.006359920501708984, "sim_compute_robot_state-ego": 0.01173007567723592, "sim_compute_robot_state-parked0": 0.009859748681386312}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 2.147696301657968, "get_ui_image": 0.05013676007588704, "step_physics": 0.156250634988149, "survival_time": 14.950000000000076, "driven_lanedir": 1.4711498499234184, "get_state_dump": 0.018523844877878823, "sim_render-ego": 0.00734493891398112, "in-drivable-lane": 2.900000000000003, "agent_compute-ego": 0.026925564606984455, "deviation-heading": 4.268163765123751, "complete-iteration": 0.30984416643778484, "set_robot_commands": 0.006378382047017415, "deviation-center-line": 0.7490025053981202, "driven_lanedir_consec": 1.4711498499234184, "sim_compute_sim_state": 0.016718459924062092, "sim_compute_performance-ego": 0.006243032614390055, "sim_compute_robot_state-ego": 0.011201943556467692, "sim_compute_robot_state-parked0": 0.0099056609471639}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 1.7851064963376069, "get_ui_image": 0.04984960635503133, "step_physics": 0.15227418740590412, "survival_time": 14.950000000000076, "driven_lanedir": 1.4332659414925517, "get_state_dump": 0.0183415420850118, "sim_render-ego": 0.007425537904103597, "in-drivable-lane": 1.7999999999999936, "agent_compute-ego": 0.02736552556355794, "deviation-heading": 4.862235691897165, "complete-iteration": 0.30576605240503946, "set_robot_commands": 0.006364326477050781, "deviation-center-line": 0.7165041507256134, "driven_lanedir_consec": 1.4332659414925517, "sim_compute_sim_state": 0.01646900812784831, "sim_compute_performance-ego": 0.006193850040435791, "sim_compute_robot_state-ego": 0.011277727286020917, "sim_compute_robot_state-parked0": 0.009991200764973958}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 2.2885218713716724, "get_ui_image": 0.04980383163843399, "step_physics": 0.16439507404963177, "survival_time": 7.79999999999998, "driven_lanedir": 1.8557350277918845, "get_state_dump": 0.01805213170173841, "sim_render-ego": 0.007298133312127529, "in-drivable-lane": 1.5499999999999945, "agent_compute-ego": 0.0249635989849384, "deviation-heading": 0.704409725468919, "complete-iteration": 0.31929617661696214, "set_robot_commands": 0.0062895065698868186, "deviation-center-line": 0.2501301754369102, "driven_lanedir_consec": 1.8557350277918845, "sim_compute_sim_state": 0.02107885403510852, "sim_compute_performance-ego": 0.006205263810280042, "sim_compute_robot_state-ego": 0.011304019353328606, "sim_compute_robot_state-parked0": 0.009699526505592542}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 2.8593305196807575, "get_ui_image": 0.05043234808225158, "step_physics": 0.14813684233536956, "survival_time": 14.100000000000064, "driven_lanedir": 2.0174201146548456, "get_state_dump": 0.017508013028625057, "sim_render-ego": 0.007417438723516802, "in-drivable-lane": 2.80000000000004, "agent_compute-ego": 0.02599955196921707, "deviation-heading": 1.4087546167112543, "complete-iteration": 0.30471358400710086, "set_robot_commands": 0.0066605222986099565, "deviation-center-line": 0.3009090359379411, "driven_lanedir_consec": 2.0174201146548456, "sim_compute_sim_state": 0.020695131721226036, "sim_compute_performance-ego": 0.006247971920256919, "sim_compute_robot_state-ego": 0.011179643319853654, "sim_compute_robot_state-parked0": 0.010223940754613131}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 2.392809653681979, "get_ui_image": 0.050560781160990394, "step_physics": 0.143872283299764, "survival_time": 14.950000000000076, "driven_lanedir": 2.3693496327252213, "get_state_dump": 0.017946475346883137, "sim_render-ego": 0.007410784562428792, "in-drivable-lane": 0.0, "agent_compute-ego": 0.02632861614227295, "deviation-heading": 3.3960339319828607, "complete-iteration": 0.30065122842788694, "set_robot_commands": 0.006676926612854004, "deviation-center-line": 0.7571995093101491, "driven_lanedir_consec": 2.3693496327252213, "sim_compute_sim_state": 0.019669408003489177, "sim_compute_performance-ego": 0.006387697060902913, "sim_compute_robot_state-ego": 0.011526474157969155, "sim_compute_robot_state-parked0": 0.010058302879333497}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 3.2581582236132167, "get_ui_image": 0.051622707843780515, "step_physics": 0.14410522063573203, "survival_time": 14.950000000000076, "driven_lanedir": 2.517426708049721, "get_state_dump": 0.017328896522521973, "sim_render-ego": 0.007277379035949707, "in-drivable-lane": 1.500000000000021, "agent_compute-ego": 0.025959150791168215, "deviation-heading": 1.1239213881756358, "complete-iteration": 0.29920177777608237, "set_robot_commands": 0.006661884784698487, "deviation-center-line": 0.2515431490629187, "driven_lanedir_consec": 2.517426708049721, "sim_compute_sim_state": 0.018732855319976805, "sim_compute_performance-ego": 0.0060963733990987146, "sim_compute_robot_state-ego": 0.010977959632873536, "sim_compute_robot_state-parked0": 0.010228984355926514}}
set_robot_commands_max0.006873358090718588
set_robot_commands_mean0.006560432258041102
set_robot_commands_median0.006587352671521775
set_robot_commands_min0.0062895065698868186
sim_compute_performance-ego_max0.006387697060902913
sim_compute_performance-ego_mean0.006209266217494807
sim_compute_performance-ego_median0.0062241482123350485
sim_compute_performance-ego_min0.005897640228271484
sim_compute_robot_state-ego_max0.01173007567723592
sim_compute_robot_state-ego_mean0.011228990195892172
sim_compute_robot_state-ego_median0.011239835421244303
sim_compute_robot_state-ego_min0.010555206298828125
sim_compute_robot_state-parked0_max0.010486424763997396
sim_compute_robot_state-parked0_mean0.010065434853304533
sim_compute_robot_state-parked0_median0.01006916570663452
sim_compute_robot_state-parked0_min0.009699526505592542
sim_compute_sim_state_max0.02107885403510852
sim_compute_sim_state_mean0.019443633623352417
sim_compute_sim_state_median0.02012462042838871
sim_compute_sim_state_min0.01646900812784831
sim_render-ego_max0.00755476713180542
sim_render-ego_mean0.0073382643663027726
sim_render-ego_median0.007339821259180704
sim_render-ego_min0.007008323669433594
simulation-passed1
step_physics_max0.16439507404963177
step_physics_mean0.14773125607358578
step_physics_median0.1473324486042591
step_physics_min0.12644690895080565
survival_time_max14.950000000000076
survival_time_mean12.990000000000052
survival_time_min6.249999999999986
No reset possible