Duckietown Challenges Home Challenges Submissions

Evaluator 1883

ID1883
evaluatorreg06-0314bee6766e-1
ownerI don't have one 😀
machinearchimede
processreg06-0314bee6766e-1
versiond-c:6.1.5;d-c-r:6.0.29
first heard
last heard
statusinactive
# evaluating
# success4 33466
# timeout
# failed
# error12 33477
# aborted
# host-error2 33662
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
336709079Bea Baselines 🐤straightaido5-LF-sim-validationLFv-simhost-errornoreg06-0314bee6766e-10:00:01
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 628, in get_cr
    os.makedirs(wd)
  File "/usr/lib/python3.8/os.py", line 213, in makedirs
    makedirs(head, exist_ok=exist_ok)
  File "/usr/lib/python3.8/os.py", line 223, in makedirs
    mkdir(name, mode)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission9079'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336628991Bea Baselines 🐤straightaido5-LF-sim-validationLFv-simhost-errornoreg06-0314bee6766e-10:05:33
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 673, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 1402, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner.py", line 1650, in upload
    copy_to_cache(realfile, sha256hex)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner_daffy-6.0.29-py3.8.egg/duckietown_challenges_runner/runner_cache.py", line 42, in copy_to_cache
    shutil.copy(fn, have)
  File "/usr/lib/python3.8/shutil.py", line 415, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 272, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/usr/lib/python3.8/shutil.py", line 163, in _fastcopy_sendfile
    raise err from None
  File "/usr/lib/python3.8/shutil.py", line 149, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 28] No space left on device: '/tmp/duckietown/DT18/evaluator/executions/aido5-LF-sim-validation/submission8991/LFv-sim-reg06-0314bee6766e-1-job33662/docker-compose.original.yaml' -> '/tmp/duckietown/DT18/evaluator/cache/by-value/sha256hex/a62c363a0880f8b7f59c0cb0afca93d8fcb88021eefda6982477fdeb4c5294ef'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336599011Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:29
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336558951Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg06-0314bee6766e-10:00:44
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336098471Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simsuccessnoreg06-0314bee6766e-10:25:09
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.9930753629457056
survival_time_median14.950000000000076
deviation-center-line_median0.4674469409107297
in-drivable-lane_median6.875000000000094


other stats
agent_compute-ego_max0.027147920926411943
agent_compute-ego_mean0.025970863113278307
agent_compute-ego_median0.025783789343508667
agent_compute-ego_min0.02527988115946452
complete-iteration_max0.29806735992431643
complete-iteration_mean0.2866469622439644
complete-iteration_median0.29062003890673316
complete-iteration_min0.26862713813781736
deviation-center-line_max1.020336560998153
deviation-center-line_mean0.5110894213190778
deviation-center-line_min0.18957859065760824
deviation-heading_max4.261182732468263
deviation-heading_mean1.757290088796203
deviation-heading_median1.418878013276471
deviation-heading_min1.00472747621318
driven_any_max1.939982430940878
driven_any_mean1.900828558615821
driven_any_median1.939947503291268
driven_any_min1.6744501849516802
driven_lanedir_consec_max1.4853654281783113
driven_lanedir_consec_mean0.987271839391184
driven_lanedir_consec_min0.6101978437936255
driven_lanedir_max1.6291504424496417
driven_lanedir_mean1.00949233605975
driven_lanedir_median0.9930753629457056
driven_lanedir_min0.6101978437936255
get_state_dump_max0.017865317662556966
get_state_dump_mean0.01650594724522179
get_state_dump_median0.016245097716649372
get_state_dump_min0.016012140115102134
get_ui_image_max0.05435611014200454
get_ui_image_mean0.0521991287638918
get_ui_image_median0.05247063716252645
get_ui_image_min0.04884436209996541
in-drivable-lane_max8.700000000000076
in-drivable-lane_mean6.315000000000076
in-drivable-lane_min0.9500000000000136
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.9336336038885336, "get_ui_image": 0.051577095985412595, "step_physics": 0.1330711317062378, "survival_time": 14.950000000000076, "driven_lanedir": 0.9560525098437468, "get_state_dump": 0.01637683629989624, "sim_render-ego": 0.007054494222005208, "in-drivable-lane": 7.250000000000097, "agent_compute-ego": 0.02566341479619344, "deviation-heading": 1.1735327738491663, "complete-iteration": 0.28715218146642046, "set_robot_commands": 0.006333903471628825, "deviation-center-line": 0.4089260349259277, "driven_lanedir_consec": 0.9560525098437468, "sim_compute_sim_state": 0.020848474502563476, "sim_compute_performance-ego": 0.005831715265909831, "sim_compute_robot_state-ego": 0.010160186290740968, "sim_compute_robot_state-parked0": 0.010043989817301433}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 1.9399354487345424, "get_ui_image": 0.05316001892089844, "step_physics": 0.14084986686706544, "survival_time": 14.950000000000076, "driven_lanedir": 1.6291504424496417, "get_state_dump": 0.01623455286026001, "sim_render-ego": 0.0068973064422607425, "in-drivable-lane": 0.9500000000000136, "agent_compute-ego": 0.02527988115946452, "deviation-heading": 4.261182732468263, "complete-iteration": 0.2948783644040426, "set_robot_commands": 0.006460875670115153, "deviation-center-line": 1.020336560998153, "driven_lanedir_consec": 1.4853654281783113, "sim_compute_sim_state": 0.020283849239349366, "sim_compute_performance-ego": 0.005728018283843994, "sim_compute_robot_state-ego": 0.00996906598409017, "sim_compute_robot_state-parked0": 0.009833961327870688}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 1.939959880106852, "get_ui_image": 0.04999591827392578, "step_physics": 0.11875320196151734, "survival_time": 14.950000000000076, "driven_lanedir": 0.8988898190499999, "get_state_dump": 0.01736413796742757, "sim_render-ego": 0.007317634423573812, "in-drivable-lane": 7.700000000000095, "agent_compute-ego": 0.02709028959274292, "deviation-heading": 1.3617009690784931, "complete-iteration": 0.27411901315053305, "set_robot_commands": 0.006903947989145915, "deviation-center-line": 0.42724237349611494, "driven_lanedir_consec": 0.8988898190499999, "sim_compute_sim_state": 0.018926126956939696, "sim_compute_performance-ego": 0.006053164005279541, "sim_compute_robot_state-ego": 0.01094543218612671, "sim_compute_robot_state-parked0": 0.010572798252105713}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 1.9399183912573532, "get_ui_image": 0.04884436209996541, "step_physics": 0.11305344978968304, "survival_time": 14.950000000000076, "driven_lanedir": 1.151959609351772, "get_state_dump": 0.017865317662556966, "sim_render-ego": 0.00744503657023112, "in-drivable-lane": 5.500000000000078, "agent_compute-ego": 0.027147920926411943, "deviation-heading": 2.010733419890854, "complete-iteration": 0.26862713813781736, "set_robot_commands": 0.006951931317647298, "deviation-center-line": 0.6324646056097352, "driven_lanedir_consec": 1.103833149815039, "sim_compute_sim_state": 0.018936734994252524, "sim_compute_performance-ego": 0.006238718827565511, "sim_compute_robot_state-ego": 0.01136962095896403, "sim_compute_robot_state-parked0": 0.010565799872080483}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 1.8205018732726304, "get_ui_image": 0.05347501171017033, "step_physics": 0.1296255622470082, "survival_time": 14.050000000000065, "driven_lanedir": 0.6295171801223429, "get_state_dump": 0.016493816816934063, "sim_render-ego": 0.006819486618041992, "in-drivable-lane": 8.700000000000076, "agent_compute-ego": 0.02601127437849486, "deviation-heading": 1.083309185883082, "complete-iteration": 0.27866044248126154, "set_robot_commands": 0.0065492701275917135, "deviation-center-line": 0.1961698698519141, "driven_lanedir_consec": 0.6295171801223429, "sim_compute_sim_state": 0.014021529845919898, "sim_compute_performance-ego": 0.005668490806932551, "sim_compute_robot_state-ego": 0.010049209051709158, "sim_compute_robot_state-parked0": 0.00976435569681731}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 1.6744501849516802, "get_ui_image": 0.05435611014200454, "step_physics": 0.131316648026691, "survival_time": 12.950000000000047, "driven_lanedir": 0.6101978437936255, "get_state_dump": 0.016031271702534444, "sim_render-ego": 0.006997061512184879, "in-drivable-lane": 7.800000000000059, "agent_compute-ego": 0.025904163890823895, "deviation-heading": 1.00472747621318, "complete-iteration": 0.2811571355031724, "set_robot_commands": 0.0066247326987130305, "deviation-center-line": 0.18957859065760824, "driven_lanedir_consec": 0.6101978437936255, "sim_compute_sim_state": 0.0141195247532318, "sim_compute_performance-ego": 0.005645234612424401, "sim_compute_robot_state-ego": 0.009941120405454891, "sim_compute_robot_state-parked0": 0.01003823777423402}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 1.939975935366391, "get_ui_image": 0.05164184888203939, "step_physics": 0.1427052887280782, "survival_time": 14.950000000000076, "driven_lanedir": 0.8648311522860861, "get_state_dump": 0.016225496133168538, "sim_render-ego": 0.006791922251383463, "in-drivable-lane": 7.900000000000093, "agent_compute-ego": 0.02600413163503011, "deviation-heading": 1.2139515050541037, "complete-iteration": 0.2946769134203593, "set_robot_commands": 0.006288852691650391, "deviation-center-line": 0.4174122038083132, "driven_lanedir_consec": 0.8648311522860861, "sim_compute_sim_state": 0.01915326992670695, "sim_compute_performance-ego": 0.005798979600270589, "sim_compute_robot_state-ego": 0.010165647665659586, "sim_compute_robot_state-parked0": 0.009713072776794434}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 1.9399682797913544, "get_ui_image": 0.051999409993489586, "step_physics": 0.14448400338490805, "survival_time": 14.950000000000076, "driven_lanedir": 1.0300982160476644, "get_state_dump": 0.016012140115102134, "sim_render-ego": 0.006734660466512044, "in-drivable-lane": 6.500000000000092, "agent_compute-ego": 0.02554655472437541, "deviation-heading": 1.5250601296480513, "complete-iteration": 0.2950431776046753, "set_robot_commands": 0.006319879690806071, "deviation-center-line": 0.5076515083253444, "driven_lanedir_consec": 1.0300982160476644, "sim_compute_sim_state": 0.01874963442484538, "sim_compute_performance-ego": 0.005542176564534505, "sim_compute_robot_state-ego": 0.009906774361928303, "sim_compute_robot_state-parked0": 0.009566120306650798}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 1.939982430940878, "get_ui_image": 0.05294186433156332, "step_physics": 0.14364439884821573, "survival_time": 14.950000000000076, "driven_lanedir": 1.071348079162972, "get_state_dump": 0.016200260321299235, "sim_render-ego": 0.006789430777231853, "in-drivable-lane": 6.400000000000091, "agent_compute-ego": 0.02541514317194621, "deviation-heading": 1.4760550574744484, "complete-iteration": 0.2940878963470459, "set_robot_commands": 0.00647572914759318, "deviation-center-line": 0.5812715956405317, "driven_lanedir_consec": 1.071348079162972, "sim_compute_sim_state": 0.01681490182876587, "sim_compute_performance-ego": 0.005757997830708822, "sim_compute_robot_state-ego": 0.01010932683944702, "sim_compute_robot_state-parked0": 0.009756869475046791}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 1.9399595578479936, "get_ui_image": 0.05399964729944865, "step_physics": 0.14501750230789184, "survival_time": 14.950000000000076, "driven_lanedir": 1.2528785084896454, "get_state_dump": 0.01625564257303874, "sim_render-ego": 0.006917595863342285, "in-drivable-lane": 4.450000000000063, "agent_compute-ego": 0.02564585685729981, "deviation-heading": 2.4626476384023883, "complete-iteration": 0.29806735992431643, "set_robot_commands": 0.006475107669830322, "deviation-center-line": 0.7298408698771363, "driven_lanedir_consec": 1.2225850156120504, "sim_compute_sim_state": 0.017438480854034422, "sim_compute_performance-ego": 0.005841570695241292, "sim_compute_robot_state-ego": 0.010369830131530762, "sim_compute_robot_state-parked0": 0.009917276700337728}}
set_robot_commands_max0.006951931317647298
set_robot_commands_mean0.006538423047472189
set_robot_commands_median0.006475418408711751
set_robot_commands_min0.006288852691650391
sim_compute_performance-ego_max0.006238718827565511
sim_compute_performance-ego_mean0.005810606649271103
sim_compute_performance-ego_median0.005778488715489706
sim_compute_performance-ego_min0.005542176564534505
sim_compute_robot_state-ego_max0.01136962095896403
sim_compute_robot_state-ego_mean0.01029862138756516
sim_compute_robot_state-ego_median0.010134756565093994
sim_compute_robot_state-ego_min0.009906774361928303
sim_compute_robot_state-parked0_max0.010572798252105713
sim_compute_robot_state-parked0_mean0.00997724819992394
sim_compute_robot_state-parked0_median0.009875619014104208
sim_compute_robot_state-parked0_min0.009566120306650798
sim_compute_sim_state_max0.020848474502563476
sim_compute_sim_state_mean0.01792925273266094
sim_compute_sim_state_median0.01883788069089254
sim_compute_sim_state_min0.014021529845919898
sim_render-ego_max0.00744503657023112
sim_render-ego_mean0.00697646291467674
sim_render-ego_median0.006907451152801514
sim_render-ego_min0.006734660466512044
simulation-passed1
step_physics_max0.14501750230789184
step_physics_mean0.13425210538672966
step_physics_median0.1369604992866516
step_physics_min0.11305344978968304
survival_time_max14.950000000000076
survival_time_mean14.660000000000071
survival_time_min12.950000000000047
No reset possible
336058459Bea Baselines 🐤template-pytorchaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:16
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336038427Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:22
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
336008367Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg06-0314bee6766e-10:00:44
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335968355Bea Baselines 🐤straightaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:03
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335948315Bea Baselines 🐤straightaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:19
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335908259Bea Baselines 🐤baseline-duckietownaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:20
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335888191Bea Baselines 🐤template-rosaido5-LF-sim-validationLFv-simerrornoreg06-0314bee6766e-10:01:00
The container "solut [...]
The container "solution" exited with code 2.


Look at the logs for the container to know more about the error.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
335336843Daniil Lisusexercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg06-0314bee6766e-10:25:38
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.724968550064051
survival_time_median14.950000000000076
deviation-center-line_median0.8004306548865809
in-drivable-lane_median1.7750000000000252


other stats
agent_compute-ego_max0.05581378777821859
agent_compute-ego_mean0.035000357354612065
agent_compute-ego_median0.027545493046442663
agent_compute-ego_min0.025982575785832775
complete-iteration_max0.3061730989245667
complete-iteration_mean0.2881798124371937
complete-iteration_median0.2876865212202875
complete-iteration_min0.2727083420753479
deviation-center-line_max1.0346905064223468
deviation-center-line_mean0.8024269881707013
deviation-center-line_min0.6124353077991213
deviation-heading_max3.1938354114521794
deviation-heading_mean1.7422450037900243
deviation-heading_median1.20364978928002
deviation-heading_min0.8317373692319064
driven_any_max2.173291474310259
driven_any_mean1.937285699147079
driven_any_median2.0235664158550897
driven_any_min1.5415207701002465
driven_lanedir_consec_max2.0693598025735063
driven_lanedir_consec_mean1.6063772779856151
driven_lanedir_consec_min0.9956196620011304
driven_lanedir_max2.0693598025735063
driven_lanedir_mean1.6063772779856151
driven_lanedir_median1.724968550064051
driven_lanedir_min0.9956196620011304
get_state_dump_max0.01824289083480835
get_state_dump_mean0.017663973765237755
get_state_dump_median0.017620593309402466
get_state_dump_min0.016980016231536867
get_ui_image_max0.05174900770187378
get_ui_image_mean0.05057932792754234
get_ui_image_median0.05053258609213558
get_ui_image_min0.04928935050964355
in-drivable-lane_max6.000000000000085
in-drivable-lane_mean2.0900000000000203
in-drivable-lane_min0.0
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.5415207701002465, "get_ui_image": 0.04989517370859782, "step_physics": 0.11369114875793455, "survival_time": 14.950000000000076, "driven_lanedir": 0.9956196620011304, "get_state_dump": 0.017631017367045084, "sim_render-ego": 0.007491623560587565, "in-drivable-lane": 3.450000000000049, "agent_compute-ego": 0.027472689151763915, "deviation-heading": 3.1938354114521794, "complete-iteration": 0.2727083420753479, "set_robot_commands": 0.007196573416392008, "deviation-center-line": 0.651301693455534, "driven_lanedir_consec": 0.9956196620011304, "sim_compute_sim_state": 0.02096242586771647, "sim_compute_performance-ego": 0.006334483623504639, "sim_compute_robot_state-ego": 0.011049575805664062, "sim_compute_robot_state-parked0": 0.010758124987284342}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 2.1393989535115017, "get_ui_image": 0.04938324530919393, "step_physics": 0.11720807631810506, "survival_time": 14.950000000000076, "driven_lanedir": 1.2428391221297068, "get_state_dump": 0.017235697905222575, "sim_render-ego": 0.007338998317718506, "in-drivable-lane": 6.000000000000085, "agent_compute-ego": 0.026851834456125896, "deviation-heading": 1.181497366784756, "complete-iteration": 0.27313582340876263, "set_robot_commands": 0.006538968086242675, "deviation-center-line": 0.6124353077991213, "driven_lanedir_consec": 1.2428391221297068, "sim_compute_sim_state": 0.021089901924133302, "sim_compute_performance-ego": 0.006210936705271403, "sim_compute_robot_state-ego": 0.010782821973164877, "sim_compute_robot_state-parked0": 0.010275647640228272}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 1.7065347479130488, "get_ui_image": 0.05174900770187378, "step_physics": 0.12381418466567994, "survival_time": 14.950000000000076, "driven_lanedir": 1.6702691272000365, "get_state_dump": 0.016980016231536867, "sim_render-ego": 0.007221704324086507, "in-drivable-lane": 0.0, "agent_compute-ego": 0.02605096499125163, "deviation-heading": 2.608509922660721, "complete-iteration": 0.2796444622675578, "set_robot_commands": 0.006840736865997315, "deviation-center-line": 0.7269599372936815, "driven_lanedir_consec": 1.6702691272000365, "sim_compute_sim_state": 0.019374917348225912, "sim_compute_performance-ego": 0.006195734341939291, "sim_compute_robot_state-ego": 0.01073224941889445, "sim_compute_robot_state-parked0": 0.010464247067769368}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 2.173291474310259, "get_ui_image": 0.051648739612463745, "step_physics": 0.12693019828411065, "survival_time": 14.850000000000076, "driven_lanedir": 1.7796679729280656, "get_state_dump": 0.017583570095023725, "sim_render-ego": 0.007421319332187024, "in-drivable-lane": 1.900000000000027, "agent_compute-ego": 0.025982575785832775, "deviation-heading": 1.2258022117752838, "complete-iteration": 0.2836847321352975, "set_robot_commands": 0.00683407992224902, "deviation-center-line": 0.6779095308405132, "driven_lanedir_consec": 1.7796679729280656, "sim_compute_sim_state": 0.01941195561829641, "sim_compute_performance-ego": 0.006143227169409344, "sim_compute_robot_state-ego": 0.010898386187826344, "sim_compute_robot_state-parked0": 0.010605015738644584}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 1.6795898801474023, "get_ui_image": 0.0501468825340271, "step_physics": 0.12842200994491576, "survival_time": 14.950000000000076, "driven_lanedir": 1.2913337734022448, "get_state_dump": 0.018000909487406413, "sim_render-ego": 0.007628575960795084, "in-drivable-lane": 2.84999999999999, "agent_compute-ego": 0.02761829694112142, "deviation-heading": 2.9032519244971797, "complete-iteration": 0.2835066469510396, "set_robot_commands": 0.006887051264444987, "deviation-center-line": 1.016304850511611, "driven_lanedir_consec": 1.2913337734022448, "sim_compute_sim_state": 0.016379794279734294, "sim_compute_performance-ego": 0.006453346411387125, "sim_compute_robot_state-ego": 0.011124602953592936, "sim_compute_robot_state-parked0": 0.01061498721440633}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 1.9589853140042184, "get_ui_image": 0.05147731701533, "step_physics": 0.13412237008412678, "survival_time": 14.950000000000076, "driven_lanedir": 1.1630645539098103, "get_state_dump": 0.017963823477427164, "sim_render-ego": 0.00774773120880127, "in-drivable-lane": 5.050000000000031, "agent_compute-ego": 0.027861289978027344, "deviation-heading": 2.551339062422611, "complete-iteration": 0.29168831030527753, "set_robot_commands": 0.007130229473114013, "deviation-center-line": 0.7809020766143737, "driven_lanedir_consec": 1.1630645539098103, "sim_compute_sim_state": 0.016412394841512044, "sim_compute_performance-ego": 0.006460503737131754, "sim_compute_robot_state-ego": 0.011443846225738526, "sim_compute_robot_state-parked0": 0.010843085447947183}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 2.0220436361390535, "get_ui_image": 0.05115439176559448, "step_physics": 0.1355197318394979, "survival_time": 14.950000000000076, "driven_lanedir": 2.0155624899896867, "get_state_dump": 0.017610169251759848, "sim_render-ego": 0.0075747235616048176, "in-drivable-lane": 0.0, "agent_compute-ego": 0.026901103655497232, "deviation-heading": 0.968859702019687, "complete-iteration": 0.29486353397369386, "set_robot_commands": 0.007017171382904053, "deviation-center-line": 1.0346905064223468, "driven_lanedir_consec": 2.0155624899896867, "sim_compute_sim_state": 0.020491468111673992, "sim_compute_performance-ego": 0.006423250834147135, "sim_compute_robot_state-ego": 0.011461811065673828, "sim_compute_robot_state-parked0": 0.010487817128499348}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 2.025089195571126, "get_ui_image": 0.05091828965024406, "step_physics": 0.12227839929204322, "survival_time": 14.950000000000076, "driven_lanedir": 1.788538679183628, "get_state_dump": 0.017487273965790917, "sim_render-ego": 0.00729365731561463, "in-drivable-lane": 1.6500000000000234, "agent_compute-ego": 0.053261303184024465, "deviation-heading": 1.0448673929257428, "complete-iteration": 0.3061730989245667, "set_robot_commands": 0.006709184933665604, "deviation-center-line": 0.819959233158788, "driven_lanedir_consec": 1.788538679183628, "sim_compute_sim_state": 0.0201188226208639, "sim_compute_performance-ego": 0.006214665728667907, "sim_compute_robot_state-ego": 0.011218977612396546, "sim_compute_robot_state-parked0": 0.010452271305199053}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 2.0527759265254875, "get_ui_image": 0.050130881468455, "step_physics": 0.11899444659550984, "survival_time": 14.950000000000076, "driven_lanedir": 2.0475175965383374, "get_state_dump": 0.01790436903635661, "sim_render-ego": 0.007514182726542155, "in-drivable-lane": 0.0, "agent_compute-ego": 0.05218972762425741, "deviation-heading": 0.912749674130178, "complete-iteration": 0.3001248629887899, "set_robot_commands": 0.0066291419665018714, "deviation-center-line": 0.8477775752863603, "driven_lanedir_consec": 2.0475175965383374, "sim_compute_sim_state": 0.0185201899210612, "sim_compute_performance-ego": 0.00638219436009725, "sim_compute_robot_state-ego": 0.011358426411946615, "sim_compute_robot_state-parked0": 0.010280157725016276}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 2.0736270932484477, "get_ui_image": 0.04928935050964355, "step_physics": 0.11097519636154177, "survival_time": 14.950000000000076, "driven_lanedir": 2.0693598025735063, "get_state_dump": 0.01824289083480835, "sim_render-ego": 0.007415723005930582, "in-drivable-lane": 0.0, "agent_compute-ego": 0.05581378777821859, "deviation-heading": 0.8317373692319064, "complete-iteration": 0.2962683113416036, "set_robot_commands": 0.007066349188486735, "deviation-center-line": 0.8560291703246831, "driven_lanedir_consec": 2.0693598025735063, "sim_compute_sim_state": 0.01842569589614868, "sim_compute_performance-ego": 0.0064005629221598305, "sim_compute_robot_state-ego": 0.011220576763153077, "sim_compute_robot_state-parked0": 0.011175991694132487}}
set_robot_commands_max0.007196573416392008
set_robot_commands_mean0.006884948649999828
set_robot_commands_median0.00686389406522115
set_robot_commands_min0.006538968086242675
sim_compute_performance-ego_max0.006460503737131754
sim_compute_performance-ego_mean0.006321890583371568
sim_compute_performance-ego_median0.006358338991800944
sim_compute_performance-ego_min0.006143227169409344
sim_compute_robot_state-ego_max0.011461811065673828
sim_compute_robot_state-ego_mean0.011129127441805126
sim_compute_robot_state-ego_median0.01117179028299474
sim_compute_robot_state-ego_min0.01073224941889445
sim_compute_robot_state-parked0_max0.011175991694132487
sim_compute_robot_state-parked0_mean0.010595734594912724
sim_compute_robot_state-parked0_median0.010546416433571966
sim_compute_robot_state-parked0_min0.010275647640228272
sim_compute_sim_state_max0.021089901924133302
sim_compute_sim_state_mean0.019118756642936614
sim_compute_sim_state_median0.01939343648326116
sim_compute_sim_state_min0.016379794279734294
sim_render-ego_max0.00774773120880127
sim_render-ego_mean0.007464823931386815
sim_render-ego_median0.007456471446387294
sim_render-ego_min0.007221704324086507
simulation-passed1
step_physics_max0.1355197318394979
step_physics_mean0.12319557621434656
step_physics_median0.12304629197886158
step_physics_min0.11097519636154177
survival_time_max14.950000000000076
survival_time_mean14.940000000000078
survival_time_min14.850000000000076
No reset possible
335267699Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:18
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334927255Bea Baselines 🐤straightaido5-LF-sim-validationLFv-simsuccessnoreg06-0314bee6766e-10:25:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.705601651532979
survival_time_median14.950000000000076
deviation-center-line_median0.35495520278373427
in-drivable-lane_median8.875000000000092


other stats
agent_compute-ego_max0.04173129558563232
agent_compute-ego_mean0.02878464092998989
agent_compute-ego_median0.02743694170643784
agent_compute-ego_min0.026769441763559977
complete-iteration_max0.300604309240977
complete-iteration_mean0.2859758870324075
complete-iteration_median0.2858221840858459
complete-iteration_min0.2752407072181988
deviation-center-line_max0.9974498371995706
deviation-center-line_mean0.442037585969931
deviation-center-line_min0.18033935477424248
deviation-heading_max2.6730577128929784
deviation-heading_mean1.4814690307051357
deviation-heading_median1.2970059812347137
deviation-heading_min1.0325902045628363
driven_any_max1.9399952639997864
driven_any_mean1.912128665796219
driven_any_median1.9399658864172735
driven_any_min1.720921634200831
driven_lanedir_consec_max1.6218350481774462
driven_lanedir_consec_mean0.8369014943606409
driven_lanedir_consec_min0.5248166652510198
driven_lanedir_max1.6218350481774462
driven_lanedir_mean0.8395432568339409
driven_lanedir_median0.705601651532979
driven_lanedir_min0.5354349815628103
get_state_dump_max0.017644805113474528
get_state_dump_mean0.016860134643735005
get_state_dump_median0.01681490484791523
get_state_dump_min0.016302065054575602
get_ui_image_max0.053785513242085775
get_ui_image_mean0.0515831122543511
get_ui_image_median0.05136059919993083
get_ui_image_min0.05041681369145711
in-drivable-lane_max10.250000000000083
in-drivable-lane_mean7.775000000000075
in-drivable-lane_min2.050000000000029
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 1.9336411119996484, "get_ui_image": 0.05041681369145711, "step_physics": 0.12380764325459798, "survival_time": 14.950000000000076, "driven_lanedir": 1.6218350481774462, "get_state_dump": 0.017644805113474528, "sim_render-ego": 0.007580073674519857, "in-drivable-lane": 2.050000000000029, "agent_compute-ego": 0.02775645176569621, "deviation-heading": 2.3154538885281197, "complete-iteration": 0.28432175000508625, "set_robot_commands": 0.0068162862459818525, "deviation-center-line": 0.9974498371995706, "driven_lanedir_consec": 1.6218350481774462, "sim_compute_sim_state": 0.02170239289601644, "sim_compute_performance-ego": 0.006282326380411784, "sim_compute_robot_state-ego": 0.011172921657562255, "sim_compute_robot_state-parked0": 0.010914701620737712}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 1.939961042155623, "get_ui_image": 0.05060870091120402, "step_physics": 0.13208277384440104, "survival_time": 14.950000000000076, "driven_lanedir": 0.5354349815628103, "get_state_dump": 0.016784117221832276, "sim_render-ego": 0.007334400018056234, "in-drivable-lane": 10.250000000000082, "agent_compute-ego": 0.028017773628234863, "deviation-heading": 1.331100107872904, "complete-iteration": 0.2885200214385986, "set_robot_commands": 0.006624979178110758, "deviation-center-line": 0.35183780258714387, "driven_lanedir_consec": 0.5248166652510198, "sim_compute_sim_state": 0.01984464089075724, "sim_compute_performance-ego": 0.005983594258626302, "sim_compute_robot_state-ego": 0.010670321782430011, "sim_compute_robot_state-parked0": 0.01035535176595052}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 1.939983425811024, "get_ui_image": 0.0510119899113973, "step_physics": 0.12390677134195964, "survival_time": 14.950000000000076, "driven_lanedir": 0.7337062669236629, "get_state_dump": 0.01657957394917806, "sim_render-ego": 0.007096319198608398, "in-drivable-lane": 8.950000000000092, "agent_compute-ego": 0.02754954179128011, "deviation-heading": 1.2629118545965232, "complete-iteration": 0.27793214639027913, "set_robot_commands": 0.0063809092839558915, "deviation-center-line": 0.3592750559474773, "driven_lanedir_consec": 0.7337062669236629, "sim_compute_sim_state": 0.01877788702646891, "sim_compute_performance-ego": 0.005807262261708577, "sim_compute_robot_state-ego": 0.010530026753743487, "sim_compute_robot_state-parked0": 0.01008171002070109}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 1.9399807707921308, "get_ui_image": 0.05141148328781128, "step_physics": 0.1247928539911906, "survival_time": 14.950000000000076, "driven_lanedir": 0.6774970361422952, "get_state_dump": 0.016964330673217773, "sim_render-ego": 0.007151861190795899, "in-drivable-lane": 9.40000000000009, "agent_compute-ego": 0.027207534313201904, "deviation-heading": 1.0529691872703482, "complete-iteration": 0.2787923733393351, "set_robot_commands": 0.006424060662587484, "deviation-center-line": 0.3272601540226831, "driven_lanedir_consec": 0.6774970361422952, "sim_compute_sim_state": 0.018226651350657146, "sim_compute_performance-ego": 0.005932646592458089, "sim_compute_robot_state-ego": 0.010411914984385172, "sim_compute_robot_state-parked0": 0.010057066281636556}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 1.8868894779817071, "get_ui_image": 0.05216108155004757, "step_physics": 0.1281786317268188, "survival_time": 14.550000000000072, "driven_lanedir": 0.6292559137452729, "get_state_dump": 0.016845692473998183, "sim_render-ego": 0.00718474551984125, "in-drivable-lane": 9.200000000000085, "agent_compute-ego": 0.027499956773318783, "deviation-heading": 1.0618057339587472, "complete-iteration": 0.2796162731459051, "set_robot_commands": 0.006555444186495752, "deviation-center-line": 0.18033935477424248, "driven_lanedir_consec": 0.6292559137452729, "sim_compute_sim_state": 0.014134774912673584, "sim_compute_performance-ego": 0.005893804773022628, "sim_compute_robot_state-ego": 0.010568828517219046, "sim_compute_robot_state-parked0": 0.010376054396744036}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 1.720921634200831, "get_ui_image": 0.052370241710117886, "step_physics": 0.12339319770497488, "survival_time": 13.300000000000054, "driven_lanedir": 0.6209905318658486, "get_state_dump": 0.016995289271935485, "sim_render-ego": 0.00715639985593638, "in-drivable-lane": 8.050000000000065, "agent_compute-ego": 0.026994916729460985, "deviation-heading": 1.0325902045628363, "complete-iteration": 0.2752407072181988, "set_robot_commands": 0.006711867518891069, "deviation-center-line": 0.1976070529759912, "driven_lanedir_consec": 0.6209905318658486, "sim_compute_sim_state": 0.014282937336685066, "sim_compute_performance-ego": 0.006028524018768081, "sim_compute_robot_state-ego": 0.010747971391319331, "sim_compute_robot_state-parked0": 0.0103424106325422}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 1.939963055749852, "get_ui_image": 0.05186940431594848, "step_physics": 0.14607282956441245, "survival_time": 14.950000000000076, "driven_lanedir": 0.963529203290399, "get_state_dump": 0.016655359268188476, "sim_render-ego": 0.007160456975301107, "in-drivable-lane": 6.950000000000089, "agent_compute-ego": 0.026769441763559977, "deviation-heading": 1.6562571087372715, "complete-iteration": 0.300604309240977, "set_robot_commands": 0.006588854789733887, "deviation-center-line": 0.5515392618339653, "driven_lanedir_consec": 0.963529203290399, "sim_compute_sim_state": 0.018890724976857506, "sim_compute_performance-ego": 0.005852749347686767, "sim_compute_robot_state-ego": 0.010450163682301838, "sim_compute_robot_state-parked0": 0.010081388155619304}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 1.93998215818689, "get_ui_image": 0.051309715112050375, "step_physics": 0.13910274267196654, "survival_time": 14.950000000000076, "driven_lanedir": 0.7554862447552759, "get_state_dump": 0.016302065054575602, "sim_render-ego": 0.007228085199991862, "in-drivable-lane": 8.800000000000091, "agent_compute-ego": 0.027373926639556886, "deviation-heading": 1.0468082949266786, "complete-iteration": 0.2933513045310974, "set_robot_commands": 0.006525132656097412, "deviation-center-line": 0.3580726029803246, "driven_lanedir_consec": 0.7554862447552759, "sim_compute_sim_state": 0.01886096000671387, "sim_compute_performance-ego": 0.005925064086914063, "sim_compute_robot_state-ego": 0.010427534580230711, "sim_compute_robot_state-parked0": 0.010081085364023844}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 1.9399687170846955, "get_ui_image": 0.053785513242085775, "step_physics": 0.13182545185089112, "survival_time": 14.950000000000076, "driven_lanedir": 1.3154624120801195, "get_state_dump": 0.016404832204182943, "sim_render-ego": 0.006981353759765625, "in-drivable-lane": 3.8500000000000543, "agent_compute-ego": 0.026945570309956868, "deviation-heading": 2.6730577128929784, "complete-iteration": 0.28732261816660565, "set_robot_commands": 0.006709875265757243, "deviation-center-line": 0.7757561803386851, "driven_lanedir_consec": 1.2996631036589117, "sim_compute_sim_state": 0.017907609144846598, "sim_compute_performance-ego": 0.005982765356699626, "sim_compute_robot_state-ego": 0.010399690469106038, "sim_compute_robot_state-parked0": 0.010170238812764486}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 1.9399952639997864, "get_ui_image": 0.050886178811391194, "step_physics": 0.12426390012105304, "survival_time": 14.950000000000076, "driven_lanedir": 0.5422349297962774, "get_state_dump": 0.017425281206766764, "sim_render-ego": 0.007422087192535401, "in-drivable-lane": 10.250000000000083, "agent_compute-ego": 0.04173129558563232, "deviation-heading": 1.3817362137049471, "complete-iteration": 0.29405736684799194, "set_robot_commands": 0.006718928813934326, "deviation-center-line": 0.32123855703922616, "driven_lanedir_consec": 0.5422349297962774, "sim_compute_sim_state": 0.017676479816436767, "sim_compute_performance-ego": 0.00615231196085612, "sim_compute_robot_state-ego": 0.011055819988250733, "sim_compute_robot_state-parked0": 0.010507595539093018}}
set_robot_commands_max0.0068162862459818525
set_robot_commands_mean0.006605633860154567
set_robot_commands_median0.006606916983922323
set_robot_commands_min0.0063809092839558915
sim_compute_performance-ego_max0.006282326380411784
sim_compute_performance-ego_mean0.005984104903715203
sim_compute_performance-ego_median0.0059577059745788575
sim_compute_performance-ego_min0.005807262261708577
sim_compute_robot_state-ego_max0.011172921657562255
sim_compute_robot_state-ego_mean0.010643519380654864
sim_compute_robot_state-ego_median0.010549427635481268
sim_compute_robot_state-ego_min0.010399690469106038
sim_compute_robot_state-parked0_max0.010914701620737712
sim_compute_robot_state-parked0_mean0.010296760258981276
sim_compute_robot_state-parked0_median0.010256324722653344
sim_compute_robot_state-parked0_min0.010057066281636556
sim_compute_sim_state_max0.02170239289601644
sim_compute_sim_state_mean0.018030505835811315
sim_compute_sim_state_median0.018502269188563028
sim_compute_sim_state_min0.014134774912673584
sim_render-ego_max0.007580073674519857
sim_render-ego_mean0.007229578258535201
sim_render-ego_median0.007172601247571179
sim_render-ego_min0.006981353759765625
simulation-passed1
step_physics_max0.14607282956441245
step_physics_mean0.12974267960722655
step_physics_median0.1264857428590047
step_physics_min0.12339319770497488
survival_time_max14.950000000000076
survival_time_mean14.745000000000076
survival_time_min13.300000000000054
No reset possible
334867219Bea Baselines 🐤template-rosaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:26
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334777171Bea Baselines 🐤template-tensorflowaido5-LFV_multi-sim-validationLFVmultibodyv-simerrornoreg06-0314bee6766e-10:01:17
Unexpected exception [...]
Unexpected exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 666, in scoring_context
    yield cie
  File "experiment_manager.py", line 607, in <module>
    wrap(cie)
  File "experiment_manager.py", line 595, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "experiment_manager.py", line 158, in main
    check_compatibility_between_agent_and_sim(robot_ci, sim_ci)
  File "experiment_manager.py", line 531, in check_compatibility_between_agent_and_sim
    raise ZException(msg, r=r)
zuper_commons.types.exceptions.ZException: Observations mismatch
│ r: CanBeUsed
│    │ result: False
│    │ why:
│    │ |Type <class 'aido_schemas.schemas.Duckiebot1ObservationsPlusState'>
│    │ | requires field "your_name"
│    │ |  of type <class 'str'>
│    │ |  but <class 'aido_schemas.schemas.Duckiebot1Observations'> does not have it.
│    │ M: Matches()
│    │ matches: {}
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
334666854Jerome Labonte 🇨🇦exercise_ros_templateaido5-LF-sim-validationLFv-simsuccessnoreg06-0314bee6766e-10:25:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median1.4755724078582633
survival_time_median14.950000000000076
deviation-center-line_median0.6610407053532739
in-drivable-lane_median4.325000000000027


other stats
agent_compute-ego_max0.027901190121968587
agent_compute-ego_mean0.026953413186772624
agent_compute-ego_median0.0269132395585378
agent_compute-ego_min0.02618771473566691
complete-iteration_max0.31371183370156264
complete-iteration_mean0.2975780533262553
complete-iteration_median0.2992613116900126
complete-iteration_min0.2805769022305806
deviation-center-line_max1.2364235842567013
deviation-center-line_mean0.6790660663442346
deviation-center-line_min0.1935303908701884
deviation-heading_max2.9093221651653156
deviation-heading_mean1.6229120424516974
deviation-heading_median1.308091848126676
deviation-heading_min0.7898261120252441
driven_any_max2.539575612032994
driven_any_mean2.028282354750454
driven_any_median2.156225621092026
driven_any_min1.1653204233840415
driven_lanedir_consec_max2.069276746605573
driven_lanedir_consec_mean1.431010696999259
driven_lanedir_consec_min0.5812064074894221
driven_lanedir_max2.069276746605573
driven_lanedir_mean1.4312980669556428
driven_lanedir_median1.4755724078582633
driven_lanedir_min0.5812064074894221
get_state_dump_max0.01858162720998128
get_state_dump_mean0.01800142149520445
get_state_dump_median0.017939265966415405
get_state_dump_min0.017136646906534832
get_ui_image_max0.051294407133239406
get_ui_image_mean0.05011420752082964
get_ui_image_median0.05026744805946552
get_ui_image_min0.04919760465621948
in-drivable-lane_max8.250000000000092
in-drivable-lane_mean4.185000000000035
in-drivable-lane_min0.5500000000000078
per-episodes
details{"ETHZ_autolab_technical_track-0-0-ego": {"driven_any": 2.128922274409704, "get_ui_image": 0.04945903460184733, "step_physics": 0.1405886435508728, "survival_time": 14.950000000000076, "driven_lanedir": 1.843293589240871, "get_state_dump": 0.018107750415802003, "sim_render-ego": 0.007721474965413411, "in-drivable-lane": 1.2999999999999954, "agent_compute-ego": 0.027901190121968587, "deviation-heading": 2.9093221651653156, "complete-iteration": 0.30152432044347127, "set_robot_commands": 0.006960483392079672, "deviation-center-line": 0.7655118948900816, "driven_lanedir_consec": 1.843293589240871, "sim_compute_sim_state": 0.021757699648539224, "sim_compute_performance-ego": 0.0065224552154541015, "sim_compute_robot_state-ego": 0.011512743631998695, "sim_compute_robot_state-parked0": 0.010766402085622151}, "ETHZ_autolab_technical_track-0-1-ego": {"driven_any": 2.1835289677743477, "get_ui_image": 0.050096678733825686, "step_physics": 0.14095280408859254, "survival_time": 14.950000000000076, "driven_lanedir": 2.007815829284879, "get_state_dump": 0.01789144198099772, "sim_render-ego": 0.007441206773122152, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.026920326550801597, "deviation-heading": 2.472818818876222, "complete-iteration": 0.29952974637349444, "set_robot_commands": 0.006815075874328613, "deviation-center-line": 0.7649017883697501, "driven_lanedir_consec": 2.007815829284879, "sim_compute_sim_state": 0.021157106558481853, "sim_compute_performance-ego": 0.006247552235921224, "sim_compute_robot_state-ego": 0.011167554060618083, "sim_compute_robot_state-parked0": 0.010634707609812418}, "ETHZ_autolab_technical_track-1-0-ego": {"driven_any": 2.0886992423219466, "get_ui_image": 0.04935272693634033, "step_physics": 0.14737154881159464, "survival_time": 14.950000000000076, "driven_lanedir": 1.1078512264756553, "get_state_dump": 0.01858162720998128, "sim_render-ego": 0.007492504119873047, "in-drivable-lane": 7.950000000000094, "agent_compute-ego": 0.02690615256627401, "deviation-heading": 1.1494069920117955, "complete-iteration": 0.3041634750366211, "set_robot_commands": 0.0066612044970194495, "deviation-center-line": 0.4486253485396576, "driven_lanedir_consec": 1.1078512264756553, "sim_compute_sim_state": 0.019374687671661377, "sim_compute_performance-ego": 0.006389362017313639, "sim_compute_robot_state-ego": 0.0114524507522583, "sim_compute_robot_state-parked0": 0.010374542872111002}, "ETHZ_autolab_technical_track-1-1-ego": {"driven_any": 2.0360394988680293, "get_ui_image": 0.0507154647509257, "step_physics": 0.146311784585317, "survival_time": 14.950000000000076, "driven_lanedir": 1.0531814818239928, "get_state_dump": 0.01780679702758789, "sim_render-ego": 0.00756322701772054, "in-drivable-lane": 8.250000000000092, "agent_compute-ego": 0.027054336071014404, "deviation-heading": 0.9994076691675496, "complete-iteration": 0.3043323008219401, "set_robot_commands": 0.00695475180943807, "deviation-center-line": 0.4787702110913151, "driven_lanedir_consec": 1.0531814818239928, "sim_compute_sim_state": 0.019534881114959716, "sim_compute_performance-ego": 0.00632443110148112, "sim_compute_robot_state-ego": 0.011185659567515056, "sim_compute_robot_state-parked0": 0.010675059159596762}, "ETHZ_autolab_technical_track-2-0-ego": {"driven_any": 1.3942304593151702, "get_ui_image": 0.05043821738510536, "step_physics": 0.1587317872930456, "survival_time": 9.45, "driven_lanedir": 0.5812064074894221, "get_state_dump": 0.01842751578679161, "sim_render-ego": 0.007810696092232195, "in-drivable-lane": 5.600000000000005, "agent_compute-ego": 0.02756302571170545, "deviation-heading": 0.7898261120252441, "complete-iteration": 0.31371183370156264, "set_robot_commands": 0.0070790840835167615, "deviation-center-line": 0.1935303908701884, "driven_lanedir_consec": 0.5812064074894221, "sim_compute_sim_state": 0.014558119748635266, "sim_compute_performance-ego": 0.006477772243439205, "sim_compute_robot_state-ego": 0.011797222510847465, "sim_compute_robot_state-parked0": 0.010616061548707346}, "ETHZ_autolab_technical_track-2-1-ego": {"driven_any": 1.1653204233840415, "get_ui_image": 0.051294407133239406, "step_physics": 0.14191516986868, "survival_time": 9.049999999999994, "driven_lanedir": 0.5930656165782315, "get_state_dump": 0.017740116593587464, "sim_render-ego": 0.007255252553613146, "in-drivable-lane": 4.9, "agent_compute-ego": 0.02619797748755355, "deviation-heading": 1.466776704241556, "complete-iteration": 0.2933115787927617, "set_robot_commands": 0.006643620643826479, "deviation-center-line": 0.2584359819030813, "driven_lanedir_consec": 0.5901919170143934, "sim_compute_sim_state": 0.014485274889192528, "sim_compute_performance-ego": 0.006203668552209001, "sim_compute_robot_state-ego": 0.011108520939863848, "sim_compute_robot_state-parked0": 0.010267880740086676}, "ETHZ_autolab_technical_track-3-0-ego": {"driven_any": 2.2210572065825316, "get_ui_image": 0.04937578996022542, "step_physics": 0.1412276562054952, "survival_time": 14.950000000000076, "driven_lanedir": 2.069276746605573, "get_state_dump": 0.01798708995183309, "sim_render-ego": 0.007401166756947835, "in-drivable-lane": 0.6500000000000092, "agent_compute-ego": 0.02665730396906535, "deviation-heading": 2.3693248232844377, "complete-iteration": 0.29728657722473145, "set_robot_commands": 0.006699818770090739, "deviation-center-line": 1.1239275487095322, "driven_lanedir_consec": 2.069276746605573, "sim_compute_sim_state": 0.01982709566752116, "sim_compute_performance-ego": 0.00629262367884318, "sim_compute_robot_state-ego": 0.01123931884765625, "sim_compute_robot_state-parked0": 0.010377391974131268}, "ETHZ_autolab_technical_track-3-1-ego": {"driven_any": 2.1901259497271734, "get_ui_image": 0.05050480763117472, "step_physics": 0.14166244586308796, "survival_time": 14.950000000000076, "driven_lanedir": 2.067277304737162, "get_state_dump": 0.017825599511464438, "sim_render-ego": 0.007378231684366862, "in-drivable-lane": 0.5500000000000078, "agent_compute-ego": 0.026800878047943115, "deviation-heading": 1.9764526704928331, "complete-iteration": 0.29899287700653077, "set_robot_commands": 0.006919557253519694, "deviation-center-line": 1.2364235842567013, "driven_lanedir_consec": 2.067277304737162, "sim_compute_sim_state": 0.01993587891260783, "sim_compute_performance-ego": 0.006168329715728759, "sim_compute_robot_state-ego": 0.011142309506734211, "sim_compute_robot_state-parked0": 0.010451952616373696}, "ETHZ_autolab_technical_track-4-0-ego": {"driven_any": 2.3353239130885997, "get_ui_image": 0.050707343419392904, "step_physics": 0.12768497069676718, "survival_time": 14.950000000000076, "driven_lanedir": 1.1027065625978487, "get_state_dump": 0.017136646906534832, "sim_render-ego": 0.007147555351257324, "in-drivable-lane": 8.100000000000094, "agent_compute-ego": 0.02618771473566691, "deviation-heading": 0.9647538035410246, "complete-iteration": 0.2805769022305806, "set_robot_commands": 0.006630369027455648, "deviation-center-line": 0.5571796223367976, "driven_lanedir_consec": 1.1027065625978487, "sim_compute_sim_state": 0.017801800568898518, "sim_compute_performance-ego": 0.0060781606038411455, "sim_compute_robot_state-ego": 0.010745305220286052, "sim_compute_robot_state-parked0": 0.010260534286499024}, "ETHZ_autolab_technical_track-4-1-ego": {"driven_any": 2.539575612032994, "get_ui_image": 0.04919760465621948, "step_physics": 0.12541305859883625, "survival_time": 14.950000000000076, "driven_lanedir": 1.8873059047227925, "get_state_dump": 0.018509629567464193, "sim_render-ego": 0.007565773328145345, "in-drivable-lane": 3.7500000000000537, "agent_compute-ego": 0.027345226605733235, "deviation-heading": 1.131030665710997, "complete-iteration": 0.2823509216308594, "set_robot_commands": 0.007192040284474691, "deviation-center-line": 0.963354292475241, "driven_lanedir_consec": 1.8873059047227925, "sim_compute_sim_state": 0.018416980107625325, "sim_compute_performance-ego": 0.006398983796437581, "sim_compute_robot_state-ego": 0.01148046334584554, "sim_compute_robot_state-parked0": 0.010621529420216878}}
set_robot_commands_max0.007192040284474691
set_robot_commands_mean0.006855600563574982
set_robot_commands_median0.006867316563924154
set_robot_commands_min0.006630369027455648
sim_compute_performance-ego_max0.0065224552154541015
sim_compute_performance-ego_mean0.0063103339160668955
sim_compute_performance-ego_median0.00630852739016215
sim_compute_performance-ego_min0.0060781606038411455
sim_compute_robot_state-ego_max0.011797222510847465
sim_compute_robot_state-ego_mean0.01128315483836235
sim_compute_robot_state-ego_median0.011212489207585653
sim_compute_robot_state-ego_min0.010745305220286052
sim_compute_robot_state-parked0_max0.010766402085622151
sim_compute_robot_state-parked0_mean0.01050460623131572
sim_compute_robot_state-parked0_median0.01053400708254052
sim_compute_robot_state-parked0_min0.010260534286499024
sim_compute_sim_state_max0.021757699648539224
sim_compute_sim_state_mean0.01868495248881228
sim_compute_sim_state_median0.019454784393310547
sim_compute_sim_state_min0.014485274889192528
sim_render-ego_max0.007810696092232195
sim_render-ego_mean0.007477708864269185
sim_render-ego_median0.007466855446497599
sim_render-ego_min0.007147555351257324
simulation-passed1
step_physics_max0.1587317872930456
step_physics_mean0.14118598695622891
step_physics_median0.1414450510342916
step_physics_min0.12541305859883625
survival_time_max14.950000000000076
survival_time_mean13.810000000000064
survival_time_min9.049999999999994
No reset possible