Duckietown Challenges Home Challenges Submissions

Evaluator 5363

ID5363
evaluatornoname
ownerDenis Medvedev
machinedenis-aspirev3571g
processnoname_denis-aspirev3571g
version6.2.20
first heard
last heard
statusinactive
# evaluating
# success2 79754
# timeout
# failed3 79755
# error
# aborted
# host-error
arm0
x86_641
Mac0
gpu available0
Number of processors4
Processor frequency (MHz)3.1 GHz
Free % of processors91%
RAM total (MB)3.7 GB
RAM free (MB)1.7 GB
Disk (MB)93.7 GB
Disk available (MB)36.1 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
8059816302Denis Medvedevtemplate-rosmooc-modconsimfailednononame0:04:27
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 219, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
8059416299Denis Medvedevobjdet exercisemooc-objdetsimsuccessnononame0:20:30
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
survival_time_median18.300000000000125
driven_lanedir_consec_median4.810667681787954


other stats
agent_compute-ego0_max0.08823605395820984
agent_compute-ego0_mean0.08109932422658556
agent_compute-ego0_median0.08109932422658556
agent_compute-ego0_min0.07396259449496127
complete-iteration_max0.7252530989167992
complete-iteration_mean0.6558843140008566
complete-iteration_median0.6558843140008566
complete-iteration_min0.5865155290849138
deviation-center-line_max0.26892281627655074
deviation-center-line_mean0.21630701637268143
deviation-center-line_median0.21630701637268143
deviation-center-line_min0.16369121646881216
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max6.73934300422758
distance-from-start_mean4.810621862412031
distance-from-start_median4.810621862412031
distance-from-start_min2.881900720596483
driven_any_max6.73934300422758
driven_any_mean4.810621862412031
driven_any_median4.810621862412031
driven_any_min2.881900720596483
driven_lanedir_consec_max6.739416375324249
driven_lanedir_consec_mean4.810667681787954
driven_lanedir_consec_min2.8819189882516576
driven_lanedir_max6.739416375324249
driven_lanedir_mean4.810667681787954
driven_lanedir_median4.810667681787954
driven_lanedir_min2.8819189882516576
get_duckie_state_max0.0806043001684812
get_duckie_state_mean0.08031869348093634
get_duckie_state_median0.08031869348093634
get_duckie_state_min0.08003308679339147
get_robot_state_max0.012879647358809369
get_robot_state_mean0.012848787340931691
get_robot_state_median0.012848787340931691
get_robot_state_min0.012817927323054016
get_state_dump_max0.0320417412503838
get_state_dump_mean0.029415613534236225
get_state_dump_median0.029415613534236225
get_state_dump_min0.026789485818088644
get_ui_image_max0.12723264319407368
get_ui_image_mean0.12722984589235273
get_ui_image_median0.12722984589235273
get_ui_image_min0.12722704859063175
in-drivable-lane_max0.0
in-drivable-lane_mean0.0
in-drivable-lane_median0.0
in-drivable-lane_min0.0
per-episodes
details{"LF-long-loop-with-duckies-000-ego0": {"driven_any": 6.73934300422758, "get_ui_image": 0.12722704859063175, "step_physics": 0.15391328688895348, "survival_time": 25.200000000000223, "driven_lanedir": 6.739416375324249, "get_state_dump": 0.026789485818088644, "get_robot_state": 0.012879647358809369, "sim_render-ego0": 0.00991440244240336, "get_duckie_state": 0.0806043001684812, "in-drivable-lane": 0.0, "deviation-heading": 0.0, "agent_compute-ego0": 0.07396259449496127, "complete-iteration": 0.5865155290849138, "set_robot_commands": 0.00840374219535601, "distance-from-start": 6.73934300422758, "deviation-center-line": 0.16369121646881216, "driven_lanedir_consec": 6.739416375324249, "sim_compute_sim_state": 0.08615684367642544, "sim_compute_performance-ego0": 0.006407205185087601}, "LF-long-loop-with-duckies-001-ego0": {"driven_any": 2.881900720596483, "get_ui_image": 0.12723264319407368, "step_physics": 0.2572354021030743, "survival_time": 11.400000000000029, "driven_lanedir": 2.8819189882516576, "get_state_dump": 0.0320417412503838, "get_robot_state": 0.012817927323054016, "sim_render-ego0": 0.014696489775545211, "get_duckie_state": 0.08003308679339147, "in-drivable-lane": 0.0, "deviation-heading": 0.0, "agent_compute-ego0": 0.08823605395820984, "complete-iteration": 0.7252530989167992, "set_robot_commands": 0.008301055066970759, "distance-from-start": 2.881900720596483, "deviation-center-line": 0.26892281627655074, "driven_lanedir_consec": 2.8819189882516576, "sim_compute_sim_state": 0.09735245059150796, "sim_compute_performance-ego0": 0.007049527230741676}}
set_robot_commands_max0.00840374219535601
set_robot_commands_mean0.008352398631163384
set_robot_commands_median0.008352398631163384
set_robot_commands_min0.008301055066970759
sim_compute_performance-ego0_max0.007049527230741676
sim_compute_performance-ego0_mean0.006728366207914638
sim_compute_performance-ego0_median0.006728366207914638
sim_compute_performance-ego0_min0.006407205185087601
sim_compute_sim_state_max0.09735245059150796
sim_compute_sim_state_mean0.0917546471339667
sim_compute_sim_state_median0.0917546471339667
sim_compute_sim_state_min0.08615684367642544
sim_render-ego0_max0.014696489775545211
sim_render-ego0_mean0.012305446108974286
sim_render-ego0_median0.012305446108974286
sim_render-ego0_min0.00991440244240336
simulation-passed1
step_physics_max0.2572354021030743
step_physics_mean0.20557434449601383
step_physics_median0.20557434449601383
step_physics_min0.15391328688895348
survival_time_max25.200000000000223
survival_time_mean18.300000000000125
survival_time_min11.400000000000029
No reset possible
8052316266Denis Medvedevtemplate-rosmooc-modconsimfailednononame0:04:31
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 219, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7975516127Denis Medvedevtemplate-rosmooc-modconsimfailednononame0:04:47
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 219, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego0: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
7975416102Denis Medvedevexercises_braitenbergmooc-BV1sim-0of5successnononame0:23:50
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean3.142889952475766


other stats
agent_compute-ego0_max0.05641459404849875
agent_compute-ego0_mean0.05641459404849875
agent_compute-ego0_median0.05641459404849875
agent_compute-ego0_min0.05641459404849875
complete-iteration_max0.7449108981955362
complete-iteration_mean0.7449108981955362
complete-iteration_median0.7449108981955362
complete-iteration_min0.7449108981955362
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max3.142889952475766
distance-from-start_median3.142889952475766
distance-from-start_min3.142889952475766
driven_any_max3.179395787507635
driven_any_mean3.179395787507635
driven_any_median3.179395787507635
driven_any_min3.179395787507635
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
get_duckie_state_max0.3264618142315063
get_duckie_state_mean0.3264618142315063
get_duckie_state_median0.3264618142315063
get_duckie_state_min0.3264618142315063
get_robot_state_max0.012922962343087738
get_robot_state_mean0.012922962343087738
get_robot_state_median0.012922962343087738
get_robot_state_min0.012922962343087738
get_state_dump_max0.05817587843320661
get_state_dump_mean0.05817587843320661
get_state_dump_median0.05817587843320661
get_state_dump_min0.05817587843320661
get_ui_image_max0.07013866047800323
get_ui_image_mean0.07013866047800323
get_ui_image_median0.07013866047800323
get_ui_image_min0.07013866047800323
in-drivable-lane_max36.40000000000007
in-drivable-lane_mean36.40000000000007
in-drivable-lane_median36.40000000000007
in-drivable-lane_min36.40000000000007
per-episodes
details{"d40-ego0": {"driven_any": 3.179395787507635, "get_ui_image": 0.07013866047800323, "step_physics": 0.1466201695885678, "survival_time": 36.40000000000007, "driven_lanedir": 0.0, "get_state_dump": 0.05817587843320661, "get_robot_state": 0.012922962343087738, "sim_render-ego0": 0.012034280146427444, "get_duckie_state": 0.3264618142315063, "in-drivable-lane": 36.40000000000007, "deviation-heading": 0.0, "agent_compute-ego0": 0.05641459404849875, "complete-iteration": 0.7449108981955362, "set_robot_commands": 0.009381384843007365, "distance-from-start": 3.142889952475766, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0, "sim_compute_sim_state": 0.045598202772101255, "sim_compute_performance-ego0": 0.006895816211674275}}
set_robot_commands_max0.009381384843007365
set_robot_commands_mean0.009381384843007365
set_robot_commands_median0.009381384843007365
set_robot_commands_min0.009381384843007365
sim_compute_performance-ego0_max0.006895816211674275
sim_compute_performance-ego0_mean0.006895816211674275
sim_compute_performance-ego0_median0.006895816211674275
sim_compute_performance-ego0_min0.006895816211674275
sim_compute_sim_state_max0.045598202772101255
sim_compute_sim_state_mean0.045598202772101255
sim_compute_sim_state_median0.045598202772101255
sim_compute_sim_state_min0.045598202772101255
sim_render-ego0_max0.012034280146427444
sim_render-ego0_mean0.012034280146427444
sim_render-ego0_median0.012034280146427444
sim_render-ego0_min0.012034280146427444
simulation-passed1
step_physics_max0.1466201695885678
step_physics_mean0.1466201695885678
step_physics_median0.1466201695885678
step_physics_min0.1466201695885678
survival_time_max36.40000000000007
survival_time_mean36.40000000000007
survival_time_median36.40000000000007
survival_time_min36.40000000000007
No reset possible