Duckietown Challenges Home Challenges Submissions

Evaluator 5750

ID5750
evaluatornoname
owneruser8212
machinewaile
processnoname_waile-6.3.2
version6.3.2
first heard
last heard
statusinactive
# evaluating
# success6 184584
# timeout
# failed1 187005
# error
# aborted1 186830
# host-error1 184579
arm0
x86_641
Mac0
gpu available0
Number of processors16
Processor frequency (MHz)4.3 GHz
Free % of processors98%
RAM total (MB)15.0 GB
RAM free (MB)7.0 GB
Disk (MB)382.1 GB
Disk available (MB)249.3 GB
Docker Hub
P11
P2
Cloud simulations1
PI Camera0
# Duckiebots0
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
18700526480user8212exercises_braitenberglx22-BV1sim-0of5failedyesnoname0:01:49
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 343, in main
    length_s = await run_episode(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 722, in run_episode
    msg = await loop.run_in_executor(executor, f)
  File "/usr/lib/python3.8/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 243, in read_one
    msgs = read_reply(self.fpout, timeout=timeout, waiting_for=waiting_for, nickname=self.nickname,
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 333, in read_reply
    others = read_until_over(fpout, timeout=timeout, nickname=nickname)
  File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper_outside.py", line 360, in read_until_over
    raise RemoteNodeAborted(m)
zuper_nodes.structures.RemoteNodeAborted: External node "ego0" aborted:

error in ego0 |Exception while handling a message on topic "get_commands".
              |
              || Traceback (most recent call last):
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 418, in loop
              ||     handle_message_node(parsed, receiver0, context0)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/wrapper.py", line 591, in handle_message_node
              ||     call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing)
              ||   File "/usr/local/lib/python3.8/dist-packages/zuper_nodes_wrapper/utils.py", line 21, in call_if_fun_exists
              ||     f(**kwargs)
              ||   File "/code/catkin_ws/src/braitenberg/packages/braitenberg_agent/agent.py", line 116, in on_received_get_commands
              ||     pwm_commands = PWMCommands(motor_left=pwm_left, motor_right=pwm_right)
              ||   File "<string>", line 5, in __init__
              ||   File "/usr/local/lib/python3.8/dist-packages/aido_schemas/schemas.py", line 78, in __post_init__
              ||     raise ValueError(msg)
              || ValueError: Expected values to be between -1 and 1. Obtained -0.11652032470703122, 1.0165203247070311
              ||

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges/cie_concrete.py", line 691, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 60, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/experiment_manager.py", line 33, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/dist-packages/duckietown_experiment_manager/code.py", line 423, in main
    raise dc.InvalidSubmission(msg) from e
duckietown_challenges.exceptions.InvalidSubmission: external protocol violation
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18686126462user8212exercises_braitenberglx22-BV1sim-0of5successyesnoname0:05:49
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean5.3260526101409145


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max5.3260526101409145
distance-from-start_median5.3260526101409145
distance-from-start_min5.3260526101409145
driven_any_max7.210732540844634
driven_any_mean7.210732540844634
driven_any_median7.210732540844634
driven_any_min7.210732540844634
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max41.0999999999998
in-drivable-lane_mean41.0999999999998
in-drivable-lane_median41.0999999999998
in-drivable-lane_min41.0999999999998
per-episodes
details{"d60-ego0": {"driven_any": 7.210732540844634, "survival_time": 41.0999999999998, "driven_lanedir": 0.0, "in-drivable-lane": 41.0999999999998, "deviation-heading": 0.0, "distance-from-start": 5.3260526101409145, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max41.0999999999998
survival_time_mean41.0999999999998
survival_time_median41.0999999999998
survival_time_min41.0999999999998
No reset possible
18683026462user8212exercises_braitenberglx22-BV1sim-0of5abortedyesnoname0:04:57
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 936, in get_cr
    uploaded = upload_files(wd, aws_config, copy_to_machine_cache=copy_to_machine_cache)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 50, in upload_files
    uploaded = upload(aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache, quiet=True)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 277, in upload
    shutil.copy(realfile0, realfile)
  File "/usr/lib/python3.8/shutil.py", line 418, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.8/shutil.py", line 275, in copyfile
    _fastcopy_sendfile(fsrc, fdst)
  File "/usr/lib/python3.8/shutil.py", line 166, in _fastcopy_sendfile
    raise err from None
  File "/usr/lib/python3.8/shutil.py", line 152, in _fastcopy_sendfile
    sent = os.sendfile(outfd, infd, offset, blocksize)
OSError: [Errno 28] No space left on device: '/tmp/nikola/duckietown-challenges-runner/evaluations/waile/noname_waile-6.3.2/lx22-BV1/submission-26462/job-186830-step-sim-0of5-a-wd/challenge-evaluation-output/episodes/d60/log.gs2.cbor.tmp' -> '/tmp/tmp31rklx94log.gs2.cbor.tmp'
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
18471026262user8212exercises_braitenberglx22-BV1sim-0of5successyesnoname0:02:47
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean1.818455756744648


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max1.818455756744648
distance-from-start_median1.818455756744648
distance-from-start_min1.818455756744648
driven_any_max2.5958786066325694
driven_any_mean2.5958786066325694
driven_any_median2.5958786066325694
driven_any_min2.5958786066325694
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max16.800000000000104
in-drivable-lane_mean16.800000000000104
in-drivable-lane_median16.800000000000104
in-drivable-lane_min16.800000000000104
per-episodes
details{"d60-ego0": {"driven_any": 2.5958786066325694, "survival_time": 16.800000000000104, "driven_lanedir": 0.0, "in-drivable-lane": 16.800000000000104, "deviation-heading": 0.0, "distance-from-start": 1.818455756744648, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max16.800000000000104
survival_time_mean16.800000000000104
survival_time_median16.800000000000104
survival_time_min16.800000000000104
No reset possible
18469826260user8212exercises_braitenberglx22-BV1sim-0of5successyesnoname0:02:19
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean1.928147510064462


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max1.928147510064462
distance-from-start_median1.928147510064462
distance-from-start_min1.928147510064462
driven_any_max2.696338508501284
driven_any_mean2.696338508501284
driven_any_median2.696338508501284
driven_any_min2.696338508501284
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max12.30000000000004
in-drivable-lane_mean12.30000000000004
in-drivable-lane_median12.30000000000004
in-drivable-lane_min12.30000000000004
per-episodes
details{"d60-ego0": {"driven_any": 2.696338508501284, "survival_time": 12.30000000000004, "driven_lanedir": 0.0, "in-drivable-lane": 12.30000000000004, "deviation-heading": 0.0, "distance-from-start": 1.928147510064462, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max12.30000000000004
survival_time_mean12.30000000000004
survival_time_median12.30000000000004
survival_time_min12.30000000000004
No reset possible
18466826253user8212exercises_braitenberglx22-BV1sim-1of5successyesnoname0:01:17
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean1.40965176613864


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max1.40965176613864
distance-from-start_median1.40965176613864
distance-from-start_min1.40965176613864
driven_any_max1.434696440669628
driven_any_mean1.434696440669628
driven_any_median1.434696440669628
driven_any_min1.434696440669628
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max4.6499999999999915
in-drivable-lane_mean4.6499999999999915
in-drivable-lane_median4.6499999999999915
in-drivable-lane_min4.6499999999999915
per-episodes
details{"d50-ego0": {"driven_any": 1.434696440669628, "survival_time": 4.6499999999999915, "driven_lanedir": 0.0, "in-drivable-lane": 4.6499999999999915, "deviation-heading": 0.0, "distance-from-start": 1.40965176613864, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max4.6499999999999915
survival_time_mean4.6499999999999915
survival_time_median4.6499999999999915
survival_time_min4.6499999999999915
No reset possible
18466226253user8212exercises_braitenberglx22-BV1sim-0of5successyesnoname0:01:06
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean0.5093163800165064


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max0.5093163800165064
distance-from-start_median0.5093163800165064
distance-from-start_min0.5093163800165064
driven_any_max0.5093452630685656
driven_any_mean0.5093452630685656
driven_any_median0.5093452630685656
driven_any_min0.5093452630685656
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max2.1500000000000004
in-drivable-lane_mean2.1500000000000004
in-drivable-lane_median2.1500000000000004
in-drivable-lane_min2.1500000000000004
per-episodes
details{"d60-ego0": {"driven_any": 0.5093452630685656, "survival_time": 2.1500000000000004, "driven_lanedir": 0.0, "in-drivable-lane": 2.1500000000000004, "deviation-heading": 0.0, "distance-from-start": 0.5093163800165064, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max2.1500000000000004
survival_time_mean2.1500000000000004
survival_time_median2.1500000000000004
survival_time_min2.1500000000000004
No reset possible
18458426246user8212exercises_braitenberglx22-BV1sim-0of5successyesnoname0:06:40
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
distance-from-start_mean5.668917750556934e-17


other stats
deviation-center-line_max0.0
deviation-center-line_mean0.0
deviation-center-line_median0.0
deviation-center-line_min0.0
deviation-heading_max0.0
deviation-heading_mean0.0
deviation-heading_median0.0
deviation-heading_min0.0
distance-from-start_max5.668917750556934e-17
distance-from-start_median5.668917750556934e-17
distance-from-start_min5.668917750556934e-17
driven_any_max6.802701300668135e-14
driven_any_mean6.802701300668135e-14
driven_any_median6.802701300668135e-14
driven_any_min6.802701300668135e-14
driven_lanedir_consec_max0.0
driven_lanedir_consec_mean0.0
driven_lanedir_consec_median0.0
driven_lanedir_consec_min0.0
driven_lanedir_max0.0
driven_lanedir_mean0.0
driven_lanedir_median0.0
driven_lanedir_min0.0
in-drivable-lane_max59.99999999999873
in-drivable-lane_mean59.99999999999873
in-drivable-lane_median59.99999999999873
in-drivable-lane_min59.99999999999873
per-episodes
details{"d60-ego0": {"driven_any": 6.802701300668135e-14, "survival_time": 59.99999999999873, "driven_lanedir": 0.0, "in-drivable-lane": 59.99999999999873, "deviation-heading": 0.0, "distance-from-start": 5.668917750556934e-17, "deviation-center-line": 0.0, "driven_lanedir_consec": 0.0}}
simulation-passed1
survival_time_max59.99999999999873
survival_time_mean59.99999999999873
survival_time_median59.99999999999873
survival_time_min59.99999999999873
No reset possible
18457925597user7802planning-exerciselx22-planning-cc-dynamic-valistep1-3of4host-erroryesnoname0:00:02
Uncaught exception: [...]
Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 139, in download_artefacts
    get_file_from_cache(fn, sha256hex)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner_cache.py", line 64, in get_file_from_cache
    raise KeyError(msg)
KeyError: 'Hash not in cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 910, in get_cr
    cr = run_single(
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1041, in run_single
    prepare_dir(wd, aws_config, steps2artefacts=steps2artefacts, steps2scores=steps2scores, use_ipfs=use_ipfs)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/runner.py", line 1474, in prepare_dir
    download_artefacts(aws_config, steps2artefacts, previous_steps_dir, use_ipfs=use_ipfs)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 155, in download_artefacts
    get_object(aws_config, bucket_name, object_key, fn)
  File "/usr/local/lib/python3.8/dist-packages/duckietown_challenges_runner/uploading.py", line 204, in get_object
    aws_object.download_file(fn)
  File "/usr/local/lib/python3.8/dist-packages/boto3/s3/inject.py", line 361, in object_download_file
    return self.meta.client.download_file(
  File "/usr/local/lib/python3.8/dist-packages/boto3/s3/inject.py", line 192, in download_file
    return transfer.download_file(
  File "/usr/local/lib/python3.8/dist-packages/boto3/s3/transfer.py", line 405, in download_file
    future.result()
  File "/usr/local/lib/python3.8/dist-packages/s3transfer/futures.py", line 103, in result
    return self._coordinator.result()
  File "/usr/local/lib/python3.8/dist-packages/s3transfer/futures.py", line 266, in result
    raise self._exception
  File "/usr/local/lib/python3.8/dist-packages/s3transfer/tasks.py", line 139, in __call__
    return self._execute_main(kwargs)
  File "/usr/local/lib/python3.8/dist-packages/s3transfer/tasks.py", line 162, in _execute_main
    return_value = self._main(**kwargs)
  File "/usr/local/lib/python3.8/dist-packages/s3transfer/download.py", line 569, in _main
    response = client.get_object(
  File "/usr/local/lib/python3.8/dist-packages/botocore/client.py", line 553, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "/usr/local/lib/python3.8/dist-packages/botocore/client.py", line 1009, in _make_api_call
    raise error_class(parsed_response, operation_name)
botocore.errorfactory.InvalidObjectState: An error occurred (InvalidObjectState) when calling the GetObject operation: The operation is not valid for the object's access tier
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible