Submission 6144

Submission

6144

Competing

yes

Challenge

aido3-LF-sim-validation

User

Moustafa Elarabi

Date submitted

2019-12-11 20:54:42+00:00

Last status update

2021-12-04 20:52:23+00:00

Complete

complete

Details

Evaluation is complete.

Sisters

Result

💚

Jobs

step1-simulation: 30157

User label

challenge-aido_LF-template-pytorch

Admin priority

Blessing

n/a

User priority

30157

Click the images to see detailed statistics about the episode.

ETHZ_autolab_technical_track-0-0

ETHZ_autolab_technical_track-1-0

ETHZ_autolab_technical_track-2-0

ETHZ_autolab_technical_track-3-0

ETHZ_autolab_technical_track-4-0

Evaluation jobs for this submission

See previous jobs for previous versions of challenges

Job ID

step

status

up to date

date started

date completed

duration

message

30157

step1-simulation

success

yes

2019-12-11 21:06:31+00:00

2019-12-11 21:13:30+00:00

0:06:59

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

driven_lanedir_consec_median	1.360826392378482
survival_time_median	4.699999999999991
deviation-center-line_median	0.24224516196696383
in-drivable-lane_median	0.3999999999999986

other stats

agent_compute-ego_max	0.2834303505877231
agent_compute-ego_mean	0.10481744729196556
agent_compute-ego_median	0.06275037513382134
agent_compute-ego_min	0.05071992994104541
deviation-center-line_max	0.4469258860841592
deviation-center-line_mean	0.2617030222261834
deviation-center-line_min	0.11484649877894616
deviation-heading_max	1.4712748406117486
deviation-heading_mean	0.9623098129266836
deviation-heading_median	0.8216016298792368
deviation-heading_min	0.5346485697962647
driven_any_max	2.9228719388203626
driven_any_mean	1.941922103930978
driven_any_median	1.6476623297504207
driven_any_min	1.385923827156404
driven_lanedir_consec_max	2.389246345849499
driven_lanedir_consec_mean	1.7032431924138005
driven_lanedir_consec_min	1.2913833402706354
driven_lanedir_max	2.389246345849499
driven_lanedir_mean	1.7032431924138005
driven_lanedir_median	1.360826392378482
driven_lanedir_min	1.2913833402706354
in-drivable-lane_max	1.499999999999995
in-drivable-lane_mean	0.539999999999998
in-drivable-lane_min	0
per-episodes	details {"ETHZ_autolab_technical_track-0-0": {"driven_any": 1.3923207933426485, "sim_physics": 0.08815350938350597, "survival_time": 4.699999999999991, "driven_lanedir": 1.360826392378482, "sim_render-ego": 0.012513632469988885, "in-drivable-lane": 0, "agent_compute-ego": 0.2834303505877231, "deviation-heading": 0.8216016298792368, "set_robot_commands": 0.008777473835234946, "deviation-center-line": 0.1651435370930435, "driven_lanedir_consec": 1.360826392378482, "sim_compute_sim_state": 0.005580876735930747, "sim_compute_performance-ego": 0.008028423532526544, "sim_compute_robot_state-ego": 0.008991137463995751}, "ETHZ_autolab_technical_track-1-0": {"driven_any": 1.385923827156404, "sim_physics": 0.07072100569220151, "survival_time": 3.399999999999996, "driven_lanedir": 1.3378282460088264, "sim_render-ego": 0.01301948813831105, "in-drivable-lane": 0, "agent_compute-ego": 0.06796897158903234, "deviation-heading": 0.7906544075390877, "set_robot_commands": 0.008245208684135885, "deviation-center-line": 0.24224516196696383, "driven_lanedir_consec": 1.3378282460088264, "sim_compute_sim_state": 0.004645375644459444, "sim_compute_performance-ego": 0.007436696220846737, "sim_compute_robot_state-ego": 0.009240693905774283}, "ETHZ_autolab_technical_track-2-0": {"driven_any": 2.3608316305850536, "sim_physics": 0.06909852657677992, "survival_time": 7.94999999999998, "driven_lanedir": 2.136931637561559, "sim_render-ego": 0.013202484298802022, "in-drivable-lane": 0.3999999999999986, "agent_compute-ego": 0.05071992994104541, "deviation-heading": 1.4712748406117486, "set_robot_commands": 0.008963586399390263, "deviation-center-line": 0.3393540272078042, "driven_lanedir_consec": 2.136931637561559, "sim_compute_sim_state": 0.004879521124018064, "sim_compute_performance-ego": 0.00801067832131056, "sim_compute_robot_state-ego": 0.009020844345572609}, "ETHZ_autolab_technical_track-3-0": {"driven_any": 1.6476623297504207, "sim_physics": 0.06981120164366975, "survival_time": 4.3499999999999925, "driven_lanedir": 1.2913833402706354, "sim_render-ego": 0.013715609736826229, "in-drivable-lane": 0.7999999999999972, "agent_compute-ego": 0.06275037513382134, "deviation-heading": 0.5346485697962647, "set_robot_commands": 0.009087839345822388, "deviation-center-line": 0.11484649877894616, "driven_lanedir_consec": 1.2913833402706354, "sim_compute_sim_state": 0.005082190721884541, "sim_compute_performance-ego": 0.008207850072575712, "sim_compute_robot_state-ego": 0.009275998192271966}, "ETHZ_autolab_technical_track-4-0": {"driven_any": 2.9228719388203626, "sim_physics": 0.06926602495127711, "survival_time": 7.249999999999982, "driven_lanedir": 2.389246345849499, "sim_render-ego": 0.012436440895343648, "in-drivable-lane": 1.499999999999995, "agent_compute-ego": 0.05921760920820565, "deviation-heading": 1.1933696168070804, "set_robot_commands": 0.008384776937550511, "deviation-center-line": 0.4469258860841592, "driven_lanedir_consec": 2.389246345849499, "sim_compute_sim_state": 0.004802541075081661, "sim_compute_performance-ego": 0.007123804092407227, "sim_compute_robot_state-ego": 0.008594887832115436}}
set_robot_commands_max	0.009087839345822388
set_robot_commands_mean	0.0086917770404268
set_robot_commands_median	0.008777473835234946
set_robot_commands_min	0.008245208684135885
sim_compute_performance-ego_max	0.008207850072575712
sim_compute_performance-ego_mean	0.007761490447933356
sim_compute_performance-ego_median	0.00801067832131056
sim_compute_performance-ego_min	0.007123804092407227
sim_compute_robot_state-ego_max	0.009275998192271966
sim_compute_robot_state-ego_mean	0.00902471234794601
sim_compute_robot_state-ego_median	0.009020844345572609
sim_compute_robot_state-ego_min	0.008594887832115436
sim_compute_sim_state_max	0.005580876735930747
sim_compute_sim_state_mean	0.0049981010602748914
sim_compute_sim_state_median	0.004879521124018064
sim_compute_sim_state_min	0.004645375644459444
sim_physics_max	0.08815350938350597
sim_physics_mean	0.07341005364948686
sim_physics_median	0.06981120164366975
sim_physics_min	0.06909852657677992
sim_render-ego_max	0.013715609736826229
sim_render-ego_mean	0.012977531107854368
sim_render-ego_median	0.01301948813831105
sim_render-ego_min	0.012436440895343648
simulation-passed	1
survival_time_max	7.94999999999998
survival_time_mean	5.529999999999989
survival_time_min	3.399999999999996

No reset possible

30154

step1-simulation

success

yes

2019-12-11 21:05:55+00:00

2019-12-11 21:15:27+00:00

0:09:32

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

30151

step1-simulation

host-error

yes

2019-12-11 21:00:42+00:00

2019-12-11 21:05:36+00:00

0:04:54

Error while running [...]

Error while running Docker Compose:

Could not run ['docker-compose', '-p', 'job30151-270142', 'up', '-d', '--renew-anon-volumes']:

   >  Command '['docker-compose', '-p', 'job30151-270142', 'up', '-d', '--renew-anon-volumes']' returned non-zero exit status 1.

stdout | 

stderr | Creating network "job30151-270142_evaluation" with the default driver
stderr | Creating volume "job30151-270142_fifos" with default driver
stderr | Creating job30151-270142_simulator_1 ...
stderr | Creating job30151-270142_evaluator_1 ...
stderr | Creating job30151-270142_scenario_maker_1 ...
stderr | Creating job30151-270142_solution_1       ...
stderr | 
stderr | ERROR: for job30151-270142_evaluator_1  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for job30151-270142_scenario_maker_1  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for job30151-270142_simulator_1  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for job30151-270142_solution_1  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for evaluator  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for scenario_maker  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for simulator  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | 
stderr | ERROR: for solution  UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)
stderr | An HTTP request took too long to complete. Retry with --verbose to obtain debug information.
stderr | If you encounter this issue regularly because of slow network conditions, consider setting COMPOSE_HTTP_TIMEOUT to a higher value (current value: 60).
stderr |

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

30149

step1-simulation

host-error

yes

2019-12-11 21:00:28+00:00

2019-12-11 21:05:14+00:00

0:04:46

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 648, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1309, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1563, in upload
    sha256hex = compute_sha256hex(realfile)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1619, in compute_sha256hex
    res: bytes = subprocess.check_output(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['shasum', '-a', '256', '/tmp/duckietown/DT18/evaluator/executions/aido3-LF-sim-validation/submission6144/step1-simulation-idsc-rudolf-22612-job30149/tmp/tmpck3yzyrj/attempts/ETHZ_autolab_technical_track-2-0.attempt2/log.gs2.cbor.tmp']' returned non-zero exit status 1.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

30147

step1-simulation

host-error

yes

2019-12-11 21:00:16+00:00

2019-12-11 21:07:56+00:00

0:07:40

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1346, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/dt_shell/env_checks.py", line 67, in check_docker_environment
    _containers = client.containers.list(filters=dict(status="running"))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 925, in list
    containers.append(self.get(r['Id']))
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 862, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/container.py", line 755, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 230, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 643, in get_cr
    server_host=server_host,
  File "/project/src/duckietown_challenges_runner/runner.py", line 795, in run_single
    write_logs(wd, project, services=config["services"])
  File "/project/src/duckietown_challenges_runner/runner.py", line 1172, in write_logs
    client = check_docker_environment()
  File "/usr/local/lib/python3.6/dist-packages/dt_shell/env_checks.py", line 74, in check_docker_environment
    raise InvalidEnvironment(msg)
dt_shell.exceptions.InvalidEnvironment: I cannot communicate with Docker:
UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Make sure the docker service is running.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 648, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1309, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1565, in upload
    copy_to_cache(realfile, sha256hex)
  File "/project/src/duckietown_challenges_runner/runner_cache.py", line 42, in copy_to_cache
    shutil.copy(fn, have)
  File "/usr/lib/python3.6/shutil.py", line 245, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
  File "/usr/lib/python3.6/shutil.py", line 122, in copyfile
    copyfileobj(fsrc, fdst)
  File "/usr/lib/python3.6/shutil.py", line 82, in copyfileobj
    fdst.write(buf)
OSError: [Errno 28] No space left on device

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

30138

step1-simulation

host-error

yes

2019-12-11 20:56:41+00:00

2019-12-11 20:59:41+00:00

0:03:00

Uncaught exception: [...]

Uncaught exception:
Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 648, in get_cr
    wd, aws_config, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1309, in upload_files
    aws_config, toupload, copy_to_machine_cache=copy_to_machine_cache
  File "/project/src/duckietown_challenges_runner/runner.py", line 1563, in upload
    sha256hex = compute_sha256hex(realfile)
  File "/project/src/duckietown_challenges_runner/runner.py", line 1619, in compute_sha256hex
    res: bytes = subprocess.check_output(cmd)
  File "/usr/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['shasum', '-a', '256', '/tmp/duckietown/DT18/evaluator/executions/aido3-LF-sim-validation/submission6144/step1-simulation-idsc-rudolf-25153-job30138/tmp/tmpw9o6xh3_/attempts/ETHZ_autolab_technical_track-2-0.attempt2/log.gs2.cbor.tmp']' returned non-zero exit status 1.

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible

30136

step1-simulation

host-error

yes

2019-12-11 20:55:55+00:00

2019-12-11 21:00:55+00:00

0:05:00

Uncaught exception w [...]

Uncaught exception while running Docker Compose:
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 384, in _make_request
    six.raise_from(e, None)
  File "<string>", line 2, in raise_from
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 380, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.6/http/client.py", line 1346, in getresponse
    response.begin()
  File "/usr/lib/python3.6/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.6/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.6/socket.py", line 586, in readinto
    return self._sock.recv_into(b)
socket.timeout: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 638, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/usr/local/lib/python3.6/dist-packages/urllib3/util/retry.py", line 367, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/packages/six.py", line 686, in reraise
    raise value
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 600, in urlopen
    chunked=chunked)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/local/lib/python3.6/dist-packages/urllib3/connectionpool.py", line 306, in _raise_timeout
    raise ReadTimeoutError(self, url, "Read timed out. (read timeout=%s)" % timeout_value)
urllib3.exceptions.ReadTimeoutError: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/duckietown_challenges_runner/runner.py", line 919, in run_one
    container = client.containers.get(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/models/containers.py", line 862, in get
    resp = self.client.api.inspect_container(container_id)
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 19, in wrapped
    return f(self, resource_id, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/container.py", line 755, in inspect_container
    self._get(self._url("/containers/{0}/json", container)), True
  File "/usr/local/lib/python3.6/dist-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/docker/api/client.py", line 230, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 546, in get
    return self.request('GET', url, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: UnixHTTPConnectionPool(host='localhost', port=None): Read timed out. (read timeout=60)

Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.

No reset possible