Evaluation Jobs

Job opportunities

Submission	Date	User	Challenge	Step	Admin priority	User priority	Previous	Avoid	Evaluator assignment
`3838`	2019-06-27 20:48:59+00:00	kene4192	`aido2-LF-real-validation-offseason`	eval0	90	50	{}	1542	No assignment yet.
`3816`	2019-06-12 12:59:03+00:00	AmaurX	`aido2-LF-real-validation-offseason`	eval0	60	50	{}		No assignment yet.
`4065`	2019-09-14 19:48:38+00:00	CharanReddy	`aido2-PRED`	step1-simulation	50	50	{}		No assignment yet.
`4064`	2019-09-09 16:05:51+00:00	bdura	`aido2-PRED`	step1-simulation	50	50	{}		No assignment yet.
`4063`	2019-09-04 08:21:26+00:00	marcus15951	`aido2-LFVI-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`4062`	2019-09-04 08:21:26+00:00	marcus15951	`aido2-LFV-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`4061`	2019-09-04 08:21:26+00:00	marcus15951	`aido2-LF-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`4060`	2019-09-04 07:36:43+00:00	marcus15951	`aido2-PRED`	step1-simulation	50	50	{}		No assignment yet.
`4059`	2019-09-03 18:27:48+00:00	lzoltan	`aido2-PRED`	step1-simulation	50	50	{}		No assignment yet.
`4058`	2019-09-03 18:26:30+00:00	lzoltan	`aido2-PRED`	step1-simulation	50	50	{}		No assignment yet.

See all job opportunities

page 2

Job ID	submission	user	user label	challenge	step	status	up to date	evaluator	date started	date completed	duration	message
`24252`	`4065`	CharanReddy	PredictorLast	`aido2-PRED`	step1-simulation	timeout	yes	`1561`	2019-09-16 19:32:45+00:00	2019-09-16 19:39:58+00:00	0:07:13	I can see how the jo [...] `I can see how the job 24252 is timeout because passed 433 seconds and the timeout is 120.0.`	(hidden)
`24251`	`4025`	dgmds17av	challenge-aido_LF-template-pytorch	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1291`	2019-08-13 23:16:12+00:00	2019-08-13 23:20:05+00:00	0:03:53		(hidden)
`24250`	`4026`	dgmds17av	challenge-aido_LF-template-pytorch	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1292`	2019-08-13 23:12:34+00:00	2019-08-13 23:17:58+00:00	0:05:24		(hidden)
`24249`	`4027`	frank_qcd_qk	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1292`	2019-08-13 23:11:35+00:00	2019-08-13 23:11:57+00:00	0:00:22		(hidden)
`24248`	`4028`	frank_qcd_qk	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1292`	2019-08-13 23:10:39+00:00	2019-08-13 23:11:04+00:00	0:00:25		(hidden)
`24247`	`4029`	frank_qcd_qk	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1292`	2019-08-13 23:09:37+00:00	2019-08-13 23:10:05+00:00	0:00:28		(hidden)
`24246`	`4030`	frank_qcd_qk	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1292`	2019-08-13 23:08:40+00:00	2019-08-13 23:09:04+00:00	0:00:24		(hidden)
`24245`	`4031`	frank_qcd_qk	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1292`	2019-08-13 23:07:03+00:00	2019-08-13 23:07:57+00:00	0:00:54		(hidden)
`24244`	`4034`	frank_qcd_qk	challenge-aido_LF-template-ros - Template solution using ROS	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1292`	2019-08-13 23:01:26+00:00	2019-08-13 23:06:55+00:00	0:05:29		(hidden)
`24243`	`4036`	frank_qcd_qk	challenge-aido_LF-template-ros - Template solution using ROS	`aido2-LFV-sim-testing-offseason`	step1-simulation	success	yes	`1290`	2019-08-13 23:01:26+00:00	2019-08-13 23:22:09+00:00	0:20:43		(hidden)
`24242`	`4033`	frank_qcd_qk	challenge-aido_LF-template-ros - Template solution using ROS	`aido2-LF-sim-testing-offseason`	step1-simulation	success	yes	`1291`	2019-08-13 23:01:24+00:00	2019-08-13 23:15:46+00:00	0:14:22		(hidden)
`24241`	`4037`	frank_qcd_qk	challenge-aido_LF-template-ros - Template solution using ROS	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1290`	2019-08-13 22:55:09+00:00	2019-08-13 23:00:57+00:00	0:05:48	The container "evalu [...] `The container "evaluator" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24240`	`4039`	frank_qcd_qk	challenge-aido_LF-template-ros - Template solution using ROS	`aido2-LFVI-sim-testing-offseason`	step1-simulation	error	yes	`1292`	2019-08-13 22:53:28+00:00	2019-08-13 23:00:55+00:00	0:07:27	The container "evalu [...] `The container "evaluator" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24239`	`4040`	frank_qcd_qk	challenge-aido_LF-template-ros - Template solution using ROS	`aido2-LFVI-sim-validation-offseason`	step1-simulation	error	yes	`1291`	2019-08-13 22:53:25+00:00	2019-08-13 23:00:55+00:00	0:07:30	The container "evalu [...] `The container "evaluator" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24238`	`4043`	riza.velioglu	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1291`	2019-08-13 22:51:07+00:00	2019-08-13 22:52:44+00:00	0:01:37		(hidden)
`24237`	`4042`	stoplight	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1292`	2019-08-13 22:51:02+00:00	2019-08-13 22:52:45+00:00	0:01:43		(hidden)
`24236`	`4041`	frank_qcd_qk	challenge-aido_LF-template-tensorflow	`aido2-LF-sim-testing-offseason`	step1-simulation	error	yes	`1290`	2019-08-13 22:50:53+00:00	2019-08-13 22:54:20+00:00	0:03:27	InvalidEvaluator: Tr [...] InvalidEvaluator: Traceback (most recent call last): File "experiment_manager.py", line 135, in main physics_dt=config.physics_dt) File "experiment_manager.py", line 305, in run_episode sim_ci.write_topic_and_expect_zero('set_robot_commands', commands) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero nickname=self.nickname) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply others = read_until_over(fpout, timeout=timeout, nickname=nickname) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over raise RemoteNodeAborted(m) zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted: error in simulator \|Exception while handling a message on topic "set_robot_commands". \| \|\| Traceback (most recent call last): \|\| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop \|\| handle_message_node(parsed, receiver0, context0) \|\| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node \|\| call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing) \|\| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists \|\| f(kwargs) \|\| File "gym_bridge.py", line 301, in on_received_set_robot_commands \|\| raise Exception(msg) \|\| Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}. \|\| The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context yield cie File "experiment_manager.py", line 441, in <module> wrap(cie) File "experiment_manager.py", line 429, in wrap main(cie, logdir, attempts) File "experiment_manager.py", line 186, in main raise dc.InvalidEvaluator(msg) from e duckietown_challenges.exceptions.InvalidEvaluator: Anomalous error while running episodes: > Traceback (most recent call last): > File "experiment_manager.py", line 135, in main > physics_dt=config.physics_dt) > File "experiment_manager.py", line 305, in run_episode > sim_ci.write_topic_and_expect_zero('set_robot_commands', commands) > File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 102, in write_topic_and_expect_zero > nickname=self.nickname) > File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply > others = read_until_over(fpout, timeout=timeout, nickname=nickname) > File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over > raise RemoteNodeAborted(m) > zuper_nodes.structures.RemoteNodeAborted: External node "simulator" aborted: > > error in simulator \|Exception while handling a message on topic "set_robot_commands". > \| > \|\| Traceback (most recent call last): > \|\| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop > \|\| handle_message_node(parsed, receiver0, context0) > \|\| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node > \|\| call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing) > \|\| File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists > \|\| f(kwargs) > \|\| File "gym_bridge.py", line 301, in on_received_set_robot_commands > \|\| raise Exception(msg) > \|\| Exception: Received invalid PWM commands. They should be between -1 and +1. Received left = {l}, right = {r}. > \|\| >	(hidden)
`24235`	`4042`	stoplight	PredictorLast	`aido2-PRED`	step1-simulation	host-error	yes	`1559`	2019-08-08 14:53:54+00:00	2019-08-08 14:53:57+00:00	0:00:03	Uncaught exception: [...] Uncaught exception: Traceback (most recent call last): File "/project/src/duckietown_challenges_runner/runner.py", line 523, in get_cr try_s3(aws_config) File "/project/src/duckietown_challenges_runner/runner.py", line 1258, in try_s3 s3_object.upload_fileobj(data) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 621, in object_upload_fileobj ExtraArgs=ExtraArgs, Callback=Callback, Config=Config) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 539, in upload_fileobj return future.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 73, in result return self._coordinator.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 233, in result raise self._exception File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 126, in __call__ return self._execute_main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 150, in _execute_main return_value = self._main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/upload.py", line 692, in _main client.put_object(Bucket=bucket, Key=key, Body=body, extra_args) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 661, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.	(hidden)
`24234`	`4025`	dgmds17av	challenge-aido_LF-template-pytorch	`aido2-LF-sim-validation-offseason`	step1-simulation	host-error	yes	`1558`	2019-07-31 15:14:21+00:00	2019-07-31 15:14:23+00:00	0:00:02	Uncaught exception: [...] Uncaught exception: Traceback (most recent call last): File "/project/src/duckietown_challenges_runner/runner.py", line 523, in get_cr try_s3(aws_config) File "/project/src/duckietown_challenges_runner/runner.py", line 1258, in try_s3 s3_object.upload_fileobj(data) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 621, in object_upload_fileobj ExtraArgs=ExtraArgs, Callback=Callback, Config=Config) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 539, in upload_fileobj return future.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 73, in result return self._coordinator.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 233, in result raise self._exception File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 126, in __call__ return self._execute_main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 150, in _execute_main return_value = self._main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/upload.py", line 692, in _main client.put_object(Bucket=bucket, Key=key, Body=body, extra_args) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 661, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.	(hidden)
`24233`	`4026`	dgmds17av	challenge-aido_LF-template-pytorch	`aido2-LF-sim-validation-offseason`	step1-simulation	host-error	yes	`1558`	2019-07-31 15:14:00+00:00	2019-07-31 15:14:05+00:00	0:00:05	Uncaught exception: [...] Uncaught exception: Traceback (most recent call last): File "/project/src/duckietown_challenges_runner/runner.py", line 523, in get_cr try_s3(aws_config) File "/project/src/duckietown_challenges_runner/runner.py", line 1258, in try_s3 s3_object.upload_fileobj(data) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 621, in object_upload_fileobj ExtraArgs=ExtraArgs, Callback=Callback, Config=Config) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 539, in upload_fileobj return future.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 73, in result return self._coordinator.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 233, in result raise self._exception File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 126, in __call__ return self._execute_main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 150, in _execute_main return_value = self._main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/upload.py", line 692, in _main client.put_object(Bucket=bucket, Key=key, Body=body, extra_args) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 661, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.	(hidden)
`24232`	`4025`	dgmds17av	challenge-aido_LF-template-pytorch	`aido2-LF-sim-validation-offseason`	step1-simulation	host-error	yes	`1557`	2019-07-30 21:36:19+00:00	2019-07-30 21:36:23+00:00	0:00:04	Uncaught exception: [...] Uncaught exception: Traceback (most recent call last): File "/project/src/duckietown_challenges_runner/runner.py", line 523, in get_cr try_s3(aws_config) File "/project/src/duckietown_challenges_runner/runner.py", line 1258, in try_s3 s3_object.upload_fileobj(data) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 621, in object_upload_fileobj ExtraArgs=ExtraArgs, Callback=Callback, Config=Config) File "/usr/local/lib/python3.6/dist-packages/boto3/s3/inject.py", line 539, in upload_fileobj return future.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 73, in result return self._coordinator.result() File "/usr/local/lib/python3.6/dist-packages/s3transfer/futures.py", line 233, in result raise self._exception File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 126, in __call__ return self._execute_main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/tasks.py", line 150, in _execute_main return_value = self._main(kwargs) File "/usr/local/lib/python3.6/dist-packages/s3transfer/upload.py", line 692, in _main client.put_object(Bucket=bucket, Key=key, Body=body, extra_args) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 357, in _api_call return self._make_api_call(operation_name, kwargs) File "/usr/local/lib/python3.6/dist-packages/botocore/client.py", line 661, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (InvalidAccessKeyId) when calling the PutObject operation: The AWS Access Key Id you provided does not exist in our records.	(hidden)
`24231`	`4024`	meanstreets	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1292`	2019-07-29 00:30:22+00:00	2019-07-29 00:30:44+00:00	0:00:22		(hidden)
`24230`	`4024`	meanstreets	PredictorLast	`aido2-PRED`	step1-simulation	success	yes	`1291`	2019-07-29 00:29:53+00:00	2019-07-29 00:30:20+00:00	0:00:27		(hidden)
`24229`	`4023`	dgmds17av	challenge-aido_LF-template-pytorch	`aido2-LF-sim-validation-offseason`	step1-simulation	error	yes	`1292`	2019-07-27 23:24:14+00:00	2019-07-27 23:25:40+00:00	0:01:26	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24228`	`4022`	dgmds17av	challenge-aido_LF-template-pytorch	`aido2-LF-sim-validation-offseason`	step1-simulation	error	yes	`1556`	2019-07-27 22:38:11+00:00	2019-07-27 22:39:05+00:00	0:00:54	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)

List of evaluation jobs

Job opportunities