Evaluation Jobs

Job opportunities

Submission	Date	User	Challenge	Step	Admin priority	User priority	Previous	Avoid	Evaluator assignment
`3838`	2019-06-27 20:48:59+00:00	kene4192	`aido2-LF-real-validation-offseason`	eval0	100	50	{}	1542	No assignment yet.
`3816`	2019-06-12 12:59:03+00:00	AmaurX	`aido2-LF-real-validation-offseason`	eval0-videos-autobots	60	50	{'eval0': 24033}		No assignment yet.
`3988`	2019-07-18 04:42:26+00:00	jagwar	`aido2-LFVI-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`3985`	2019-07-18 04:42:25+00:00	jagwar	`aido2-LFV-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`3982`	2019-07-18 04:42:25+00:00	jagwar	`aido2-LF-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`3963`	2019-07-16 22:34:37+00:00	jagwar	`aido2-LFVI-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`3960`	2019-07-16 22:34:37+00:00	jagwar	`aido2-LFV-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`3957`	2019-07-16 22:34:36+00:00	jagwar	`aido2-LF-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`3935`	2019-07-13 23:22:14+00:00	kene4192	`aido2-LFVI-real-validation-offseason`	eval0	50	50	{}		No assignment yet.
`3932`	2019-07-13 23:22:14+00:00	kene4192	`aido2-LFV-real-validation-offseason`	eval0	50	50	{}		No assignment yet.

See all job opportunities

page 2

Job ID	submission	user	user label	challenge	step	status	up to date	evaluator	date started	date completed	duration	message
`24217`	`4011`	kene4192	CNN-48by96-less-one-50-aug	`aido2-LF-sim-validation-offseason`	step1-simulation	failed	yes	`1290`	2019-07-19 04:44:56+00:00	2019-07-19 04:46:15+00:00	0:01:19	InvalidSubmission: T [...] InvalidSubmission: Traceback (most recent call last): File "experiment_manager.py", line 297, in run_episode r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands') File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one nickname=self.nickname) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply others = read_until_over(fpout, timeout=timeout, nickname=nickname) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over raise RemoteNodeAborted(m) zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted: error in agent \|Exception while handling a message on topic "get_commands". \| \|\| Traceback (most recent call last): \|\| File "/workspace/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop \|\| handle_message_node(parsed, receiver0, context0) \|\| File "/workspace/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node \|\| call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing) \|\| File "/workspace/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists \|\| f(*kwargs) \|\| File "imitation_agent.py", line 83, in on_received_get_commands \|\| pwm_left, pwm_right = self.compute_action(self.config.current_image) \|\| File "imitation_agent.py", line 52, in compute_action \|\| graph = load_graph(frozen_model_filename) \|\| File "/workspace/graph_utils.py", line 15, in load_graph \|\| tf.import_graph_def(graph_def, name="prefix") \|\| File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func \|\| return func(args, **kwargs) \|\| File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 426, in import_graph_def \|\| graph._c_graph, serialized, options) # pylint: disable=protected-access \|\| tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'ImageProjectiveTransformV2' in binary running on 8c10866f3f0e. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed. \|\| The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context yield cie File "experiment_manager.py", line 441, in <module> wrap(cie) File "experiment_manager.py", line 429, in wrap main(cie, logdir, attempts) File "experiment_manager.py", line 135, in main physics_dt=config.physics_dt) File "experiment_manager.py", line 301, in run_episode raise dc.InvalidSubmission(msg) from e duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.	(hidden)
`24216`	`4010`	kene4192	CNN-48by96-less-one-50-aug	`aido2-LFV-sim-validation-offseason`	step1-simulation	failed	yes	`1291`	2019-07-19 04:43:19+00:00	2019-07-19 04:46:15+00:00	0:02:56	InvalidSubmission: T [...] InvalidSubmission: Traceback (most recent call last): File "experiment_manager.py", line 297, in run_episode r: MsgReceived = agent.write_topic_and_expect('get_commands', expect='commands') File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 93, in write_topic_and_expect ob: MsgReceived = self.read_one(expect_topic=expect, timeout=timeout) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 174, in read_one nickname=self.nickname) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 255, in read_reply others = read_until_over(fpout, timeout=timeout, nickname=nickname) File "/project/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper_outside.py", line 283, in read_until_over raise RemoteNodeAborted(m) zuper_nodes.structures.RemoteNodeAborted: External node "agent" aborted: error in agent \|Exception while handling a message on topic "get_commands". \| \|\| Traceback (most recent call last): \|\| File "/workspace/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 274, in loop \|\| handle_message_node(parsed, receiver0, context0) \|\| File "/workspace/src/zuper-nodes/src/zuper_nodes_wrapper/wrapper.py", line 428, in handle_message_node \|\| call_if_fun_exists(agent, expect_fn, data=ob, context=context, timing=timing) \|\| File "/workspace/src/zuper-nodes/src/zuper_nodes_wrapper/utils.py", line 18, in call_if_fun_exists \|\| f(*kwargs) \|\| File "imitation_agent.py", line 83, in on_received_get_commands \|\| pwm_left, pwm_right = self.compute_action(self.config.current_image) \|\| File "imitation_agent.py", line 52, in compute_action \|\| graph = load_graph(frozen_model_filename) \|\| File "/workspace/graph_utils.py", line 15, in load_graph \|\| tf.import_graph_def(graph_def, name="prefix") \|\| File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py", line 507, in new_func \|\| return func(args, **kwargs) \|\| File "/usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/importer.py", line 426, in import_graph_def \|\| graph._c_graph, serialized, options) # pylint: disable=protected-access \|\| tensorflow.python.framework.errors_impl.NotFoundError: Op type not registered 'ImageProjectiveTransformV2' in binary running on ef0495743c75. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) `tf.contrib.resampler` should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed. \|\| The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/project/src/duckietown-challenges/src/duckietown_challenges/cie_concrete.py", line 610, in scoring_context yield cie File "experiment_manager.py", line 441, in <module> wrap(cie) File "experiment_manager.py", line 429, in wrap main(cie, logdir, attempts) File "experiment_manager.py", line 135, in main physics_dt=config.physics_dt) File "experiment_manager.py", line 301, in run_episode raise dc.InvalidSubmission(msg) from e duckietown_challenges.exceptions.InvalidSubmission: Trouble with communication to the agent.	(hidden)
`24215`	`4009`	kene4192	CNN-8logs-60by80-latest	`aido2-LFV-sim-validation-offseason`	step1-simulation	success	yes	`1291`	2019-07-19 00:07:39+00:00	2019-07-19 00:24:16+00:00	0:16:37		(hidden)
`24214`	`4008`	jagwar	Baseline solution using reinforcement learning	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1291`	2019-07-18 22:34:49+00:00	2019-07-18 22:43:25+00:00	0:08:36		(hidden)
`24213`	`4007`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1291`	2019-07-18 21:54:13+00:00	2019-07-18 21:54:38+00:00	0:00:25	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24212`	`4006`	jagwar	Baseline solution using reinforcement learning	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1291`	2019-07-18 21:45:12+00:00	2019-07-18 21:49:50+00:00	0:04:38		(hidden)
`24211`	`4005`	jagwar	Baseline solution using reinforcement learning	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1291`	2019-07-18 21:38:10+00:00	2019-07-18 21:41:03+00:00	0:02:53		(hidden)
`24210`	`4004`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1292`	2019-07-18 21:33:49+00:00	2019-07-18 21:34:17+00:00	0:00:28	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24209`	`4003`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1292`	2019-07-18 21:24:54+00:00	2019-07-18 21:26:43+00:00	0:01:49	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24208`	`4002`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1292`	2019-07-18 20:59:24+00:00	2019-07-18 20:59:50+00:00	0:00:26	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24207`	`4001`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1290`	2019-07-18 20:55:26+00:00	2019-07-18 20:55:52+00:00	0:00:26	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24206`	`4000`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1291`	2019-07-18 20:40:59+00:00	2019-07-18 20:41:27+00:00	0:00:28	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24205`	`3999`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1291`	2019-07-18 20:37:22+00:00	2019-07-18 20:37:50+00:00	0:00:28	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24204`	`3998`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1290`	2019-07-18 20:01:36+00:00	2019-07-18 20:02:01+00:00	0:00:25	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24203`	`3998`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	host-error	yes	`1291`	2019-07-18 19:59:17+00:00	2019-07-18 20:00:52+00:00	0:01:35	Uncaught exception w [...] Uncaught exception while running Docker Compose: Traceback (most recent call last): File "/project/src/duckietown_challenges_runner/runner.py", line 759, in run_one run_docker(wd, project, cmd) File "/project/src/duckietown_challenges_runner/runner.py", line 1100, in run_docker subprocess.check_call(cmd0, cwd=cwd, stdout=tee_stdout, stderr=tee_stderr) File "/usr/lib/python3.6/subprocess.py", line 286, in check_call retcode = call(popenargs, kwargs) File "/usr/lib/python3.6/subprocess.py", line 267, in call with Popen(popenargs, **kwargs) as p: File "/usr/lib/python3.6/subprocess.py", line 709, in __init__ restore_signals, start_new_session) File "/usr/lib/python3.6/subprocess.py", line 1344, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-validation-offseason/submission3998/step1-simulation-ip-172-31-42-167-7227-job24203': '/tmp/duckietown/DT18/evaluator/executions/aido2-LFV-sim-validation-offseason/submission3998/step1-simulation-ip-172-31-42-167-7227-job24203'	(hidden)
`24202`	`3997`	kene4192	CNN-8logs-60by80-300	`aido2-LFV-sim-validation-offseason`	step1-simulation	success	yes	`1292`	2019-07-18 18:34:46+00:00	2019-07-18 18:46:54+00:00	0:12:08		(hidden)
`24201`	`3996`	kene4192	CNN-8logs-48by96-400	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1290`	2019-07-18 18:28:18+00:00	2019-07-18 18:36:50+00:00	0:08:32		(hidden)
`24200`	`3995`	jagwar	Baseline solution using reinforcement learning	`aido2-LF-sim-validation-offseason`	step1-simulation	success	yes	`1292`	2019-07-18 18:08:57+00:00	2019-07-18 18:14:48+00:00	0:05:51		(hidden)
`24199`	`3994`	kene4192	CNN-8logs-48by96-300-same	`aido2-LFV-sim-validation-offseason`	step1-simulation	success	yes	`1290`	2019-07-18 18:08:32+00:00	2019-07-18 18:22:33+00:00	0:14:01		(hidden)
`24198`	`3993`	kene4192	CNN-8logs-48by96-400	`aido2-LFV-sim-validation-offseason`	step1-simulation	success	yes	`1290`	2019-07-18 17:42:04+00:00	2019-07-18 17:58:23+00:00	0:16:19		(hidden)
`24197`	`3992`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	error	yes	`1290`	2019-07-18 17:13:09+00:00	2019-07-18 17:14:31+00:00	0:01:22	The container "solut [...] `The container "solution" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24196`	`3991`	Inderpal	Baseline solution using reinforcement learning	`aido2-LFV-sim-validation-offseason`	step1-simulation	success	yes	`1292`	2019-07-18 15:17:17+00:00	2019-07-18 15:21:32+00:00	0:04:15		(hidden)
`24195`	`3983`	jagwar	Baseline solution using reinforcement learning	`aido2-LF-sim-testing-offseason`	step1-simulation	error	yes	`1292`	2019-07-18 04:56:57+00:00	2019-07-18 05:00:18+00:00	0:03:21	The container "evalu [...] `The container "evaluator" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24194`	`3986`	jagwar	Baseline solution using reinforcement learning	`aido2-LFV-sim-testing-offseason`	step1-simulation	error	yes	`1290`	2019-07-18 04:49:38+00:00	2019-07-18 05:00:59+00:00	0:11:21	The container "evalu [...] `The container "evaluator" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)
`24193`	`3989`	jagwar	Baseline solution using reinforcement learning	`aido2-LFVI-sim-testing-offseason`	step1-simulation	error	yes	`1291`	2019-07-18 04:48:00+00:00	2019-07-18 05:00:27+00:00	0:12:27	The container "evalu [...] `The container "evaluator" exited with code 1. Look at the logs for the container to know more about the error.`	(hidden)

List of evaluation jobs

Job opportunities