Duckietown Challenges Home Challenges Submissions

Evaluator 1346

ID1346
evaluatorduckietown3-20424
ownerPascal Auf der Maur 🇨🇭
machineduckietown3
processduckietown3-20424
versiond-c:4.0.30;d-c-r:4.0.46;d-s:4.0.22
first heard
last heard
statusinactive
# evaluating
# success4 23365
# timeout1 23373
# failed
# error
# aborted
# host-error
arm
x86_64
Mac
gpu available
Number of processors
Processor frequency (MHz)
Free % of processors
RAM total (MB)
RAM free (MB)
Disk (MB)
Disk available (MB)
Docker Hub
P1
P2
Cloud simulations
PI Camera
# Duckiebots
Map 3x3 avaiable
Number of duckies
gpu cores
AIDO 2 Map LF public
AIDO 2 Map LF private
AIDO 2 Map LFV public
AIDO 2 Map LFV private
AIDO 2 Map LFVI public
AIDO 2 Map LFVI private
AIDO 3 Map LF public
AIDO 3 Map LF private
AIDO 3 Map LFV public
AIDO 3 Map LFV private
AIDO 3 Map LFVI public
AIDO 3 Map LFVI private
AIDO 5 Map large loop
ETU track
for 2021, map is ETH_small_inter
IPFS mountpoint /ipfs available
IPNS mountpoint /ipns available

Evaluator jobs

Job IDsubmissionuseruser labelchallengestepstatusup to dateevaluatordate starteddate completeddurationmessage
233733547Julian ZillyBaseline-IL-logs-tensorflowaido2-LF-real-validationeval1timeoutyesduckietown3-204245:05:52
I can see how the jo [...]
I can see how the job 23373 is timeout because passed 18352 seconds and the timeout is 18000.0.
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
233723593Victor Adolfo Romero Cano 🇨🇴aido2-LF-real-validationeval1successyesduckietown3-204240:27:03
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.58
survival_time_median2


other stats
01_start
details{"1_01attemptDate": {"name": "1_01attemptDate", "output": "2019-05-19", "readable_string": "Attempt date"}, "1_03autolabName": {"name": "1_03autolabName", "output": "ETHZ", "readable_string": "Autolab"}, "1_04operatorName": {"name": "1_04operatorName", "output": "Pascal", "readable_string": "Operator's name"}}
02_autolab_setup
details{"2_04ledLightsOn": {"name": "2_04ledLightsOn", "output": true, "readable_string": "Ceiling led lightbulbs are on with a bright white color"}, "2_01mapConformity": {"name": "2_01mapConformity", "output": true, "readable_string": "Conformity of the map"}, "2_05curtainsClosed": {"name": "2_05curtainsClosed", "output": true, "readable_string": "The curtains are closed"}, "2_02pairedHostnames": {"name": "2_02pairedHostnames", "output": true, "readable_string": "Pairing hostnames to duckiebot-IDs"}, "2_03ceilingLightsOff": {"name": "2_03ceilingLightsOff", "output": true, "readable_string": "Ceiling default neon lights are off"}, "2_multi01_duckiebot_select_autobot01": {"name": "2_multi01_duckiebot_select_autobot01", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot02": {"name": "2_multi01_duckiebot_select_autobot02", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot03": {"name": "2_multi01_duckiebot_select_autobot03", "output": "active", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot04": {"name": "2_multi01_duckiebot_select_autobot04", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot05": {"name": "2_multi01_duckiebot_select_autobot05", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot06": {"name": "2_multi01_duckiebot_select_autobot06", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot07": {"name": "2_multi01_duckiebot_select_autobot07", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot08": {"name": "2_multi01_duckiebot_select_autobot08", "output": "not_used", "readable_string": "Selection among available duckiebot"}}
03_setup_validation
details{"3_11freeMemory": {"name": "3_11freeMemory", "output": true, "readable_string": "All agents have enough free memory"}, "3_01robotsAreOn": {"name": "3_01robotsAreOn", "output": true, "readable_string": "All duckiebots are on"}, "3_09agentsPinged": {"name": "3_09agentsPinged", "output": true, "readable_string": "All agents have been pinged"}, "3_03duckiesOnBots": {"name": "3_03duckiesOnBots", "output": true, "readable_string": "All autobots received yeelow or black duckies (according to active of passive)"}, "3_08attemptNumber": {"name": "3_08attemptNumber", "output": "2", "readable_string": "Attempt number"}, "3_04batteryChecked": {"name": "3_04batteryChecked", "output": true, "readable_string": "All duckiebots have enough battery"}, "3_06agentPlacement": {"name": "3_06agentPlacement", "output": true, "readable_string": "Placement of the duckiebots"}, "3_02aprilTagsPlates": {"name": "3_02aprilTagsPlates", "output": true, "readable_string": "All duckiebots have their april tag backplate, in the right orientation"}, "3_05agentsPluggedIn": {"name": "3_05agentsPluggedIn", "output": false, "readable_string": "All non-movable agents (watchtowers and trafficlights) are plugged in (alimentation and ethernet)"}, "3_10readWriteMemory": {"name": "3_10readWriteMemory", "output": true, "readable_string": "All agents have been tested for read and write on their logging device (USB stick)"}, "3_07lightCheckedReport": {"name": "3_07lightCheckedReport", "output": false, "readable_string": "Report on light conditions"}, "3_hidden_autolab_helper_command": {"name": "3_hidden_autolab_helper_command", "output": "python3 autolab_helper_script.py 3593 aido2-LF-real-validation autobot03 ", "readable_string": "The autolab helper script launch command"}}
04_initialization
details{"4_01duckiebotInterfaceStarted": {"name": "4_01duckiebotInterfaceStarted", "output": true, "readable_string": "All duckiebots have stared their duckiebot interface container"}, "4_02submissionContainerLoaded": {"name": "4_02submissionContainerLoaded", "output": true, "readable_string": "All active duckiebots have loaded the submission container"}, "4_04agentsStartedLoggerContainers": {"name": "4_04agentsStartedLoggerContainers", "output": true, "readable_string": "All agents have loaded and started the logger container"}, "4_03passiveDuckiebotsContainerLoaded": {"name": "4_03passiveDuckiebotsContainerLoaded", "output": false, "readable_string": "All passive duckiebots have loaded the passive behavior container"}}
05_confirmation
details{"5_01operatorIsReady": {"name": "5_01operatorIsReady", "output": true, "readable_string": "The operator is ready to start"}, "5_02serverInitializationError": {"name": "5_02serverInitializationError", "output": "", "readable_string": "The server initialization error message"}}
06_execution
details{"6_06visualCheck": {"name": "6_06visualCheck", "output": "", "readable_string": "Comment from operator on visual check of launching"}, "6_03beginTimeStamp": {"name": "6_03beginTimeStamp", "output": "1558280332.470414", "readable_string": "Beginning timestamp (useful for attempt evaluation)"}, "6_01launchedAttempt": {"name": "6_01launchedAttempt", "output": true, "readable_string": "The attempt was successfully launched"}, "6_02serverLaunchError": {"name": "6_02serverLaunchError", "output": "", "readable_string": "The server launch error message"}, "6_09watchtowerLoggings": {"name": "6_09watchtowerLoggings", "output": true, "readable_string": "Watchtowers started logging data"}, "6_10serverLoggingCheck": {"name": "6_10serverLoggingCheck", "output": "", "readable_string": "Logging error message from the server"}, "6_07activeDuckiebotLogging": {"name": "6_07activeDuckiebotLogging", "output": true, "readable_string": "Active duckiebots started logging data"}, "6_08passiveDuckiebotLogging": {"name": "6_08passiveDuckiebotLogging", "output": false, "readable_string": "Passive duckiebots started logging data"}, "6_04activeDuckiebotsAreRunningVisualCheck": {"name": "6_04activeDuckiebotsAreRunningVisualCheck", "output": true, "readable_string": "Active duckiebots started moving after launch"}, "6_05passiveDuckiebotsAreRunningVisualCheck": {"name": "6_05passiveDuckiebotsAreRunningVisualCheck", "output": false, "readable_string": "Passive duckiebots started moving after launch"}}
07_monitoring_and_termination
details{"7_05errorReason": {"name": "7_05errorReason", "output": "no_error", "readable_string": "Reason for error"}, "7_01endTimeStamp": {"name": "7_01endTimeStamp", "output": "1558280340.716525", "readable_string": "end of the experiment timestamp"}, "7_03failureReason": {"name": "7_03failureReason", "output": "no_failure", "readable_string": "Reason for failure"}, "7_06errorComments": {"name": "7_06errorComments", "output": "", "readable_string": "Comment on error reason"}, "7_02attemptSuccess": {"name": "7_02attemptSuccess", "output": "success", "readable_string": "Outcome of the attempt"}, "7_04failureComments": {"name": "7_04failureComments", "output": "", "readable_string": "Comment on failure reason"}, "7_07outcomechecking": {"name": "7_07outcomechecking", "output": false, "readable_string": "Checked that the outcome matched the given reasons"}}
08_retrieving_and_checking_logs
details{"8_01bagsCopied": {"name": "8_01bagsCopied", "output": true, "readable_string": "All bags were successfully copied"}, "8_03bagsClipped": {"name": "8_03bagsClipped", "output": false, "readable_string": "All bags were clipped with the time stamps"}, "8_02bagsVerified": {"name": "8_02bagsVerified", "output": true, "readable_string": "The contents of all bags was verified"}, "8_04agentsCleaned": {"name": "8_04agentsCleaned", "output": true, "readable_string": "All agents were cleaned from bags"}}
09_evaluation
details{"9_02timeTravelled": {"name": "9_02timeTravelled", "output": "2", "readable_string": "How many seconds did the Duckiebot successfully travel"}, "9_01tilesTravelled": {"name": "9_01tilesTravelled", "output": "1", "readable_string": "How many tiles did the Duckiebot successfully travel"}}
10_saving_data
details{"10_02dataIpfsHash": {"name": "10_02dataIpfsHash", "output": "QmWq8BeqLSiY6sfsjDSXrRULVixN9wq3cS3nMnBSzvJ82U", "readable_string": "IPFS hash of the experiment"}, "10_04cleanupCheck": {"name": "10_04cleanupCheck", "output": false, "readable_string": "The IPFS Docker terminal and the local storage were cleaned"}, "10_05autolabCleaned": {"name": "10_05autolabCleaned", "output": true, "readable_string": "Autolab was cleaned after the end of the experiment"}, "10_01dataUploadCheck": {"name": "10_01dataUploadCheck", "output": false, "readable_string": "The data was uploaded to the IDSC NAS"}, "10_03dataPinningCheck": {"name": "10_03dataPinningCheck", "output": false, "readable_string": "The data is accessible through IPFS"}}
No reset possible
233703616jiang pengFourth test - THaido2-LF-real-validationeval0successyesduckietown3-204240:34:59
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0
survival_time_median0


other stats
01_start
details{"1_01attemptDate": {"name": "1_01attemptDate", "output": "2019-05-19", "readable_string": "Attempt date"}, "1_03autolabName": {"name": "1_03autolabName", "output": "ETHZ", "readable_string": "Autolab"}, "1_04operatorName": {"name": "1_04operatorName", "output": "Pascal", "readable_string": "Operator's name"}}
02_autolab_setup
details{"2_04ledLightsOn": {"name": "2_04ledLightsOn", "output": true, "readable_string": "Ceiling led lightbulbs are on with a bright white color"}, "2_01mapConformity": {"name": "2_01mapConformity", "output": true, "readable_string": "Conformity of the map"}, "2_05curtainsClosed": {"name": "2_05curtainsClosed", "output": true, "readable_string": "The curtains are closed"}, "2_02pairedHostnames": {"name": "2_02pairedHostnames", "output": true, "readable_string": "Pairing hostnames to duckiebot-IDs"}, "2_03ceilingLightsOff": {"name": "2_03ceilingLightsOff", "output": true, "readable_string": "Ceiling default neon lights are off"}, "2_multi01_duckiebot_select_autobot01": {"name": "2_multi01_duckiebot_select_autobot01", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot02": {"name": "2_multi01_duckiebot_select_autobot02", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot03": {"name": "2_multi01_duckiebot_select_autobot03", "output": "active", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot04": {"name": "2_multi01_duckiebot_select_autobot04", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot05": {"name": "2_multi01_duckiebot_select_autobot05", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot06": {"name": "2_multi01_duckiebot_select_autobot06", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot07": {"name": "2_multi01_duckiebot_select_autobot07", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot08": {"name": "2_multi01_duckiebot_select_autobot08", "output": "not_used", "readable_string": "Selection among available duckiebot"}}
03_setup_validation
details{"3_11freeMemory": {"name": "3_11freeMemory", "output": true, "readable_string": "All agents have enough free memory"}, "3_01robotsAreOn": {"name": "3_01robotsAreOn", "output": true, "readable_string": "All duckiebots are on"}, "3_09agentsPinged": {"name": "3_09agentsPinged", "output": true, "readable_string": "All agents have been pinged"}, "3_03duckiesOnBots": {"name": "3_03duckiesOnBots", "output": true, "readable_string": "All autobots received yeelow or black duckies (according to active of passive)"}, "3_08attemptNumber": {"name": "3_08attemptNumber", "output": "1", "readable_string": "Attempt number"}, "3_04batteryChecked": {"name": "3_04batteryChecked", "output": true, "readable_string": "All duckiebots have enough battery"}, "3_06agentPlacement": {"name": "3_06agentPlacement", "output": true, "readable_string": "Placement of the duckiebots"}, "3_02aprilTagsPlates": {"name": "3_02aprilTagsPlates", "output": true, "readable_string": "All duckiebots have their april tag backplate, in the right orientation"}, "3_05agentsPluggedIn": {"name": "3_05agentsPluggedIn", "output": true, "readable_string": "All non-movable agents (watchtowers and trafficlights) are plugged in (alimentation and ethernet)"}, "3_10readWriteMemory": {"name": "3_10readWriteMemory", "output": true, "readable_string": "All agents have been tested for read and write on their logging device (USB stick)"}, "3_07lightCheckedReport": {"name": "3_07lightCheckedReport", "output": false, "readable_string": "Report on light conditions"}, "3_hidden_autolab_helper_command": {"name": "3_hidden_autolab_helper_command", "output": "python3 autolab_helper_script.py 3616 aido2-LF-real-validation autobot03 ", "readable_string": "The autolab helper script launch command"}}
04_initialization
details{"4_01duckiebotInterfaceStarted": {"name": "4_01duckiebotInterfaceStarted", "output": true, "readable_string": "All duckiebots have stared their duckiebot interface container"}, "4_02submissionContainerLoaded": {"name": "4_02submissionContainerLoaded", "output": true, "readable_string": "All active duckiebots have loaded the submission container"}, "4_04agentsStartedLoggerContainers": {"name": "4_04agentsStartedLoggerContainers", "output": true, "readable_string": "All agents have loaded and started the logger container"}, "4_03passiveDuckiebotsContainerLoaded": {"name": "4_03passiveDuckiebotsContainerLoaded", "output": false, "readable_string": "All passive duckiebots have loaded the passive behavior container"}}
05_confirmation
details{"5_01operatorIsReady": {"name": "5_01operatorIsReady", "output": true, "readable_string": "The operator is ready to start"}, "5_02serverInitializationError": {"name": "5_02serverInitializationError", "output": "", "readable_string": "The server initialization error message"}}
06_execution
details{"6_06visualCheck": {"name": "6_06visualCheck", "output": "The container exited", "readable_string": "Comment from operator on visual check of launching"}, "6_03beginTimeStamp": {"name": "6_03beginTimeStamp", "output": "1558278230.476936", "readable_string": "Beginning timestamp (useful for attempt evaluation)"}, "6_01launchedAttempt": {"name": "6_01launchedAttempt", "output": false, "readable_string": "The attempt was successfully launched"}, "6_02serverLaunchError": {"name": "6_02serverLaunchError", "output": "Error in node PytorchRLBaseline", "readable_string": "The server launch error message"}, "6_09watchtowerLoggings": {"name": "6_09watchtowerLoggings", "output": true, "readable_string": "Watchtowers started logging data"}, "6_10serverLoggingCheck": {"name": "6_10serverLoggingCheck", "output": "", "readable_string": "Logging error message from the server"}, "6_07activeDuckiebotLogging": {"name": "6_07activeDuckiebotLogging", "output": true, "readable_string": "Active duckiebots started logging data"}, "6_08passiveDuckiebotLogging": {"name": "6_08passiveDuckiebotLogging", "output": false, "readable_string": "Passive duckiebots started logging data"}, "6_04activeDuckiebotsAreRunningVisualCheck": {"name": "6_04activeDuckiebotsAreRunningVisualCheck", "output": false, "readable_string": "Active duckiebots started moving after launch"}, "6_05passiveDuckiebotsAreRunningVisualCheck": {"name": "6_05passiveDuckiebotsAreRunningVisualCheck", "output": false, "readable_string": "Passive duckiebots started moving after launch"}}
07_monitoring_and_termination
details{"7_05errorReason": {"name": "7_05errorReason", "output": "no_error", "readable_string": "Reason for error"}, "7_01endTimeStamp": {"name": "7_01endTimeStamp", "output": "1558278230.576936", "readable_string": "end of the experiment timestamp"}, "7_03failureReason": {"name": "7_03failureReason", "output": "other", "readable_string": "Reason for failure"}, "7_06errorComments": {"name": "7_06errorComments", "output": "", "readable_string": "Comment on error reason"}, "7_02attemptSuccess": {"name": "7_02attemptSuccess", "output": "failure", "readable_string": "Outcome of the attempt"}, "7_04failureComments": {"name": "7_04failureComments", "output": "Exception: Error in node PytorchRLBaseline", "readable_string": "Comment on failure reason"}, "7_07outcomechecking": {"name": "7_07outcomechecking", "output": false, "readable_string": "Checked that the outcome matched the given reasons"}}
08_retrieving_and_checking_logs
details{"8_01bagsCopied": {"name": "8_01bagsCopied", "output": true, "readable_string": "All bags were successfully copied"}, "8_03bagsClipped": {"name": "8_03bagsClipped", "output": false, "readable_string": "All bags were clipped with the time stamps"}, "8_02bagsVerified": {"name": "8_02bagsVerified", "output": true, "readable_string": "The contents of all bags was verified"}, "8_04agentsCleaned": {"name": "8_04agentsCleaned", "output": true, "readable_string": "All agents were cleaned from bags"}}
09_evaluation
details{"9_02timeTravelled": {"name": "9_02timeTravelled", "output": "0", "readable_string": "How many seconds did the Duckiebot successfully travel"}, "9_01tilesTravelled": {"name": "9_01tilesTravelled", "output": "0", "readable_string": "How many tiles did the Duckiebot successfully travel"}}
10_saving_data
details{"10_02dataIpfsHash": {"name": "10_02dataIpfsHash", "output": "QmS6XnL6BSP3PExEkDEuUNu9nXwbJxAh6zs9QL2zoTw6KY", "readable_string": "IPFS hash of the experiment"}, "10_04cleanupCheck": {"name": "10_04cleanupCheck", "output": false, "readable_string": "The IPFS Docker terminal and the local storage were cleaned"}, "10_05autolabCleaned": {"name": "10_05autolabCleaned", "output": true, "readable_string": "Autolab was cleaned after the end of the experiment"}, "10_01dataUploadCheck": {"name": "10_01dataUploadCheck", "output": false, "readable_string": "The data was uploaded to the IDSC NAS"}, "10_03dataPinningCheck": {"name": "10_03dataPinningCheck", "output": false, "readable_string": "The data is accessible through IPFS"}}
No reset possible
233693605Riza Velioglu 🇩🇪aido2-LF-real-validationeval0successyesduckietown3-204240:36:35
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median0.29
survival_time_median14


other stats
01_start
details{"1_01attemptDate": {"name": "1_01attemptDate", "output": "2019-05-19", "readable_string": "Attempt date"}, "1_03autolabName": {"name": "1_03autolabName", "output": "ETHZ", "readable_string": "Autolab"}, "1_04operatorName": {"name": "1_04operatorName", "output": "Pascal", "readable_string": "Operator's name"}}
02_autolab_setup
details{"2_04ledLightsOn": {"name": "2_04ledLightsOn", "output": true, "readable_string": "Ceiling led lightbulbs are on with a bright white color"}, "2_01mapConformity": {"name": "2_01mapConformity", "output": true, "readable_string": "Conformity of the map"}, "2_05curtainsClosed": {"name": "2_05curtainsClosed", "output": true, "readable_string": "The curtains are closed"}, "2_02pairedHostnames": {"name": "2_02pairedHostnames", "output": true, "readable_string": "Pairing hostnames to duckiebot-IDs"}, "2_03ceilingLightsOff": {"name": "2_03ceilingLightsOff", "output": true, "readable_string": "Ceiling default neon lights are off"}, "2_multi01_duckiebot_select_autobot01": {"name": "2_multi01_duckiebot_select_autobot01", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot02": {"name": "2_multi01_duckiebot_select_autobot02", "output": "active", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot03": {"name": "2_multi01_duckiebot_select_autobot03", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot04": {"name": "2_multi01_duckiebot_select_autobot04", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot05": {"name": "2_multi01_duckiebot_select_autobot05", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot06": {"name": "2_multi01_duckiebot_select_autobot06", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot07": {"name": "2_multi01_duckiebot_select_autobot07", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot08": {"name": "2_multi01_duckiebot_select_autobot08", "output": "not_used", "readable_string": "Selection among available duckiebot"}}
03_setup_validation
details{"3_11freeMemory": {"name": "3_11freeMemory", "output": true, "readable_string": "All agents have enough free memory"}, "3_01robotsAreOn": {"name": "3_01robotsAreOn", "output": true, "readable_string": "All duckiebots are on"}, "3_09agentsPinged": {"name": "3_09agentsPinged", "output": true, "readable_string": "All agents have been pinged"}, "3_03duckiesOnBots": {"name": "3_03duckiesOnBots", "output": true, "readable_string": "All autobots received yeelow or black duckies (according to active of passive)"}, "3_08attemptNumber": {"name": "3_08attemptNumber", "output": "1", "readable_string": "Attempt number"}, "3_04batteryChecked": {"name": "3_04batteryChecked", "output": true, "readable_string": "All duckiebots have enough battery"}, "3_06agentPlacement": {"name": "3_06agentPlacement", "output": true, "readable_string": "Placement of the duckiebots"}, "3_02aprilTagsPlates": {"name": "3_02aprilTagsPlates", "output": true, "readable_string": "All duckiebots have their april tag backplate, in the right orientation"}, "3_05agentsPluggedIn": {"name": "3_05agentsPluggedIn", "output": true, "readable_string": "All non-movable agents (watchtowers and trafficlights) are plugged in (alimentation and ethernet)"}, "3_10readWriteMemory": {"name": "3_10readWriteMemory", "output": true, "readable_string": "All agents have been tested for read and write on their logging device (USB stick)"}, "3_07lightCheckedReport": {"name": "3_07lightCheckedReport", "output": false, "readable_string": "Report on light conditions"}, "3_hidden_autolab_helper_command": {"name": "3_hidden_autolab_helper_command", "output": "python3 autolab_helper_script.py 3605 aido2-LF-real-validation autobot02 ", "readable_string": "The autolab helper script launch command"}}
04_initialization
details{"4_01duckiebotInterfaceStarted": {"name": "4_01duckiebotInterfaceStarted", "output": true, "readable_string": "All duckiebots have stared their duckiebot interface container"}, "4_02submissionContainerLoaded": {"name": "4_02submissionContainerLoaded", "output": true, "readable_string": "All active duckiebots have loaded the submission container"}, "4_04agentsStartedLoggerContainers": {"name": "4_04agentsStartedLoggerContainers", "output": true, "readable_string": "All agents have loaded and started the logger container"}, "4_03passiveDuckiebotsContainerLoaded": {"name": "4_03passiveDuckiebotsContainerLoaded", "output": false, "readable_string": "All passive duckiebots have loaded the passive behavior container"}}
05_confirmation
details{"5_01operatorIsReady": {"name": "5_01operatorIsReady", "output": true, "readable_string": "The operator is ready to start"}, "5_02serverInitializationError": {"name": "5_02serverInitializationError", "output": "", "readable_string": "The server initialization error message"}}
06_execution
details{"6_06visualCheck": {"name": "6_06visualCheck", "output": "", "readable_string": "Comment from operator on visual check of launching"}, "6_03beginTimeStamp": {"name": "6_03beginTimeStamp", "output": "1558276364.438780", "readable_string": "Beginning timestamp (useful for attempt evaluation)"}, "6_01launchedAttempt": {"name": "6_01launchedAttempt", "output": true, "readable_string": "The attempt was successfully launched"}, "6_02serverLaunchError": {"name": "6_02serverLaunchError", "output": "", "readable_string": "The server launch error message"}, "6_09watchtowerLoggings": {"name": "6_09watchtowerLoggings", "output": true, "readable_string": "Watchtowers started logging data"}, "6_10serverLoggingCheck": {"name": "6_10serverLoggingCheck", "output": "", "readable_string": "Logging error message from the server"}, "6_07activeDuckiebotLogging": {"name": "6_07activeDuckiebotLogging", "output": true, "readable_string": "Active duckiebots started logging data"}, "6_08passiveDuckiebotLogging": {"name": "6_08passiveDuckiebotLogging", "output": false, "readable_string": "Passive duckiebots started logging data"}, "6_04activeDuckiebotsAreRunningVisualCheck": {"name": "6_04activeDuckiebotsAreRunningVisualCheck", "output": true, "readable_string": "Active duckiebots started moving after launch"}, "6_05passiveDuckiebotsAreRunningVisualCheck": {"name": "6_05passiveDuckiebotsAreRunningVisualCheck", "output": false, "readable_string": "Passive duckiebots started moving after launch"}}
07_monitoring_and_termination
details{"7_05errorReason": {"name": "7_05errorReason", "output": "no_error", "readable_string": "Reason for error"}, "7_01endTimeStamp": {"name": "7_01endTimeStamp", "output": "1558276388.114300", "readable_string": "end of the experiment timestamp"}, "7_03failureReason": {"name": "7_03failureReason", "output": "no_failure", "readable_string": "Reason for failure"}, "7_06errorComments": {"name": "7_06errorComments", "output": "", "readable_string": "Comment on error reason"}, "7_02attemptSuccess": {"name": "7_02attemptSuccess", "output": "success", "readable_string": "Outcome of the attempt"}, "7_04failureComments": {"name": "7_04failureComments", "output": "", "readable_string": "Comment on failure reason"}, "7_07outcomechecking": {"name": "7_07outcomechecking", "output": false, "readable_string": "Checked that the outcome matched the given reasons"}}
08_retrieving_and_checking_logs
details{"8_01bagsCopied": {"name": "8_01bagsCopied", "output": true, "readable_string": "All bags were successfully copied"}, "8_03bagsClipped": {"name": "8_03bagsClipped", "output": false, "readable_string": "All bags were clipped with the time stamps"}, "8_02bagsVerified": {"name": "8_02bagsVerified", "output": true, "readable_string": "The contents of all bags was verified"}, "8_04agentsCleaned": {"name": "8_04agentsCleaned", "output": true, "readable_string": "All agents were cleaned from bags"}}
09_evaluation
details{"9_02timeTravelled": {"name": "9_02timeTravelled", "output": "14", "readable_string": "How many seconds did the Duckiebot successfully travel"}, "9_01tilesTravelled": {"name": "9_01tilesTravelled", "output": "0.5", "readable_string": "How many tiles did the Duckiebot successfully travel"}}
10_saving_data
details{"10_02dataIpfsHash": {"name": "10_02dataIpfsHash", "output": "QmXky7DQXxxxyMpuMj6e7nSnCHYKRzauVaBquuLdXTSwi5", "readable_string": "IPFS hash of the experiment"}, "10_04cleanupCheck": {"name": "10_04cleanupCheck", "output": false, "readable_string": "The IPFS Docker terminal and the local storage were cleaned"}, "10_05autolabCleaned": {"name": "10_05autolabCleaned", "output": true, "readable_string": "Autolab was cleaned after the end of the experiment"}, "10_01dataUploadCheck": {"name": "10_01dataUploadCheck", "output": false, "readable_string": "The data was uploaded to the IDSC NAS"}, "10_03dataPinningCheck": {"name": "10_03dataPinningCheck", "output": false, "readable_string": "The data is accessible through IPFS"}}
No reset possible
233653549Bea Baselines 🐤Running the real lane following -- without challenge containeraido2-LF-real-validationeval1successyesduckietown3-204241:09:18
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.7
survival_time_median35


other stats
01_start
details{"1_01attemptDate": {"name": "1_01attemptDate", "output": "2019-05-19", "readable_string": "Attempt date"}, "1_03autolabName": {"name": "1_03autolabName", "output": "ETHZ", "readable_string": "Autolab"}, "1_04operatorName": {"name": "1_04operatorName", "output": "Pascal", "readable_string": "Operator's name"}}
02_autolab_setup
details{"2_04ledLightsOn": {"name": "2_04ledLightsOn", "output": true, "readable_string": "Ceiling led lightbulbs are on with a bright white color"}, "2_01mapConformity": {"name": "2_01mapConformity", "output": true, "readable_string": "Conformity of the map"}, "2_05curtainsClosed": {"name": "2_05curtainsClosed", "output": true, "readable_string": "The curtains are closed"}, "2_02pairedHostnames": {"name": "2_02pairedHostnames", "output": true, "readable_string": "Pairing hostnames to duckiebot-IDs"}, "2_03ceilingLightsOff": {"name": "2_03ceilingLightsOff", "output": true, "readable_string": "Ceiling default neon lights are off"}, "2_multi01_duckiebot_select_autobot01": {"name": "2_multi01_duckiebot_select_autobot01", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot02": {"name": "2_multi01_duckiebot_select_autobot02", "output": "active", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot03": {"name": "2_multi01_duckiebot_select_autobot03", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot04": {"name": "2_multi01_duckiebot_select_autobot04", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot05": {"name": "2_multi01_duckiebot_select_autobot05", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot06": {"name": "2_multi01_duckiebot_select_autobot06", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot07": {"name": "2_multi01_duckiebot_select_autobot07", "output": "not_used", "readable_string": "Selection among available duckiebot"}, "2_multi01_duckiebot_select_autobot08": {"name": "2_multi01_duckiebot_select_autobot08", "output": "not_used", "readable_string": "Selection among available duckiebot"}}
03_setup_validation
details{"3_11freeMemory": {"name": "3_11freeMemory", "output": true, "readable_string": "All agents have enough free memory"}, "3_01robotsAreOn": {"name": "3_01robotsAreOn", "output": true, "readable_string": "All duckiebots are on"}, "3_09agentsPinged": {"name": "3_09agentsPinged", "output": true, "readable_string": "All agents have been pinged"}, "3_03duckiesOnBots": {"name": "3_03duckiesOnBots", "output": true, "readable_string": "All autobots received yeelow or black duckies (according to active of passive)"}, "3_08attemptNumber": {"name": "3_08attemptNumber", "output": "7", "readable_string": "Attempt number"}, "3_04batteryChecked": {"name": "3_04batteryChecked", "output": true, "readable_string": "All duckiebots have enough battery"}, "3_06agentPlacement": {"name": "3_06agentPlacement", "output": true, "readable_string": "Placement of the duckiebots"}, "3_02aprilTagsPlates": {"name": "3_02aprilTagsPlates", "output": true, "readable_string": "All duckiebots have their april tag backplate, in the right orientation"}, "3_05agentsPluggedIn": {"name": "3_05agentsPluggedIn", "output": true, "readable_string": "All non-movable agents (watchtowers and trafficlights) are plugged in (alimentation and ethernet)"}, "3_10readWriteMemory": {"name": "3_10readWriteMemory", "output": true, "readable_string": "All agents have been tested for read and write on their logging device (USB stick)"}, "3_07lightCheckedReport": {"name": "3_07lightCheckedReport", "output": false, "readable_string": "Report on light conditions"}, "3_hidden_autolab_helper_command": {"name": "3_hidden_autolab_helper_command", "output": "python3 autolab_helper_script.py 3549 aido2-LF-real-validation autobot02 ", "readable_string": "The autolab helper script launch command"}}
04_initialization
details{"4_01duckiebotInterfaceStarted": {"name": "4_01duckiebotInterfaceStarted", "output": true, "readable_string": "All duckiebots have stared their duckiebot interface container"}, "4_02submissionContainerLoaded": {"name": "4_02submissionContainerLoaded", "output": true, "readable_string": "All active duckiebots have loaded the submission container"}, "4_04agentsStartedLoggerContainers": {"name": "4_04agentsStartedLoggerContainers", "output": true, "readable_string": "All agents have loaded and started the logger container"}, "4_03passiveDuckiebotsContainerLoaded": {"name": "4_03passiveDuckiebotsContainerLoaded", "output": false, "readable_string": "All passive duckiebots have loaded the passive behavior container"}}
05_confirmation
details{"5_01operatorIsReady": {"name": "5_01operatorIsReady", "output": true, "readable_string": "The operator is ready to start"}, "5_02serverInitializationError": {"name": "5_02serverInitializationError", "output": "", "readable_string": "The server initialization error message"}}
06_execution
details{"6_06visualCheck": {"name": "6_06visualCheck", "output": "", "readable_string": "Comment from operator on visual check of launching"}, "6_03beginTimeStamp": {"name": "6_03beginTimeStamp", "output": "1558273872.893051", "readable_string": "Beginning timestamp (useful for attempt evaluation)"}, "6_01launchedAttempt": {"name": "6_01launchedAttempt", "output": true, "readable_string": "The attempt was successfully launched"}, "6_02serverLaunchError": {"name": "6_02serverLaunchError", "output": "", "readable_string": "The server launch error message"}, "6_09watchtowerLoggings": {"name": "6_09watchtowerLoggings", "output": false, "readable_string": "Watchtowers started logging data"}, "6_10serverLoggingCheck": {"name": "6_10serverLoggingCheck", "output": "", "readable_string": "Logging error message from the server"}, "6_07activeDuckiebotLogging": {"name": "6_07activeDuckiebotLogging", "output": false, "readable_string": "Active duckiebots started logging data"}, "6_08passiveDuckiebotLogging": {"name": "6_08passiveDuckiebotLogging", "output": false, "readable_string": "Passive duckiebots started logging data"}, "6_04activeDuckiebotsAreRunningVisualCheck": {"name": "6_04activeDuckiebotsAreRunningVisualCheck", "output": true, "readable_string": "Active duckiebots started moving after launch"}, "6_05passiveDuckiebotsAreRunningVisualCheck": {"name": "6_05passiveDuckiebotsAreRunningVisualCheck", "output": false, "readable_string": "Passive duckiebots started moving after launch"}}
07_monitoring_and_termination
details{"7_05errorReason": {"name": "7_05errorReason", "output": "no_error", "readable_string": "Reason for error"}, "7_01endTimeStamp": {"name": "7_01endTimeStamp", "output": "1558273908.068892", "readable_string": "end of the experiment timestamp"}, "7_03failureReason": {"name": "7_03failureReason", "output": "no_failure", "readable_string": "Reason for failure"}, "7_06errorComments": {"name": "7_06errorComments", "output": "", "readable_string": "Comment on error reason"}, "7_02attemptSuccess": {"name": "7_02attemptSuccess", "output": "success", "readable_string": "Outcome of the attempt"}, "7_04failureComments": {"name": "7_04failureComments", "output": "", "readable_string": "Comment on failure reason"}, "7_07outcomechecking": {"name": "7_07outcomechecking", "output": false, "readable_string": "Checked that the outcome matched the given reasons"}}
08_retrieving_and_checking_logs
details{"8_01bagsCopied": {"name": "8_01bagsCopied", "output": true, "readable_string": "All bags were successfully copied"}, "8_03bagsClipped": {"name": "8_03bagsClipped", "output": false, "readable_string": "All bags were clipped with the time stamps"}, "8_02bagsVerified": {"name": "8_02bagsVerified", "output": true, "readable_string": "The contents of all bags was verified"}, "8_04agentsCleaned": {"name": "8_04agentsCleaned", "output": true, "readable_string": "All agents were cleaned from bags"}}
09_evaluation
details{"9_02timeTravelled": {"name": "9_02timeTravelled", "output": "35", "readable_string": "How many seconds did the Duckiebot successfully travel"}, "9_01tilesTravelled": {"name": "9_01tilesTravelled", "output": "15", "readable_string": "How many tiles did the Duckiebot successfully travel"}}
10_saving_data
details{"10_02dataIpfsHash": {"name": "10_02dataIpfsHash", "output": "QmRXxgMubahGgXoLU6ZBNfiX4nrYstK8kDybgmyoaVAuwy", "readable_string": "IPFS hash of the experiment"}, "10_04cleanupCheck": {"name": "10_04cleanupCheck", "output": false, "readable_string": "The IPFS Docker terminal and the local storage were cleaned"}, "10_05autolabCleaned": {"name": "10_05autolabCleaned", "output": true, "readable_string": "Autolab was cleaned after the end of the experiment"}, "10_01dataUploadCheck": {"name": "10_01dataUploadCheck", "output": false, "readable_string": "The data was uploaded to the IDSC NAS"}, "10_03dataPinningCheck": {"name": "10_03dataPinningCheck", "output": false, "readable_string": "The data is accessible through IPFS"}}
No reset possible