Duckietown Challenges Home Challenges Submissions

Submission 12820

Submission12820
Competingyes
Challengeaido5-LFV_multi-sim-testing
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFVmultibodyv-sim: 59013
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

Show only up-to-date jobs
Job IDstepstatusup to datedate starteddate completeddurationmessage
59013LFVmultibodyv-simsuccessyes1:53:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.976460407066156
survival_time_median59.99999999999873
deviation-center-line_median3.643712486051149
in-drivable-lane_median1.9999999999999896


other stats
agent_compute-ego0_max0.013313889205703928
agent_compute-ego0_mean0.012595597621050472
agent_compute-ego0_median0.012948468563260882
agent_compute-ego0_min0.01136672089836381
agent_compute-ego1_max0.0135876850522031
agent_compute-ego1_mean0.012872101455280418
agent_compute-ego1_median0.013031916852596895
agent_compute-ego1_min0.011714705618890894
complete-iteration_max1.0645389868556014
complete-iteration_mean0.8600311573227079
complete-iteration_median1.0183725182360952
complete-iteration_min0.3594917437997282
deviation-center-line_max4.951713374087057
deviation-center-line_mean3.2933392906892194
deviation-center-line_min0.2491217844218268
deviation-heading_max21.0771131273286
deviation-heading_mean15.218213712907389
deviation-heading_median17.26085774001794
deviation-heading_min1.7858835330295706
driven_any_max12.516347955938697
driven_any_mean9.747668595554172
driven_any_median9.936540278099628
driven_any_min6.07200364361221
driven_lanedir_consec_max11.79460956076771
driven_lanedir_consec_mean8.098042258097774
driven_lanedir_consec_min0.666293928599336
driven_lanedir_max11.79460956076771
driven_lanedir_mean8.348164515655974
driven_lanedir_median9.09158767403523
driven_lanedir_min0.666293928599336
get_duckie_state_max1.92700178795909e-06
get_duckie_state_mean1.6793266317242605e-06
get_duckie_state_median1.914892268121292e-06
get_duckie_state_min1.3507077537111002e-06
get_robot_state_max0.01614329955857759
get_robot_state_mean0.014365840545860525
get_robot_state_median0.016081167795973753
get_robot_state_min0.00793745111210555
get_state_dump_max0.01060802712230063
get_state_dump_mean0.00963075090703912
get_state_dump_median0.010506369092879346
get_state_dump_min0.006760962364677277
get_ui_image_max0.053217307018499194
get_ui_image_mean0.046350004261439405
get_ui_image_median0.05104658407136661
get_ui_image_min0.03451641583772299
in-drivable-lane_max55.999999999998735
in-drivable-lane_mean8.12499999999975
in-drivable-lane_min0.0
per-episodes
details{"LFV_multi-norm-loop-000-ego0": {"driven_any": 10.122675501661174, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 9.573289904678772, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 0.0, "deviation-heading": 17.444805795259008, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 3.3854545748246996, "driven_lanedir_consec": 9.573289904678772, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-loop-000-ego1": {"driven_any": 12.193804966173126, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 10.811187895028771, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 1.8499999999999768, "deviation-heading": 19.791777784069374, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 3.931158479905904, "driven_lanedir_consec": 10.811187895028771, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-loop-000-ego2": {"driven_any": 12.143477667536065, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 11.299781796357363, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 0.7499999999999973, "deviation-heading": 18.39744440012714, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 3.54987611552523, "driven_lanedir_consec": 11.299781796357363, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-loop-000-ego3": {"driven_any": 9.750405054538083, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 8.609885443391688, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 14.499999999999288, "deviation-heading": 14.457715975377733, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 2.973087100023152, "driven_lanedir_consec": 8.609885443391688, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-zigzag-000-ego0": {"driven_any": 11.996922830083548, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 10.300386849070758, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 4.799999999999946, "deviation-heading": 18.985088690042527, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 4.176243355245407, "driven_lanedir_consec": 9.343035370740626, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-zigzag-000-ego1": {"driven_any": 11.955498309628249, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 10.749281310083909, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 2.050000000000029, "deviation-heading": 18.98220965800725, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 3.8709863485577687, "driven_lanedir_consec": 10.749281310083909, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-zigzag-000-ego2": {"driven_any": 11.908476706990283, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 10.451982649124009, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 1.9499999999999504, "deviation-heading": 20.84845558514661, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 4.312477462651137, "driven_lanedir_consec": 10.042282445808183, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-zigzag-000-ego3": {"driven_any": 6.455377214915915, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 0.666293928599336, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 55.999999999998735, "deviation-heading": 1.7858835330295706, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 0.2491217844218268, "driven_lanedir_consec": 0.666293928599336, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-techtrack-000-ego0": {"driven_any": 8.539738710842798, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 7.9768845231250385, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 0.0, "deviation-heading": 13.364022481677608, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 4.183435485422187, "driven_lanedir_consec": 7.9768845231250385, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-techtrack-000-ego1": {"driven_any": 12.516347955938697, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 11.79460956076771, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 0.0, "deviation-heading": 17.076909684776872, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 3.737548856577068, "driven_lanedir_consec": 11.79460956076771, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-techtrack-000-ego2": {"driven_any": 8.278649008068825, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 6.2786529847335135, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 26.599999999998488, "deviation-heading": 9.920710356300509, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 2.0315163381036108, "driven_lanedir_consec": 6.2786529847335135, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-techtrack-000-ego3": {"driven_any": 8.190008926824136, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 7.186048706895708, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 2.600000000000037, "deviation-heading": 21.0771131273286, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 4.951713374087057, "driven_lanedir_consec": 5.05138878272686, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-small_loop-000-ego0": {"driven_any": 6.343973840945295, "get_ui_image": 0.03451641583772299, "step_physics": 0.2549436733286868, "survival_time": 32.50000000000029, "driven_lanedir": 5.905091735960328, "get_state_dump": 0.006760962364677277, "get_robot_state": 0.00793745111210555, "sim_render-ego0": 0.004237389601137597, "sim_render-ego1": 0.004386013737105737, "get_duckie_state": 1.3700828024868592e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.127922087203382, "agent_compute-ego0": 0.012911026012696062, "agent_compute-ego1": 0.01343609513958112, "complete-iteration": 0.3594917437997282, "set_robot_commands": 0.002499534604003719, "deviation-center-line": 2.401490567668618, "driven_lanedir_consec": 5.905091735960328, "sim_compute_sim_state": 0.010835464465819565, "sim_compute_performance-ego0": 0.002219030933995401, "sim_compute_performance-ego1": 0.0021986382714431225}, "LFV_multi-norm-small_loop-000-ego1": {"driven_any": 6.07200364361221, "get_ui_image": 0.03451641583772299, "step_physics": 0.2549436733286868, "survival_time": 32.50000000000029, "driven_lanedir": 5.27092593136673, "get_state_dump": 0.006760962364677277, "get_robot_state": 0.00793745111210555, "sim_render-ego0": 0.004237389601137597, "sim_render-ego1": 0.004386013737105737, "get_duckie_state": 1.3700828024868592e-06, "in-drivable-lane": 2.650000000000031, "deviation-heading": 10.79493282235721, "agent_compute-ego0": 0.012911026012696062, "agent_compute-ego1": 0.01343609513958112, "complete-iteration": 0.3594917437997282, "set_robot_commands": 0.002499534604003719, "deviation-center-line": 2.3526402266354154, "driven_lanedir_consec": 5.27092593136673, "sim_compute_sim_state": 0.010835464465819565, "sim_compute_performance-ego0": 0.002219030933995401, "sim_compute_performance-ego1": 0.0021986382714431225}}
set_robot_commands_max0.0026103858646008495
set_robot_commands_mean0.0024983336811002484
set_robot_commands_median0.002600831850482264
set_robot_commands_min0.002283182866765895
sim_compute_performance-ego0_max0.002470941567401108
sim_compute_performance-ego0_mean0.0022921555297054368
sim_compute_performance-ego0_median0.0023948183464666487
sim_compute_performance-ego0_min0.00204726897310357
sim_compute_performance-ego1_max0.002302568818409179
sim_compute_performance-ego1_mean0.002155711099058601
sim_compute_performance-ego1_median0.0022384632040717816
sim_compute_performance-ego1_min0.0019046376885025825
sim_compute_sim_state_max0.047120913379297566
sim_compute_sim_state_mean0.03233408946558101
sim_compute_sim_state_median0.035296194956364185
sim_compute_sim_state_min0.010835464465819565
sim_render-ego0_max0.004398492254087272
sim_render-ego0_mean0.004179163271923542
sim_render-ego0_median0.004344039912227786
sim_render-ego0_min0.0037658444848485432
sim_render-ego1_max0.0044544350594704
sim_render-ego1_mean0.004300349083098747
sim_render-ego1_median0.004446103908338713
sim_render-ego1_min0.003957675954483629
simulation-passed1
step_physics_max0.8598917588703241
step_physics_mean0.6765027194197674
step_physics_median0.801531524086475
step_physics_min0.2549436733286868
survival_time_max59.99999999999873
survival_time_mean56.07142857142753
survival_time_min32.50000000000029
No reset possible
59012LFVmultibodyv-simsuccessyes1:47:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
59011LFVmultibodyv-simfailedyes0:02:52
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego1: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible