Duckietown Challenges Home Challenges Submissions

Submission 12820

Submission12820
Competingyes
Challengeaido5-LFV_multi-sim-testing
UserYishu Malhotra 🇨🇦
Date submitted
Last status update
Completecomplete
DetailsEvaluation is complete.
Sisters
Result💚
JobsLFVmultibodyv-sim: 59013
Next
User labelsim-exercise-2
Admin priority50
Blessingn/a
User priority50

The highlights are available only to the owner and the admins.

Evaluation jobs for this submission

See previous jobs for previous versions of challenges
Job IDstepstatusup to datedate starteddate completeddurationmessage
59013LFVmultibodyv-simsuccessyes1:53:54
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
driven_lanedir_consec_median8.976460407066156
survival_time_median59.99999999999873
deviation-center-line_median3.643712486051149
in-drivable-lane_median1.9999999999999896


other stats
agent_compute-ego0_max0.013313889205703928
agent_compute-ego0_mean0.012595597621050472
agent_compute-ego0_median0.012948468563260882
agent_compute-ego0_min0.01136672089836381
agent_compute-ego1_max0.0135876850522031
agent_compute-ego1_mean0.012872101455280418
agent_compute-ego1_median0.013031916852596895
agent_compute-ego1_min0.011714705618890894
complete-iteration_max1.0645389868556014
complete-iteration_mean0.8600311573227079
complete-iteration_median1.0183725182360952
complete-iteration_min0.3594917437997282
deviation-center-line_max4.951713374087057
deviation-center-line_mean3.2933392906892194
deviation-center-line_min0.2491217844218268
deviation-heading_max21.0771131273286
deviation-heading_mean15.218213712907389
deviation-heading_median17.26085774001794
deviation-heading_min1.7858835330295706
driven_any_max12.516347955938697
driven_any_mean9.747668595554172
driven_any_median9.936540278099628
driven_any_min6.07200364361221
driven_lanedir_consec_max11.79460956076771
driven_lanedir_consec_mean8.098042258097774
driven_lanedir_consec_min0.666293928599336
driven_lanedir_max11.79460956076771
driven_lanedir_mean8.348164515655974
driven_lanedir_median9.09158767403523
driven_lanedir_min0.666293928599336
get_duckie_state_max1.92700178795909e-06
get_duckie_state_mean1.6793266317242605e-06
get_duckie_state_median1.914892268121292e-06
get_duckie_state_min1.3507077537111002e-06
get_robot_state_max0.01614329955857759
get_robot_state_mean0.014365840545860525
get_robot_state_median0.016081167795973753
get_robot_state_min0.00793745111210555
get_state_dump_max0.01060802712230063
get_state_dump_mean0.00963075090703912
get_state_dump_median0.010506369092879346
get_state_dump_min0.006760962364677277
get_ui_image_max0.053217307018499194
get_ui_image_mean0.046350004261439405
get_ui_image_median0.05104658407136661
get_ui_image_min0.03451641583772299
in-drivable-lane_max55.999999999998735
in-drivable-lane_mean8.12499999999975
in-drivable-lane_min0.0
per-episodes
details{"LFV_multi-norm-loop-000-ego0": {"driven_any": 10.122675501661174, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 9.573289904678772, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 0.0, "deviation-heading": 17.444805795259008, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 3.3854545748246996, "driven_lanedir_consec": 9.573289904678772, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-loop-000-ego1": {"driven_any": 12.193804966173126, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 10.811187895028771, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 1.8499999999999768, "deviation-heading": 19.791777784069374, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 3.931158479905904, "driven_lanedir_consec": 10.811187895028771, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-loop-000-ego2": {"driven_any": 12.143477667536065, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 11.299781796357363, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 0.7499999999999973, "deviation-heading": 18.39744440012714, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 3.54987611552523, "driven_lanedir_consec": 11.299781796357363, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-loop-000-ego3": {"driven_any": 9.750405054538083, "get_ui_image": 0.040702915906310574, "step_physics": 0.5788643983480436, "survival_time": 59.99999999999873, "driven_lanedir": 8.609885443391688, "get_state_dump": 0.009212750777118313, "get_robot_state": 0.01408724899990771, "sim_render-ego0": 0.0037658444848485432, "sim_render-ego1": 0.003957675954483629, "sim_render-ego2": 0.003957432175953124, "sim_render-ego3": 0.003982096488628658, "get_duckie_state": 1.3507077537111002e-06, "in-drivable-lane": 14.499999999999288, "deviation-heading": 14.457715975377733, "agent_compute-ego0": 0.01136672089836381, "agent_compute-ego1": 0.011714705618890894, "agent_compute-ego2": 0.01175042989351271, "agent_compute-ego3": 0.011929589445446056, "complete-iteration": 0.7474516736379174, "set_robot_commands": 0.002283182866765895, "deviation-center-line": 2.973087100023152, "driven_lanedir_consec": 8.609885443391688, "sim_compute_sim_state": 0.025334472560961975, "sim_compute_performance-ego0": 0.00204726897310357, "sim_compute_performance-ego1": 0.0019046376885025825, "sim_compute_performance-ego2": 0.0019424259414482275, "sim_compute_performance-ego3": 0.0019227360607086868}, "LFV_multi-norm-zigzag-000-ego0": {"driven_any": 11.996922830083548, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 10.300386849070758, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 4.799999999999946, "deviation-heading": 18.985088690042527, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 4.176243355245407, "driven_lanedir_consec": 9.343035370740626, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-zigzag-000-ego1": {"driven_any": 11.955498309628249, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 10.749281310083909, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 2.050000000000029, "deviation-heading": 18.98220965800725, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 3.8709863485577687, "driven_lanedir_consec": 10.749281310083909, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-zigzag-000-ego2": {"driven_any": 11.908476706990283, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 10.451982649124009, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 1.9499999999999504, "deviation-heading": 20.84845558514661, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 4.312477462651137, "driven_lanedir_consec": 10.042282445808183, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-zigzag-000-ego3": {"driven_any": 6.455377214915915, "get_ui_image": 0.053217307018499194, "step_physics": 0.801531524086475, "survival_time": 59.99999999999873, "driven_lanedir": 0.666293928599336, "get_state_dump": 0.01060802712230063, "get_robot_state": 0.016081167795973753, "sim_render-ego0": 0.004344039912227786, "sim_render-ego1": 0.0044544350594704, "sim_render-ego2": 0.004452805038693545, "sim_render-ego3": 0.00453622970454004, "get_duckie_state": 1.914892268121292e-06, "in-drivable-lane": 55.999999999998735, "deviation-heading": 1.7858835330295706, "agent_compute-ego0": 0.012948468563260882, "agent_compute-ego1": 0.013031916852596895, "agent_compute-ego2": 0.01315478798153994, "agent_compute-ego3": 0.013339345004536726, "complete-iteration": 1.0183725182360952, "set_robot_commands": 0.0026103858646008495, "deviation-center-line": 0.2491217844218268, "driven_lanedir_consec": 0.666293928599336, "sim_compute_sim_state": 0.047120913379297566, "sim_compute_performance-ego0": 0.002470941567401108, "sim_compute_performance-ego1": 0.0022384632040717816, "sim_compute_performance-ego2": 0.0022721651888806854, "sim_compute_performance-ego3": 0.00226553830377863}, "LFV_multi-norm-techtrack-000-ego0": {"driven_any": 8.539738710842798, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 7.9768845231250385, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 0.0, "deviation-heading": 13.364022481677608, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 4.183435485422187, "driven_lanedir_consec": 7.9768845231250385, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-techtrack-000-ego1": {"driven_any": 12.516347955938697, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 11.79460956076771, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 0.0, "deviation-heading": 17.076909684776872, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 3.737548856577068, "driven_lanedir_consec": 11.79460956076771, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-techtrack-000-ego2": {"driven_any": 8.278649008068825, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 6.2786529847335135, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 26.599999999998488, "deviation-heading": 9.920710356300509, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 2.0315163381036108, "driven_lanedir_consec": 6.2786529847335135, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-techtrack-000-ego3": {"driven_any": 8.190008926824136, "get_ui_image": 0.05104658407136661, "step_physics": 0.8598917588703241, "survival_time": 59.99999999999873, "driven_lanedir": 7.186048706895708, "get_state_dump": 0.010506369092879346, "get_robot_state": 0.01614329955857759, "sim_render-ego0": 0.004398492254087272, "sim_render-ego1": 0.004446103908338713, "sim_render-ego2": 0.0045104038705436715, "sim_render-ego3": 0.0044636295598115055, "get_duckie_state": 1.92700178795909e-06, "in-drivable-lane": 2.600000000000037, "deviation-heading": 21.0771131273286, "agent_compute-ego0": 0.013313889205703928, "agent_compute-ego1": 0.0135876850522031, "agent_compute-ego2": 0.013301661766935247, "agent_compute-ego3": 0.013700959485933048, "complete-iteration": 1.0645389868556014, "set_robot_commands": 0.002600831850482264, "deviation-center-line": 4.951713374087057, "driven_lanedir_consec": 5.05138878272686, "sim_compute_sim_state": 0.035296194956364185, "sim_compute_performance-ego0": 0.0023948183464666487, "sim_compute_performance-ego1": 0.002302568818409179, "sim_compute_performance-ego2": 0.002299370118521532, "sim_compute_performance-ego3": 0.002304946850181916}, "LFV_multi-norm-small_loop-000-ego0": {"driven_any": 6.343973840945295, "get_ui_image": 0.03451641583772299, "step_physics": 0.2549436733286868, "survival_time": 32.50000000000029, "driven_lanedir": 5.905091735960328, "get_state_dump": 0.006760962364677277, "get_robot_state": 0.00793745111210555, "sim_render-ego0": 0.004237389601137597, "sim_render-ego1": 0.004386013737105737, "get_duckie_state": 1.3700828024868592e-06, "in-drivable-lane": 0.0, "deviation-heading": 10.127922087203382, "agent_compute-ego0": 0.012911026012696062, "agent_compute-ego1": 0.01343609513958112, "complete-iteration": 0.3594917437997282, "set_robot_commands": 0.002499534604003719, "deviation-center-line": 2.401490567668618, "driven_lanedir_consec": 5.905091735960328, "sim_compute_sim_state": 0.010835464465819565, "sim_compute_performance-ego0": 0.002219030933995401, "sim_compute_performance-ego1": 0.0021986382714431225}, "LFV_multi-norm-small_loop-000-ego1": {"driven_any": 6.07200364361221, "get_ui_image": 0.03451641583772299, "step_physics": 0.2549436733286868, "survival_time": 32.50000000000029, "driven_lanedir": 5.27092593136673, "get_state_dump": 0.006760962364677277, "get_robot_state": 0.00793745111210555, "sim_render-ego0": 0.004237389601137597, "sim_render-ego1": 0.004386013737105737, "get_duckie_state": 1.3700828024868592e-06, "in-drivable-lane": 2.650000000000031, "deviation-heading": 10.79493282235721, "agent_compute-ego0": 0.012911026012696062, "agent_compute-ego1": 0.01343609513958112, "complete-iteration": 0.3594917437997282, "set_robot_commands": 0.002499534604003719, "deviation-center-line": 2.3526402266354154, "driven_lanedir_consec": 5.27092593136673, "sim_compute_sim_state": 0.010835464465819565, "sim_compute_performance-ego0": 0.002219030933995401, "sim_compute_performance-ego1": 0.0021986382714431225}}
set_robot_commands_max0.0026103858646008495
set_robot_commands_mean0.0024983336811002484
set_robot_commands_median0.002600831850482264
set_robot_commands_min0.002283182866765895
sim_compute_performance-ego0_max0.002470941567401108
sim_compute_performance-ego0_mean0.0022921555297054368
sim_compute_performance-ego0_median0.0023948183464666487
sim_compute_performance-ego0_min0.00204726897310357
sim_compute_performance-ego1_max0.002302568818409179
sim_compute_performance-ego1_mean0.002155711099058601
sim_compute_performance-ego1_median0.0022384632040717816
sim_compute_performance-ego1_min0.0019046376885025825
sim_compute_sim_state_max0.047120913379297566
sim_compute_sim_state_mean0.03233408946558101
sim_compute_sim_state_median0.035296194956364185
sim_compute_sim_state_min0.010835464465819565
sim_render-ego0_max0.004398492254087272
sim_render-ego0_mean0.004179163271923542
sim_render-ego0_median0.004344039912227786
sim_render-ego0_min0.0037658444848485432
sim_render-ego1_max0.0044544350594704
sim_render-ego1_mean0.004300349083098747
sim_render-ego1_median0.004446103908338713
sim_render-ego1_min0.003957675954483629
simulation-passed1
step_physics_max0.8598917588703241
step_physics_mean0.6765027194197674
step_physics_median0.801531524086475
step_physics_min0.2549436733286868
survival_time_max59.99999999999873
survival_time_mean56.07142857142753
survival_time_min32.50000000000029
No reset possible
59012LFVmultibodyv-simsuccessyes1:47:02
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible
59011LFVmultibodyv-simfailedyes0:02:52
InvalidSubmission: T [...]
InvalidSubmission:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/duckietown_challenges/cie_concrete.py", line 681, in scoring_context
    yield cie
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 68, in go
    wrap(cie)
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/experiment_manager.py", line 34, in wrap
    asyncio.run(main(cie, logdir, attempts), debug=True)
  File "/usr/local/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/local/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.8/site-packages/duckietown_experiment_manager/code.py", line 190, in main
    raise InvalidSubmission(msg)
duckietown_challenges.exceptions.InvalidSubmission: Timeout during connection to ego1: <SignalTimeout in state: 2>
Artefacts hidden. If you are the author, please login using the top-right link or use the dashboard.
No reset possible