Render the Best Model Tutorial 4

CommonRoad-RL
1

Hello,

When I run the first cell of render the best model, I got the output like this;

/Users/utkupolat/miniconda3/envs/cr37/lib/python3.7/site-packages/stable_baselines/init.py:33: UserWarning: stable-baselines is in maintenance mode, please use [Stable-Baselines3 (SB3)](https://github.com/DLR-RM/stable-baselines3) for an up-to-date version. You can find a migration guide in SB3 documentation.
“stable-baselines is in maintenance mode, please use Stable-Baselines3 (SB3) for an up-to-date version. You can find a migration guide in SB3 documentation.”
[INFO] commonroad_rl.gym_commonroad.commonroad_env - Testing on commonroad_rl/tutorials/data/highD/pickles/problem_test with 16 scenarios
Testing a maximum of -1 scenarios
/Users/utkupolat/miniconda3/envs/cr37/lib/python3.7/site-packages/commonroad_route_planner/route.py:309: RuntimeWarning: invalid value encountered in true_divide
** return (x_d * y_dd - x_dd * y_d) / ((x_d ** 2 + y_d ** 2) ** (3. / 2.))**

1 ) There are 16 scenarios but it gives the output “testing a maximum of -1 scenarios”. How can I fix it?

2 ) Also I want to ask other issues that I remarked bold, why they are always appearing, when I try to change stable baselines to stable_baseline version 3 in running “.py” folders it gives an error.

I try to optimize the model parameters but I don’t know the commonroad-rl works well because whatever I changed, the output a little bit change or it gives the same values. Another thing that I want to ask, the output images of the tutorial 4, there exist only off_road images the folders which are “collision”, “goal_reached”, “time._out” and “other” are fully empty.

3 ) However, when I run the last cell of the tutorial 4, program check the 16 scenarios but never reach the goal, is this because the bad model or there are something to are going to wrong.

/Users/utkupolat/miniconda3/envs/cr37/lib/python3.7/site-packages/stable_baselines/init.py:33: UserWarning: stable-baselines is in maintenance mode, please use Stable-Baselines3 (SB3) for an up-to-date version. You can find a migration guide in SB3 documentation.
“stable-baselines is in maintenance mode, please use Stable-Baselines3 (SB3) for an up-to-date version. You can find a migration guide in SB3 documentation.”
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Initialization started
/Users/utkupolat/miniconda3/envs/cr37/lib/python3.7/site-packages/commonroad/scenario/lanelet.py:1279: ShapelyDeprecationWarning: STRtree will be changed in 2.0.0 and will not be compatible with versions < 2.
self._strtee = STRtree(list(self._buffered_polygons.values()))
[INFO] commonroad_rl.gym_commonroad.commonroad_env - Testing on commonroad_rl/tutorials/data/highD/pickles/problem_test with 16 scenarios
/Users/utkupolat/miniconda3/envs/cr37/lib/python3.7/site-packages/gym/logger.py:34: UserWarning: WARN: Box bound precision lowered by casting to float32
warnings.warn(colorize("%s: %s" % (“WARN”, msg % args), “yellow”))
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Meta scenario path: commonroad_rl/tutorials/data/highD/pickles/meta_scenario
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Training data path: /Users/utkupolat/Downloads/commonroad-rl/pickles/problem_train
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Testing data path: commonroad_rl/tutorials/data/highD/pickles/problem_test
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Initialization done
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 16
Step: 1, Reward: [5.611194], Done: [False]
Step: 2, Reward: [5.602102], Done: [False]
Step: 3, Reward: [5.591236], Done: [False]
Step: 4, Reward: [5.5809393], Done: [False]
Step: 5, Reward: [5.5708294], Done: [False]
Step: 6, Reward: [5.5610847], Done: [False]
Step: 7, Reward: [5.551802], Done: [False]
Step: 8, Reward: [5.543106], Done: [False]
Step: 9, Reward: [5.535149], Done: [False]
Step: 10, Reward: [5.5281043], Done: [False]
Step: 11, Reward: [5.4495], Done: [False]
Step: 12, Reward: [4.7012577], Done: [False]
Step: 13, Reward: [4.625826], Done: [False]
Step: 14, Reward: [4.552849], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 15
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [4.393524], Done: [False]
Step: 2, Reward: [4.3737574], Done: [False]
Step: 3, Reward: [4.3817167], Done: [False]
Step: 4, Reward: [4.3870473], Done: [False]
Step: 5, Reward: [4.39257], Done: [False]
Step: 6, Reward: [4.3979244], Done: [False]
Step: 7, Reward: [4.403191], Done: [False]
Step: 8, Reward: [3.9534569], Done: [False]
Step: 9, Reward: [3.8752522], Done: [False]
Step: 10, Reward: [3.8054426], Done: [False]
Step: 11, Reward: [3.7347736], Done: [False]
Step: 12, Reward: [3.6630707], Done: [False]
Step: 13, Reward: [3.5902486], Done: [False]
Step: 14, Reward: [3.5162194], Done: [False]
Step: 15, Reward: [3.4435008], Done: [False]
Step: 16, Reward: [3.3813648], Done: [False]
Step: 17, Reward: [3.3359473], Done: [False]
Step: 18, Reward: [3.311189], Done: [False]
Step: 19, Reward: [3.3046253], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 14
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [5.4802194], Done: [False]
Step: 2, Reward: [5.494081], Done: [False]
Step: 3, Reward: [5.4699016], Done: [False]
Step: 4, Reward: [5.4459286], Done: [False]
Step: 5, Reward: [5.4225335], Done: [False]
Step: 6, Reward: [5.399827], Done: [False]
Step: 7, Reward: [5.3779593], Done: [False]
Step: 8, Reward: [5.357095], Done: [False]
Step: 9, Reward: [5.3374224], Done: [False]
Step: 10, Reward: [5.023212], Done: [False]
Step: 11, Reward: [4.5783634], Done: [False]
Step: 12, Reward: [4.4931784], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 13
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [7.4473853], Done: [False]
Step: 2, Reward: [7.42394], Done: [False]
Step: 3, Reward: [7.443462], Done: [False]
Step: 4, Reward: [7.4590287], Done: [False]
Step: 5, Reward: [7.475418], Done: [False]
Step: 6, Reward: [7.4918942], Done: [False]
Step: 7, Reward: [7.508417], Done: [False]
Step: 8, Reward: [7.524862], Done: [False]
Step: 9, Reward: [7.541142], Done: [False]
Step: 10, Reward: [7.5571995], Done: [False]
Step: 11, Reward: [7.5729976], Done: [False]
Step: 12, Reward: [7.5885243], Done: [False]
Step: 13, Reward: [7.2083845], Done: [False]
Step: 14, Reward: [7.0501285], Done: [False]
Step: 15, Reward: [7.0105076], Done: [False]
Step: 16, Reward: [6.9716797], Done: [False]
Step: 17, Reward: [6.9344707], Done: [False]
Step: 18, Reward: [6.8994055], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 12
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [3.5045264], Done: [False]
Step: 2, Reward: [3.5644011], Done: [False]
Step: 3, Reward: [3.6031544], Done: [False]
Step: 4, Reward: [3.6601584], Done: [False]
Step: 5, Reward: [3.7156425], Done: [False]
Step: 6, Reward: [3.7734358], Done: [False]
Step: 7, Reward: [3.8339133], Done: [False]
Step: 8, Reward: [3.897405], Done: [False]
Step: 9, Reward: [3.9642756], Done: [False]
Step: 10, Reward: [4.0231953], Done: [False]
Step: 11, Reward: [4.0747275], Done: [False]
Step: 12, Reward: [4.1293054], Done: [False]
Step: 13, Reward: [4.1704354], Done: [False]
Step: 14, Reward: [4.1839232], Done: [False]
Step: 15, Reward: [4.18037], Done: [False]
Step: 16, Reward: [4.1652355], Done: [False]
Step: 17, Reward: [4.141314], Done: [False]
Step: 18, Reward: [4.109934], Done: [False]
Step: 19, Reward: [4.067365], Done: [False]
Step: 20, Reward: [4.023191], Done: [False]
Step: 21, Reward: [3.191795], Done: [False]
Step: 22, Reward: [2.8384023], Done: [False]
Step: 23, Reward: [2.777189], Done: [False]
Step: 24, Reward: [2.716944], Done: [False]
Step: 25, Reward: [2.6574333], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 11
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [5.1869907], Done: [False]
Step: 2, Reward: [5.169395], Done: [False]
Step: 3, Reward: [5.1731305], Done: [False]
Step: 4, Reward: [5.1742973], Done: [False]
Step: 5, Reward: [5.1745734], Done: [False]
Step: 6, Reward: [5.1736107], Done: [False]
Step: 7, Reward: [5.1713114], Done: [False]
Step: 8, Reward: [5.1675835], Done: [False]
Step: 9, Reward: [5.162375], Done: [False]
Step: 10, Reward: [5.1556826], Done: [False]
Step: 11, Reward: [4.6022954], Done: [False]
Step: 12, Reward: [4.3202963], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 10
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [4.7719874], Done: [False]
Step: 2, Reward: [4.7807164], Done: [False]
Step: 3, Reward: [4.7537694], Done: [False]
Step: 4, Reward: [4.7290983], Done: [False]
Step: 5, Reward: [4.704983], Done: [False]
Step: 6, Reward: [4.68161], Done: [False]
Step: 7, Reward: [4.6590505], Done: [False]
Step: 8, Reward: [4.63741], Done: [False]
Step: 9, Reward: [4.6167994], Done: [False]
Step: 10, Reward: [4.5973353], Done: [False]
Step: 11, Reward: [4.202247], Done: [False]
Step: 12, Reward: [3.8024266], Done: [False]
Step: 13, Reward: [3.716151], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 9
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [5.8866887], Done: [False]
Step: 2, Reward: [5.8490787], Done: [False]
Step: 3, Reward: [5.838011], Done: [False]
Step: 4, Reward: [5.820877], Done: [False]
Step: 5, Reward: [5.80502], Done: [False]
Step: 6, Reward: [5.7889915], Done: [False]
Step: 7, Reward: [5.7730923], Done: [False]
Step: 8, Reward: [5.757281], Done: [False]
Step: 9, Reward: [5.7415648], Done: [False]
Step: 10, Reward: [5.7259526], Done: [False]
Step: 11, Reward: [5.7104464], Done: [False]
Step: 12, Reward: [5.695054], Done: [False]
Step: 13, Reward: [5.6797853], Done: [False]
Step: 14, Reward: [5.66464], Done: [False]
Step: 15, Reward: [5.6496367], Done: [False]
Step: 16, Reward: [5.6347814], Done: [False]
Step: 17, Reward: [5.620087], Done: [False]
Step: 18, Reward: [5.605563], Done: [False]
Step: 19, Reward: [5.5912223], Done: [False]
Step: 20, Reward: [5.577077], Done: [False]
Step: 21, Reward: [5.563139], Done: [False]
Step: 22, Reward: [5.5494246], Done: [False]
Step: 23, Reward: [5.535946], Done: [False]
Step: 24, Reward: [5.522725], Done: [False]
Step: 25, Reward: [5.5097895], Done: [False]
Step: 26, Reward: [5.497165], Done: [False]
Step: 27, Reward: [5.4848833], Done: [False]
Step: 28, Reward: [5.472978], Done: [False]
Step: 29, Reward: [5.4614906], Done: [False]
Step: 30, Reward: [5.4504695], Done: [False]
Step: 31, Reward: [5.4399624], Done: [False]
Step: 32, Reward: [5.2487936], Done: [False]
Step: 33, Reward: [4.9312353], Done: [False]
Step: 34, Reward: [4.875348], Done: [False]
Step: 35, Reward: [4.8204145], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 8
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [4.711434], Done: [False]
Step: 2, Reward: [4.7200665], Done: [False]
Step: 3, Reward: [4.684526], Done: [False]
Step: 4, Reward: [4.6471734], Done: [False]
Step: 5, Reward: [4.609608], Done: [False]
Step: 6, Reward: [4.5718813], Done: [False]
Step: 7, Reward: [4.5340533], Done: [False]
Step: 8, Reward: [4.496206], Done: [False]
Step: 9, Reward: [4.4584384], Done: [False]
Step: 10, Reward: [4.420891], Done: [False]
Step: 11, Reward: [4.3836827], Done: [False]
Step: 12, Reward: [4.0923963], Done: [False]
Step: 13, Reward: [3.573014], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 7
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [6.339526], Done: [False]
Step: 2, Reward: [6.338615], Done: [False]
Step: 3, Reward: [6.3218646], Done: [False]
Step: 4, Reward: [6.306953], Done: [False]
Step: 5, Reward: [6.2922764], Done: [False]
Step: 6, Reward: [6.2781267], Done: [False]
Step: 7, Reward: [6.264632], Done: [False]
Step: 8, Reward: [6.2519765], Done: [False]
Step: 9, Reward: [6.2403674], Done: [False]
Step: 10, Reward: [6.2299724], Done: [False]
Step: 11, Reward: [5.4814353], Done: [False]
Step: 12, Reward: [5.396439], Done: [False]
Step: 13, Reward: [5.31811], Done: [False]
Step: 14, Reward: [5.2426257], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 6
/Users/utkupolat/miniconda3/envs/cr37/lib/python3.7/site-packages/commonroad_route_planner/route.py:309: RuntimeWarning: invalid value encountered in true_divide
return (x_d * y_dd - x_dd * y_d) / ((x_d ** 2 + y_d ** 2) ** (3. / 2.))
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [5.448282], Done: [False]
Step: 2, Reward: [5.417126], Done: [False]
Step: 3, Reward: [5.420941], Done: [False]
Step: 4, Reward: [5.419803], Done: [False]
Step: 5, Reward: [5.418675], Done: [False]
Step: 6, Reward: [5.4163985], Done: [False]
Step: 7, Reward: [5.4130507], Done: [False]
Step: 8, Reward: [5.4085274], Done: [False]
Step: 9, Reward: [5.4028454], Done: [False]
Step: 10, Reward: [5.396073], Done: [False]
Step: 11, Reward: [5.388318], Done: [False]
Step: 12, Reward: [5.379713], Done: [False]
Step: 13, Reward: [5.3704085], Done: [False]
Step: 14, Reward: [5.36059], Done: [False]
Step: 15, Reward: [5.350702], Done: [False]
Step: 16, Reward: [5.0283566], Done: [False]
Step: 17, Reward: [4.5332565], Done: [False]
Step: 18, Reward: [4.451507], Done: [False]
Step: 19, Reward: [4.3702602], Done: [False]
Step: 20, Reward: [4.2907243], Done: [False]
Step: 21, Reward: [4.213277], Done: [False]
Step: 22, Reward: [4.1338954], Done: [False]
Step: 23, Reward: [4.054512], Done: [False]
Step: 24, Reward: [3.981595], Done: [False]
Step: 25, Reward: [3.9140027], Done: [False]
Step: 26, Reward: [3.8490827], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 5
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [5.410066], Done: [False]
Step: 2, Reward: [5.421934], Done: [False]
Step: 3, Reward: [5.413297], Done: [False]
Step: 4, Reward: [5.4088855], Done: [False]
Step: 5, Reward: [5.404072], Done: [False]
Step: 6, Reward: [5.399627], Done: [False]
Step: 7, Reward: [5.395442], Done: [False]
Step: 8, Reward: [5.391511], Done: [False]
Step: 9, Reward: [5.387764], Done: [False]
Step: 10, Reward: [5.3840194], Done: [False]
Step: 11, Reward: [5.374512], Done: [False]
Step: 12, Reward: [5.360182], Done: [False]
Step: 13, Reward: [5.3470936], Done: [False]
Step: 14, Reward: [5.3347793], Done: [False]
Step: 15, Reward: [5.3232284], Done: [False]
Step: 16, Reward: [5.3123507], Done: [False]
Step: 17, Reward: [5.3020473], Done: [False]
Step: 18, Reward: [5.292203], Done: [False]
Step: 19, Reward: [5.282693], Done: [False]
Step: 20, Reward: [5.2733827], Done: [False]
Step: 21, Reward: [5.2641306], Done: [False]
Step: 22, Reward: [5.254796], Done: [False]
Step: 23, Reward: [5.244968], Done: [False]
Step: 24, Reward: [5.2352953], Done: [False]
Step: 25, Reward: [5.2261496], Done: [False]
Step: 26, Reward: [5.2169485], Done: [False]
Step: 27, Reward: [5.2077813], Done: [False]
Step: 28, Reward: [5.1987934], Done: [False]
Step: 29, Reward: [5.190191], Done: [False]
Step: 30, Reward: [5.1822176], Done: [False]
Step: 31, Reward: [5.1751494], Done: [False]
Step: 32, Reward: [4.627959], Done: [False]
Step: 33, Reward: [4.4470043], Done: [False]
Step: 34, Reward: [4.4008307], Done: [False]
Step: 35, Reward: [4.3587413], Done: [False]
Step: 36, Reward: [4.3211746], Done: [False]
Step: 37, Reward: [4.2879095], Done: [False]
Step: 38, Reward: [4.2584567], Done: [False]
Step: 39, Reward: [4.2321343], Done: [False]
Step: 40, Reward: [4.2081537], Done: [False]
Step: 41, Reward: [4.1857333], Done: [False]
Step: 42, Reward: [4.164202], Done: [False]
Step: 43, Reward: [4.1430726], Done: [False]
Step: 44, Reward: [4.122579], Done: [False]
Step: 45, Reward: [4.106648], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 4
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [5.2949095], Done: [False]
Step: 2, Reward: [5.2930326], Done: [False]
Step: 3, Reward: [5.287066], Done: [False]
Step: 4, Reward: [5.2868977], Done: [False]
Step: 5, Reward: [5.2863297], Done: [False]
Step: 6, Reward: [5.2859387], Done: [False]
Step: 7, Reward: [5.285457], Done: [False]
Step: 8, Reward: [5.2847853], Done: [False]
Step: 9, Reward: [5.2838235], Done: [False]
Step: 10, Reward: [5.282481], Done: [False]
Step: 11, Reward: [5.28066], Done: [False]
Step: 12, Reward: [5.278294], Done: [False]
Step: 13, Reward: [5.275315], Done: [False]
Step: 14, Reward: [5.271667], Done: [False]
Step: 15, Reward: [5.267588], Done: [False]
Step: 16, Reward: [5.262861], Done: [False]
Step: 17, Reward: [5.257646], Done: [False]
Step: 18, Reward: [5.2520885], Done: [False]
Step: 19, Reward: [5.2463727], Done: [False]
Step: 20, Reward: [5.240738], Done: [False]
Step: 21, Reward: [5.235468], Done: [False]
Step: 22, Reward: [5.23097], Done: [False]
Step: 23, Reward: [5.2275105], Done: [False]
Step: 24, Reward: [5.2256255], Done: [False]
Step: 25, Reward: [5.2257857], Done: [False]
Step: 26, Reward: [5.2284684], Done: [False]
Step: 27, Reward: [5.234152], Done: [False]
Step: 28, Reward: [5.243284], Done: [False]
Step: 29, Reward: [5.2562556], Done: [False]
Step: 30, Reward: [5.273362], Done: [False]
Step: 31, Reward: [5.294792], Done: [False]
Step: 32, Reward: [5.3206077], Done: [False]
Step: 33, Reward: [5.3507524], Done: [False]
Step: 34, Reward: [5.3850646], Done: [False]
Step: 35, Reward: [5.423285], Done: [False]
Step: 36, Reward: [5.4650693], Done: [False]
Step: 37, Reward: [5.510011], Done: [False]
Step: 38, Reward: [5.557668], Done: [False]
Step: 39, Reward: [5.607571], Done: [False]
Step: 40, Reward: [5.659243], Done: [False]
Step: 41, Reward: [5.7122145], Done: [False]
Step: 42, Reward: [5.766035], Done: [False]
Step: 43, Reward: [5.820277], Done: [False]
Step: 44, Reward: [5.8745484], Done: [False]
Step: 45, Reward: [5.928486], Done: [False]
Step: 46, Reward: [5.98177], Done: [False]
Step: 47, Reward: [6.034115], Done: [False]
Step: 48, Reward: [6.084478], Done: [False]
Step: 49, Reward: [6.1260133], Done: [False]
Step: 50, Reward: [6.151257], Done: [False]
Step: 51, Reward: [6.1615357], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 3
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [6.966807], Done: [False]
Step: 2, Reward: [6.9896617], Done: [False]
Step: 3, Reward: [6.9883466], Done: [False]
Step: 4, Reward: [6.9974413], Done: [False]
Step: 5, Reward: [7.0066824], Done: [False]
Step: 6, Reward: [7.017523], Done: [False]
Step: 7, Reward: [7.029731], Done: [False]
Step: 8, Reward: [7.0432167], Done: [False]
Step: 9, Reward: [7.057702], Done: [False]
Step: 10, Reward: [7.072892], Done: [False]
Step: 11, Reward: [7.08849], Done: [False]
Step: 12, Reward: [7.1043086], Done: [False]
Step: 13, Reward: [7.1201444], Done: [False]
Step: 14, Reward: [7.135953], Done: [False]
Step: 15, Reward: [6.7898746], Done: [False]
Step: 16, Reward: [6.297317], Done: [False]
Step: 17, Reward: [6.2415304], Done: [False]
Step: 18, Reward: [6.184029], Done: [False]
Step: 19, Reward: [6.1248684], Done: [False]
Step: 20, Reward: [6.063432], Done: [False]
Step: 21, Reward: [5.999122], Done: [False]
Step: 22, Reward: [5.931394], Done: [False]
Step: 23, Reward: [5.859936], Done: [False]
Step: 24, Reward: [5.7847986], Done: [False]
Step: 25, Reward: [5.708361], Done: [False]
Step: 26, Reward: [5.640582], Done: [False]
Step: 27, Reward: [5.591554], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 2
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [6.2411785], Done: [False]
Step: 2, Reward: [6.2865915], Done: [False]
Step: 3, Reward: [6.3070426], Done: [False]
Step: 4, Reward: [6.3280845], Done: [False]
Step: 5, Reward: [6.348218], Done: [False]
Step: 6, Reward: [6.3676457], Done: [False]
Step: 7, Reward: [6.3864164], Done: [False]
Step: 8, Reward: [6.4045925], Done: [False]
Step: 9, Reward: [6.3574815], Done: [False]
Step: 10, Reward: [6.356443], Done: [False]
Step: 11, Reward: [6.34217], Done: [False]
Step: 12, Reward: [5.818576], Done: [False]
Step: 13, Reward: [5.55376], Done: [False]
Step: 14, Reward: [5.4697585], Done: [False]
Step: 15, Reward: [5.38279], Done: [False]
Step: 16, Reward: [5.2930593], Done: [False]
Step: 17, Reward: [5.2008657], Done: [False]
Step: 18, Reward: [5.1066747], Done: [False]
Step: 19, Reward: [5.010533], Done: [False]
Step: 20, Reward: [4.913201], Done: [False]
Step: 21, Reward: [4.8236337], Done: [False]
Step: 22, Reward: [4.75504], Done: [False]
Step: 23, Reward: [4.7077017], Done: [False]
Step: 24, Reward: [4.6738195], Done: [False]
Step: 25, Reward: [4.646255], Done: [False]
Step: 26, Reward: [4.6212125], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 1
Step: 0, Reward: [-10.], Done: [ True]
Step: 1, Reward: [3.5411036], Done: [False]
Step: 2, Reward: [3.5695624], Done: [False]
Step: 3, Reward: [3.561743], Done: [False]
Step: 4, Reward: [3.5588188], Done: [False]
Step: 5, Reward: [3.5550294], Done: [False]
Step: 6, Reward: [3.5514534], Done: [False]
Step: 7, Reward: [3.547994], Done: [False]
Step: 8, Reward: [3.5446467], Done: [False]
Step: 9, Reward: [3.5413723], Done: [False]
Step: 10, Reward: [3.5381262], Done: [False]
Step: 11, Reward: [3.5348544], Done: [False]
Step: 12, Reward: [3.5314894], Done: [False]
Step: 13, Reward: [3.5279508], Done: [False]
Step: 14, Reward: [3.5241387], Done: [False]
Step: 15, Reward: [3.51993], Done: [False]
Step: 16, Reward: [3.5151834], Done: [False]
Step: 17, Reward: [3.5097642], Done: [False]
Step: 18, Reward: [3.530262], Done: [False]
Step: 19, Reward: [3.5738769], Done: [False]
Step: 20, Reward: [3.6139538], Done: [False]
Step: 21, Reward: [3.6530566], Done: [False]
Step: 22, Reward: [3.6907516], Done: [False]
Step: 23, Reward: [3.7268763], Done: [False]
Step: 24, Reward: [3.7614567], Done: [False]
Step: 25, Reward: [3.7944915], Done: [False]
Step: 26, Reward: [3.8259706], Done: [False]
Step: 27, Reward: [3.8558943], Done: [False]
Step: 28, Reward: [3.8842556], Done: [False]
Step: 29, Reward: [3.910957], Done: [False]
Step: 30, Reward: [3.935646], Done: [False]
Step: 31, Reward: [3.9580996], Done: [False]
Step: 32, Reward: [3.9785693], Done: [False]
Step: 33, Reward: [3.9971046], Done: [False]
Step: 34, Reward: [4.0134377], Done: [False]
Step: 35, Reward: [4.027315], Done: [False]
Step: 36, Reward: [4.038483], Done: [False]
Step: 37, Reward: [4.0515275], Done: [False]
Goal not reached
Termination reason: is_off_road
[DEBUG] commonroad_rl.gym_commonroad.commonroad_env - Number of scenarios left 0
Step: 38, Reward: [-10.], Done: [ True]

Thanks!

2

Hi,

  1. Testing a maximum of -1 scenarios meaning that it evaluates on all the scenarios in the specified folder. -1 is the default value for the option specifying how many scenarios should be evaluated. Sorry for the confusion about this printout. We will improve this in the future release.

  2. That is a warning from stable baselines because they stopped maintaining stable baselines2. Only switing the import won’t work, you would have to adapt the syntax and parameters for the models as well. But since it’s only a warning instead of an error, you could ignore it or commented it out in /Users/utkupolat/miniconda3/envs/cr37/lib/python3.7/site-packages/stable_baselines/**init** .py:33

  3. Did you enable the observations for road boundaries? Actually for highD scenarios it should be super easy to learn staying on road since it’s only straight roads. The agent should learn very fast to associate the negative reward for off-road and the observations for the distances to road boundaries.

Best,
Xiao

3

Hi,

Thanks for your detailed response.

For your thirth answer, I’ve already enabled road boundaries. I do not know what I am missing. I will compare my configs once again with your configurations.

Also, are there any possibility for the train_model.py is not working full capacity in Tutorial 2&3? Because, when I checked your shared models and configurations in GitLab of the project. Your console outputs does not seem like my outputs in the Tutorials. So, I tried to run with python command below, some part of the output seem like your console outputs but I also got a lot of errors and some of the functions did not give result like best model hyper-parameters and environment configurations.

!python -m commonroad_rl.train_model.py --env='commonroad-v1' --algo=ppo --n_envs=1 -n=1000000 --save-     freq=50000 --seed=0 --save_path='/path/to/your/saving/location/'--hyperparams_path='/path/to/your/file/model_hyperparameters.yml' --configs-path='path/to/your/file/environment_configs.yml

Thanks!

4

Hi,

train_model.py works. What errors did you get when calling it?

Best,
Xiao

5

Hi

Actually, I have to run it again because I cleaned the output of it. I will share it.

Is it necessary to run train_model.py with python command to get optimised model hyper-parameters and environment configurations? Because after I run the tutorials, I couldn’t get any optimised hyper-parameters from optima package.

Best,
Utku

6

Yes, if you want to perform an extensive hyperparameter search, you need to run train_model.py. The tutorials are meant for getting familiar with the functionalities of CommonRoad-RL, not doing actual training etc.