Problem with training environment in commonroad-rl tutorial 'vanilla learning'

CommonRoad-RL
1

Hello,

I have a problem when creating the training environment in the second commonroad-rl tutorial on vanilla learning. I get following error:

AttributeError Traceback (most recent call last)
in
11 meta_scenario_path=meta_scenario_path,
12 train_reset_config_path= training_data_path,
—> 13 **env_configs)
14
15 # Wrap the environment with a monitor to keep an record of the learning process

~/miniconda3/envs/cr37/lib/python3.7/site-packages/gym/envs/registration.py in make(id, **kwargs)
674 # fmt: on
675 def make(id: str, **kwargs) → “Env”:
→ 676 return registry.make(id, **kwargs)
677
678

~/miniconda3/envs/cr37/lib/python3.7/site-packages/gym/envs/registration.py in make(self, path, **kwargs)
518 spec = self.spec(path)
519 # Construct the environment
→ 520 return spec.make(**kwargs)
521
522 def all(self):

~/miniconda3/envs/cr37/lib/python3.7/site-packages/gym/envs/registration.py in make(self, **kwargs)
131 “(HINT: is there a newer registered version of this env?)”
132 )
→ 133 _kwargs = self.kwargs.copy()
134 _kwargs.update(kwargs)
135

AttributeError: ‘NoneType’ object has no attribute ‘copy’

env_configs is of type dict and seems to be alright when printing it.
I guess the problem is similar to the one mentioned in the topic Pytest for the lastest version issues.
So, I made sure to use the most recent versions of commonroad-rl and commonroad-io. I have gym version 0.23.1.

Do you have any ideas how to solve this issue?

2

Dear annim,

Please try to downgrade gym to 0.21.0. I’ve updated the environment.yml accordingly.

Best,
Xiao

3

Hi Xiao,

thank you, it worked!
However, now I am facing a problem when training the model (for different algorithms it’s the same error):

----> 8 model.learn(n_timesteps, eval_callback)

[…]

…/commonroad-rl/commonroad_rl/gym_commonroad/commonroad_env.py in _set_scenario_problem(self, benchmark_id, scenario, planning_problem)
397 problem_dict = self.all_problem_dict[benchmark_id]
398 elif scenario is None or planning_problem is None:
→ 399 self.benchmark_id, problem_dict = random.choice(list(self.all_problem_dict.items()))

~/miniconda3/envs/cr37/lib/python3.7/random.py in choice(self, seq)
259 i = self._randbelow(len(seq))
260 except ValueError:
→ 261 raise IndexError(‘Cannot choose from an empty sequence’) from None
262 return seq[i]

IndexError: Cannot choose from an empty sequence

It would be great if you also had a solution for this issue!

Best,
Annabel

4

Dear Annabel,

It seems like the environment did not load any scenarios. Did you check if the path to the scenario folder is correct and the folder contains pickle files of the scenarios?

Best,
Xiao

5

Dear Xiao,

thanks for the suggestion! I am not sure what I changed but somehow it does work now and the model starts training.

Though, I noticed that sometimes during training after some thousand time steps an error raises saying that the input for the dynamics of the ego vehicle is not within bounds. This error occurs at different time steps if you run the training several times. Sometimes it is also possible to finish the training process before the error occurs.

Do you have any idea what’s the reason behind this behavior?

Best,
Annabel

6

Dear Annabel,

Which vehicle model did you use here?

Best,
Xiao

7

Dear Xiao,

I use vehicle model 4 (as it is the default value).

Best,
Annabel

8

This might be an edge case for this vehicle model. It would be great if you could provide the data to reproduce this error. Otherwise, I’ll try myself if I can reproduce it.

9

Meanwhile, you can try with other vehicle models. Point-mass model and KS model should be more stable.

10

Which data do you need exactly?
I was doing the tutorial from github without any changes to it.

And thanks for the suggestion!