logologo
Sign In

Scenarios

The scenario files have the same format as the existing scenarios from the CommonRoad scenario database and provide maps and bounding boxes for all traffic participants in the environment. The scenarios are located in urban and highway locations.

We distinguish between two scenario formats: non-interactive and interactive scenarios:

Non-interactive scenarios have a predefined trajectory for all traffic participants like in a recorded dataset. Therefore, the prediction of each traffic participant is provided for the complete planning horizon which facilitates the development in the first competition stage.

Interactive scenarios simulate the behavior of other traffic participants in a closed loop with the ego vehicle using the traffic simulator SUMO. Only the current bounding boxes are provided to the planner at each time step, which requires the implementation of a trajectory prediction. A package to execute the interactive scenarios is provided under Interactive Scenarios.

 

Submissions

Each scenario comes with a planning problem consisting of an initial state and a goal state. The task for the participants of the competition is to submit solution files with feasible trajectories while minimizing an objective given by a cost function.

Feasible trajectory

  • The trajectory must start in the initial state and end in the goal state.
  • The trajectory is valid with respect to the vehicle dynamics of the kinematic single-track model with vehicle parameters for FORD_ESCORT. Details and implementations can be found under here.
  • No collisions with other traffic participants are accepted.
  • Driving is allowed on the road only.

The parameters of the vehicle model need to be defined in the solution files as well. The following settings need to be used when creating the solution file using the CommonRoadSolutionWriter in order to obtain the benchmark id KS1:TR1: <scenario_name> :2020a.

  • vehicle_type = VehicleType.FORD_ESCORT
  • vehicle_model = VehicleModel.KS
  • cost_function = CostFunction.TR1 (more details under Evaluation )

A demo for creating solution files from a trajectory using the CommonRoadSolutionWriter is provided in Tutorial 2 - CommonRoad Input-Output.

We suggest checking the feasibility of solutions before uploading them using the Drivability Checker. More tutorials for helpful software packages can be found under Getting Started.

Uploading solutions

To submit solutions, enroll in the challenge of the respective competition stage and upload the solutions on the challenge page. Note that in both competition stages, solutions are submitted differently. In the first stage, the solution XML files are uploaded directly, while in the second stage, a link to a docker image must be provided that is executed on our server. After uploading, the solutions are evaluated on the server, and the results can be found under My Submissions from where they can be published. Only public solutions are counted in the leaderboard.

Create and submit Docker images

A documentation for the creation of docker images can be found in the template repository. To submit docker images, navigate to the leaderboard page of the 2nd phase and click new submission. After the successful evaluation on the server, the submission also needs to be published under the challenge submissions tab on My Submissions

 

Evaluation

Solutions for each scenario are checked for feasibility and assessed using the cost function TR1 that evaluates the driving performance and safety metrics. This function is documented under Cost Functions and implemented in the Drivability Checker.

In the second stage o0f the competition, the cost function will be extended by traffic rules:

  • speed limit (see doc how to retrieve the speed limit from a lanelet)
  • safe distance of the ego vehicle (e) to the leading vehicle (o), taken from [1]:

 

Competition stages

The competition is divided into two stages. The purpose of the first stage is to get familiar with the scenario formats and only the second stage is relevant for determining the winner. Each stage has its own submission page with a leaderboard, scenario visualizations.

1st stage: July 1st - July 31st

  • Mix of non-interactive & interactive scenarios.
  • Scenarios are public and can be downloaded here. The folder contains 300 interactive and 146 non-interactive scenarios.
  • Submission of solution trajectories as XML files. The submission is extended until September 1st.
  • No prizes and everyone can participate in the 2nd stage.

2nd stage: August 1st - September 1st

  • Only interactive scenarios. The scenarios are unknown, but similar to those of the interactive scenarios from the first stage.
  • Submission of motion planning software as docker images and evaluation on a server. See section "submissions" above for more details.
  • The submissions are evaluated on unknown 200 scenarios. The evaluation is timed out after 6 hours and at most two cores can be used. Only solutions evaluated within this time frame are counted.
  • Winner: most no. 1 solutions among all scenarios of stage 2.

Make sure your submissions appear in the leaderboard. Only solutions that are set public are counted in the leaderboard.

 

Participation

Participation is open for single persons or teams. However, submissions must be made from a single account. The best participants will be invited to give a presentation on their approach in our virtual ITSC workshop on September 19th. Registration for the final workshop is open here.

Forum

Feel free to give feedback regarding the competition in the forum.