Test Execution
You can run the tests in simulation environments. For the moment we focus on the BeamNG.tech simulator. As input, you need to have a directory containing the test specifications in JSON format. In case you don’t have yet any test specification then you should take a look at the Test Generation section to know more about generating test specifications automatically.
Note
SDC-Scissor supports currently only BeamNG.tech v0.24.0.2!
The general command to run the tests is lable-tests
, as illustrated in the following listing:
Usage: sdc-scissor label-tests [OPTIONS]
Execute the tests in simulation to label them as safe or unsafe scenarios.
Options:
-t, --tests PATH Path to the directory containing the test
specifications
--home PATH The home directory of the BeamNG.tech
simulator containing the executable
--user PATH The user directory of BeamNG.tech containing
the tech.key file and levels files
--rf FLOAT Risk factor of the AI driving the car
--oob FLOAT The out-of-bound parameter specifying how
much a car is allowed to drive off the lane
--max-speed FLOAT The maximum speed the AI is allowed to drive
--interrupt / --no-interrupt Indicator if the test executions should stop
when the car violates the OOB criteria
--obstacles / --no-obstacles Indicator if there should be obstacles in
the virtual environment
--bump-dist INTEGER The distance between the speed bumps
('obstacles' needs to be true)
--delineator-dist INTEGER The distance between the delineators
('obstacles' needs to be true)
--tree-dist INTEGER The distance between the trees ('obstacles'
needs to be true)
-fov, --field-of-view INTEGER The field of view angle
--canbus / --no-canbus Enable CAN messages
--can-stdout / --no-can-stdout Output CAN messages to stdout
--can-dbc PATH Path to CAN database file
--can-dbc-map PATH Path to CAN database map json file
--can-interface TEXT CAN interface
--can-channel TEXT CAN channel
--can-bitrate PATH CAN bitrate
--influxdb-bucket TEXT InfluxDB bucket to write CAN message to
--influxdb-org TEXT InfluxDB organization
--help Show this message and exit.
Note
The command is called label-tests
since the main purpose for running the tests is to have a labeled dataset.
With a dataset of tests with the according test outcome (i.e., the label), SDC-Scissor can train machine learning models on it.
More information you will find in the Test Outcome Prediction section.
Options
The command label-tests
comes with several options.
Those options are mainly about to configure the virtual environment and not the actual road specifications since these are set in the JSON files.