1National Highway Traffic Safety Administration, United States
2Southwest Research Institute, United States
Automated Driving Systems (ADS) are being developed to perform the primary functions of the dynamic driving task (DDT). These technologies hold great promise to improve safety and mobility for transportation. Test scenarios are critical for assessing the safety assurance of ADS in a range of operational environments and roadway conditions. The development of testing scenarios for ADS is proving to be an important challenge for the development of safety assurance requirements, certification and licensing frameworks, testbed services, standards, and international harmonization.
This paper summarizes foundational research undertaken to identify a sample preliminary, objective testing and evaluation approach for ADS. The paper considers technologies of interest that fall within Level 3 through Level 5 of the SAE International levels of driving automation and identifies a cross-section of prototype and conceptual ADS that are then categorized into seven generic ADS features.
This research also takes the first steps to partition the ADS performance space by identifying and assessing the primary variables that comprise an ADS test scenario. Those primary variables are described in detail, and include:
Tactical and operational maneuver capabilities largely focus on the control-related elements of the DDT (i.e., lateral and longitudinal control) that enable an ADS to navigate to reach its destination (e.g., lane centering / following, turning). A working list of these capabilities is presented. The ODD represents the operating conditions under which an ADS is designed to function (e.g., roadway types, weather conditions, etc.). A notional hierarchical ODD taxonomy is presented and described. OEDR capabilities include the elements of the DDT that involve monitoring the driving environment and implementing appropriate responses to relevant objects and events. A working list of OEDR capabilities is presented. Failure mode behaviors include fail-safe (FS) and fail-operational (FO) strategies that will allow an ADS to respond to a variety of failures, including DDT performance-relevant system failures that require the ADS or a DDT fallback-ready user to achieve a minimal risk condition.
The paper also considers the implementation of the proposed evaluation framework using existing test methods, including modeling and simulation (M&S), closed track testing, and open road testing. It further seeks to examine how each of the testing methods can be logically used to minimize the complexity of comprehensive safety assessments of ADS by leveraging each method’s strengths to maximize the knowledge gained from each test. It also includes extensive discussion of challenges associated with testing ADS, including challenges related to the technology itself as well as challenges associated with test execution. This paper is based on research completed by NHTSA and its contractors, and is more fully documented in NHTSA Report DOT HS 812 623, “A Framework for Automated Driving System and Testable Cases and Scenarios”; September 2018.