To date, there is not a standardized method for consistently quantifying the performance of an automated driving system (ADS)-equipped vehicle (AV). The purpose of this dissertation is to contribute to a framework for such an approach referred to throughout as the operational safety assessment (OSA) methodology. Through this research, safety metrics are identified, researched, and analyzed to capture aspects of the operational safety of AVs, interacting with other salient objects. This dissertation outlines the approach for developing this methodology through a series of key steps including:​ (1) comprehensive literature review;​ (2) research and refinement of OSA metrics;​ (3) generation of MATLAB script for metric calculations;​ (4) generation of simulated events for analysis;​ (5) collection of real-world data for analysis;​ (6) review of OSA methodology results;​ and (7) discussion of future work to expand complexity, fidelity, and relevance aspects of the OSA methodology.

The detailed literature review includes the identification of metrics historically used in both traditional and more recent evaluations of vehicle performance. Subsequently, the metric formulations are refined, and robust severity evaluations are proposed. A MATLAB script is then presented which was generated to calculate the metrics from any given source assuming proper formatting of the data. To further refine the formulations and the MATLAB script, a variety of simulated scenarios are discussed including car-following, intersection, and lane change situations. Additionally, a data collection activity is presented, leveraging the SMARTDRIVE testbed operated by Maricopa County Department of Transportation in Anthem, AZ to collect real-world data from an active intersection.

Lastly, the efficacy of the OSA methodology with respect to the evaluation of vehicle performance for a set of scenarios is evaluated utilizing both simulated and real-world data. This assessment provides a demonstration of the ability and robustness of this methodology to evaluate vehicle performance for a given scenario. At the conclusion of this dissertation, additional factors including fidelity, complexity, and relevance are explored to contribute to a more comprehensive evaluation.