The automobile industry, universities, and automotive research institutes in Europe have started an initiative for cooperative research regarding assessment of real-world safety benefits of advanced driver assistance systems (ADAS) and active safety systems. A ‘Harmonization Group’ was established in 2012 whose motivation is the development of a comprehensive, reliable, transparent, and thus accepted methodology for quantitative assessment of these systems by virtual numerical simulation. One aim of this group, so-called P.E.A.R.S. (Prospective Effectiveness Assessment for Road Safety), was to provide a review of the current practices for this prospective effectiveness assessment of ADAS and active safety systems. This paper’s objective is to present this review.

As a complement to a literature review, five workshops were held with a dozen of P.E.A.R.S. members to collect qualitative in-depth information about their approaches concerning the effectiveness evaluation of ADAS and active safety systems via simulation. During the workshops, non-directive interviews and discussions were held to gather information on the research questions, metrics, methods and simulation techniques employed by the P.E.A.R.S. members. Subsequently, the approaches for prospective effectiveness assessment were classified into four levels according to their use of simulation. Finally, criteria for evaluating the approaches were identified.

The overall evaluation approach consists of:​ 1) identifying the target accident situations (TS) that the system could potentially address (usually by using crash databases), for example pedestrian crashes;​ 2) establishing, for each TS, reference situations (RS) such as driving, pre-crash or crash situations in which the system was not present, for example all configurations of pedestrian crashes or critical situations involving a vehicle and a pedestrian;​ 3) adding the system to the reference situations in order to establish what would have happened if the system had been present, generating potentially modified situations (MS);​ and 4) comparing the outputs of the two situations to estimate the effectiveness of the system in terms of crash avoidance and injury mitigation.

Additionally, approaches were classified in four levels depending on their sparse, limited or intensive use of numerical simulation to establish the reference situations and the modified situations. The zero level uses expert opinion instead of simulation to roughly estimate the safety benefit of a system on crash situations. The first level uses simulation to add the system (and simulate its effect) to reference situations that are usually real-life crashes recorded in crash databases. The second level uses simulation to modify parameters of real-life situations and generate more reference situations, and also to add the system and generate the modified situations. The third level characterizes the processes involved in the target situations, then uses simulation to generate reference situations (which are not exclusively based on real-life situations), and the modified situations.

Lastly, fourteen evaluation criteria were identified to assess the performance of the different approaches:​ Thoroughness and exhaustiveness, completeness, understandability and interpretability, operation capability usability, degree of automation, generalizability, flexibility, fidelity, accuracy, time consideration and ability to go back in time before the collision or critical situation, required resources, validation, and granularity.

This paper provides a taxonomy of approaches and use of simulation to estimate the safety benefits of ADAS and active safety systems but does not provide quantitative evaluation on the performance of the different approaches. Future work focuses on applying various approaches on a same case study (Round Robin) in order to compare them relative to their effectiveness assessment outputs and the evaluation criteria.

This review offers insights into the categories of current approaches for estimating potential benefits of ADAS and active safety systems via simulation. In order to develop a harmonized methodology, stakeholders acknowledge that simulation can be used at several levels with various degrees of data description. Moreover, the evaluation criteria can be used to determine which approach is more suitable for a specific need.