With the Federal Automated Vehicles Policy, the U.S. National Highway Traffic Safety Administration (NHTSA) has provided an outline that can be used to guide the development and validation of Automated Driving Systems (ADS). Acknowledging that the Human-Machine-Interface (HMI) – identified as one of the 12 priority safety design elements in this voluntary guidance – will be crucial for the success of ADSs, we developed a two-step iterative test procedure that serves to evaluate the conformity of SAE level 3 ADS HMIs with the requirements outlined in NHTSA’s Auomated Vehicles policy. The aim of this assessment is to evaluate whether minimum HMI requirements are met that facilitate a safe and efficient use of AVs. The present contribution describes the development of an expert-based checklist, how it was compiled from existing literature, how its content and application were refined in simulator and real-word studies, and how it can be employed as a complimentary or stand-alone tool to assess the conformity of SAE Level 3 ADS HMIs with NHTSA’s AV policy. It also discusses boundary conditions for the application of the method and the generalization of findings. The described method can be employed in a variety of settings to evaluate SAE Level 3 ADS HMIs, therefore making it a valuable tool for both researchers and practitioners alike.