The Visual-Manual NHTSA Driver Distraction Guidelines for In-Vehicle Electronic Devices includes the Eye Glance Measurement Using Driving Simulator Testing (EGDS) task acceptance test. This paper describes the outcome of two EGDS tests, where both tests assessed the same set of 10 in-vehicle tasks, though each test employed a different group of 24 test participants, randomly sampled according to the proscribed EGDS procedure. Thus, in total 48 test participants drove on a simulated motorway in a lead vehicle following scenario while performing tasks such as changing temperature, setting destination in the navigation system and selecting and playing music using a prototype in-vehicle infotainment system located on the center panel. When comparing the test outcomes between the two groups, it was found that for 6 of the 10 tested tasks, pass/fail outcomes differed between the two groups on one or more of the proposed criteria. This high level of inconsistency in outcome between two identical tests using ten identical tasks raises questions regarding the repeatability of the proposed NHTSA EGDS procedure.