Highly automated driving (HAD) is under rapid development and will be available for customers within the next years. However the evidence that HAD is at least as safe as human driving has still not been produced. The challenge is to drive hundred millions of test kilometers without incidents to show that statistically HAD is significantly safer. One solution is to let a HAD function run in parallel with human drivers in customer cars to utilize a fraction of the billions of kilometers driven every year. To guarantee safety, the function under test (FUT) has access to sensors but its output is not executed, which results in an open loop problem. To overcome this shortcoming, the proposed method consists of four steps to close the loop for the FUT. First, sensor data from real driving scenes is fused in a world model and enhanced by incorporating future time steps into original measurements. Second, recorded time-shifted data is used to identify intentions of each driver of the surrounding traffic. Third, the real scene is used as start scenario for simulation. Reactions of surrounding traffic to the FUT output are simulated using identified intentions from step two. Fourth, simulation output is evaluated for erroneous behavior of the FUT to give developers input for further improvement and provide statistics about the safely driven distance. Main contribution is the first complete method for passive HAD safety assessment with a solution to the open loop problem. Results are presented for driver intention identification and criticality assessment.