In this paper, a sensor fusion approach is introduced to estimate lane departure. The proposed algorithm combines the camera, inertial navigation sensor, and GPS data with the vehicle dynamics to estimate the vehicle path and the lane departure time. The lane path and vehicle path are estimated by using Kalman filters. This algorithm can be used to provide early warning for lane departure in order to increase driving safety. By integrating inertial navigation sensor and GPS data, the inertial sensor biases can be estimated and the vehicle path can be estimated where the GPS data is not available or is poor. Additionally, the algorithm can be used to reduce the latency of information embedded in the controls, so that the vehicle lateral control performance can be significantly improved during lane keeping in Advanced Driver Assistance Systems (ADAS) or autonomous vehicles. Furthermore, it improves lane detection reliability in situations when camera fails to detect lanes.