As one of the key technologies in autonomous driving, the lateral planning module guides the lateral movement during the driving process. An integrated lateral planning module should consider the non-holonomic constraints of a vehicle, the optimization of the generated trajectory and the applicability to various scenarios. However, the current lateral planning methods can only meet parts of these requirements. In order to satisfy all the performance requirements above, a novel Potential Field (PF) based lateral planning method is proposed in this paper. Firstly, a PF model is built to describe the potential risk of the traffic entities, including the obstacles, road boundaries and lines. The potential fields of these traffic entities are determined by their properties and the traffic regulations. Secondly, the planning algorithm is presented, which comprises three modules: state prediction, state search and trajectory generation. The state prediction is realized through the lateral dynamics and kinematics equations of the vehicle. Then based on the PF model, a cost function is designed, which takes the potential risk and comfort requirements into consideration. With the cost function and vehicle states, a heuristic search algorithm is employed for the state search and the resulting optimal trajectory is achieved by the trajectory generation. The proposed method can escape the local-minimal effectively and meet the non-holonomic constraints. Besides, thanks to its generality of the problem formulation, this method gives the possibility to adapt to different traffic scenarios. The performance of this method is verified in the bench test.