In the design or match process of vehicle powertrain system, gearbox rattle is a common NVH problem which directly affects passengers’ judgment on the quality and performance of vehicle. During the development process of a passenger car, prototype vehicles have serious gear rattle problem. In order to efficiently and fundamentally control this problem, this work first studied the characteristics and mechanisms of the gearbox rattle. The study results revealed that the torsional vibration of powertrain system was the root cause of gearbox rattle. Then a simulation model of the full vehicle was built with the aid of Simulink® toolbox, which is a graphical extension to MATLAB® for modeling and simulation of variety of systems. With this model, the sensitivity analysis and parametrical optimization were performed, and the simulation results indicated that the dual-mass flywheel (DMF) was the best measure to control the rattle. In order to verify this conclusion, this work developed a DMF system and embedded it into the powertrain of the prototype vehicle. Both subjective evaluation and the objective measurement on the vehicle indicated that the gearbox rattle problem was successfully solved. The main significance of this study is to establish an effective and easy approach to analyze and control the gearbox rattle. The indicative conclusions of this work could be referred by NVH engineers in the early phase of vehicle development.