Engine calibration on the powertrain test bed with transient mode is proposed with dynamic powertrain test bed having low inertia dynamometer. Automated ECU (Engine Control Unit) calibration system is completed with the combination of experimental design software, powertrain test bed, evaluation tools and their electrical interfaces. The process is composed up of the system interface definition, test design using DoE skill, test proceedings by step sequence of connecting systems, measured data collecting, mathematical model and optimization result extraction at the end. All the processes are automated by interfaces between the systems. Acceleration surge is minimized by proposed process by optimizing combustion control labels and tip in driveability is maximized by manipulating torque filter labels of EMS (Engine Management System) logic. Their detailed steps from the problem definition to the verification test results of improved design with vehicle test are presented. The validation of this methodology is proven by not only good correlation between bench test and vehicle test but also better engine acceleration surge and tip in driveability.