To achieve high power output and good efficiency and to comply with increasingly stricter emission standards, modern combustion engines require a more complex engine design, which results in a higher number of control parameters. As the measurement effort and the number of sensors for engine development at the test bed continue to increase, it is becoming nearly impossible for the test bed engineer to manually check measurement data quality. As a result, automated methods for analysis and plausibility checks of measurement data are necessary in order to find faults as soon as they occur and to obtain test results of the highest possible quality. This paper presents a methodology for automated fault diagnosis on engine test beds. The methodology allows reliable detection of measurement faults as well as the identification of the root cause of faults. This methodology is based on a modular concept that combines several modules and methods consisting of physical principles as well as statistical or data-based models. The theoretical principles and functionality of the different modules and the combinational logic between the modules are discussed in detail. A standardized setup of the modules facilitates the combination of module results, thereby improving the expandability and flexibility of the system. Finally, a thorough evaluation of the fault diagnosis system using various parameters is provided and the performance of the individual modules are compared and discussed. In conclusion, the best diagnosis results can be obtained with a combination of all the modules presented in the paper.