As technology for both military and civilian aviation systems mature into a new era, techniques to test and evaluate these systems have become of great interest. To achieve a general understanding as well as save time and cost, the use of computer modeling and simulation for component, subsystem or integrated system testing has become a central part of technology development programs. However, the evolving complexity of the systems being modeled leads to a tremendous increase in the complexity of the developed models. To gain confidence in these models there is a need to evaluate the risk in using those models for decision making. Statistical model validation techniques are used to assess the risk of using a given model in decision making exercises.In this paper, we formulate a transient model validation challenge problem for an air cycle machine (ACM) and present a hardware test bench used to generate experimental data relevant to the model. A set of design requirements are provided as guidelines for the development of candidate validation methods. To reduce ACM test bench cost and development time, commercial off the shelf components were used where possible. Measured temperature, pressure, and mass flow results for the ACM test bench are compared to the simulated computer model. The ACM test bench provides a platform for testing different transient validation techniques.