To ensure the safety of the vehicle and driver while testing the performance of road departure mitigation (warning (RDW) and road keeping assistance (RKA)) systems and to standardize the test environment, a set of roadside test surrogates are being developed, which includes grass, curbs, metal guardrail, concrete divider and traffic barrel/cones. This paper describes how to determine the representative color of these roadside surrogates. Firstly, 44,000 stratified road locations in the US were generated from randomly selected 820,000 locations based on road levels defined in ArcGIS which is a geographic information system for maps. Among them, 24,762 locations have Google street view images, which are used for the color determination of roadside objects. To mitigate the effect of the brightness for the representative color determination, the images in good weather, daylight and not under shade were manually selected. Then, the RGB values of the roadside objects in these images were extracted. To obtain the representative color of the roadside surrogates, the K-means clustering algorithm was applied to find the color clusters of each type of roadside objects. In this method, the color space of RGB was converted to the modified CIE LUV color space (which is a color space adopted by the International Commission on Illumination) because of the perceptual uniformity of modified CIE LUV colors. The Silhouette index was applied to determine the optimal number of clusters. Finally, the primary colors of the clusters covering 90% of the sampled locations were chosen as the representative colors of the roadside objects.