Study of the spray formation in the vicinity of the nozzle is essential to better understand and predict the physical processes involved in the diesel atomization, in detail. In this work, a long distance microscope coupled with a high speed camera was adopted to capture the spray structure at different injection pressures. The initial spray patterns were found to be different from one injection to another: It contained a clear single mushroom, tail region and intact liquid column, or had a "tail" in front of the mushroom without changing its direction and had a high speed. Occasionally, it presented as a double-mushroom shape, or did not include a clear mushroom. The difference of spray patterns may be due to the residual fuel/air, turbulent nature of the flow and the unsteady movement of the injector needle, while the reasons are not identified. Thus, effects of the distribution of the residual air bubble inside the nozzle surviving from the last injection on spray patterns at the start of the next spray were further investigated with a combination of the LES method and VOF model coupling the cavitating flow inside the orifice. Our findings confirm that the initial mushroom was generated by the residual air bubble. The diameter, distribution and amount of the residual bubble were responsible for the variation of initial spray structures between different injections. Moreover, the injection pressure and the cavitation model were shown to play a role in the development of spray formation at the opening stage, especially the spray cone angle.