Most of the present electric vehicle (EV) on-board chargers utilize a conventional design, i.e., a boost-type Power Factor Correction (PFC) controller followed by an isolated DC/DC converter. Such design usually yields a ~94% wall-to-battery efficiency and 2~3kW/L power density at most, which makes a high-power charger, e.g., 20kW module difficult to fit in the vehicle. As described in this paper, first, an E-mode GaN HEMT based 7.2kW single-phase charger was built. Connecting three such modules to the three-phase grid allows a three-phase >20kW charger to be built, which compared to the conventional three-phase charger, saves the bulky DC-bus capacitor by using the indirect matrix converter topology. To push the efficiency and power density to the limit, comprehensive optimization is processed to optimize the single-phase module through incorporating the GaN HEMT switching performance and securing its zero-voltage switching. Close-loop control is implemented to minimize the output current ripple and balance the power among three single-phase modules. Simulation and experimental results validated that the proposed charger has an efficiency of ~98%, and a power density of ~5kW/L@20kW.