BBW (Brake-by-wire) can increase the electric and hybrid vehicles performance and safety. This paper proposes a novel mechatronic booster system, which includes APS (active power source), PFE (pedal feel emulator), ECU (electronic control unit). The concrete configuration and working principle of booster is proposed. A test bench which consists of booster, master cylinder, wheel cylinder and braking pipelines is set up. PI (Proportional-integral) control algorithm is used in the closed-loop system. Through the test bench, dynamic characteristic of the system are tested. But the system is easily disturbed when the outside conditions changes. The system performance is weakened. The cascade control technique can be used to solve the problems. This paper develops an novel adaptive cascade optimum braking control algorithm based on the mechatronic booster system. The system is divided into main loop and servo loop, both of them are closed-loop system. The servo-loop system can eliminate the disturbance which exists in the servo loop. So the robustness of the cascade control system is improved than which of the general closed-loop control system. Different control object is respectively chosen. The control-oriented mathematical model is designed. Based on the control-oriented model, optimum control algorithm is formed to realize the hydraulic pressure highly accurate control. The correction factor is proposed to compensate the steady state deviation according to different operating point. The performance of the novel booster system is evaluated by bench tests. Experimental results prove that the system fulfills the requirements of the brake system for automotive. The adaptive cascade optimum control algorithm can improve the robustness and less performance degradation occurs than PI control algorithm is used.