In the next 20 years fully autonomous vehicles are expected to be in the market. The advance on their development is creating paradigm shifts on different automotive related research areas. Vehicle interiors design and human vehicle interaction are evolving to enable interaction flexibility inside the cars. However, today’s vehicle manufacturers’ autonomous car concepts maintain the steering wheel as a control element. While this approach allows the driver to take over the vehicle route if needed, it causes a constraint in the previously mentioned interaction flexibility. Other approaches, such as the one proposed by Google, enable interaction flexibility by removing the steering wheel and accelerator and brake pedals. However, this prevents the users to take control over the vehicle route if needed, not allowing them to make on-route spontaneous decisions, such as stopping at a specific point of interest. This article presents VoGe, a human vehicle interaction system based on voice and pointing gestures that enables the user making spontaneous decisions over the route and communicate them to the car, while maintaining the interaction flexibility. We describe the system’s design, its main functions as well as the first proof of concept in a driving simulator that showcases its main functions.