There is little agreement in the field of driving safety as to how to define cognitive distraction, much less how to measure it. Without a definition and metric, it is impossible to make scientific and engineering progress on determining the extent to which cognitive distraction causes crashes, and ways to mitigate it if it does. We show here that different studies are inconsistent in their definitions of cognitive distraction. For example, some definitions do not include cellular conversation, while others do. Some definitions confound cognitive distraction with visual distraction, or cognitive distraction with cognitive workload. Other studies define cognitive distraction in terms of a state of the driver, and others in terms of tasks that may distract the driver. It is little wonder that some studies find that cognitive distraction is a negligible factor in causing crashes, while others assert that cognitive distraction causes more crashes than drunk driving. Perhaps the largest problem however is that the definitions typically refer to distraction as a reduction of attention, but never define attention. It is recommended that a new definition of cognitive distraction be created, based on observational data of actual rather than simulated driving. This definition should have associated with it a clear metric for assessing the amount of cognitive distraction for a wide range of tasks, involving different sensory and motor modalities (visual-manual, auditory-vocal, tactile, etc.) A new foundation, grounded in on-road driving data and the experimental study of attention within cognitive neuroscience, may improve the speed at which new and effective countermeasures are developed, crashes are reduced, and driving safety is improved for all.