In contrast to the usual approach to operating self-driving cars, we did not program any explicit object detection, mapping, path planning or control components into this car. Instead, the car learns on its own to create all necessary internal representations necessary to steer, simply by observing human drivers.

The car successfully navigates the construction site while freeing us from creating specialized detectors for cones or other objects present at the site. Similarly, the car can drive on the road that is overgrown with grass and bushes without the need to create a vegetation detection system. All it takes is about twenty example runs driven by humans at different times of the day. Learning to drive in these complex environments demonstrates new capabilities of deep neural networks.

The car also learns to generalize its driving behavior. This video includes a clip that shows a car that was trained only on California roads successfully driving itself in New Jersey .