How Do Autonomous Cars Work?


We’ve all seen cars that drive themselves in books and movies, but the automotive industry is working hard now to make them a reality. But how do they work?

For an autonomous car to work it must “see”; that is, build up a picture of the world around it. To do this it uses sensors to gather data, which is then processed and analysed using artificial intelligence (AI) algorithms running on powerful and efficient hardware so that the vehicle to react to its surroundings instantly.

Cameras, lidar, or radar?

The best way to accumulate this data is currently a point of contention in the industry. Each technology has its strength and benefits and every company looking at autonomous driving has a differing view on which would be best to use. Tesla is well known for relying only on standard vision cameras. Cameras are highly capable, offering very high-resolution, colour-rich, images that can be discreetly placed all around the car so the vehicle can see all around in a 360° view. Crucially, cameras also have the advantage of being low cost, which is important when you’re trying to mass-produce a car at an affordable price.

Other companies focused on self-driving vehicles believe that “light detection and ranging”, known as lidar, is the way forward. Lidar sensors send out pulses of photons to create highly accurate three-dimensional pictures of the car’s surroundings at distances up to 300 metres. Some significant autonomous players, such as Cruise and Waymo, rely exclusively on lidar. The downsides of the technology, however, are three-fold. First, despite significantly dropping in price over the last five years, lidar sensors are still much more expensive than vision cameras. Secondly, lidar can be deceived by poor weather conditions, and third, the sensors are bulkier, making them harder to integrate into the car.

There is a third way. Some are pushing radar as the primary means of enabling a vehicle to self-navigate. Radar has been around since the Second World War, so is a familiar and lower-cost technology to implement. It offers good depth perception and, crucially, has the advantage of working in poor weather conditions. However, radar suffers from relatively low resolution, especially at distance.

So which type of autonomous car sensor is best? Different car companies are taking different approaches, and some believe that just as humans rely on multiple senses, more than one technology will be needed to create a vehicle that can safely drive itself in any scenario.

Moving the data around

Whatever sensors are used, the amount of data required is going to be large, with multiple gigabytes, and potentially terabytes, of information required to be moved around efficiently to ensure the data is processed quickly. Therefore, autonomous cars will increasingly come to rely on automotive Ethernet as the standard for data transfer around the car. Ethernet is a long-established open standard and offers much greater bandwidth than current data transport solutions such as CANbus.

AI at the heart

However, the data is captured and moved around, the real magic of self-driving vehicles will be delivered through artificial intelligence (AI). It is the AI software solutions that will enable vehicles to recognise roads, road signs, other vehicles and pedestrians, and predict their actions to ensure that the vehicle can react correctly and safely.

The primary tool for doing this are deep learning algorithms produced using convolutional neural networks (CNNs). These are used to process the data and then take decisions that cause physical actions in the car, such as braking, accelerating, and steering. CNNs are deep learning tools that attempt to mimic the behaviour of the human visual cortex to understand an image, so that specific features can be identified.

While we do not yet have fully autonomous cars, neural networks are already widely deployed in the advanced driver assistance (ADAS) features found in many of today’s cars, delivering features such as automatic lane keep assist, automatic emergency braking and adaptive cruise control. Eventually, drivers will be able to let go of the wheel under certain conditions all let the car drive itself, while eventually, it will not need a human driver at all. These different stages are described as the six levels of autonomous driving below.

  • Level 0: No Driving Automation
  • Level 1: Driver Assistance
  • Level 2: Partial Driver Assistance
  • Level 3: Conditional Driving Automation
  • Level 4: High Driving Automation
  • Level 5: Full Driving Automation

Go to our What are the six levels of autonomous driving technology? page to learn more about the six levels in more depth.

The Benefits of Self Driving Cars

Self-driving cars will provide a host of benefits to society. Once they are commonplace they will:

  • saves lives. In the United States alone, over 36,000 people were killed in motor vehicle crashed in 2019. Autonomous cars will be able to respond faster to road conditions than a human and will not be at risk of getting drunk or falling asleep.
  • reduce congestion. Thanks to “smart city” technologies such as 5G, vehicles will talk to each other and to roadside infrastructure to coordinate traffic to help alleviate traffic jams.
  • reduce dependence on humans for freight transportation. “Platooning” lorries, as in driving at convoy at speed will optimise road use reducing congestion, improving safety and increasing product distribution efficiency. In the UK, much was made of the lorry driver shortage holding up goods transportation following “Brexit”. These are the sorts of issues that autonomous transport will help alleviate.