On the Way to Solving the Big Data Problem in Autonomous Driving

The appearance of self-driving vehicles is closely related to progress in the sphere of artificial intelligence and robotics. However, self-driving cars need more than just artificial intelligence and machine learning.

To build its route, each self-driving car sends and receives plenty of data from various sensors like GPS, sonars, radars, lidars, cameras, and so on.

Information from all these sources must be processed very quickly, almost in real-time. And the amount of data is really massive here.

The car’s built-in computer cannot process all data. Some info is transferred for further analysis to peripheral servers and data centers.

Not only the artificial intelligence algorithms are crucial, but also the capabilities of installed computers and the cloud. The speed of receiving and sending data by car and low latency are vital these days.

Information amount issues

Even ordinary vehicles which are driven by people are generating huge amounts of data. Self-driving vehicles create up to 1 terabyte of data each hour. This tremendous amount of data represents a barrier to the quick adoption of autonomous cars.

All data generated by self-driving cars cannot be processed by peripheral servers or in the cloud as this causes severe delays. Each 100-ms delay can draw a line between the life or death of a pedestrian or passenger.

To reduce delays in sending and receiving data and responding to new situations, some parts of data are analyzed by the car’s computer. Modern vehicles are equipped with up to 50 processing cores that serve such functions as blind-spot monitoring, cruise control, obstacle warning, auto braking, and so on.

Car nodes communicate with each other and external servers. As a result, self-driving cars represent a hybrid network that includes the cloud, local data centers, and a lot of peripheral nodes.

Communication nodes located in charging stations, traffic lights, and control posts should provide the cars additional assistance for secure autonomous driving. And as security professionals from VPNBrains.com say, all communication should be protected with the help of secure private tunnels.

Interaction with infrastructure and each other

The range of the environment that cars can analyze is increasing day by day. Still, each vehicle can only store and collect a limited amount of data. Information exchange is necessary. The more significant data set generated by the autonomous vehicle fleet helps better analyze the surroundings.

Vehicle-to-vehicle communication technologies buildup mesh networks constituted of cars located in the same area.

V2V can be extended and share data with traffic lights and other connected objects. So, we are slowly moving to vehicle-to-infrastructure (V2I).

Self-driving vehicles are constantly learning. They can send helpful info to local peripheral servers integrated into traffic lights or charging stations. Those charging stations may use AI to analyze the new data received from vehicles and offer the best route solutions.

With the help of the cloud, this info can be sent to other self-driving cars in the same network. Again, this model of data exchange means exabytes of new data being generated every day.

5G as the starting point

Unmanned vehicles should receive info about other cars, cyclists, and pedestrians not only from their own sensors but also from other cars and other urban infrastructure.

Opportunities offered by 5G come in handy here. 5G can provide the necessary speed, low latency, and support plenty of simultaneous connections.

Actually, self-driving cars will not be able to become fully autonomous without 5G.

Mobile carriers have spent billions on building 5G networks. It is high time to integrate this technology and help cars use it in everyday situations.

Daimler, BMW, Hyundai, Toyota and other manufacturers are already using 5G in their cars.

Again, each unmanned car can generate up to 20TB per day. The mobile network should be able to transfer all this data. Experiments with 5G and autonomous driving will not be successful unless 5G infrastructure is entirely ready.

Processing and storing exabytes of information

To fully introduce autonomous driving, it is vital to solve all issues with storing and processing huge amounts of data. The car’s computers have limited storage and performance capabilities.

At the same time, not all data requires immediate processing. Data that is unimportant right now can be accumulated and analyzed later by peripheral servers and the cloud.

Old data, such as info about the speed and location of the car several hours ago, is helpful in training autonomous driving algorithms. Developers of machine learning systems must have enough data to train their networks. Road safety units need information about cars’ location and speed at the time of accidents.

Again, we will be facing exabytes of data. To store it, we will need flexible, malware-free, and high-performance edge infrastructure. The problem of using an optimal data storage architecture becomes evident.

Fresh data that should be analyzed immediately can be temporarily stored on high-capacity HAMR drives and SSDs. They offer very low latency, good throughput, and support multi-drive technologies.

After the initial analysis stage, data should be stored more efficiently on low-cost nearline storage units. Old data that must be kept for the future can be moved to archives.

Data is being increasingly processed at the edge, bringing us closer to the era of Industry 4.0. Edge computing means data will be processed where it was collected rather than in a traditional cloud. It will allow data to be analyzed much quicker, responding to changing situations immediately.

Crucial big data

In this article, I have shed light on how crucial bid data is in the sphere of autonomous driving. The adoption of self-driving cars involves huge amounts of data that must be processed by onboard computers, edge servers, and the cloud. Necessary infrastructure should be ready beforehand.

With the help of 5G, self-driving cars will generate enough data to make smart cities safe and effective. Self-driving vehicles are at the forefront of such technologies as robotics, machine learning, and artificial intelligence. Achieving the goal of mass adoption of fully autonomous driving is not easy.

We should continue developing new technologies in order to open a new page in the history of transportation.

About the author: Alex Vakulov is a cybersecurity researcher with over 20 years of experience in malware analysis. Alex has strong malware removal skills. He is writing for numerous tech-related publications sharing his security experience.