CES 2024: How Provizio Optimised 5D Perception^® in a Single Week

Imagine a vehicle that can constantly learn and improve over time, just like we drivers do. That's the future of perception technology and ubiquitous autonomy, and Provizio is at the forefront of this exciting revolution.

Traditionally, radar systems were limited by the hardware they were built with, but with the rise of software-defined radars, we now have the ability to adapt and improve radar performance through fine-tuning and over-the-air (OTA) updates.

At CES 2024, we leveraged these technologies to continually optimise the performance of our 5D Perception® system using the unique driving environment that is Las Vegas. In the short couple of days we were demonstrating our latest technology developments to our current and future partners, our perception performance was continually learning, adapting and improving. How? Let’s take a look.

A closer look at our technology

5D Perception^®

5D Perception® refers to the unique capability of Provizio radar sensors to “see” and perceive the world in five dimensions:

• Third dimension: High-resolution 3D world view
• Fourth dimension: Relative velocity & distance of all objects in the vehicle’s environment
• Fifth dimension: Edge Intelligence: making sense of everything in the vehicle’s environment and determining a safe driving path utilising Artificial Intelligence (AI), Machine Learning (ML) and Digital Signal Processing (DSP) to deliver object classification & drivable freespace mapping.

When speaking of optimising our 5D Perception® system at CES, our main focus was to improve performance with respect to object detection, classification, tracking and freespace mapping - the 5th dimension of our perception system. To do this, we needed to re-train our Convolutional Neural Net (CNN) algorithms based on data recorded from Las Vegas roads. The catch? We had to do it LIVE!

Fine-Tuning our Neural Networks

Think of our 5D Perception® system like a student. It needs to be able to learn and adapt to different situations (generalisation), but it also needs to be good at specific tasks (fine-tuning). For example, our radars are trained on a massive dataset of roads from Europe. But the US is a different environment, with wider roads, different markings, and even unfamiliar vehicles like large semi-trucks or pickup trucks.

To address this, we continued to fine-tune our system using data captured directly in Las Vegas while demonstrating to our customers. Using our custom automated data pipeline, data collected from the vehicle was uploaded directly to the cloud. From here, the pipeline extracted and processed the data, making it available to use for fine-tuning our neural network models.

With this data, our engineers created a new dataset by combining a mix of data captured from Las Vegas with data from our general balanced dataset. Our engineers then used this data to fine-tune our existing generalised model overnight, seamlessly delivering improved performance for the next day of captures and demos.

The result: In addition to providing 35% less false-negative vehicle detections, our fine-tuning of the neural networks improved vehicle detection range by over 20m on Las Vegas streets. In addition, we also noted a 45% improvement in pedestrian tracking stability and a 25% improvement in stability for the tracking of cars and large-vehicles.

Optimising Freespace

The Freespace microservice acts as a virtual guide, identifying safe driving paths and highlighting potential obstacles on the road. Since the service relies primarily on data from point-cloud clustering, road boundary estimation and odometry, the optimisation process at CES consisted of finding the perfect balance between raising the clustering threshold while maintaining reliable road boundary estimation.

As a result of these optimisations, the freespace detection distance for Las-Vegas boulevards and large interstate roads went from an average maximum range of 53m to an optimised maximum range of 86m. This is a 62% increase in average maximum range for the detection of freespace - day and night, in all weather conditions.

Conclusion

By leveraging the power of our software-defined radars and OTA updates, we are able to perform near real-time optimisation of our perception system to provide higher performance in a variety of areas - from classification & tracking, to radar-based odometry and freespace detection.

In just a few days, our system demonstrated:

• The ability to detect vehicles up to 20m further away and with 35% less false negatives than before.
• An improvement of 45% in pedestrian tracking stability and 25% for cars and large-vehicles.
• An improvement in average maximum freespace detection range from 53m to 86m - an increase of 62%.

This is just a glimpse of the future of radar-based perception technology, where software systems working on-the-edge are constantly learning and adapting. We believe that making the transition to scalable L3+ a reality is a challenge that can be solved by continuously learning and improving our 5D Perception® solution.

To learn more, read our white paper.