How We Leverage AI in Our Products

The integration of Artificial Intelligence (AI) in vehicle perception systems has revolutionised the landscape of autonomous vehicles (AVs). At Provizio, AI is a core component of our innovative 5D Perception technology, enabling on-the-edge object detection, classification, and tracking, as well as enhancing the performance of our sensors. At Autosens this week, our Senior Machine Learning Engineer, Dane Mitrev, took to the stage to present the latest progress on how AI is being leveraged within Provizio. Let’s dive in to some key takeaways below:

The Provizio Approach

At Provizio, we are leveraging AI to deliver on three core goals:

1. To deliver LiDAR-level resolution performance, while leveraging the robust sensing and cost advantages of radar technology.
2. To deliver powerful on-the-edge perception capabilities while minimising resource demands.
3. To deliver a solution that can improve over time, leveraging crowdsourced datasets to improve perception capabilities and deliver enhanced Software Over The Air (SOTA) features to our customers.

Our Implementation

With 5D Perception, AI is utilised in a unique Tri-Level design to deliver compound enhancement from the signal level, through to the point cloud, and finally at the fusion level. In this way, we make the most of our hardware systems by using intelligent software to squeeze out the best possible performance from each layer of the stack. Let’s start with the first layer - point cloud denoising.

Neural Networks for Point Cloud Denoising and Enhancement

Stage 1: Training Dataset

The first phase in creating an effective AI model starts with good quality training data. At Provizio, we generate such data using a 3-stage process:

1. Sensor Synchronisation: Inputs from radar, LiDAR & camera are used to create an accurate source of ground-truth data. Inputs from each respective sensor need to be synchronised, such that objects and their locations are consistent.
2. Point Cloud Filtering Based on Lidar Data and Camera Semantic Segmentation: During this process, LiDAR and camera data is compared. Using semantic segmentation, each individual pixel from a video frame is classified into categories like “road”, “pedestrian”, “vehicle” and so on. This is cross referenced with 3D LiDAR point cloud data to ensure any “noise” (erroneous points on the point cloud) are removed.
3. Manual Corrections: Human visual inspection is used to ensure the resultant training data is as accurate as possible.

Stage 2: Lightweight Models for Real-Time denoising

2D Convolutional Neural Networks (CNN): With this method, 3D radar point cloud data is transformed into 2D projections, which simplifies the training process. A unique CNN architecture is then trained on a dataset where noise in the radar data has been identified and labelled. During this process, the CNN learns to identify patterns in the data that represent noise and as a result, once training is complete, the CNN can be used to identify and filter noise from previously unseen data.

Stage 3: Point Cloud Super Resolution

In a similar way to how noise patterns are identified and removed using CNNs, patterns that denote real objects can also used to improve the resolution of point cloud outputs. In this case, during the training process, the CNN learns to understand the spatial relationships within high-resolution ground truth point cloud datasets and predict where additional points should be added to increase resolution. Once trained, the CNN can take lower resolution point clouds and enhance them by adding additional points in a way that increases detail and accuracy.

Neural Networks for Object Detection, Tracking and Freespace Estimation

Once the data from our radar sensors is de-noised and enhanced as per the above systems, a further set of neural networks is used to process this data with the goal of understanding the real-world environment it represents. In this respect, our hardware and software teams worked closely together to develop an understanding of how to build a neural network that could extract the most information from the radar point clouds. In doing so, several efficiencies in the process were identified to create a lightweight system, capable of performing advanced perception tasks on-the-edge.

The Provizio Advantage

The above provides a high-level outline of the modular process we use in Provizio to maximise the value output of our products. Not only does this approach enable greater maintainability over time, but by developing both the hardware and software for our devices, we posses a unique ability to produce high quality outputs at a fraction of the cost of our competitors. By leveraging AI within our 5D Perception system, we deliver:

• Robust reliability in challenging environmental conditions.
• Native support for a triple redundancy safety approach, incorporating VizioPrime, Plex and camera sensors.
• Real-time, on-the-edge processing that slashes hazard response times and reduces manufacturing cost and complexity.
• Scalable, over-the-air improvements that deliver improved safety and unique features throughout the lifetime of the product.

Conclusion

The application of AI in vehicle perception is a field rich with innovation and challenges. As AI continues to evolve, Provizio is at the forefront of addressing the complex technical hurdles affecting the safety and real-world variability of autonomous systems, such that a future of zero accidents will become possible for all.