In map development and navigation, there are several rare and challenging situations that developers might encounter. These situations often require special handling or innovative solutions to ensure accurate and effective mapping and navigation. One example might be temporary closures or changes in road networks due to construction, which can significantly alter navigation routes. Furthermore, navigation in areas with many high-rise buildings can be challenging due to signal interference and the need for 3D mapping. Or think of mapping large indoor spaces like malls, airports, or hospitals where GPS signals are weak or unavailable. Martin Pfeifle, Chairman of NDS and CTO at NNG LLC, addressed this topic at the ATZ Conference 2024 in his talk “Enhancing Digital Maps with AI derived confidence information”.
Rare map challenges can be overcome with the help of smart and innovative technologies and AI-based approaches. An innovative network of like-minded automotive experts and flexibility, as offered by the NDS Association and NDS.Live, help to develop new approaches. One challenge in this context: Autonomous driving situations, which are increasingly coming into focus, must be reliably covered in order to ensure the safety of the driver.
Enhancing map features with uncertainty information
Digital maps play a crucial role as a sensor for self-driving cars. They not only facilitate Lane Keeping Assist (LKA) functionality and provide mission statements to self-driving cars but are also indispensable for triggering takeover requests well in advance. Maps stand out as the only sensor capable of anticipating road conditions ten or more seconds ahead. Similar to other sensors, it is desirable for maps to provide information about the confidence level in their data.
Map providers can enhance map features with uncertainty information, which can naturally be derived from crowd sourcing or timestamps indicating the last verification of map features. This information serves as a quality indicator for maps.
Self-driving cars may need to trigger a takeover request not only when the map could be wrong but also when encountering new map situations. Consider a scenario where a self-driving car encounters a situation it has never seen before in the training data. Even if the map data is correct, it’s highly likely that the car doesn’t know how to handle the situation properly. It is desired to mathematically describe the ‘degree of unusuality’ with the goal of enabling self-driving vehicles to trigger takeover requests when encountering unusual map situations.
Unsupervised learning methods, such as Variational Autoencoders, can learn mean and standard deviation values of latent map features that are useful for describing maps. If a particular map situation deviates significantly from the mean values of latent map features, it is highly likely that the self-driving car has not encountered such situations during training, warranting a takeover request.
Map Data Representations
One of the key questions to be answered is how the map should be represented so that an AI module can determine the likelihood of a certain map situation. In Figure 3 below, three different basic approaches are shown.
One approach to providing map data to a neural network is through a rendered map (cf. top row in Figure 3). This method was utilized by Amini, Rosman, Karaman and Rus (Variational End-to-End Navigation and Localization, IEEE International Conference on Robotics and Automation (ICRA), New York: 2021) to provide the mission statement, i.e., the computed route path, to an end-to-end self-driving neural network. Using a rendered map as input has the advantage that convolutional layers can process the input information using a well-known technology.
In the middle row of Figure 3, the use of a Graph Neural Network as an input module is shown. Graphs naturally represent road networks. They are, for instance, used by Derrow-Pinion et al. (ETA Prediction with Graph Neural Networks in Google Maps. In: CIKM ’21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 3767–3776, Association for Computing Machinery, New York: 2021) to improve the ETA (expected time of arrival) in Google Maps.
In the bottom row of Figure 3, the map is represented by a binary data stream. These binary streams are encoded in formal languages such as Protobuf or Zserio, the language used by the NDS Association. These binary streams can naturally be regarded as a high-dimensional feature vector. Additionally, these binary sequence streams of map data share similarities with text streams in large language models (LLM) and might, therefore, be encoded by suitable feature embeddings.
Variational Auto Encoders (VAEs). Figure 4 depicts the basic scheme of a variational autoencoder (VAE). The model takes x as input. The encoder compresses it into a latent space, and the decoder creates the output x’, aiming to be as similar as possible to x. The crucial aspect is that x is represented in the latent space with less data than x. The latent representation of x can be considered a form of compression.
Now, let’s assume we do this with millions of datapoints, x. The autoencoder learns to represent these millions of datapoints in a compact way in the latent space while minimizing the reconstruction error. The latent space can be seen as a set of latent variables described by a mean value µ and a standard distribution σ. These latent variables serve as an aggregated representation of the dataset.
In this example on GitHub, for instance, VAEs are used to represent human faces. The latent space can then be used for generating artificial new faces by simply sampling in the latent space and letting the decoder create new instances x’.
One of the key achievements of VAEs is that the latent variables do not need to be defined. The system decides on its own what is important and what is not. In the face generation example, for instance, it decides on its own whether there should be a latent variable describing the nose, ears, eyes, chin, mouth, etc. The latent variables and their distribution are created in such a way that they optimally approximate the training set.
In our map use case, the latent space of a VAE is exactly the same as the output layer in Figure 3. We are not interested in creating artificial map situations x’. Essentially, only the first part (Input, Encoder, Latent Space) of Figure 4 is important for us. The latent space consists of a set of latent variables (cf. the output layer of Figure 3) that describe all map situations. Each particular map situation can now be represented by a latent variable of the latent space. If these latent variables are close to the mean values of the distribution, the map situation can be regarded as commonly known. If the latent variables deviate significantly from the mean values, the map situation can be considered an outlier, and a takeover request should be triggered.
Quality attributes in NDS.Live
Reliable digital maps need quality indicators. In NDS.Live, each attribute can have a confidence value. This includes, for example, confidence in the existence of the feature, confidence in the accuracy of the feature’s geometry, or confidence in the correct values of a feature’s properties.
In addition, the NDS association considers right now to represent unusual map situations in NDS.Live as outlined in this blog post. Both confidence and unusualness indicators can trigger take over requests in autonomous driving scenarios, which can be the difference between life and death for the driver.
In the NDS.Live Developer Portal you can find out more about the possibilities and functions of NDS.Live.
Further interesting info can also be found in NNG’s whitepaper “Enhancing Digital Maps with AI derived confidence information”.
Back to news →