Enterprises building AI applications recognize that mastering anomaly detection and test case generation brings incredible value to improving machine learning model performance.
In this session, Eric Chu, Senior Manager, Data Science at Zoox, and Anshuman Patnaik, Deep Learning Lead at Embark Trucks, unveil the challenges and opportunities of identifying data anomalies and generating test cases to optimize machine learning.
Defining and Handling Anomalies
An edge case is an anomalous event that is out of the expected distribution. In the context of autonomous driving, a category of anomalies that needs to be commonly handled is surprising driving, such as drivers who are about to miss their turns and cut several lanes of traffic. To overcome those, the ML team at Zoox uses retrofitted vehicles and safe drivers to observe and collect this kind of data.
Once more data about the anomalies is collected, organizations can recreate the event in a structured testing or simulated scenario. The ML model can then be vetted against this set of anomalous events. If the ML system cannot handle the anomaly, there needs to be a change in the process. Once the new processes are set up, it’s a matter of iteration to solve each case individually.
Some edge cases are considerably more challenging. For example, edge cases where the appropriate reaction is ’reverse’ are difficult to work with from a planning point of view. This may be the case when driving on a slope on a narrow road, where one car needs to reverse to allow the other one to pass before continuing.
It is impossible to prepare an AI system for all real-world scenarios. Even when gathering a diverse dataset, you can only generalize as far as your training data, so the more you discover from the tail, the better and more robust the model will be. By definition, in the base training data, you have the average case (the non-anomalous case), overrepresented.
“Tails become the bane of our existence.”
– Anshuman Patnaik, Deep Learning Lead at Embark Trucks
Anomalies are a function of where you are on the road, which makes geospatial data a critical part of the planning process. For example, construction zones are well-known sources of edge cases, as their location and time vary constantly.
Anomaly Sharing
For an anomaly to be addressed throughout the whole ML system, the entire MLOps and engineering teams must be able to communicate and collaborate. For example, a multi-modal stack uses different sensors. An anomaly to one sensor might not be an anomaly to a different one. Sharing information across teams working on the multimodal stack helps define an anomaly from the end-to-end point of view rather than looking at sensors individually. Working together the systems can be calibrated and used as failover.
“By improving edge cases, we also reduce bias.”
– Anshuman Patnaik, Deep Learning Lead at Embark Trucks
A human-in-the-loop, such as a safety driver is also important in finding and solving edge cases. A driver should be aware of where the system is today and the future direction of the system. Safety drivers also provide a baseline behavior of a human that tackles the anomaly. They provide a lot of data about responding to these anomalies, such as the matter in which they react depending on the anomalous event and driving conditions.
Anomalies in the wild
There are two factors to keep in mind when encountering an anomaly in the wild. Firstly, we must identify we are in an anomalous setting. This may prove difficult as anomalies are often never-before-seen situations. Second, we must react safely. This is the number one priority for handling any kind of situation in an autonomous driving scenario.
To react safely, we can envision two outcomes.
- The AI could handle the situation – this is the preferred option, where an edge case can be tackled with the tools the system already has at its disposal.
- The AI requests help – in an anomalous state, an AI can make a decision such as slowing down or swerving while phoning home to ask for guidance about the situation.
“All the anomalies are unique, but they can be categorized.”
– Eric Chu, Senior Manager, Data Science at Zoox
Humans and autonomous vehicles have access to the same controls. They can either change the velocity or change direction. Humans are very good at handling anomalies, which means that collecting more information and understanding taxonomies can help devise systems to handle those situations.
Upon discovering an anomaly it should be recorded and stored for future recalling and reproduction. Calling back on the details of an anomaly can help reproduce them in a synthetic environment to rerun them, and address each anomaly one by one. The ML team needs to keep a track record of the growing dataset of anomalies to maintain accountability. Another important aspect is that with new releases, the newly updated model must be tested against the same bank to make sure there are no regressions.
