The Robotaxi market globally is expected to grow at a CAGR of 137% from 2021 to 2030, with vehicle units increasing from 617 to 1,445,822, as per a report by MarketsandMarkets. Fuel efficiency, emission- control, road safety, and traffic management are a few factors driving its growth. The surging demand of the segment requires Robotaxi companies to improve the safety of their autonomous vehicles, which depends on the performance of their ML models.
While developers at Robotaxi companies are improving and refining their models, they need to know how accurate the ground truth data is. iMerit has been working with a leading Robotaxi, with more than 1200 employees, to improve ground truth data for improved safety and control of autonomous vehicles.
Better Quality Control with Industry Expertise
Before iMerit, the Robotaxi company worked with large-scale auto-labeling partners but found quality issues with the output. Expertise and experience are crucial for a successful data annotation partnership in the autonomous vehicles industry. Due to high volumes, the company needed expertise in quality control to ensure accurate ground truth data.
Along with the high volume of data, the Robotaxi company came with complex annotation requirements.
- Multiple data types, including 2D, 3D LiDAR, and Video on multi-frame annotation
- Semantic segmentation, Cuboids, and Polygon, with Point density ranges from 1-5, on upwards to over 1,000
- 30-40 object classes, with often changing taxonomy, classifications, and attributes
- Spanning multiple cities and countries across the US, EU, and APAC
The team from iMerit reviewed different types of annotations, such as full-scene semantic segmentation, mapping scenarios, different simulation stages, and prediction work types.
QC Audit Services for Autonomous Vehicles by iMerit
iMerit worked collaboratively with the Robotaxi company on the quality control of annotated data and shared expertise on QC for production data.
95% Accuracy
The human-in-the-loop workflows by the iMerit team, with weekly calibrations, helped exceed an accuracy threshold as per client requirements. For instance, the LiDAR workflows started at 80% accuracy and are now running above 95%.
250% Improvement in Efficiency
In-house autonomous vehicle experts in the iMerit team could identify issues and areas of improvement with the client tool. The team also shared insights and recommendations for taxonomy upgrades by adding new classes and removing unused ones. With these changes, the Robotaxi company saw a 250% improvement in annotation efficiency. For example, the semantic segmentation workflow started with 200 QAs per batch but is currently doing 500 QAs per batch with the same set of resources.
Overachieving SLAs
With a 2-step rigorous quality control process, iMerit could deliver high-quality production-ready data for their ML models. The team that worked on the project was curated carefully and trained rigorously with solution architects and industry experts to improve throughput. Within a few weeks of the project, the team could exceed the client’s expectations for SLAs and priorities.
Relentlessly Pursuing High-quality
iMerit’s partnership with the Robotaxi company started with LiDAR and 2D segmentation, and now it is moving into much more complex scenarios of scene hunting and masking.
When the Robotaxi came to iMerit with its requirements, it stated what was needed exactly. However, as the engagement progressed, the company saw the quality of the work, and now they ask the team to find the exceptions in their data and share insights.
With iMerit’s experts and human-in-the-loop, the client is able to improve the quality of raw production data to improve the performance of their ML models for better safety of their Robotaxis.
