Scaling Annotative Capacity
Kitware leverages iMerit to rapidly annotate terabytes of video data and achieve high-quality annotation.
Challenge
Kitware provides advanced technical computing, state-of-the-art artificial intelligence (AI), and tailored software solutions to its customers. While working with Intelligence Advanced Research Projects Activity (IARPA), Kitware needed to scan almost 200 hours of video, along with a significant amount of unreleased, sequestered data, to identify 47 unique activity types. IARPA then used this annotated data to support the development of an AI solution that could be used by many institutions to detect specific activities performed by people or vehicles. To annotate this data, Kitware sought support for their in-house annotation team. With limited time to hire, train, and onboard new talent, Kitware began evaluating AI data solutions vendors who could help. After evaluating vendors, Kitware chose to work with iMerit.
“iMerit brought a lot of experience to the table that helped us. They pushed through the pandemic when the world was on hold, complimented our tool set, and expanded our capacity and capabilities.”
Solution
Kitware assigned iMerit with 189.5 hours of video content, which included nearly 3000 5-minute video clips. Annotation guidelines were established seamlessly, and iMerit quickly developed a sequential production and quality control workflow to optimize efficiency while maintaining quality. iMerit annotators immediately began viewing, segmenting, and annotating the video footage.
After iMerit completed initial annotations, Kitware provided feedback, which was integrated into iMerit's annotation workflow. The result of this strong feedback loop allowed iMerit to fine-tune the annotation, achieving an overall clip quality of 97.5%, with only 2.5% of the video clips requiring rework.
“iMerit functions as an arm of Kitware, complimenting our toolset and expanding our ability to take on more business.”
Result
iMerit annotators dedicated over 60K hours annotating video footage, with an average of 26 hours on each clip. This yielded a final result of 132k activity instances, 186k track instances, and 84 million bounding boxes applied. Kitware annotators reviewed 100% of the testing and evaluation quality data, resulting in an exceptional 2.5% clip return rate.
iMerit continues to be an essential partner for Kitware. As the IARPA engagement continues, iMerit functions as an arm of Kitware, using the same tools, philosophy, and professionalism that Kitware’s customers demand.
