Improved Field-Extraction Accuracy
Hand - annotated
To guarantee a best-in-class user experience, Bill.com leverages iMerit annotation to optimize the accuracy and performance of their field-extraction algorithms.
Challenge
Bill.com leverages field-annotation algorithms to optimize and ease the experiences of their users. IVA, the intelligent virtual assistant, leverages computer-vision and deep-learning to
extract data from user-uploaded documents, effectively sparing users from manual entry. After noticing that users were changing inputs extracted by IVA, Bill.com's AI team decided it was time to evaluate annotation vendors who could improve IVA’s accuracy.
After consulting with the iMerit team, the IVA team was attracted to iMerit’s collaborative approach, social-impact mission, security protocols, and ease of feedback. As iMerit began analyzing Bill.com’s data in relation to Bill.com's business goals, the team began ideating on annotative solutions that stood to improve IVA’s accuracy.
“Our previous annotation vendor was very transactional. We needed a vendor who understood our business needs and would work closely with us to accomplish them.”
Solution
To better understand where IVA wasn’t performing optimally, iMerit began by evaluating IVA’s performance across a series of documentation including invoices, statements, receipts and more. After running several annotations and checking with the Bill.com team, iMerit discovered exactly how to improve IVA’s accuracy through improved annotation.
iMerit’s tool-agnostic approach guaranteed integration with Bill.com’s proprietary toolset, which guaranteed the secrecy of sensitive user documentation. As the annotation progressed, Eitan and his team would spot check to see if IVA’s accuracy was improving.
On each occasion, the AI team provided feedback to iMerit annotators that helped to continually improve the accuracy of IVA. After hand-annotating every document, iMerit came up with a solution that stood to improve the accuracy of IVA: combine iMerit annotated high-error user documentation with normal user documentation, and feed this combination of data into the model.
“It wasn’t a static engagement. iMerit took the time to collaborate and calibrate their annotation approach with our team, and make this project successful.”
Result
iMerit’s combining of training data successfully improved IVA’s field-extraction accuracy by 5%. Instead of manually correcting IVA, users are now processing documents faster than before and with markedly higher satisfaction. Thanks to iMerit, the Bill.com team can now pinpoint exactly how well their field-extraction algorithms are performing, allowing them to continuously and easily identify areas for improvement.
Today, Bill.com is expanding their relationship with iMerit to continuously improve the performance of their field-annotation algorithms.
