Evaluation And Categorization
of Text Summary Relevance
Leading professional social network improves search relevance by switching from crowd-based vendor to iMerit domain experts for advanced text evaluation.
Challenge
A premier professional social networking platform sought to enhance user engagement and search relevance across its products, including its learning platform and AI-driven coaching tools. The project involved sophisticated content categorization and the evaluation of text summary relevance, demanding domain expertise to help significantly refine user experiences.
After working with a generic crowd vendor, this social media company needed higher quality data. Under this process, data validation required evaluation between five to seven technicians, resulting in high costs, poor efficiency, and missed project timelines. Realizing they needed a better solution to scale operations and meet their ambitions, this company began evaluating AI data solutions providers.
“Our old approach wasn’t delivering quality in the timeframe we needed and couldn’t keep up with our changing requirements.”
Solution
To enhance search relevance, a custom strategy was developed where domain experts would evaluate summaries of articles and tag them based on their content topics. This extra criteria enabled this company’s algorithm to suggest more relevant content, resulting in a personalized user experience and more user time being spent on the platform.
For project staffing, iMerit created an in-depth guideline review and specialized training sessions for 200+ in-house experts to align with the project's end goals. Utilizing this company’s previous crowd vendor’s tooling environment, iMerit devised a multi-tiered workflow featuring thematic categorization, summary relevance assessment, and optimization of search relevance, all through the end-user's lens.
To ensure quality, a dedicated team of Project Managers and Team Leads managed the 200+ experts. A standout innovation was iMerit’s real-time quality control (QC) mechanism, leveraging an internal task tracking tool to dynamically assess and accelerate project timelines.
“A huge kudos to the team for such outstanding results. You achieved 91% binary accuracy and 94% category classification accuracy. That’s amazing.”
Result
After finishing the project, iMerit helped this client complete their project 37% faster while improving their machine learning model’s accuracy. These workflows are now fully automated and no longer require any human intervention. The project’s growth trajectory over four years underscores the tangible impact of iMerit’s solutions and how it aligns with this company’s product roadmap.
