Segment Faster on the iMerit Ango Hub platform with Smart Segment Tool

October 05, 2023

In data labeling, efficiency matters, and quality data must be delivered as fast as possible so that model training, validation, and deployment may follow. However, labeled data often becomes the biggest bottleneck in deploying ML models. Here’s where Smart Segment comes into play.

Image Segmentation is one of the areas where this bottleneck is particularly acute due to the pixel-level accuracy required. At iMerit, we aim to reduce the time it takes to label a data set while maintaining accuracy. To achieve this, we have an arsenal of AI assistance tools that speed up the annotation workflow. 

Smart Segment is another addition to our suite of AI assistance tools. The tool makes a rough mask (polygon or segmentation) and processes it to create a highly accurate and tightly bounding mask for the object of interest.


Segmenting a moderately complex object by its boundaries up to a pixel level can take multiple minutes. Smart Segment draws a rough mask in 2-3 seconds and refines it to capture its boundaries while reducing the time per label.

Based on our practical experiments on a variety of data, we have observed the following:

  • For complex objects, reduction of segmentation time from 60-100 seconds to 5-10 seconds.
  • For simple ones, reduction of segmentation time from 20-30 seconds to 5-10 seconds.

Reduction of annotation time per object consequently results in the following benefits:

  • Faster preparation of dataset, thus faster time to model training
  • Lower mental load for annotators
  • Lower cost per label
  • Tighter segmentations, thus more accurately segmented (higher quality) dataset.

How Smart Segment Works

Smart Segment employs a deep learning model trained on a wide variety of images from multiple open-source datasets with more than 1000 classes for segmentation. Using the training data, the model can accurately predict the pixels that make the object against pixels that don’t.

The model has a ResNet 101 backbone, with an FPN to retrieve features within images of different scales. The model performs well class agnostically as it was trained to learn “objectness” (Object class only) rather than classifying pixels into multiple classes.

Smart Segment Model Inference

How to use Smart Segment

The tool is easy to use:

  • Open an asset on iMerit Ango Hub.
  • Select the Smart Segment tool.
  • Draw a rough boundary around the object of interest.
  • Let Smart Segment refine this boundary for higher accuracy.

Use Cases

Smart Segment works well with common world objects and can be used for a variety of segmentation tasks in various industries:


Plants, fruits, and crops often have complex boundaries, and annotations can take many minutes. With Smart Segment, it is a matter of seconds.

Autonomous Vehicles

Common objects encountered in AV datasets, such as cars, pedestrians, traffic lights, and signs, have a moderate complexity in terms of segmentation. Using Smart Segment, this complexity can be alleviated, allowing us to create AV datasets faster.


While the tool is trained to understand the definition of objects in a real-world context (common objects), for the medical domain, where objects are discernible, such as skin lesions and microscopy, the refiner works well in getting tight boundaries.

The quality assurance use cases of this tool are: 
  • It can be used on already labeled datasets to take images in batches and refine the labeled masks. The model performance is such that the resultant IOU (compared to the ground truth)is almost always higher. It means for an already good mask, the tool increases tightness and accuracy. 
  • It can be used on bounding box projects(object detection rather than segmentation) to obtain tighter bounding boxes, increasing the overall IOU.

The Smart Segment tool is available on our all-in-one data labeling platform, the iMerit Ango Hub, used by industry leaders to annotate millions of data points in mission-critical environments.

Try iMerit Ango Hub now for a free trial. Book a demo with us to talk about how to solve your data labeling needs.