Ango Hub has plenty of features that make annotation easy and less time-consuming. Our experts continuously work towards bringing more eventful changes and adding features in Ango Hub to simplify the process of annotation. Out of all the other features, one crucial feature is Ango Hub’s Rank Tool.
In this post, we’ll explore the Rank Tool and how to integrate it into your Ango Hub project and populate ranking options.
What is the Ranking Classification Tool?
The Ranking or Rank Tool in Ango Hub is ideal for Reinforced Learning from Human Feedback (RLHF) projects and workflows. Ideal for scenarios where multiple LLM answers need to be ranked or rated, this versatile and user-friendly solution has become essential for many generative AI and LLM training processes. The tool allows annotators to rank different options including texts, images, videos, or any custom HTML prompts and outputs.
Annotators can:
- Identify high-quality outputs for further exploration
- Fine-tune models for better performance
This tool is designed to enhance your project’s interactivity and provide a structured way to prioritize and evaluate various assets. It also helps in decision-making and improves project outcomes.
How to Add a Rank Tool to Your Project
To integrate the Rank Tool into your project, follow these simple steps:
- Navigate to the Settings Tab: Access your project’s settings
- Enter the Category Schema Section: Here, you can manage various categories for your project
- Add the Rank Tool:
- Click on “Add Category”
- Select Rank
- This action will add the Rank Tool to your project, ready for further customization
By default, adding the Rank Tool will display only the option titles to users. For instance, if you upload an asset without including context data, the rank tool will look like this:
To enrich the ranking options with content, you need to populate them using specific instructions.
Populating Ranking Options
1. Automatic Ordering
To automatically order and enrich ranking options with content of any HTML-supported type, follow these steps:
-
- Upload Your Asset Using JSON Import
- Include contextData in the following format:
Once JSON is imported, Ango Hub assumes that ranking options are presented in order. For example, if you have three ranking options (“Response 1,” “Response 2,” “Response 3”), the first item in the responses list will be “Response 1,” the second will be “Response 2,” and so on.
2. Manual Ordering
If you prefer not to follow the automatic order or wish to maintain a specific or manual order in Ango Hub, use the following JSON format:
Here, the id corresponds to the name of the option as created in the Category Schema and the label is the content of the ranking option.
Once the ranking options are populated, they will be displayed clearly and concisely, enhancing the user’s ability to evaluate and prioritize them effectively. Prompt and response ranking of outputs, drag and drop placement for ranking. Here’s an example:
Conclusion
Ango Hub’s Rank Tool is essential for projects that require structured evaluation and prioritization of assets. By following the steps outlined above, you can easily integrate and customize it to meet your project’s needs, ensuring a seamless and interactive user experience. This tool allows you to leverage human expertise to rank outputs from your RLHF models. The invaluable feedback it provides guides your model toward generating superior responses that align with your desired outcomes.
Learn more about Ango Hub and its features here.
