Blog

Communication is Key, even in Dataset Creation

In our previous blog post about working with external teams, we talked about the importance of knowledge transfer when creating, enhancing, and cleaning datasets. Take a look here. This week, we’re talking about communication. When we work on projects with co-workers, we do a great deal of communicating: meetings, calls, stand-ups, check-ins, you know the drill. When work with external teams – from consultants to BPOs to crowds – it’s […]

In our previous blog post about working with external teams, we talked about the importance of knowledge transfer when creating, enhancing, and cleaning datasets. Take a look here. This week, we’re talking about communication.

When we work on projects with co-workers, we do a great deal of communicating: meetings, calls, stand-ups, check-ins, you know the drill. When work with external teams – from consultants to BPOs to crowds – it’s important to remember that though there may be fewer in-house team members, communication is still key.

In practice, this first means establishing good knowledge transfer, as we talked about in our last segment. However, that one-way channel is not enough.

It’s important to create feedback channels from your external teams to in-house teams.

When working with a managed team – whether it’s a group of consultants or an iMerit team – this can be straightforward. Between email, phone calls, Slack, Skype – the list goes on – channels are established and it’s just a matter of making and sticking to a schedule.

When working with an anonymous crowd, however, you need to get creative.

The “team” you’re working with could be ever-shifting, making it hard (but not impossible) to gather unified feedback.

One method to try is adding a “task” in your process that asks for the workers’ feedback. You can gather feedback on the task structure, the task documentation, and see if there is anything you can change to make the task more straightforward for them, and more useful for you.

Encourage honesty, and then you can iterate your tasks based on the feedback you get. Over time, your tasks will be clearer and easier for the crowd to complete, ensuring even more accuracy for you!

What does this look like in reality?

Perhaps you have an online clothing store, and are entering new items into your retail site’s taxonomy. As you go through the data coming from your crowd, you notice that there is an item with markedly low inner-agreement rates. Different crowd members keep placing it in different categories, there is no agreement on where it should belong.

hoodie_blog_imgTake a look at it on the left.

Your crowd is baffled.

Some call it a “hoodie” – it does have a hood, after all – while others are placing it in the “sleeveless top” category. If they were sitting in-house, they could ask you which of these categories is more important to your categorization, or they could suggest placing it in two categories. But, they’re not in your office, so you need to anticipate their thoughts.

To avoid this confusion, design tasks in a way that makes it easy for workers to voice their thoughts.

Going forward, there are many interventions you could take:

  • Add a checkbox workers can tick to mark that they are “unsure of the category”
  • Include a free-text field that workers can fill in with any questions they have about the categorization of each particular item
  • Place a question at the end of the tasks asking your workers if they faced any confusion at an overall level
  • Require a final question where workers can offer suggestions for tasks or instructions

Get creative with the questions you ask your external teams, and remember they’re team members just like those you see in your office everyday!

Stay tuned for more tips on using external teams.

Author


Avatar