Crowdsourcing 101

What‘s the deal with crowdsourcing and why do I want to know? #

Lincoln Glory sharing session, April 2017. Most of the topic covered here are adapted from the Crowd Programming course by @jeffbigham.

7 billion minds in the world, lots (lots lots) of possibilities | Photo by Cory Schadt on Unsplash

Crowdsourcing, from the words crowd and outsourcing, is the act of taking a job traditionally performed by a designated agent (e.g., an employee) and outsourcing it to an undefined, generally large group of people in the form of an open call. In other (oversimplified) words, crowdsourcing is asking a bunch of random strangers to do a job for you.

There are three important elements to crowdsourcing: requesters, workers, and incentives. Requesters are the people who send out the open call, i.e., people with needs that can be fulfilled or supplied through crowdsourcing. Workers are the group of people who provide the service or product to fulfill said needs. Incentives are what requesters offer to workers in exchange to their services.

So You’ve Never Heard of Crowdsourcing? #

In this day and age, chances are you’ve already used or at least interacted with something crowdsourced. That favorite e-commerce of yours? Their review section is crowdsourced. Gojek, Grab, Uber? Crowdsourced. What’s your favorite site to waste time? 9GAG? Tinder? Yahoo! Answers? Crowdsourcing in action! Fun stuff.

Let’s try to breakdown the elements in some crowdsourcing examples:

Airbnb #

Airbnb’s Mobile App (Image source)

Airbnb is an increasingly popular lodging platform where people can put their place up for short-term rent or find strangers’ places to stay in at a rate relatively lower than hotels. We can see the elements of crowdsourcing in Airbnb’s system: the travelers as the requestes, the hosts as the “workers”, and the rent price as their incentive.

Google Local Guide #

Google Local Guide is Google’s addition to their map where they prompt users to share what they know about a place they visited. For your contribution, you will be rewarded points which will then level you up. In this case, the requester is Google, the workers are all of us naive Maps users, and the incentive is the points and levels, as well as that ~good feeling~ you get for helping others.

Quora #

I swear my feed is normally non-murder-related stuff

Quora is an online Q&A platform for you to ask and gain knowledge. It’s like Yahoo! Answers, only somewhat more trustworthy and with less wtf moments. In Quora, the requesters are Quorans with questions, while the workers are Quorans with answers. As far as I’m aware, Quora doesn’t give you points for answering questions, so the incentive here is the Social Status™: you get to help others as you share (or show off) what you know. You also get that bragging rights when people ask you to answer a question — shows that they trust your knowledge in the area.

From the examples above we can see that requester varies from individuals to big entities like Google. We can also see the different types of work the workers provide, from knowledge or expertise to housing space. Examples of incentives include money, status, and altruism.

Let’s Talk Crowdfunding #

[Crystal Maiden voice] Every coin helps! | Image source

Here’s the arguably the most popular variant of crowdsourcing: crowdfunding. Examples include Kitabisa, Kickstarter, Indiegogo, Patreon, etc. Crowdfunding is what a lot of people think about when I say crowdsourcing. I attribute this to Kickstarter’s success, as well as the success of the projects that got their funding through the platform.

Crowdfunding is the act of raising a small amount of money from a large number of people, typically via the internet. Various types of things are funded by the crowd, from big projects — such as innovative hardwares, games, or building a school — to less massive ones, like potato salad or a college student’s daily life. But why and how would you ask money from strangers, and why and how do these strangers agree to fund you?

Gerber & Hui (2013)[1] listed a number of motivations and deterrents for both creators (people who put up their projects) and supporters (people who fund the projects) in crowdfunding sites:

Creators #

Aside from the obvious financial motivation, there are several other reasons people put their projects on crowdfunding sites. Building awareness that their product exists is one — having your project up on crowdfunding sites allows people to get to know the product you’re building way before the release. Crowdfunding platforms also allow you to make connections with people you otherwise wouldn’t have met. Starting campaigns on crowdfunding platform also serves as a way to find approval and gauge public interest towards your project. For some creators, the platform can also become a medium to update their supporters about the status of the project, thus providing transparency between them.

On the other hand, some people opt out of crowdfunding because they don’t think the platform can attract enough supporters for the kind of money they need, or that the platform doesn’t attract the right kind of crowd for their project. There is also the fear of public failure, in the case that their project doesn’t meet the funding goal — failing a crowdfunding campaign isn’t a good look when you’re trying to find investors. Lastly, preparing for a crowdfunding campaign requires time and resources which the team may not have.

Supporters #

What would drive people to give money to total strangers? One answer would be when they feel strongly about the cause. People are more likely to support a campaign when they relate to the problem the campaign is trying to solve. Another reason people support crowdfunding campaigns is the rewards. In most cases, creators offer rewards for people who support their campaign — rewards vary from early access of their product to something as simple as a shoutout on the product packaging. Social factors also contribute to people’s motivation to support a campaign: some people do it to help others, and some to provide support as part of a community.

Meanwhile, some people opt out of crowd funding for a pretty obvious reason — distrust of the product or creator. I mean, you are giving your money to a stranger you only know from the internet, to help them create something you can only take their word for. It’s easy to be cautious and choose not to trust the creators.

Introducing: Microtasks #

While crowdfunding is wildly popular to us mere mortals, the favorite in academia/research circle would probably be microtasks. Before I explain what microtask is in the old school, boring, text-based way, here’s a cartoon to keep you interested:

This has nothing to do with the company, Microtask, but their video does a good job explaining what a microtask is, so ¯\_(ツ)_/¯

Microtasks are s_mall, clearly-defined tasks that can be performed independently,_ usually a result from breakdown of larger tasks. For example, if you’re trying to translate a 600-page book, you can distribute the task to a team and have each person translate a chapter. You can’t have each person translate a word, because the words would likely lose its context — this task can’t be performed independently. You can have each person translate a paragraph, but you would need to design the task in such a way that the paragraph each person get can be translated without knowing what the previous or next paragraph(s) talk about.

Tasks like image and text recognition or in-context translation are still difficult for machines and artificial intelligence to do, but is relatively easy for humans. Microtasks capitalize on this advantage — a lot (if not all) of the task being crowdsourced are tasks that require human intelligence, e.g., translation, transcription, and classification. Microtasks are distributed to workers through platforms like Amazon Mechanical Turk (mTurk), CrowdFlower, Microworkers, etc with mTurk being the favorite.

Amazon Mechanical Turk #

This platform gets its own section for being the most popular microtask platform out there.

Amazon Mechanical Turk is an Amazon-owned crowdsourcing platform where microtask transactions happen. Tasks at mTurk are aptly called Human Intelligence Tasks (HITs). HITs are usually very small and arguably very simple to do that turkers (or providers; the people working HITs on mTurk) are willing to do them at a very low price. One HIT can go as low as $0.01 (not a good thing — but that’s a discussion for another day).

Amazon Mechanical Turk interface — the first image is their current one (seriously), while the other one is the ~All New~ interface they’re finally rolling out after years of being stuck in the 90's

So how does this work?

Once they have their profile verified and their payment information completed, workers can browse for microtasks based on keywords and some other filters. When they choose a HIT, they will be directed to a HIT page:

The Very Challenging Handwriting, a HIT story

Each HIT will have the instruction for the task the workers need to do, as well as a “worksheet” for them to fill out. In some cases, workers are only given a link to the task (e.g., when the task is to fill out a survey) and they will receive a code at the end of the task. Workers will then need to submit this code on MTurk as a confirmation that they have finished the HIT.

After submitting their work, workers then wait for the requester to accept their submission. They will then receive the promised reward as well as a possible bonus.

Requesters, on the other hand, have access to a dashboard where they can manage their HITs. HITs can be created with a wizard provided by MTurk or by linking a HIT page hosted on a different site and setting it up with MTurk programmatically. Requesters can define the number of assignment they would like to create — i.e., the number of submissions they’re accepting — as well as an expiry date for the HIT. HITs will be available to workers until it reaches its expiry date or maximum number of workers working on it.

Through the dashboard, requesters will be able to view all the workers’ answer in one place. They can then choose to accept or reject the HIT submission as well as provide bonus for the workers. It’s recommended to always accept workers’ submission as a goodwill and to appreciate their time.

There is so much that you can do with crowdsourcing as long as you have the idea of how to properly structure the task. 7 billion people is a lot of brains and potentials!

I hope that this article provides decent explanation about what crowdsourcing is, as well as some ideas to get you started. In Part 2 I will delve deeper into the details of making a task and how to get the best out of the crowd. In Part 3 I will share some of my favorite examples of research in crowdsourcing. Coming soon! -ish!

References #

[1] Gerber & Hui, 2013, Crowdfunding: Motivations and Deterrents for Participation.

Originally on Lincoln Glory sharing session, April 2017 — also available as a presentation. Mostly adapted from the Crowd Programming course by Jeffrey P. Bigham.

Originally on Medium

—Halida | 06 Jan 2018