UPDATED 14:15 EDT / APRIL 28 2011

Zappos.com Puts Crowdsourcing where its Mouth is with Amazon’s Mechanical Turk

zapposcom-mechanical-turk The idea behind crowdsourcing has always been taking the combined efforts of a large group and directing them towards a single task. In a lot of venues, crowdsourcing can be used for charity by taking small donations from large numbers of people, it can be used to rapidly develop a lot of information about a particular area of expertise, or provide complex actions that only humans can do across large data sets. In the case of one online retailer, Zappos.com, it’s become an engine of revenue increase.

When it comes to language, humans are still the bomb; plain English query and other linguistic lexicon still provide a great deal of trouble for computer applications. As a result, when Zappos realized that well spelled and spoken reviews of products upped the likelihood that people would make purchases they turned to crowdsourcing for the solution.

Panos Ipeirotis, a computer science and business blogger, caught onto this story about leveraging Amazon’s Mechanical Turk to copyedit reviews independent of the content,

An online retailer noticed that, indeed, products with high-quality reviews are selling well. So, they decided to take action. The retailer used Amazon Mechanical Turk to improve the quality of the reviews posted on its own website. Using the Find-Fix-Verify pattern, the retailed used Mechanical Turk to examine millions of product reviews. (Here are the archived versions of the HITs: Find, Fix, Verify. And if you have not figured out the firm name by now, the retailer is Zappos.) For the reviews with mistakes, they fixed the spelling and grammar errors! Thus they effectively improved the quality of the reviews on their website. And, correspondingly, they improved the demand for their products.

For the curious readers, Zappos has been doing this at least since April of 2009, which means that they were doing it even before being bought by Amazon.

At its core, Find-Fix-Verify is a process by which processes can be passed through crowdsourcing with a very low error rate. Conceptually, it splits tasks into three components: finding the error, fixing the error, and finally a verifying the fix. Each of these three tasks would be provided by a separate process (in this case a person) and possibly done several times by different people (in the case of finding and verifying.) By this pattern, a document could be passed through several hands to be checked for errors and then finally dropped after the errors drops to zero or something so low that it’s not worth another pass; and after fixes are made, if enough people verify that the fix is proper it can be committed or sent back to be fixed again if too many mark it as wrong.

According to the article, after the reviews had been edited by the crowdsourcing engine—Zppos.com reports it greatly impacted their sales and revenue. At about 10¢ per review and almost 5 million reviews, that would make out to a few hundred thousand dollars. That suggests the improvement must have been substantial in order to generate an actual impact. Although the results are yet uncorroborated, it does appear to merit a case study.

The ethics of this process has been questioned, of course, as it affects the earnest reactions of reviewers buy subjecting them to editing. In fact, copyediting for grammar and spelling could change the tone and content of a message. However, by using crowdsourcing and passing it through the hands of millions of anonymous users, it absolves Zappos from being called out on introducing bias into the reviews. Instead of changing their polarity or tone, fixes to grammar and spelling would only work to make them more uniform in language and easier for readers to access their content.

In fact, it seems like the entire Internet might be a better place if public reviews were easier to digest.

We can add this to the growing list of crowdsourcing technologies that generate results, but this is the first one with a direct economic impact. In the past we’ve seen AudioDraft use crowdsourcing for music and sound effects contests; Google has gone literally to ground for local experts to crowdsource their knowledge of locales with citizen cartography; and on the charity angle, Internet famous Craig Newmark of Craigslist started Craigconnects, a crowdsourced social empowerment and volunteer network.


A message from John Furrier, co-founder of SiliconANGLE:

Your vote of support is important to us and it helps us keep the content FREE.

One click below supports our mission to provide free, deep, and relevant content.  

Join our community on YouTube

Join the community that includes more than 15,000 #CubeAlumni experts, including Amazon.com CEO Andy Jassy, Dell Technologies founder and CEO Michael Dell, Intel CEO Pat Gelsinger, and many more luminaries and experts.

“TheCUBE is an important partner to the industry. You guys really are a part of our events and we really appreciate you coming and I know people appreciate the content you create as well” – Andy Jassy

THANK YOU