In an effort to clamp down on the number of erroneous or joke edits made on Wikipedia.com, the foundation that runs the website has built its own AI filter to keep tabs on new entries.

The sheer number of daily edits on Wikipedia is staggering – there are approximately ten new entries made every second, with an additional 20,000 new articles created each month. Simply put, keeping up with it all is no job for humans. To make matters worse, MIT Technology Review points out that Wikipedia is going through something of a manpower crisis, with the number of volunteers having declined by forty percent to just 30,000 in the last eight years.

In order to compensate for the shortfall and the growing amount of work that’s needed, the Wikimedia Foundation’s senior research scientist Aaron Halfaker has developed a new AI system called the Objective Revision Evaluation Service (ORES) to take on the job of editing people’s edits.

ORES uses machine learning to revise its algorithms continually, so it’s better able to spot whether erroneous edits are simple mistakes, jokes, or worse, deliberate vandalism.

Here’s a brief overview of how it works, as described by Halfaker in the official Wikimedia blog:

“ORES brings automated edit and article quality classification to everyone via a set of open Application Programming Interfaces (APIs). The system works by training models against edit- and article-quality assessments made by Wikipedians and generating automated scores for every single edit and article.

What’s the predicted probability that a specific edit be damaging? You can now get a quick answer to this question. ORES allows you to specify a project (e.g. English Wikipedia), a model (e.g. the damage detection model), and one or more revisions. The API returns an easily consumable response in JSON format:

As soon as ORES identifies what looks like a particularly glaring mistake, a human editor will be alerted to correct the new entry, if needed. Additionally, any editors who’ve been deemed to have made a mistake will be informed so they can learn from it.

To date, ORES is being used to monitor new entries made in English, Farsi, Portuguese and Turkish. The system uses previous edits made in those languages to build an algorithm the AI can use, so it’s aware of the challenges it will likely face.

“Our hope is that ORES will enable critical advancements in how we do quality control—changes that will both make quality control work more efficient and make Wikipedia a more welcoming place for new editors,” explained Halfaker.

Wikipedia is planning to roll out ORES slowly across other language versions of Wikipedia. For now, editors will be given the choice of opting in to work with the AI if they choose.

Photo Credit: golanlevin via Compfight cc