Facebook Inc. removed 22.1 million pieces of hate speech content on its namesake social network in the third quarter, 94.7% of which was detected automatically by artificial intelligence models.

The company shared the figures in the latest edition of its Community Standards Enforcement Report that was published today. On the occasion, it also detailed several of the new AI innovations behind the improved detection accuracy, including a technology called Linformer that could significantly enhance the computational efficiency of natural-language processing models.

“AI has been a powerful tool to support our content moderation work at scale,” Facebook Chief Technology Officer Mike Schroepfer said on a press call this morning.

The 95% detection rate Facebook’s AI models achieved in the third quarter represents a major improvement over the previous three-month period. The figure is up from just over 80% in the second quarter and 24% in 2017, according to the news Community Standards Enforcement Report. The company recorded similar results on Instagram: AI models detected 95% of the 6.5 million pieces of hate speech content Facebook had identified on the photo sharing service in the third quarter. 

Behind the scenes, the increased detection rate is made possible by several new software systems developed by the company’s engineers. Some of those innovations are new AI models, while others are supporting technologies designed to facilitate the development of better neural networks.

Falling into the latter category is a system dubbed Reinforcement Integrity Optimizer or RIO. It automatically optimizes the accuracy of the neural networks that review content users upload to Facebook and Instagram by training them on millions of “current pieces of content,” the company said. Using fresh posts to train models helps keep the models up to date, ensuring they can detect new types of harmful content that emerge.

“The framework’s data sampler estimates the value of the training examples, deciding which ones will produce the most effective hate speech classifier,” Facebook researchers explained in a blog post. 

SimSearchNet++ is one of the newest models Facebook uses to find harmful content. The AI, detailed today, automatically detects cases when an image flagged as containing hate speech is reuploaded elsewhere in an altered form. The company said that the AI is better at overcoming image manipulations such as crops and blurs than the earlier model it used for the task.

Facebook today also detailed RoBERTa and XLM-R, two other new models developed by its engineers. RoBERTa is a natural-language processing model that has been shown to deliver “state of the art performance,” while XLM-R likewise promises industry-leading speeds for cross-lingual understanding. Cross-lingual understanding is the process wherein a model is trained in one language and then applies its learnings to other languages, an important feature for applications such as translation services.

But from a research standpoint, the perhaps most significant technology Facebook revealed today is Linformer. It’s a new approach to building Transformers, a widely used class of natural language processing models, that promises to reduce the amount of computational resources required for AI training. 

“When you increase the input size from, say, to 4,000 to 8,000, the number of computations doesn’t double,” Facebook researchers explained. “It goes from about 16,000,000 to 64,000,000” with traditional Transformer models. With Linformer, in contrast, the number of computations increases linearly, meaning models can be trained using significantly less infrastructure.

Taken together, the AI work helps reduce the load on thousands of human moderators still needed to make judgment calls. “We can amplify the work of human reviewers,” Schroepfer said. “People can spend time on the harder, more nuanced and subtle things.”

In response to a press question, Facebook also addressed a letter sent Wednesday by more than 200 moderators demanding a safer work environment, since many have been called back into offices despite the pandemic. “We’re not able to route the most sensitive and graphic content to people at home,” said Guy Rosen, Facebook’s vice president of integrity. “This is not content you want people reviewing from home around family.”

Facebook’s efforts to improve how it tackles hate speech using AI come in a time when it’s facing mounting scrutiny over the issue, as well as over its business practices. The Washington Post reported today that state and federal investigators are planning to file antitrust charges against the social network for alleged anticompetitive practices.

With reporting from Robert Hof

Photo: Eston Bond/Flickr