OpenAI LP today detailed internal research that found GPT-4 can help online platforms moderate user content more effectively.

According to OpenAI, its research determined that GPT-4 can perform some content moderation tasks better than humans in some cases. The company says its technology offers other benefits as well. In particular, OpenAI determined that large language models enable online platforms to more quickly roll out new content moderation rules and refine existing ones.

After a company releases a new content policy, it must train its moderators how to enforce the policy. Then, it has to collect feedback to determine whether the new rule is effective. Any issues that the feedback may reveal have to be manually corrected.

That process can take months in some cases, according to OpenAI. Its newly detailed research found that GPT-4 can complete the task in a few hours. 

OpenAI says that using GPT-4 to refine a content moderation policy involves a multistep workflow.

First, a company has to create a dataset that includes examples of harmful content. Then, it must instruct GPT-4 to tag each piece of harmful content with a label that describes why it breaches moderation policies. In the final step of the workflow, the labels generated by the model are reviewed to find issues.

Moderation mistakes made by GPT-4 can reveal shortcomings in a company’s content policies. Such mistakes can, for example, be a sign that GPT-4 failed to understand a company’s policies because they’re too vague. This information can be used to make improvements. 

“By examining the discrepancies between GPT-4’s judgments and those of a human, the policy experts can ask GPT-4 to come up with reasoning behind its labels, analyze the ambiguity in policy definitions, resolve confusion and provide further clarification in the policy accordingly,” OpenAI researchers wrote in a blog post today. 

Besides speeding up the process of creating and refining content policies, OpenAI says that GPT-4 can help companies enforce those policies more consistently. 

In an internal test, OpenAI compared GPT-4 with a team of human content moderators. The startup determined that the model can outperform moderators who had received “light training.” However, GPT-4 is currently less accurate than “experienced, well-trained human moderators.”

OpenAI said today that any user with access to the GPT-4 application programming interface can build a content moderation system like the one its new research details. On Monday, Popular Science reported that one Iowa school district is already using OpenAI’s technology to enforce content rules.

According to the publication, the Mason City Community School District has removed 19 books from its library to comply with recently enacted state legislation. The legislation specifies that books in Iowa school libraries must be “age appropriate” and shouldn’t include “descriptions or visual depictions of a sex act.” The school district reportedly used OpenAI’s ChatGPT to speed up the process of deciding which books to remove.

“Frankly, we have more important things to do than spend a lot of time trying to figure out how to protect kids from books,” Bridgette Exman, Mason City’s assistant superintendent of curriculum and instruction, told Popular Science,

The original version of ChatGPT that launched last year was based on OpenAI’s GPT-3 model. The current iteration is powered by GPT-4, the AI at the center of the content moderation research the startup released today. 

Image: OpenAI