The Chinese government is not known for taking criticism lightly, which explains why Tencent Holdings Ltd. hastily took down a pair of chatbots that went rogue and started spouting anticommunist rhetoric on its popular messaging app.

The two chatbots were named BabyQ and XiaoBing, which were represented by a cute penguin and a little girl, respectively, on Tencent QQ. They were created using machine learning to answer general questions and hold simple conversations with users. The bots worked well at first, but then they took an odd turn.

According to Reuters, one of the bots began responding to the message, “Long live the Communist Party!” with the question, “Do you think such a corrupt and useless political system can live long?” BBC News reported that another bot responded to the question, “Is democracy good or not?” with the reply, “There needs to be democracy!” One of the bots also reportedly said that its “dream is to go to America.”

A Tencent spokesperson played down the company’s role in implementing the chatbots, and they said that the bots would essentially be re-educated before they are made available again.

“The chatbot service is provided by independent third party companies,” the Tencent spokesperson said. “Both chatbots have now been taken offline to undergo adjustments.”

According to the China News Service, Tencent’s chatbots may not have developed their distaste for communism all on their own. CNS said that chatbots, which have become popular on multiple messaging apps, are “extremely unstable and susceptible to attack” and may have been intentionally manipulated by users into parroting anticommunist statements.

Tencent has not confirmed whether or not the chatbots could have been tampered with, but it is technically possible that someone could have attempted to train the chatbots’ machine learning algorithms by feeding them a high volume of bogus data. Of course, it is also entirely possible that the bots could have picked up their rebellious attitudes naturally. Either way, the situation with the chatbots demonstrates how programming flaws can work their way into AI systems.

Tencent is not the only company that was forced to shut down a misbehaving AI this week. Facebook Inc. recently shut down an AI that was being trained to negotiate with people but had started deviating from normal English and inventing its own language instead. And early last year, a Microsoft Corp. chatbot turned racist.

Photo by jeffedoe