With increasing number of cheats and misbehaving people online, Microsoft has come up with a plan to safeguard its users. To avoid the harassment of “cheats and jerks” online with the next console generation, Microsoft has created a new and enhanced reputation algorithm for Xbox One.

Moving away from the Xbox Live’s survey option, Microsoft is updating its player reputation model for Xbox One. This will include incorporating direct feedback–like blocking or muting troublesome players. As a result, the behavior of other people around the “suspect” user will be used to produce a better model of their behavior–to see if a user really does tend to cross social boundaries rather than just gets reported a lot.

Apparently, there will be three categories of players: “good,” “needs improvement,” and “avoid me.” The assessments will be reflected through color-coded indicators on their gamer card, and there will be penalty for players who are repeatedly disruptive on Xbox Live.

“The algorithm is sophisticated and won’t penalize you for a few bad reports. Even good players might receive a few player feedback reports each month and that is OK. The algorithm weighs the data collected so if a dozen people suddenly reporting a single user, the system will look at a variety of factors before docking their reputation. The system will use additional verification methods in an attempt to provide accurate reputation scores,” said Michael Dunn, program manager for Xbox Live.

“Looking at someone’s gamer card you’ll be able to quickly see their reputation. And, your reputation score is ultimately up to you. The more hours you play online without being a jerk, the better your reputation will be; similar to the more hours you drive without an accident, the better your driving record and insurance rates will be,” Dunn added.

While this may be a good step by Microsoft to deal with people who say odd things online, we recently had an incident where the step went in an all wrong direction. Justin Carter has been being held under suicide watch in a Comal County Jail, just outside of San Antonio, Texas, for words he said during an argument on Facebook, over a disagreement with players from the online game League of Legends.

“People on online forums say the darndest things,” says Kyt Dotson, assistant editor at SiliconAngle, “and gamers often must rely on their own judgement for determining how to interact with others. Due to the general malaise of ‘loud mouths’ on Xbox Live users have tended to stick to party chat and only speak with their friends–partially due to the noise level and partially due to fears that what they say might get them in trouble. Microsoft would rather people actually interact on their service rather than form tiny cliques; a working and less exploitable reputation system will help.”

Dotson went on to describe why “less exploitable” is important.

“Reputation systems come with an inherent problem of being exploited,” he added. “We’ve seen rep systems such as those in The Sims Online used by users in a sort of extortion ‘mafia’ threatening new players out of house and home if they didn’t give into essentially blackmail. Otherwise good players on Xbox Live would find themselves reported for ‘bad sportsmanship’ or ‘bad language’ without ever even having a microphone–mostly by sore losers.”

To this end, Microsoft’s reputation engine will need to take into account if another player reports one immediately after losing a game to them (especially if a user does this often it should lower the impact of their reporting.) Players have even been caught exiting games against players with higher stats to then just report them. The reputation algorithm will need to be adjusted to catch and weight this sort of exploitative behavior and weaken its impact.

In the end, the rise of machine-aided moderation of communities will give an interesting insight into how to best cultivate an online gamer community and Xbox Live is a good experimental ground.