Facebook Inc.-owned Instagram unveiled on Thursday its new method to weed out toxic content and spam: an artificial intelligence system called DeepText.

The improved content filter goes a step further than the tools Instagram had previously used to block certain content. Before, users could either turn on a filter that would block content containing pre-selected words or phrases, or they could choose specific words or phrases they didn’t want to see.

Instagram wrote in blog post that it has been training its system to “recognize certain types of offensive and spammy comments” but didn’t get into the details of how the system actually works. Wired added some pieces to the puzzle of what DeepText does.

The problem with algorithmic censorship is its prone to condemning innocent content, because it isn’t able to recognize context. This results in a lot of false positives and upset innocent scribes. DeepText reportedly is trained to understand the context of a phrase using a concept called “word embeddings,” which like the human brain has the ability to understand a word from those that surround it. “I hate fags” for instance, is perfectly fine in the U.K. as it usually means someone doesn’t like cigarettes. In the U.S., it’s hate speech. Programming an algorithm to understand this is a very steep hill to climb.

The filter is turned on by default, but users can turn it off in their settings. If a comment does get flagged, no one will be able to see it except the person that wrote it, and apparently that person won’t be informed his or her comment has been blocked.

The system was first tested and used to block spam, which started last October. It was so successful at eliminating spam, in multiple languages, that Instagram turned its focus toward abusive language. DeepText was trained by a team who went through millions of comments, sorting them into categories labelled under such things as “bullying” or “racism,” and then tweaked and tested the algorithm until it was moderating fairly.

Unlike the spam buster, it only works in English at the moment. According to Instagram, the system will occasionally get it wrong. “The reality is it’s going to happen,” an Instagram representative told Wired, “so the question is: Is that margin of error worth it for all the really bad stuff that’s blocked?”

Image: Jens karlsson via Flickr