Amazon.com Inc.’s controversial face recognition technology, “Rekognition,” heralds to some a dystopian future of omnipresent surveillance, and recent tests of the product have made skeptics even more worried.

Those tests were conducted by the American Civil Liberties Union of Northern California. Some 25,000 mugshots of people who have been arrested for a crime were used, and then 535 members of Congress were run through the software to see if there were any matches. It turned out that 28 lawmakers were supposedly dead ringers for criminals.

“The false matches were disproportionately of people of color, including six members of the Congressional Black Caucus, among them civil rights legend Rep. John Lewis (D-Ga.),” said ACLU. “These results demonstrate why Congress should join the ACLU in calling for a moratorium on law enforcement use of face surveillance.” ACLU added that 40 percent of the false matches were people of color, while they only make up 20 percent of Congress.

Face recognition software has shown false positives for African-Americans in the past, which is one of the reasons it’s so controversial. Amazon has come under tremendous pressure for selling Rekognition to police departments in the U.S., resulting in widespread petitions and voiced concerns from company shareholders.

The tests, said ACLU, cost $12.33 – “less than a large pizza” – and yet these quick tests “could cost people their freedom or even their lives.” ACLU concluded, “Congress must take these threats seriously, hit the brakes, and enact a moratorium on law enforcement use of face recognition.”

Amazon responded to the news, telling The Verge that those tests used an 80 percent default level of confidence. For matters of law, Amazon recommends that the software is set to 95 percent.

“While 80 percent confidence is an acceptable threshold for photos of hot dogs, chairs, animals or other social media use cases,” the representative told The Verge, “it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty.”

Doug Aley, chief executive of Ever AI, which is also a face recognition platform, told SiliconANGLE that such software is a long way from being totally accurate – a matter of profound importance when police departments are using it.

“Many companies in the field of face recognition still suffer from a lack of diversity in the datasets they use, leading to very real bias concern,” said Aley. “Any technology used by the police can be abused.”

The system is not foolproof, he said, saying that anyone identified by such technology should be presumed innocent and human intervention is a must. “Regulation and training is needed for sure, but we also need to understand that having these tools will indeed save lives in the long run,” said Aley.

Three members of Congress, Sen. Markey (D-Mass.), Rep. Gutiérrez (D-Ill.) and Rep. DeSaulnier (D-Calif.) – all of whom were matched with the mugshots – have already written an open letter to Amazon Chief Executive Jeff Bezos.

The letter expresses concern regarding the accuracy of the technology and asks Amazon to share results of internal Rekognition tests, including information of possible racial bias. The lawmakers also want to know which law enforcement agencies are now using the software and if Amazon has any scruples about selling it to agencies with a history of “unlawful or discriminatory policing practices.”

Image: ACLU