Amazon's facial recognition AI wrongly identifies 28 politicians as criminals
What's the story
Confirming long-standing fears about potential loopholes, the American Civil Liberties Union (ACLU) conducted a study using Amazon's facial recognition technology, and found that it incorrectly matched people to faces. It even seemed to have a racial bias.
In the study, Amazon Rekognition misidentified 28 of the 435 members of Congress as criminals, thereby raising serious questions about the technology's adoption in surveillance.
Study
Details of the ACLU study involving Rekognition
For the study, the ACLU built a face database and search tool using 25,000 publicly available arrest photos, and searched that database with publicly available photos of all serving Congressmen, using Rekognition's default setting of 80% confidence.
It was found that 28 were wrongly matched to criminals and, among them, 39% happened to be people of color, while only 5% were white.
Amazon's response
Amazon cites calibration issues as a reason for the mismatch
Amazon responded to the ACLU study saying that the errors had happened owing to faulty calibration of Rekognition.
The e-commerce giant recommends an error-tolerance (confidence) setting of 95% for use of Rekognition in law enforcement, but ACLU used only 80% (meaning, the system was 80% sure of the matches it made).
However, this minor caveat doesn't exempt Amazon from the larger questions surrounding Rekognition.
No governance
There's no law governing the use of facial recognition tech
Notably, there's no law in the US governing facial recognition technology, and there's no law mandating that a 95% or 99% confidence setting be used for Rekognition in law enforcement.
Consequently, with Amazon marketing the product aggressively to government agencies and police across the US, the potential for misuse in government surveillance and law enforcement, especially along racial lines, remains massive.
Marketing
Several police departments are already using Rekognition
Amazon is already marketing Rekognition to the police and government agencies in the US, touting low costs, real-time tracking of people, image/video analysis capabilities, and the ability to identify over 100 faces from a single image.
Several police departments across the US have already started using Rekognition, without the issue ever going into public debate.
Considering the lack of legal barriers, this is alarming.
Risks
Widespread, unregulated use of Rekognition could result in racial violence
Law enforcement in the US doesn't really have the best track record when it comes to people of color and immigrants, and to this day, such people face disproportionate harm.
Considering that there's a margin of error when using Rekognition, the use of Amazon's technology in law enforcement, without fool-proofing and proper regulation, could cost people their freedom, and even their lives.
Opposition
Amazon continues marketing Rekognition, despite strong opposition
While this concern seems to have escaped Amazon, but its employees and investors are worried.
The marketing of Rekognition to government agencies and police has seen strong opposition from Amazon's employees and shareholders, and from 70 civil rights groups, over 400 members of the academic community, and more than 150,000 members of the public.
Will Amazon hit the brakes? We don't know.
Regulation
Lack of regulation could result in a perverse Orwellian future
All those who are opposed to Amazon marketing Rekognition to the government are hoping that the Congress takes the threat of potential misuse of the technology seriously, especially after the ACLU study.
If the Congress doesn't enact a moratorium on law enforcement's use of the tech before proper regulation comes up, the US could be facing at a perverse Orwellian future ahead.