AI has seen a renaissance over the last year, with developments in driverless vehicle technology, voice recognition, and the mastery of the game “Go,” revealing how much machines are capable of.
But with all of the successes of AI, it’s also important to pay attention to when, and how, it can go wrong, in order to prevent future errors. A recent paper by Roman Yampolskiy, director of the Cybersecurity Lab at the University of Louisville, outlines a history of AI failures which are “directly related to the mistakes produced by the intelligence such systems are designed to exhibit.” According to Yampolskiy, these types of failures can be attributed to mistakes during the learning phase or mistakes in the performance phase of the AI system.
Here is TechRepublic’s top 10 AI failures from 2016, drawn from Yampolskiy’s list as well as from the input of several other AI experts.
The company Northpointe built an AI system designed to predict the chances of an alleged offender to commit a crime again. The algorithm, called “Minority Report-esque” by Gawker (a reference to the dystopian short story and movie based on the work by Philip K. Dick), was accused of engaging in racial bias, as black offenders were more likely to be marked as at a higher risk of committing a future crime than those of other races. Another media outlet, ProPublica, found that Northpointe’s software wasn’t an “effective predictor in general, regardless of race.”
In June, an AI-fueled video game called Elite: Dangerous exhibited something the creators never intended: The AI had the ability to create superweapons that were beyond the scope of the game’s design. According to one gaming website, “[p]layers would be pulled into fights against ships armed with ridiculous weapons that would cut them to pieces.” The weapons were later pulled from the game’s developers.
A so-called “crime fighting robot,” created by the platform Knightscope, crashed into a child in a Silicon Valley mall in July, injuring the 16-month-old boy. The Los Angeles Times quoted the the company as saying that incident was a ” freakish accident.”
As previously reported by TechRepublic, Joshua Brown was driving a Tesla engaged in Autopilot mode when his vehicle collided with a tractor-trailer on a Florida highway, in the first-reported fatality of the feature. Since the accident, Telsa has announced major upgrades to its Autopilot software, which Elon Musk claimed would have prevented that collision. There have been other fatalities linked to Autopilot, including one in China, although none can be directly tied to a failure of the AI system.
In an attempt to form relationships with younger customers, Microsoft launched an AI-powered chatbot called “Tay.ai” on Twitter last spring. “Tay,” modeled around a teenage girl, morphed into, well, a ” Hitler-loving, feminist-bashing troll”—within just a day of her debut online. Microsoft yanked Tay off the social media platform and announced it planned to make “adjustments” to its algorithm.
In “The First International Beauty Contest Judged by Artificial Intelligence,” a robot panel judged faces, based on “algorithms that can accurately evaluate the criteria linked to perception of human beauty and health,” according to the contest’s site. But by failing to supply the AI with a diverse training set, the contest winners were all white. As Yampolskiy said, “Beauty is in the pattern recognizer.”
After the release of the massively popular Pokémon Go in July, several users noted that there were fewer Pokémon locations in primarily black neighborhoods. According to Anu Tewary, chief data officer for Mint at Intuit, it’s because the creators of the algorithms failed to provide a diverse training set, and didn’t spend time in these neighborhoods.
In March 2016, Google’s AI, AlphaGo, was beaten in game four of a five-round series of the game Go by Lee Sedol, a 18-time world champion of the game. And though the AI program won the series, Sedol’s win proved AI’s algorithms aren’t flawless yet.
“Lee Sedol found a weakness, it seems, in Monte Carlo tree search,” said Toby Walsh, professor of AI at the University of New South Wales. But while this can be considered a failure of AI, Yampolskiy also makes the point that the loss “could be considered by some to be within normal operations specs.”
Two researchers at China’s Shanghai Jiao Tong University published a study entitled “Automated Inference on Criminality using Face Images.” According to the Mirror, they “fed the faces of 1,856 people (half of which were convicted violent criminals) into a computer and set about analysing them.” In the work, the researchers concluded that there are “some discriminating structural features for predicting criminality, such as lip curvature, eye inner corner distance, and the so-called nose-mouth angle.” Many in the field questioned the results and the report’s ethics underpinnings.