The problem of Bias in Artificial Intelligence.

I'm about to show the world how bad the homework I just submitted looks like, and older me is probably going to Mickey-Mouse-puncture his eyes when he reads this. But here goes nothing.

“Algorithms are everywhere. They sort and separate the winners from the losers,” says Cathy O’Neil, a mathematician, Data Scientist, and author of New York Times Best-Seller (of course) Weapons of Math Destruction in her TED talk, ‘The era of faith in big data must end.’ She posits that this Harry Potter-esque sorting of real people into different socio-economic cohorts is not objective by construction, but is instead the algorithm’s creators’ “opinion embedded in code.”

As many experts point out, the biggest ramifications of how algorithms affect the real world stem from the data used to train the models. Data come from historical records by definition, and thus, predicting the future using data explicitly depends on past trends. This is the core of using data science for machine learning. However, in a massive data frame with possibly millions or billions of data points, the algorithm’s creators get to decide the variables that define success as well as the factors that contribute to those results. This effectively allows a handful of people to dictate social, cultural, and even financial practices on an unprecedented scale. On top of that, models trained with data from and allowed to interact with humans in social contexts show that AI starts to mirror the cracks in our social interactions. A good example of this is language processing systems. It has been shown that some implicit biases regarding race and gender in human psychology are readily acquired by algorithms which use mathematical modeling to compute the contextual meaning of a word. One article argues that “This purely statistical approach appears to capture the rich cultural and social context of what a word means in a way that a dictionary definition would be incapable of.” It seems like even without malicious intent by the developers, the datasets themselves carry potential for future destruction.

Demonstrably, the data used in modeling seem to carry the most potential for socio-economic and political destruction. Here we can look at facial recognition, perhaps one of the most concerning applications of artificial intelligence today. If the training data do not fully represent an entire demographic, the models built from it will be seriously, yet unsurprisingly defective. Take for example Jacky Alcine’s case in 2015, where Google Photos wrongly classified him as a gorilla, or Joy Boulamwini’s case where she had to put on a white mask to do her Computer Vision assignment because the camera could not recognize her face. It is easy to argue that these examples, and many others like them, represent a case of explicit racism, but instead an abundance of biases embedded in training datasets when you consider a study by the National Institute of Standards and Technology (NIST) which found that some algorithms developed in China perform better with Asian faces(pdf).

The highlighted drawbacks become particularly troubling when you start to apply faulty models in very serious, even life-changing practices like law enforcement and the justice system. Not too long ago (as of May 2021), Equivant, an AI company committed to advancing justice came under fire for the application of its tool, Correctional Offender Management Profiling for Alternative Sanctions (COMPAS) in law enforcement. The algorithm supposedly predicts an offender’s propensity to commit another crime based on a series of 137 questions. Two researchers at Dartmouth showed that the model is no better at predicting recidivism than any group of random unqualified people with just a small amount of information about the offender and challenged its employment in courts in the first place. To top it off, ProPublica assessed a group of 7000 arrestees and claimed that COMPAS is biased against African Americans. Of course, Equivant strongly challenged this finding by arguing that the model predicted recidivism in both Black and White Americans at the same rate. I would argue that behind this Math jargon lies a single conclusion that holds true for both claims, which The Washington Post supports. The post notes that, “If the recidivism rate for white and black defendants is the same within each risk category, and if black defendants have a higher overall recidivism rate, then a greater share of black defendants will be classified as high risk,” and that “If [Equivant’s] definition of fairness holds, and if the recidivism rate for black defendants is higher than for whites, the imbalance ProPublica highlighted will always occur.”

We could look at another case, a specific example of natural language processing systems we call chatbots. As we highlighted earlier, these use sophisticated algorithms to formulate mathematical models based on millions of examples usually collected from public data sources on the internet. A good example of this would be Microsoft’s Tay, a Twitter bot that learned to be shockingly insolent after a very short time on the internet. While this might be problematic, Urwa Muaz notes that a moderated version of a wild chatbot could have just as bad social outcomes because of what constitutes inclusivity with regards to politics, religion, and race. He claims that because of how moderation is done, we can end up with problems of equity, inclusion, and denial of services if chatbots are deployed to provide services in urban environments.

In the grand scheme of things, it is not the machine’s fault.

Joanna Bryson in an interview by The Guardian argues that “we’re prejudiced, and AI is learning it,” which I will concatenate with Urwa Muaz’s claim that using public data without check teaches AI to incorporate the worst traits in humanity and end up embodying prejudices in society. The problem is, however, while humans have the capacity to consciously counteract learned bias, machines do not. And since they have such a wide application on almost all facets of our daily lives where we interact with digital technology, and increasingly even when we do not (like walking down the street), they can reinforce their learned biases.

However, there seems to be light all over the tunnel, except for where we are standing. A study by the Center for Strategic and International Studies (CSIS) concluded that improvements in facial recognition technology, especially in how algorithms are developed and trained will continue to reduce the risk of error and bias. They note that like all new technologies, continued improvement reduces risk. Google was able to react quickly to fix the facial recognition problem, and a lot of work is being done to correct the errors presented by AI across the board. Many experts also encourage installing rules and safeguards appropriate for any model’s application. One way of doing this is to advocate for emphasis on the importance of national privacy legislation as a foundation for protections in the use of facial recognition.

On the overall, it seems that using better data to build better models is the biggest step in fixing the problem. This is indubitably a challenging task, but since AI is an unstoppable progression towards the future, I think reacting to concerns as quickly as they present themselves is the right way to go.