Skip to content

Bias in AI: Does your AI really work?

Artificial Intelligence (AI) can be depicted as computer systems’ ability to perform tasks that were previously done by humans. 1These systems receive data from the environment and take actions that affect that data by generating outputs such as content, predictions, recommendations, classifications, or decisions influencing the environments they interact.2

While AI has the outstanding potential as a transformative technology, it also presents inherent risks that could present it from being so and for AI Systems not to be trustworthy. 3

Trust is cultivated via presence of the following characteristics in these systems: accuracy, explainability and interpretability, privacy, reliability, robustness, safety, and security resilience. Therefore, risks such as harmful Biases in AI have to be mitigated or controlled.4

Figure 1

This article explains what Artificial Intelligence Bias is, the reasons why it could occur and the consequences of such occurrence.

Fig 2

AI bias also referred to as machine learning bias depicts a situation whereby the AI system produces distorted results and false statistical representation. This sustains human biases and leads to wrong decision making. 5 Furthermore, Bias can be described as the degree to which a reference value deviates from the truth.

AI Bias can be classified into three categories which are:

(I) Systemic Bias; (II) Human Bias; (III)Statistical / Computational Bias 6

(I) Systematic Bias

Over the years, particular institutions have operated in ways that result into certain groups being advantaged or favored and others disadvantaged or devalued.

These norms and rules of operations have been passed down over years and therefore reflect in the institutions’ historical datasets.

These datasets are then used in training AI Algorithms/Models and the institutional norms, practices and processes embedded across the AI lifecycle, resulting in AI Bias.

(II) Statistical and Computational Bias

Statistical and computational biases occur when there are errors that result in the data sample not a representative of real-life population.

These biases are present in the selected datasets and algorithms used in the development of the AI applications. They occur when the algorithms are trained on a particular dataset but when then used on other datasets, produces misleading results.

This could be due to Overfitting and underfitting the data: Overfitting occurs when a model is trained with a lot of data that it learns excessive data and inaccurate data entries in the used dataset. Underfitting occurs when a model fails to find patterns within the dataset because the dataset is too small.

Other factors leading to this type of bias include data cleaning and imputation factors.

(III) Human Bias

Human biases reflect systematic human errors in decision making due to the fact that humans are limited by time they have and their mental ability to understand information such as automated AI lifecycle.

These biases are present in individuals, group, and institutional decision-making processes across the AI lifecycle and in the use of AI applications deployed.

Examples of Real-Life AI Bias

Bias via AI Predictive policing tools – AI Predictive policing tools were found to be biased due to fact that historical police data used in training the algorithms within these tools was biased. The historical police data was biased due to the following:

Based on the Police Arrest Data, police were known to arrest more people in black and minority communities.

There were communities whereby even though high number of crimes exist, only a few of them were reported. Meanwhile there were communities with a smaller number of crimes but a higher rate of reporting those crimes.

Based on the above, it was found that these tools lead to biased outcomes that do not improve public safety.

Example of such is the misallocation of police patrols with neighborhoods being unfairly designated as hotspots while others are under policed.7

Bias via AI Image Generators – Based on a 6-month research study of an AI Image generator called Midjourney, which involved analyzing 100 AI-generated images outputted by this tool over this period, some of the findings were:

(i) For non-specialized job titles, images of only younger men and women were displayed.

(ii) For specialized roles, images of both younger and older persons were displayed but the older persons were only men.

(ii) There was difference in displayed images of women and men. Women were displayed as younger and wrinkle-free while images of men included those of men with wriggles.

(iii) Technology was represented by images from past era such as typewriters, printing press and oversized vintage cameras rather than images depicting digital technology.

(iv) Outputted images were found to be conservative in appearance with none having tattoos, piercing, unconventional hairstyles or attributes that might distinguish them from conservative mainstream descriptions.

(v)All the images returned for search words such as “journalist”, “reporter” or “correspondent” were only those of light-skinned people. 

It was concluded that the reasons for the above biases are:

The inputted data into the AI Image Generators influenced the outputs. The inputted data used in training the algorithm within the Image generator was not broad enough, thereby lacking diversity and representation of the true real-life population.

Also, another contributing factor is how the underlying algorithm was designed whereby it had a default tendency to return certain kinds of outputs. 8

Bias in AI Hiring System – In 2015, it was found that the Artificial Intelligence (AI) algorithm used for hiring employees was biased against women. Further investigation showed that this was because the development of the algorithm was based on the number of resumes submitted over the past 10 years.

Over the past years, these resumes were sent mainly by men and therefore the algorithm was trained to have a preference for men over women.9


References

  1. Maria Diaz; ‘What is AI?, Everything to know about artificial intelligence’, ZDNET, April 21 2003, https://www.zdnet.com/article/what-is-ai-heres-everything-you-need-to-know-about-artificial-intelligence/ ↩︎
  2. R Schwartz;A Vassilev;K Greene; L Perine; A Burt;P Hall;
    “Towards a Standard for Identifying and Managing Bias in Artificial Intelligence”, NIST Special Publication 1270, March 2022, https://doi.org/10.6028/NIST.SP.1270
    ↩︎
  3. (2) ↩︎
  4. (2) ↩︎
  5. (4) ↩︎
  6. (4) ↩︎
  7. Will Douglas Heaven,’Predictive policing is still racist—whatever data it uses’, MIT Technology Review, February 5, 2021, https://www.technologyreview.com/subscribe?itm_source=nav-button&itm_medium=onsite&itm_campaign=subscribe-BAU ↩︎
  8. Chris Calimlim;’Ageism, sexism, classism and more: 7 examples of bias in AI-generated images’, THE CONVERSATION, July 9 2023,https://theconversation.com/ageism-sexism-classism-and-more-7-examples-of-bias-in-ai-generated-images-208748#:~:text=There%20were%20also%20notable%20differences,of%20more%20fluid%20gender%20expression. ↩︎
  9. “Real-life Examples of Discriminating Artificial Intelligence”, Datatron,
    https://datatron.com/real-life-examples-of-discriminating-artificial-intelligence/
    ↩︎