Discrimination and Bias

The word bias is often used to refer to prejudice against a group of people. While this differs from the statistical bias which has been the focus of this chapter, there is significant overlap between these types of bias. A model may act in a biased way against a group of people, which can lead to discriminatory outcomes.

Data science and AI will always be at risk of bias - biased data leads to biased models, which may lead to biased decision-making. Datasets contain biases which reflect the personal and societal biases which were present during data collection. They also become biased due to difficulties in sampling.

Examples of discrimination in AI include racism in predictive policing, sexism in recruiting, and multiple forms of discrimination in generative AI. These algorithms form negative feedback loops, with historic discrimination contributing to discrimination in future decision-making. There is no one-size fits all solution to ensure fairness in AI, though guidance is available through the Information Commissioner’s Office (ICO) and the Alan Turing Institute.

Further information can also be found through the Turing-Roche Knowledge Share Event: Fairness in AI for Health.