Fair treatment with AI

Today, artificial intelligence (AI) and machine learning are increasingly being used in our healthcare system. Doctors and radiologists use, for example, algorithms to support the decisions they make when diagnosing the patient. The only problem is that algorithms can be just as biased and prejudiced as people because they are based on data from previous observations. For example, if an algorithm has seen more examples of lung diseases in men than in women, it will be better trained to detect lung diseases in men.

Health data often contains a bias, a misrepresentation of different demographic population groups that ends up influencing the decisions that a health algorithm is able to make. This can lead to underdiagnosis of some population groups because there is a predominance of data from a specific group.

“If the algorithm is trained by data that reflects social prejudices and stereotypes, there will also be imbalances or biases in the artificial intelligence that reproduces the data—and that’s not necessarily fair,” says Aasa Feragen, who is professor at DTU Compute, and continues:

"The idea that artificial intelligence will make important decisions about my state of health is frightening, considering that AI can be just as discriminatory as the most diehard racists or sexists,” she says.

How to ensure fair treatment?

Aasa Feragen is the manager of a research project which will investigate bias and fairness in artificial intelligence in medical applications over the next three years. The aim is to develop fair algorithms to help provide fair treatment to everyone in the public health service. Researchers from the University of Copenhagen, Rigshospitalet, and the Swiss university for science and technology ETH participate in the project.

In the project, the researchers will, for example, analyse the demographic data of all Danes diagnosed with depression in recent years. The researchers will test a hypothesis that the data contains imbalances—for example in how often Danes are diagnosed and the type of treatment they receive, based on gender, age, geography, and income. The researchers will then examine—together with ethicist and Associate Professor Sune Hannibal Holm from UCPH—how to develop a fair health algorithm.

The issues to be discussed include questions like: ‘When is a decision discriminatory?’, ‘Can you develop methods for detection and removal of discriminatory biases in algorithms before they are taken into use?', and ‘How can you define which decisions are fair in medicine?’

“One could argue that because the biased predictions of the AI algorithm are simply a result of the biased decisions being made in our current healthcare system, the use of AI in the public health service will not cause new problems,” says Aasa Feragen.

But—according to Aasa Feragen—it is about exploiting that AI has the potential to detect these biases before the tool has made a single decision. This will make it possible to take the best elements from AI and find new ways to minimize bias through transparent and secure solutions.

The research project ‘Bias and fairness in medicine’ receives funding from Independent Research Fund Denmark.