According to new research, deep learning models based on artificial intelligence can identify someone's race merely by looking at their X-rays, which would be impossible for a human doctor looking at the same photos.
An international team of health researchers from the U.S., Canada, and Taiwan tested their AI on X-ray images that the computer program had never seen before after training it with hundreds of thousands of existing X-ray images annotated with specifics of the patient's race (and had no additional information about).
The AI could predict the reported racial identity of the patient on these images with surprising accuracy, even when the scans were taken from people of the same age and the same sex. The system hit levels of 90 percent with some groups of images.
"We aimed to conduct a comprehensive evaluation of the ability of AI to recognize a patient's racial identity from medical images," the researchers wrote in their published paper.
The findings raise some unsettling questions regarding the role of AI in medical diagnosis, assessment, and treatment: may computer software inadvertently apply racial bias when viewing images like these?
The findings echo those of a previous study, which discovered that artificial intelligence scans of X-ray pictures were more likely to miss indicators of sickness in Black persons. To prevent this from happening again, scientists must first understand why it is happening in the first place.
AI, by definition, mimics human reasoning in order to detect patterns in data. However, this also implies that it is susceptible to the same kind of biases. Worse, their complexities make it difficult to unravel the preconceptions we've weaved into them.
Scientists are now unsure why the AI system is so good at identifying race from photographs that don't appear to contain such information. Even with minimal information, such as omitting hints about bone density or focusing on a tiny portion of the body, the models were very good at predicting the race represented in the file.
It's likely that the system is detecting melanin, the pigment that gives skin its color, in ways that science has yet to discover.
There are many unsolved questions from the study, but for the time being, it's crucial to be conscious of the possibility of racial bias in artificial intelligence systems - especially if we're going to give them more responsibility in the future.
