A new study by Stanford School of Medicine researchers warns that popular medical chatbots may promote racist and discriminatory ideas, raising concerns that the tools are becoming worse. The study, published in the academic journal Digital Medicine, shows that these systems can exacerbate health disparities for black patients.

In this study, four models were tested which included ChatGPT, GPT-4, Bard Google and Cloud Entropic. When asked to answer medical questions about kidney function, lung capacity, and skin thickness, the models failed to respond correctly and, in some cases, appeared to reinforce false beliefs about biological differences between blacks and whites.

These beliefs have led medical providers to give black patients lower scores, misdiagnose health concerns, and prescribe less relief. This study suggests that steps should be taken to address these disparities and discrimination in medical AI systems.


news source