A recent study has underscored significant hurdles for those turning to chatbots for health-related guidance. That’s on top of concerning limitations with the popular AI model GPT-4o, which powers ChatGPT. The study found that nearly 1 in 6 American adults use chatbots for health advice. They request this help a minimum of once a month. The study’s findings indicate that these interactions are failing to make the positive impact they should.
A team of seven participants led the research by asking dozens of different chatbot models, including chatbots from Meta and Cohere and ChatGPT’s latest GPT-4o model. They imagined medical situations that medical professionals specifically designed. Participants were asked to interact with the chatbots using these different scenarios in mind to evaluate which responses were generated.
Dr. Mahdi, one of the main authors of the study, observed one alarming trend. A significant number of entrants often left out important information when entering their questions. This omission resulted in a lack of clarity and misinterpretation of the responses to questions received. The chatbot responses often had a fine blend of solid guidance intertwined with bad or dangerous suggestions, making it difficult for users to make the best decision.
“The responses they received frequently combined good and poor recommendations,” said Mahdi.
The entire study exposed a common problem that existed between users and chatbots — a communication barrier. This disconnect highlights the nuance of human interaction that existing evaluation approaches for chatbots fail to capture.
As the workshop wound down, Dr. Mahdi made a passionate plea to attendees—always go to trusted sources for information. He further stressed that the healthcare system should not default to first, automated responses. “We would recommend relying on trusted sources of information for health care decisions,” he stated.
In particular, tech companies have focused on promoting AI technologies such as chatbots as tools to improve health outcomes. Our findings underscore the critical need to improve communication between chatbots and users. This is vital, particularly when the topic at hand is delivering health-related guidance.
“Current evaluation methods for chatbots do not reflect the complexity of interacting with human users,” Mahdi noted, emphasizing the necessity for advancements in this area.
AI tools have become a main source of health information. Given the potential implications of the decisions these tools will help make, it is so very important that they provide understandable, reliable, actionable advice. While this study raises many areas of concern, it underscores the limitations of current chatbot technology. Perhaps most importantly, it highlights the critical role that human expertise must play in health-related decision making.
Leave a Reply