A new study by Giskard uncovers a key finding that illustrates this point—when you ask chatbots for short answers, inaccurate information is more likely to occur. This phenomenon is commonly known as hallucinations. TechCrunch’s AI Editor Kyle Wiggers explains why this happens—because the inherent probabilistic nature of language models means they get things wrong sometimes. These models do not output responses that they know are true, false or even misleading.
The Giskard research reaffirms this in highlighting how users frequently prompt chatbots for clear and direct responses. These models often still fail to return context, allowing unchecked false premises and factual inaccuracies to occur. With that in mind, these models are usually trained to favor concise responses over correctness, which in turn causes hallucinations to be more frequent.
“When forced to keep it short, models consistently choose brevity over accuracy.” – Giskard researchers
As Giskard’s research highlights, there are more fascinating revelations about how users interact with AI models. What the research found may be surprising. Once users can boldly go making dubious assertions, models stop debunking them. This behavior raises concerns over the reliability of chatbots. During crises like natural disasters, outbreaks, or conflicts, misinformation can contribute to loss of life.
The conclusions indicate that users’ choices are statistically biased toward models considered more reliable. This preference doesn’t often align with the accuracy of the information generated by such models. The crux of the issue is that language models, no matter how powerful, will always generate false information because of their underlying architecture and functionality.
“Optimization for user experience can sometimes come at the expense of factual accuracy.” – Giskard researchers
Researchers at Giskard believe all LMs are inherently probabilistic. As a result, all but the most sophisticated systems are prone to generating false information. We hope the study will bring attention to a major flaw in AI technology as it exists today. It uncovers that hallucination is an intrinsic property of language models.
Kyle Wiggers, who lives in Manhattan with his partner, a fellow music therapist, is passionate about raising awareness of these limitations. Chatbots are quickly becoming a standard feature of our favorite consumer applications. Users must be ever-vigilant for misinterpretation, particularly when information is distilled down to simple statements.
Leave a Reply