In recent debates surrounding artificial intelligence, a contentious issue has emerged: the use of IQ tests as benchmarks for AI capabilities. Human brains grapple with a myriad of complexities when solving problems, whereas AI systems receive significant assistance. This contrast raises questions about the appropriateness of using IQ tests, originally designed for humans, to evaluate AI. These tests, which carry cultural biases and historical links to eugenics, do not comprehensively measure human intelligence. As AI's "IQ" reportedly advances rapidly, experts urge a reevaluation of how AI is assessed.
Humans face numerous variables in problem-solving, unlike AI, which often benefits from clear parameters and guidance. IQ tests aim to assess general problem-solving abilities in humans, but their applicability to AI is questionable. AI systems do not learn in the manner humans do; they experience no noise or signal loss, processing information with perfect clarity. As a result, AI's strong performance on IQ tests highlights flaws in the tests rather than AI's true capabilities.
"Very roughly, it feels to me like — this is not scientifically accurate, this is just a vibe or spiritual answer — every year we move one standard deviation of IQ." – Sam Altman
IQ tests were originally linked to eugenics, a discredited theory, and are biased towards Western cultural norms. They require a strong working memory to excel, which does not account for the complexity of human intelligence. The idea of using IQ as a benchmark for AI is a new and contested concept.
“It can be very tempting to use the same measures we use for humans to describe capabilities or progress, but this is like comparing apples with oranges.” – Sandra Wachter
The use of IQ tests for AI evaluation is part of a broader trend where AI systems are judged by benchmarks that are themselves debated. The direct comparison of AI and human intelligence remains controversial, as Heidy Khlaaf notes.
“This idea that we directly compare systems’ performance against human abilities is a recent phenomenon that is highly contested, and what surrounds the controversy of the ever-expanding — and moving — benchmarks being created to evaluate AI systems.” – Khlaaf
Human intelligence encompasses far more than what IQ tests can measure. Sandra Wachter emphasizes that these tests are merely tools for measuring human capabilities, based on scientific assumptions about intelligence.
“IQ is a tool to measure human capabilities — a contested one no less — based on what scientists believe human intelligence looks like.” – Sandra Wachter
Experts argue that the complexity of human intelligence cannot be equated with AI's performance on specific tasks. Heidy Khlaaf draws parallels to vehicles and submarines outperforming humans in speed and diving, respectively, without implying superior intelligence.
“But you can’t use the same measure to describe AI capabilities. A car is faster than humans, and a submarine is better at diving. But this doesn’t mean cars or submarines surpass human intelligence. You’re equivocating one aspect of performance with human intelligence, which is much more complex.” – Heidy Khlaaf
The tendency to compare AI models' performance with human abilities can be misleading. Mike Cook underscores how repeated patterns in tests can inflate perceived intelligence.
“Tests tend to repeat very similar patterns — a pretty foolproof way to raise your IQ is to practice taking IQ tests, which is essentially what every [model] has done.” – Mike Cook
The debate extends beyond mere testing to encompass broader implications for understanding both AI and human intelligence. As AI's capabilities grow, stakeholders must develop better methods to evaluate these systems beyond traditional human metrics.
Leave a Reply