Erik Brynjolfsson, director of the Stanford Digital Economy Lab, has released the results of a study examining the comparative intelligence of humans and artificial intelligence in predictive environments. The research, which involved adults recruited from the San Francisco Bay Area, tasked participants with forecasting the outcomes of real-world events within a 60-minute timeframe. The study aimed to test the common assumption that AI will handle routine data processing while humans provide superior judgment and creativity.
The experiment utilized scenarios from Polymarket, a decentralized prediction market platform. Polymarket serves as a benchmark for accuracy because its forecasts are driven by the collective wisdom of thousands of participants who have financial incentives to be correct. The platform currently maintains a data partnership with Dow Jones, the parent company of The Wall Street Journal. This partnership allows for a rigorous comparison between individual or group predictions and the broader market consensus.
According to the study findings, human-only teams performed the most poorly among all tested groups. These participants frequently relied on personal intuition or the most recent information available on their social media feeds rather than conducting systematic analysis. Their predictions consistently fell short of the benchmarks set by both the AI models and the prediction market.
The AI models tested in the study included OpenAI’s ChatGPT and Google’s Gemini. These large language models performed significantly better than the human-only teams. However, their accuracy still did not reach the levels established by the Polymarket platform. While the AI was capable of synthesizing information more rapidly than the human subjects, it could not fully replicate the predictive power of the incentivized market collective.
The research also analyzed the performance of human-AI hybrid teams, where individuals were given access to AI tools to assist in their forecasting. The results showed that these teams generally failed to improve upon the AI’s solo performance. In most instances, the human participants did not use the machine to augment their own judgment; instead, they deferred to the AI’s output and submitted it as their own. This behavior resulted in hybrid performance metrics that were nearly identical to those of the AI models working independently.
Brynjolfsson, who has worked in the field of artificial intelligence for 30 years, noted that the findings suggest a fundamental misunderstanding of how humans and machines interact. The study concludes that the mere presence of AI does not automatically enhance human decision-making, as the tendency to rely entirely on the machine’s first draft can negate the potential benefits of human oversight and creativity.