Google Revealed That Best AI Chatbots Are Barely 69% Accurate
That's a bit depressing.
ioda, Shutterstock
AI chatbots' popularity has obvious reasons: why waste time on research when ChatGPT or Gemini can do the hard work for you? However, by now we are all aware of AI hallucinations, which create facts that are simply not true, and the situation is even worse than you might think.
Google created the FACTS Benchmark Suite to test how factually accurate chatbots are, and the results are underwhelming, to say the least.
The suite contains 4 parameters:
- Multimodal, which measures the factuality of responses to image-based questions;
- Parametric, which assesses models' world knowledge by answering closed-book factoid questions from internal parameters;
- Search, which evaluates factuality in information-seeking scenarios, where the model must use a search API;
- Grounding, which evaluates whether long-form responses are grounded in the provided documents.
Based on Google's research, the most "correct" chatbot overall is Gemini 3 Pro, but even it shows only 68.8% accuracy, which is substantially lower than one expects from a know-it-all system.
It is followed by Gemini 2.5 Pro (62.1%) and GPT 5 (61.8%). The least accurate model is Grok 4 Fast, with only 36 points in the FACTS Leaderboard. Not surprisingly, Multimodal tasks are the hardest for AI to deal with.
This research shows that you shouldn't blindly trust chatbots to give you correct information: we are still far away from AI having all the answers. The FACTS Leaderboard is a useful tool to measure chatbots' worth, however.
Don't forget to subscribe to our Newsletter and join our 80 Level Talent platform, follow us on Twitter, LinkedIn, Telegram, and Instagram, where we share breakdowns, the latest news, awesome artworks, and more.
