Are Chatbots Actually Correct? The Complicated Question of AI Accuracy and AI-Generated Information
Artificial intelligence chatbots have quickly moved from technological novelty to everyday adviser. People ask them about finances, careers, technology, relationships, health concerns, business decisions and countless other subjects. In many cases, users are willing to disclose information to an AI system that they might hesitate to tell another person.
But increasing trust in artificial intelligence raises a much more fundamental question: Are chatbots actually correct?
The answer is more complicated than simply assigning AI a percentage.
A recent MIT Sloan School of Management study illustrates an important distinction between how trustworthy or comfortable an AI system feels and whether the information it provides is actually accurate. Researchers examining AI and human advisers found that people generally considered human advisers more competent. However, that preference could change dramatically when embarrassing information was involved.
Researchers Eric So of MIT Sloan, Abigail Sussman and Fiona Yang of the University of Chicago Booth School of Business examined how people decide between human and AI advisers in areas including financial, technological, medical and career advice.
Across three experiments, participants generally preferred human advisers when human and AI competence appeared comparable. When people had to reveal embarrassing information, however, their preferences shifted toward AI.
That finding tells us something important about the future of chatbots: People may trust and use AI for reasons that have little to do with whether AI is more accurate.
Comfort Is Not the Same as Accuracy
One experiment presented 965 participants with hypothetical situations such as having substantial credit card debt. The cause might be something beyond the person’s control, such as medical expenses, or something potentially embarrassing, such as frivolous spending.
Participants were then asked whether they would prefer explaining their situation to a human or an AI adviser.
Most preferred humans — until embarrassment became a factor.
Another experiment involving 744 participants manipulated both the level of embarrassment and the apparent competence of the advisers. When AI was portrayed as particularly capable, participants became more likely to choose it. High levels of embarrassment also pushed people toward AI.
Perhaps most revealing was an experiment in which participants described actual embarrassing financial or technology-related situations. People felt considerably more compelled to justify themselves when communicating with another person.
Participants offered justifications to human advisers 33% of the time, compared with only 15% when communicating with AI.
That could provide AI with an unexpected advantage. If people provide more complete and candid information to a chatbot, the system potentially has better information upon which to formulate its answer.
But that still doesn’t establish that its answer is correct.
So How Often Are Chatbots Correct?
There is no universal percentage that answers this question.
Chatbot accuracy depends upon the AI model, the question being asked, the information available to the model, whether current external sources are being consulted and how “correct” is being measured.
A chatbot answering a straightforward factual question represents a very different accuracy problem from one being asked to diagnose a complicated business problem, interpret medical symptoms, analyze an investment or predict what might happen next.
This distinction matters because conversational AI can produce extraordinarily convincing language.
An answer can be clear, detailed, grammatically flawless and logically structured — and still contain incorrect information.
Consequently, fluency should never be treated as proof of factual accuracy.
The MIT Sloan research reinforces this distinction indirectly. Participants’ choices involved perceived competence and fear of judgment rather than an independent measurement showing that the AI adviser actually produced more accurate advice.
That difference becomes increasingly important as people become more comfortable using AI for sensitive problems.
Where Does AI’s Information Come From?
There is another question that deserves considerably more attention: What happens when the internet itself becomes increasingly filled with AI-generated information?
Large AI systems are developed using enormous collections of information. The modern internet, meanwhile, increasingly contains material that has itself been generated or substantially assisted by artificial intelligence.
That creates an important information-quality problem.
Suppose an AI system produces an inaccurate statement. Someone copies that answer into a blog, social-media post, article or website. The information is then indexed, copied, summarized or repeated elsewhere.
The original AI mistake has now become internet content.
If future AI systems, search systems or retrieval tools encounter that material, distinguishing original human research from repeated machine-generated assertions can become increasingly important.
The MIT Sloan material provided here does not establish that a particular chatbot routinely trains on its own answers or give a percentage of training material that is AI-generated. Those would require separate evidence from AI developers and technical research.
Nevertheless, the distinction between original evidence and repeated information is crucial when evaluating chatbot answers.
Ten websites repeating the same statement do not necessarily constitute ten independent sources. They could ultimately trace back to one source — potentially even one erroneous source.
AI Can Be Useful Without Being Infallible
None of this means chatbots are inherently unreliable.
The MIT Sloan research points toward a genuine advantage of conversational AI: people may be more willing to disclose uncomfortable information when they do not feel they are being judged.
Professor Eric So describes the decision as a trade-off between competence and social judgment. AI could therefore encourage someone to seek advice who otherwise might avoid seeking assistance altogether.
That is significant.
An individual embarrassed about debt, a cybersecurity mistake or a career problem might explain the situation more openly to a machine. Better disclosure could help identify the actual cause of a problem.
The strongest system, however, may not require choosing exclusively between humans and machines.
AI can collect information, organize complicated problems, explain concepts and help users formulate questions. Humans can contribute experience, accountability, contextual judgment and continuing support.
Verification remains important in both cases.
Trust, but Verify
The central question surrounding chatbots therefore shouldn’t simply be, “Is AI correct?”
A better question is: How can I determine whether this particular AI answer is correct?
Users should distinguish between an AI-generated explanation and the underlying evidence supporting it. Important factual claims can be checked against primary documents, peer-reviewed research, government databases, corporate filings, original statistics or other authoritative sources.
The greater the consequences of an incorrect answer, the more important that verification becomes.
There is also an important lesson for AI companies. The MIT Sloan research found that users may disclose more sensitive information to machines precisely because they perceive them as less judgmental. That makes transparency concerning how personal information is stored, processed and potentially reviewed by humans especially important.
Artificial intelligence may become one of the most useful information tools ever developed. But usefulness and correctness are not synonymous.
Chatbots can help people ask questions they might otherwise be afraid to ask. They can make complicated information accessible and rapidly analyze enormous amounts of material.
What they cannot provide is an automatic guarantee that every confident-sounding statement is true.
As AI-generated material becomes a larger part of the digital information environment, understanding the difference between an answer, a source and verified evidence may become one of the most important information-literacy skills of the AI era.
Sources and Further Reading
- MIT Sloan School of Management — “Clients prefer AI to human advisers when the details are embarrassing”
This is the original August 31, 2026 article underlying the material you supplied. It covers Eric So, Abigail Sussman, and Fiona Yang’s research on perceived AI competence, embarrassment, disclosure, and why users sometimes prefer AI advisers.
Read the original MIT Sloan article - Original research paper — “AI Advisers and the Competence-Judgment Tradeoff in Information Disclosure”
This is the academic study discussed by MIT Sloan and is the better citation when you want to reference the actual experiments rather than the MIT news article.
Read the research paper on SSRN - Stanford Institute for Human-Centered AI — AI Index Report 2025
Particularly useful for your question, “How often are chatbots correct?” Stanford discusses factuality and hallucination benchmarks. One benchmark summarized in the report found hallucination rates ranging around 1.3%–2.9% among the listed models in a document-summarization test. That should not be interpreted as meaning chatbots are 97–99% accurate in general; accuracy varies substantially with the task and benchmark.
Read the Stanford AI Index Report - OpenAI — “Why language models hallucinate”
Useful background on why an AI can confidently produce a false answer. The research argues that conventional training and evaluation can reward guessing rather than acknowledging uncertainty.
Read Why Language Models Hallucinate - ACL — FactBench: A Dynamic Benchmark for In-the-Wild Language Model Factuality Evaluation
This 2025 research is especially relevant because it evaluates factuality using realistic prompts rather than assuming one accuracy number applies to every chatbot question. Researchers found that factual precision declined as questions became harder.
Read the FactBench research - Nature — “AI produces gibberish when trained on too much AI-generated data”
This directly addresses the second major issue in your article: what happens when AI-generated information becomes part of the data environment used by later AI systems. It discusses research showing degradation—often called model collapse—when models are recursively trained on generated data without adequately preserving real data.
Read the Nature article - ACM Web Conference 2026 — “Retrieval Collapses When AI Pollutes the Web”
This may be the most interesting additional source for your specific argument. Rather than looking only at model training, the researchers examine what happens when search engines and retrieval-augmented AI systems retrieve AI-generated material from the web. Their experiments demonstrate how synthetic content can increasingly dominate retrieved evidence and reduce source diversity.
Read the ACM research - EMNLP 2025 — “Demystifying Synthetic Data in LLM Pre-training”
This provides an important counterbalance. AI-generated data isn’t automatically harmful. Researchers studying more than 1,000 LLMs found that results depended heavily on the kind of synthetic data and how it was mixed with natural web data. In other words, the issue isn’t simply “AI trained on AI = bad”; data quality, proportions and training methodology matter.
Read the EMNLP/ACL paper
