One wrong number costs more than a dull paragraph
Fact-checking AI content means pulling every checkable claim out of a draft, matching each one to a named primary source, confirming it on that source’s page, and cutting whatever does not confirm, all before the piece goes live.
A dull paragraph loses a little attention. An invented statistic loses the one reader who knew the real number, and a B2B audience is full of those readers. They work in the field you are writing about. They recognize the figure, the study, the case name. Once a single claim turns out to be made up, the rest of the page stops reading as evidence and starts reading as guesswork.
Nothing in the prose warns them first. A language model states a fabricated figure in the same register as a correct one: same syntax, same confidence, same clean sentence. An editor reading for flow will pass it. The only thing that catches it is someone checking the claim against a source.
A fabricated figure arrives in the same clean sentence as a correct one.
This is not a hypothetical failure. It has a public record.
What the record already shows
| Case | What was published | What was wrong | Consequence |
|---|---|---|---|
| Mata v. Avianca, S.D.N.Y., June 2023 | A filed brief citing and quoting judicial decisions | Six opinions did not exist: “non-existent judicial opinions with fake quotes and citations created by the artificial intelligence tool ChatGPT” | Judge Castel imposed a $5,000 penalty and ordered letters to every judge named as author of a fake opinion |
| Moffatt v. Air Canada, BC Civil Resolution Tribunal, February 2024, reported by the Guardian | An answer from the chatbot on Air Canada’s website | It told a customer he could claim a bereavement fare within 90 days of ticket issue. The airline’s policy page said the opposite | Air Canada was held responsible and ordered to pay the fare difference. Tribunal member Christopher Rivers: “It should be obvious to Air Canada that it is responsible for all the information on its website” |
| CNET Money, January 2023, per CNET’s own editors’ note | 77 short financial explainers produced with an in-house AI engine | An audit “identified additional stories that required correction, with a small number requiring substantial correction”; borrowed sentences had also got past the plagiarism check | CNET corrected the stories, paused the tool, and moved the AI disclosure out from behind a byline hover |
| Deloitte Australia, October 2025, reported by the Associated Press | A commissioned report to the federal Department of Employment and Workplace Relations | “A fabricated quote from a federal court judgment and references to nonexistent academic research papers” | The department said Deloitte “confirmed some footnotes and references were incorrect” and agreed to repay the final instalment of its contract |
| Chicago Sun-Times, May 2025, reported by NPR | A syndicated summer reading list in the print edition | Titles attributed to real authors who never wrote them, among them “The Rainmakers” under Percival Everett’s name | Chicago Public Media said it is “unacceptable for any content we provide to our readers to be inaccurate” and said it was investigating. The list also ran in the Philadelphia Inquirer |
| Google Bard launch ad, February 2023, reported by NPR | Google’s own promotional GIF for Bard | Bard credited the James Webb Space Telescope with the first pictures of exoplanets. Those came from the European Southern Observatory’s Very Large Telescope in 2004 | The error shipped inside the advertisement and social media users quickly pointed it out |
Read the middle column again. None of these were sloppy prose. They were fluent, on-topic, correctly formatted claims that nobody checked. And the consequence landed on the publisher every time: the law firm, the airline, the newspaper, the consultancy. Not on the model.
Every time, the publisher carried the consequence. The model carried none.
Why the confident wrong answer is the architecture
A language model does not look anything up while it writes. It produces the most plausible continuation of the text so far, and a plausible continuation of “according to a 2024 study” is a study-shaped sentence, whether or not the study exists.
The reason it prefers a guess to an admission sits in the training loop. In “Why Language Models Hallucinate”, Adam Tauman Kalai and three co-authors argue that “language models hallucinate because the training and evaluation procedures reward guessing over acknowledging uncertainty” (arXiv, September 2025). OpenAI’s own summary of the paper is blunter: “when models are graded only on accuracy, the percentage of questions they get exactly right, they are encouraged to guess rather than say ‘I don’t know.’”
So the fluency of a sentence carries no information about its truth, and neither does the model’s confidence. Matthew Dahl, Varun Magesh, Mirac Suzgun and Daniel E. Ho measured what that costs on checkable questions. Asked specific, verifiable questions about random federal court cases, ChatGPT 4 hallucinated 58% of the time (Journal of Legal Analysis, 2024). Their second finding matters more for an editing process: models “cannot always predict, or do not always know, when they are producing legal hallucinations”, so asking the model whether it is sure adds nothing.
Those models are from 2023 and 2024, and error rates have moved since. The mechanism has not. Training still pays better for a confident answer than for an abstention, which is why this is a property of the architecture rather than a bug on someone’s roadmap.
What a fact-check actually is
Not a careful reread. Fact-checking AI content is claim-level work, and it happens before publication rather than after a reader writes in.
- Pull every checkable claim out of the draft as a list: statistics, dates, named studies, quotes, attributions, causal statements.
- Match each one to a named primary source, meaning the court record, the paper, the outlet that reported it, or the company’s own page. Not a summary of one.
- Open the source and find the sentence. Confirm the figure matches on units, period, population and definition, not only on digits.
- Cut what you cannot confirm. Do not soften it into “some research suggests” and publish it anyway.
- Log which claim was verified against which URL, so the next editor can audit the pass instead of repeating it.
Step 4 is the one that gets negotiated away, and it carries the whole point. A claim that survives as a hedge is still unverified, and it still runs under your byline.
A hedged claim is still an unchecked claim, and your byline still carries it.
By hand, or built into the pipeline
Our own experience, not a sourced claim: on the drafts we edit, verifying every claim takes roughly as long as writing the piece did. That is a real cost, and it explains what happens under deadline. Verification is the last step in the process and the only one whose absence is invisible on the page.
What that looks like at scale is measurable. We measured 531 pages ranking in Google’s organic top 20 for 40 B2B content-marketing queries (US, September 2026). The median page carries 0.9 statistics per 1,000 words, and 27% name no source at all. Among the pages that do cite something, 22% cite a single host. And 13% carry at least one link returning 404 or 410; only those two codes count as dead here, and timeouts are reported separately. Ranking pages are not a random sample of the web. They are the competitive set.
A step can be skipped. A stage cannot. When verification is a fixed stage in a publishing pipeline, an article either passes it or waits, and the decision about a shaky claim gets made before the URL is public instead of after.
When verification is a stage, a shaky claim is decided before the URL is public.
What to ask when you choose a tool
Output quality is the wrong lead question: every current generator writes clean sentences. Two questions separate tools that verify from tools that only write:
- Is there a separate claim-verification stage running against named sources, or does quality control end at grammar and style?
- What happens when a claim fails? Does the pipeline block, flag it for a human, quietly drop the sentence, or publish anyway?
Then ask for the artifact. A real verification stage produces something you can read: the claims, the source matched to each, and the ones that failed. If a vendor cannot show that list, treat verification as absent.
No tool can promise that every sentence it produces is true, and a vendor claiming otherwise is describing something a language model cannot do. What a pipeline can do is refuse to publish a claim nobody checked. If the answer to both questions above is “that is your job”, the accuracy risk stays with the publisher, however good the draft reads.
Frequently asked
What are AI hallucinations in content marketing?
Claims a model states with full confidence and no basis: an invented statistic, a study that was never published, a quote nobody said, a case that does not exist. The mechanism is ordinary. A model completes text with the most plausible continuation instead of looking facts up, and per Kalai and co-authors, training and evaluation “reward guessing over acknowledging uncertainty”. In marketing copy the result is a fluent, specific, checkable sentence that fails the moment a reader checks it.
How to verify AI-generated content without rereading it five times?
Work from a list, not from the page. Extract every checkable claim, match each to a named primary source, open the source and confirm the number matches on units and period, then cut whatever will not confirm. Rereading catches tone and typos. Only the claim-by-claim pass catches a citation that was never real, which is what the court found in Mata v. Avianca, where six cited opinions did not exist.
Who is liable when an AI-generated claim is wrong?
The publisher carries it in practice. Air Canada argued to the BC Civil Resolution Tribunal that its chatbot was a separate entity; the tribunal held the airline responsible for all the information on its website and ordered it to pay the fare difference. Deloitte Australia agreed to repay the final instalment of a government contract after its report was found to contain a fabricated court quote and references to papers that do not exist.
Do newer models make fact-checking AI content unnecessary?
No, because the incentive that produces the errors is still in place. Models are optimized to answer, and an answer scores better than an abstention, so a plausible fabrication remains the profitable output. Rates improve between model generations; the failure mode does not disappear. Dahl and co-authors also found that models cannot reliably tell when they are hallucinating, so a self-check by the model is not a verification step.
Sources
9 sources
Methodology: every claim and figure above was checked against the page it is cited from on October 13, 2026. Items 1 to 8 are primary: one court order, one publisher’s own editors’ note, four reports from the Guardian, the Associated Press and NPR, and two research papers. Item 9 is our own measurement, and its rules are stated with it.
- 01Mata v. Avianca, Inc., No. 22-cv-1461 (PKC), Opinion and Order on Sanctions, S.D.N.Y., June 22, 2023. https://storage.courtlistener.com/recap/gov.uscourts.nysd.575368/gov.uscourts.nysd.575368.54.0.pdf.Supports: six non-existent opinions created by ChatGPT, the $5,000 penalty, and the letters to the judges named as their authors.
- 02“Air Canada ordered to pay customer who was misled by airline’s chatbot.” The Guardian, February 16, 2024. https://www.theguardian.com/world/2024/feb/16/air-canada-chatbot-lawsuit.Supports: the chatbot’s bereavement-fare answer, the order to pay the fare difference, and the quote from tribunal member Christopher Rivers.
- 03Guglielmo, C. “CNET Is Testing an AI Engine. Here’s What We’ve Learned, Mistakes and All.” CNET, January 25, 2023. https://www.cnet.com/tech/cnet-is-testing-an-ai-engine-heres-what-weve-learned-mistakes-and-all/.Supports: 77 stories published with an internally designed AI engine, the audit that identified stories requiring correction, and the plagiarism-checker miss.
Show all 9 sourcesShow fewer
- 04McGuirk, R. “Deloitte Australia to partially refund $290,000 report filled with suspected AI-generated errors.” Associated Press, October 7, 2025. https://apnews.com/article/australia-ai-errors-deloitte-ab54858680ffc4ae6555b31c8fb987f3.Supports: the fabricated court quote, the references to nonexistent papers, the department’s statement, and the repayment of the final instalment.
- 05“How an AI-generated summer reading list got published in major newspapers.” NPR, May 20, 2025. https://www.npr.org/2025/05/20/nx-s1-5405022/fake-summer-reading-list-ai.Supports: the fabricated titles under real authors’ names, the Sun-Times statement, and the syndication to the Philadelphia Inquirer.
- 06“Google’s AI chatbot, Bard, sparks a $100 billion loss in Alphabet shares.” NPR, February 9, 2023. https://www.npr.org/2023/02/09/1155650909/google-chatbot–error-bard-shares.Supports: the exoplanet claim in Google’s Bard advertisement and the correct attribution to the European Southern Observatory’s Very Large Telescope in 2004.
- 07Kalai, A. T., Nachum, O., Vempala, S. S., Zhang, E. “Why Language Models Hallucinate.” arXiv:2509.04664, September 4, 2025. https://arxiv.org/abs/2509.04664. Plain-language version: OpenAI, “Why language models hallucinate”, https://openai.com/index/why-language-models-hallucinate/.Supports: training and evaluation rewarding guessing over acknowledging uncertainty, and the “I don’t know” line.
- 08Dahl, M., Magesh, V., Suzgun, M., Ho, D. E. “Large Legal Fictions: Profiling Legal Hallucinations in Large Language Models.” Journal of Legal Analysis 16(1), 2024, pp. 64-93. arXiv:2401.01301. https://arxiv.org/abs/2401.01301.Supports: ChatGPT 4 hallucinating on 58% of specific, verifiable questions about random federal court cases, and the finding that models cannot always tell when they are hallucinating.
- 09Avoid Content corpus measurement, September 2026. The instrument we ran over the corpus covered 531 articles ranking in Google organic top 20 for 40 B2B content-marketing queries (US, English), SERP pulled September 7, 2026. Dead links counted as HTTP 404 and 410 only.Supports: median 0.9 statistics per 1,000 words, 27% naming zero sources, 22% of citing pages citing a single host, 13% carrying a dead link. Dataset not yet public; thresholds stated inline.