Hallucinations and Brand Safety

What a hallucination is, in a brand context

In the research literature, a hallucination is a fluent, confident statement produced by a generative system that is not supported by its sources or by any verifiable record. For an organization the question is narrower and more practical: it is the subset of those statements that describe you. Your prices, your policies, your ownership, your staff, your product capabilities, your safety record, your regulatory or legal standing.

The reason this matters now is exposure rather than novelty. Assistants and AI answer surfaces have become a routine intermediary between a question about a company and the company's own material, and a person reading a synthesized answer usually does not see the page it was drawn from. The largest published test of this to date, coordinated by the European Broadcasting Union and led by the BBC in October 2025, evaluated roughly 3,000 assistant responses across 22 public service media organizations, 18 countries and 14 languages. It found that 45 percent of responses contained at least one significant issue and 81 percent contained some issue, with 31 percent showing serious sourcing problems and 20 percent showing major accuracy problems including hallucinated detail. That study measured news questions, not questions about companies, and it should not be read as an error rate for brand queries. It is evidence that the failure is systemic rather than anecdotal, which is a different and more useful claim.

Four shapes recur often enough to be worth naming separately, because they have different causes and different remedies.

• Fabrication: a fact that exists nowhere, such as a policy, a product feature or a founding date that was never published.
• Misattribution: a real fact attached to the wrong source, or a citation that does not contain the claim it is offered for.
• Conflation: a real fact about a different entity attached to yours, common where names, sectors or locations are similar.
• Staleness: a fact that was true and is no longer, presented without any indication of when it was true.

Mechanism: why confident wrong statements occur at all

A 2025 paper by researchers at OpenAI and Georgia Tech argues that hallucinations persist in part because of how models are scored. Under binary grading, a guess can outscore an admission of uncertainty, so systems optimized against those benchmarks are, in the authors' framing, optimized to be good test takers. The paper's proposed remedy is to change how the dominant benchmarks score abstention. This is an argument advanced by researchers about the systems they build. It is a well specified account rather than a settled or complete one, and it does not describe the internals of any system it does not cover.

The second relevant mechanism is retrieval. Google's own documentation for site owners describes its generative features as using retrieval augmented generation, which it calls grounding, to retrieve relevant, up to date web pages through its core Search ranking systems, along with a query fan out technique that issues concurrent related queries. Retrieval changes where the text comes from. It does not by itself guarantee that the generated summary is faithful to what was retrieved, and the evidence indicates that retrieval based systems still misattribute at meaningful rates: the Tow Center for Digital Journalism tested 200 queries across eight AI search products in March 2025, a total of 1,600 tests, and reported that the systems failed to return the correct source more than 60 percent of the time, ranging from 37 percent incorrect for one product to 94 percent for another.

Both mechanisms point the same way for an organization. If the material about you is thin, contradictory, undated or easily confused with someone else's, a system doing retrieval and synthesis has more room to produce something plausible and wrong, and less to check itself against.

What the measurements show, and what they do not

Public benchmarks measure specific, narrow things. Vectara's Hallucination Leaderboard evaluates faithfulness in a constrained task: models are given a document and asked to summarize it using only the facts it contains, and the score is how often unsupported content appears. Read on 2026-08-02, the top entry on that leaderboard showed a hallucination rate of 1.8 percent. That figure describes closed book faithfulness to a supplied document. It says nothing about what a system will assert when asked an open question about a company it has no document for.

The distance between those two situations is the whole problem. Low single digit error on grounded summarization and substantial error on open questions about real entities are not in conflict, because they are different tasks. A published rate is also a snapshot: models, retrieval layers and product surfaces change, and a number measured in one quarter is not evidence about the next one.

AIOFacts is not aware of any public benchmark that measures how often AI systems are accurate about arbitrary named organizations. That gap is worth stating plainly, because it means an organization cannot cite an industry figure as its own risk level. The only measurement specific to you is one you take yourself.

The failure modes that actually reach organizations

The publicly documented incidents cluster into a small number of patterns. Each has a different point of leverage, and treating them as one undifferentiated problem tends to produce generic advice.

• Entity conflation. Similar names in the same sector or region are the most common route to a damaging false statement, because the wrong fact is genuinely true of someone.
• Adverse synthesis. A sector wide event gets narrowed onto a named company. In a suit filed in Minnesota state court in March 2025, solar installer Wolf River Electric alleged that a Google AI Overview stated the state attorney general had sued it, when the attorney general's action named four solar lending companies and not Wolf River. The company alleged cancelled contracts and sought more than 100 million dollars. Those are allegations in a case that was removed to federal court and remanded to state court in January 2026. Nothing here is an adjudicated finding.
• Stale facts. Prices, policies, hours, leadership and product capabilities that changed without a dated, canonical published record of the change.
• Third party dominance. Where forums, aggregators and directories carry more retrievable text about you than you do, the retrievable picture is largely written by other people.
• Superseded content that persists. A retracted claim, an old press release or an archived page can remain retrievable long after you stopped standing behind it.
• Your own assistant. A chatbot on your own property is the one system in this list whose outputs you own outright.

Accountability runs in two directions, and they are not equal

When the wrong statement comes from your own system, the accountability question has at least one documented answer. In Moffatt v. Air Canada, decided by the British Columbia Civil Resolution Tribunal in February 2024, a customer relied on the airline's website chatbot for its bereavement fare policy and was given incorrect information. The tribunal treated this as negligent misrepresentation, found that Air Canada had not exercised reasonable care to ensure the information's accuracy, rejected the argument that the chatbot was a separate entity responsible for its own statements, and awarded damages of 812.02 Canadian dollars. This is a single small claims level decision in one jurisdiction and it does not create a general rule anywhere else, but it is a clear, checkable instance of an operator being held to what its assistant said.

When the wrong statement comes from a third party system, the position is genuinely unsettled. The questions raised by cases like the Wolf River filing, including who is a publisher of generated text and what the standards for fault and remedy should be, do not have settled answers, and litigation moves on a timescale that is of little use to an organization whose customers are reading the wrong thing this week.

The practical consequence is a matter of order. Fix the surface you own, because you are answerable for it and you can change it today. Treat the surfaces you do not own as something to be measured, evidenced and disputed through documented channels, on the understanding that no channel guarantees an outcome.

Practices that are available and honestly described

None of the following is documented by any operator as changing what a model will say. They are things an organization can control, chosen because they reduce the ambiguity a retrieval and synthesis process has to resolve.

• Publish a canonical, dated, unambiguous statement of the facts most likely to be misstated: legal name, former names, ownership, locations, what you do and do not sell, current pricing structure, current policies. Dated, because staleness is a distinct failure mode from fabrication and needs a distinct fix.
• Disambiguate the entity explicitly. Name the similarly named organizations you are not, and state what distinguishes you. Conflation is the failure that unambiguous published text is best positioned to address.
• Keep facts consistent across every surface that carries them, including your own site, your profiles and any third party listing you control. Contradiction between your own sources is something you can remove without anyone's permission.
• Correct in public and keep the record. Dated correction notes are retrievable evidence. Silently editing a page removes the wrong claim and also removes any evidence that you addressed it.
• Use structured data with a clear understanding of what it does. Google's guidance states that structured data is not required for its generative AI features and that no special schema.org markup is needed for them, while recommending it as part of overall SEO for rich result eligibility. Marking up your facts is defensible. Claiming it makes AI systems accurate about you is not.
• Use the documented feedback paths. Google publishes a process for submitting feedback on content about you in Knowledge Panels, which includes claiming the panel through a verified property. These are request channels, not correction guarantees.
• Understand what crawler controls do and do not do. Google documents Google-Extended as a robots.txt token governing whether crawled content may be used for training Gemini models and for grounding in Gemini Apps and Grounding with Google Search on Vertex AI, and states it does not affect Search ranking. OpenAI documents three separate agents with separate purposes: GPTBot for model training, OAI-SearchBot for surfacing sites in ChatGPT search, and ChatGPT-User for user triggered fetches, which it notes is not governed by robots.txt in the same way. These govern access to your content. They do not govern what a system says about you, and blocking retrieval can remove your material from the answer while leaving other people's descriptions of you in it.
• Measure rather than assume. Run a fixed set of questions about your organization across several systems on a schedule, and log the verbatim output with the date, the system and the exact prompt. A dated verbatim log is the only evidence that supports either a correction request or a legal one.

Where this practice ends

No operator known to AIOFacts publishes a correction service level for statements about a third party organization, and none publishes a mechanism that guarantees a specific claim will stop appearing. Feedback channels exist and are worth using. They are requests.

Outputs also vary. The same question asked twice, or asked of two products, can return materially different answers, and platform behavior varies between systems and over time. This means a single clean test result is weak evidence and a single bad one is not proof of a systemic problem. Sampling repeatedly and dating everything is what turns either into something usable.

The honest summary is that an organization can materially reduce ambiguity about itself in the public record, can own and repair its own systems outright, and can document what third party systems say. It cannot control the output of a system it does not operate, and any framework, including this one, that implies otherwise is overstating what is known.