AI hallucinations: 7 ways to reduce the risk of wrong answers in your company

A hallucination is an AI answer that sounds plausible but is untrue. It might be an invented regulation, a court ruling that does not exist, a wrong amount or a procedure the company never had. The model states it in the same confident tone as true information.

An AI model does not look up facts in a database. It composes the answer that statistically best fits the question. When it does not know the answer, it often guesses rather than admit it does not know.

In a 2025 paper, OpenAI researchers explain this by the way models are trained and evaluated. The benchmarks used to compare models usually reward a lucky guess and give no points for “I don’t know”. So the model behaves like a student in a multiple-choice test who would rather guess than leave a blank.

Newer models make fewer mistakes, but the problem does not go away. It most often affects questions about things that are rare or specific to your company: internal procedures, individual customer terms, current prices.

Two high-profile cases show clearly who pays for hallucinations. In both, liability fell on the company using AI, not on the model provider.

In Moffatt v. Air Canada, the airline’s chatbot gave a customer incorrect rules on a bereavement discount. The airline argued that the correct information was on another page of its website. The tribunal ruled that the company is responsible for what its chatbot says and ordered it to refund the price difference.

In Mata v. Avianca, lawyers filed a brief in court citing judgments invented by ChatGPT. The court fined them and ordered them to inform the judges to whom the fictitious rulings had been attributed. The reputational cost was far higher than the fine itself.

In a mid-sized company, the stakes are usually less newsworthy, but real. Below are typical consequences of AI errors in day-to-day work.

Not every AI use case needs the same safeguards. A draft internal note that someone will read anyway can contain minor errors. A transfer amount or an answer to a customer about contract terms cannot.

A simple approach is to calculate the expected cost of errors. Assume, hypothetically, that AI prepares 1,000 answers a month and 2 in 100 contain a significant error. That is 20 errors a month. If each one costs an average of PLN 500 in corrections and lost time, the risk is PLN 10,000 a month.

You compare this amount with the cost of safeguards. If human review costs less than the errors it prevents, it is worth introducing. The table below shows a typical match between safeguards and types of use.

The first and most effective method is answering on the basis of company documents. This technique is called RAG: the AI first finds the relevant passages in your procedures, price lists or contracts, and then answers based only on them. It works like an employee who checks the right file before answering.

It is important that the AI cites the source of every answer, e.g. the document name and section. An employee can then check the answer quickly, and you can work out afterwards where an error came from. RAG will not help, however, if your documents contain three conflicting versions of the price list.

The second method is good instructions for the model. The AI should be explicitly allowed to answer “I don’t know” and told to use only the documents provided. It is also worth narrowing the task: “list the payment deadline from the contract” produces fewer errors than “analyse the contract”.

The third method is automatic rules that check the output before it goes any further. They do not catch every error, but they do catch the most costly ones. Below are examples of such rules.

The fourth method is a second, independent check of the answer. A separate query to a model – often a different one – checks whether the answer follows from the documents provided. If the check finds a mismatch, the answer does not go to the user but to a person.

Such a check increases response time and model usage fees, because every question is processed twice. It makes sense where an error is expensive, e.g. in answers to customers about contract terms. In the most demanding use cases, the same question can be sent to two or three different models. If the answers differ, the case goes to a person.

The fifth method is human approval above a set threshold. The AI prepares a proposal and a short justification, and an employee accepts or corrects it. You set the threshold according to risk: the amount, the type of customer or whether the AI was confident in its answer.

Well-designed approval is quick, because the employee receives a ready summary rather than a raw document. For high-risk systems, human oversight is also a requirement of the AI Act (Article 14).

The sixth method is a set of test questions with correct answers, prepared before launch. It should include typical questions, edge cases and trick questions to which the correct answer is “I don’t know”. You compare the AI’s answers with the reference answers and record the result.

You run the same set after every change: to the model, the instructions or the source documents. Providers regularly retire older models, and switching to a new one can change answer quality even when nothing in the process has changed.

The seventh method is ongoing monitoring after launch. Without it, you find out about errors from customer complaints – in other words, too late. Below are the elements worth introducing from day one.

The AI Act has no specific provision on hallucinations. Several obligations do, however, relate directly to the quality and transparency of answers. Which of them apply to you depends on what you use AI for.

From 2 August 2026, every AI system that talks to people must tell them they are interacting with AI (Article 50). This applies, for example, to a website chatbot or a voice assistant.

High-risk systems – e.g. AI that assesses job candidates or creditworthiness – are subject to more requirements. Following the 2026 amendments, they apply from 2 December 2027. Most of them are met by the system’s provider, while the company deploying it is responsible for, among other things, human oversight and use in line with the instructions (Article 26). The list below shows the requirements most closely linked to wrong answers.