November 9, 2026

Who Is Liable When Your AI Chatbot Gets It Wrong?

Air Canada paid for its chatbot's invented policy. What an AI hallucination is, who is liable for chatbot mistakes, and how to reduce them.

  • AI hallucination
  • chatbot liability
  • AI Act
A dark card with the eyebrow AI HALLUCINATION, the headline "When your chatbot gets it wrong, you pay" and the subtitle "What Air Canada learned in court in 2024".

In short: when an AI chatbot gives a customer false information, the business that deployed it is liable, not the machine. Air Canada learned this in 2024. AI hallucinations never disappear completely, but you can cut them sharply by making the AI answer from your own documents, cite its sources and stay inside a defined scope.

In November 2022, a Canadian passenger asked Air Canada’s website chatbot how to get a bereavement fare after his grandmother died. The chatbot told him he could buy a full-price ticket and claim a partial refund within 90 days.

That was wrong. The airline refused the refund. In front of the tribunal, it tried an unusual defense: the chatbot was, it suggested, “a separate legal entity that is responsible for its own actions.”

The tribunal did not buy it.

What is an AI hallucination?

An AI hallucination is a false answer, delivered with confidence, that a language model produces because it predicts the most likely next words, not the most accurate ones.

A language model is the engine behind ChatGPT and most chatbots. Think of your phone’s autocomplete, trained on an enormous library: it knows exactly what a good answer looks like. On its own, it does not know whether that answer is true.

That is why a hallucination does not look like a bug. It has the tone, structure and confidence of a correct answer. An invented refund policy sounds exactly like a real one.

Why do chatbots make things up?

They make things up because they are asked to answer without being given the right information, and they were not built to stay silent. Three causes come up again and again.

  • No reliable source. The chatbot cannot see the actual policy, or sees it buried among dozens of conflicting pages.
  • Scope that is too wide. It was connected to the whole website, or the open web, and it mixes contexts.
  • No permission to be unsure. Nothing allows it to say, “I don’t know, let me connect you with a person.”

The lawyers in Mata v. Avianca lived the extreme version: ChatGPT handed them entirely fictional court rulings, which they cited before a federal judge. The sanction: $5,000. We look at what that case teaches about hiring help in How to choose an AI consultant.

Who is liable when an AI chatbot is wrong?

The business that put it online. That is what the Canadian tribunal concluded in the Air Canada case, and it is the principle emerging everywhere else.

On February 14, 2024, British Columbia’s Civil Resolution Tribunal ruled in Moffatt v. Air Canada that the airline is responsible for all the information on its website, whether it comes from a static page or a chatbot. It found negligent misrepresentation. The award came to roughly CAD 812, including interest and fees.

The amount is small. The precedent is not: you cannot blame the tool.

What about the EU in 2026?

The rules are tightening. The new EU Product Liability Directive explicitly brings software, including AI systems, into the definition of a “product.” Member states must transpose it by December 9, 2026.

Meanwhile, the EU AI Act requires, among other things, that people be told when they are interacting with an AI. This is not legal advice, but the direction is clear: liability stays with whoever deploys the system.

What do other documented cases teach us?

They show that errors take very different forms, but the bill always lands in the same place.

Case What happened What it cost
Air Canada (2024) Invented refund policy Ruling against the airline and a legal precedent
NYC MyCity (2024) Illegal advice to business owners Months online despite warnings, then planned shutdown
DPD (2024) Chatbot swore at customers and mocked the company Viral reputational damage
Mata v. Avianca (2023) Invented case law $5,000 sanction
McDonald’s / IBM (2024) Drive-thru order errors Test ended after about two years

In New York, the MyCity chatbot told business owners they could take a cut of workers’ tips and refuse tenants with housing vouchers, both illegal. It stayed online for months with a disclaimer. In early 2026, the new city administration announced plans to take it down.

At DPD, a single update was enough for the chatbot to swear and mock its own company; one post was viewed 800,000 times in 24 hours. And McDonald’s ended its AI drive-thru test with IBM, run in more than 100 restaurants, after order mistakes went viral.

The common thread: each time, the AI was answering without verifiable guardrails.

Book a call

30 min · Free · No commitment

To position your project: what can be automated, what should not be, and where to start.

Book a call

How do you reduce chatbot hallucinations?

You reduce them by changing the question you ask the AI: instead of “what do you know?”, you ask “what do these specific documents say?” This approach is called grounding.

A grounded answer is one built from specific documents supplied to the model, rather than from its general memory. The most common technique for this is called RAG (retrieval-augmented generation); we walk through it step by step in RAG explained: how AI answers from your documents.

The 5-guardrail test

Before you put a chatbot in front of customers, patients or citizens, check these five points:

  1. Source required. Every answer cites the document it came from, with a link. No source, no answer.
  2. Closed scope. The AI only consults a defined, up-to-date set of documents, not the web or the entire intranet.
  3. Permission to say “I don’t know.” If the information is missing, the system says so and routes to a human.
  4. Human sign-off on commitments. Anything that creates a right, a price, a refund or legal advice goes through review.
  5. Searchable log. Every exchange is kept, so you can understand and fix an error instead of discovering it in court.

These guardrails do not make AI infallible. They make its errors visible, checkable and recoverable. That is exactly what was missing in every case above.

How does Sceptra apply these guardrails?

These five guardrails sit at the heart of the architecture we built Sceptra on: an AI workspace that runs on your own infrastructure.

Answers are grounded in your documents, files and business knowledge, not the open web, and every answer cites its sources: you click, you check. Work is organized in spaces, by client, team or project. The AI working on Client A’s file sees Client A and nothing else, so closed scope is built in, not bolted on.

You define your standards, formats and checklists, and Sceptra applies them every time, for the whole team. Data leaves only through connections your administrator has created, everything that leaves is logged, and everything is reversible.

AI can still be wrong. But with a source under every answer, the error shows up before it reaches your customer.

Frequently asked questions

Is a company liable for its chatbot’s mistakes? Generally, yes. In Moffatt v. Air Canada (2024), the tribunal held that the airline was responsible for information its chatbot gave, just as for any other page on its website. The argument that the chatbot was a separate entity was rejected.

Can AI hallucinations be eliminated completely? No. A language model predicts likely text, so it can always produce an error. You can, however, sharply reduce how often errors happen and how much damage they do: grounded answers, cited sources, closed scope, human review on commitments and a log of every exchange.

Does an “AI can make mistakes” disclaimer protect the business? Not reliably. New York’s MyCity chatbot displayed a disclaimer and was still heavily criticized for illegal advice. In the Air Canada case, the tribunal found a customer could not be expected to double-check elsewhere what the website itself told them.

What is the difference between a hallucination and a bug? A bug is a programming defect, often reproducible and visible. A hallucination is a false but plausible answer produced by the model working normally. It looks like a good answer, which makes it harder to catch unless there is a source to check.

What changes in EU law for chatbots in 2026? The new Product Liability Directive covers software and AI, with transposition due by December 9, 2026. The AI Act also adds transparency duties. Have a lawyer review your specific situation before deploying a customer-facing chatbot.

Key takeaways

  • A chatbot is not a legal entity: the business that deploys it is liable.
  • A hallucination is a false but plausible answer produced by the model working normally.
  • Disclaimers are no substitute for guardrails.
  • Source, scope, permission to say “I don’t know,” human sign-off, log: those are the five guardrails.
  • AI grounded in your documents makes its errors checkable before they get expensive.

Planning to put an AI assistant in front of customers or staff? Book a 30-minute call and we will run your project through the 5-guardrail test. You can also try the domain-by-domain simulator at sceptra.ai.