How do I put an AI chatbot on my website without it making things up?

Last updated 14 October 2026. Written by Oyekola Obajuwon, founder of Growveloper, a growth studio that builds the website, the marketing and the follow up for owner run businesses in the United States, the United Kingdom and Nigeria.

Give it one written page of your own answers, switch off everything else it could draw on, and send every question about money, promises or anything missing from the page to a person. Then test it before launch with questions it should refuse. No setting stops an AI guessing; the page and the test keep it honest.

That test is the part almost nobody does. We call it the trap list: the questions your chatbot must refuse, asked before it goes live and again every time you change what it knows. The rest of this page explains why you need one, and how to set the bot up so it passes.

Why does an AI chatbot make things up?

Because it was trained to guess. In a September 2025 paper, Why Language Models Hallucinate, researchers at OpenAI and Georgia Tech explain that "large language models sometimes guess when uncertain, producing plausible yet incorrect statements", because "the training and evaluation procedures reward guessing over acknowledging uncertainty."

Think of a multiple choice exam with no penalty for wrong answers. A blank scores nothing. A guess sometimes scores. The paper puts it the same way: models "are optimized to be good test-takers, and guessing when uncertain improves test performance." So when a customer asks your chatbot something it does not know, its first instinct is not "I don't know." It is a confident sentence that sounds like your business.

That habit lives in the model, not in your settings. You cannot switch it off. You can only make sure the bot has the right answer in front of it, and a safe way out when it does not.

Does training a chatbot on my own website stop it making things up?

It helps a lot. It does not stop it.

In 2024, researchers at Stanford and Yale tested the AI research tools that lawyers pay for, tools that answer from their own libraries of real cases and whose makers had described them with words like "hallucination-free". Their paper, Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, found the tools from LexisNexis and Thomson Reuters "each hallucinate between 17% and 33% of the time." That was better than a general chatbot. It was still roughly one answer in six, at best, on hard legal questions. A bot answering your opening hours from a short page has a much easier job, but the lesson carries: a vendor's promise is not a test result.

The closest warning for a small business came from New York City. Its MyCity chatbot was launched to "access trusted information from more than 2,000 NYC Business web pages", the city's own guidance for owners. In March 2024, The Markup asked it questions a business owner might ask. Asked whether an employer could take a share of staff tips, it answered: "Yes, you can take a cut of your worker's tips." New York's law says otherwise. Asked whether buildings had to accept housing vouchers, it told ten separate staff members they did not, which is also wrong in New York, and gave one reporter the opposite answer.

The bot had 2,000 official pages behind it. It still produced a confident yes to a question whose honest answer was no.

Who is responsible when a website chatbot gets it wrong?

You are. The chatbot is your business talking. MyCity's page carried a warning that it "may occasionally produce incorrect, harmful or biased content", and the story ran anyway.

When an airline told a Canadian tribunal that its chatbot was responsible for its own mistakes, it lost and had to pay the customer. We tell that case, Moffatt v Air Canada, in our guide to which repetitive jobs AI can actually do. The short version: anything your chatbot says, you said.

How do I set up a chatbot that only answers from my own answers?

Four steps, in this order. The first one does most of the work.

Write the answer page first

Go through the last three months of WhatsApp messages, emails and calls, and write down every question customers actually asked, in their words, with your answer in yours. Prices you publish, hours, areas you cover, how booking works, what you do not do. Twenty to forty questions covers most small businesses. If it is not written down, the chatbot should not say it. This page is worth having even if you never add a bot: put it on your website as a questions page and it answers people on its own.

Lock the bot to that page

Choose a tool that answers only from content you give it, and check the setting yourself. Vendors describe this plainly on their own pages. Tidio says its Lyro assistant "works exclusively with the information you provide" and "won't generate random responses or draw from web-based data". Intercom says its Fin agent "builds answers using only the most relevant information from multiple knowledge sources" that you give it. Treat those as claims to test, not results. If your tool offers to fall back on general AI knowledge when your page has no answer, switch that off.

Decide what it never answers

Some questions go to a person even when the answer is on the page: refunds, discounts and custom quotes, any promise about a date or a result, anything legal, medical or about safety, and every complaint. A wrong answer there costs money or trust, and a few minutes' wait costs nothing.

Make the hand off a real person

"Someone will be in touch" is not a hand off. Name how the person reaches the customer (WhatsApp, a call or an email), and when. Outside your hours, the bot should say when you will reply and take a number. Our guide to automating lead follow up covers the instant reply and booking link that make this quick, and connecting your website to WhatsApp, a CRM and a calendar covers where the conversation lands.

What is the trap list?

A short set of questions your chatbot must not answer from its own head. Write about twenty, in your customers' words, across these six kinds.

Ask each question three times, in fresh chats. The Markup's experience is the reason: the same bot gave ten people one answer and another person the opposite. A bot passes only when every trap question gets either the answer from your page or a clean hand off, every time. Run the list again whenever you change the answer page or the tool's settings, and once a month regardless.

How do I keep checking the chatbot after launch?

Read a sample of real conversations every week, twenty is enough for a small business, and look for two things: an answer that is not on your page, and a customer who asked twice. Each wrong answer is fixed in the page, not by arguing with the bot, and the question that caught it goes on the trap list. Over a few months the list becomes the best record you have of what customers really ask.

What did building RideOn teach us about rules that fail?

RideOn Nigeria had no chatbot. When I started, customers booked through WhatsApp and the admin team ran on spreadsheets and chat threads. The platform I built for them had four portals, and the case study describes one design choice in a sentence: "The same access rule is enforced four times, so a mistake in one layer is caught by the next."

That is the right way to think about a chatbot. The answer page will have a gap. The lock will sometimes let something through. The never answer list will miss a topic. The trap list will not think of every question. None of them has to be perfect, as long as each one catches what the one before it missed.

Does a small business need a website chatbot at all?

Often not yet. Our judgement: if most questions arrive in working hours and you reply within the hour, a clear questions page plus a WhatsApp or call button does the job, and it cannot make anything up. A chatbot earns its place when the same questions keep arriving at night or at weekends and nobody is there to answer. Our AI and automation page lists customer support among the work we automate, and our guide to what to automate first helps you decide whether a chatbot is the first job or the fifth. If you want a second pair of eyes on a bot you are trialling, book a free 30 minute call and bring your trap list.

Write the page, lock the bot to it, and ask it the questions you hope a customer never will.