How to Check an AI Answer Step by Step

The answer looks finished, and you're tempted to paste it in and move on. Don't. A short routine can help catch slips before you use an answer. That is our own suggestion, not something a source measures.

Australia's privacy regulator is blunt about it. It says AI systems are known to give results that are inaccurate or false.¹ The same guidance wants a person built into the process, not left out of it.¹ This page is one way to do that.

Step one: decide how much checking it needs

A tip for a staff lunch needs less than a paragraph about a refund rule. Ask what happens if a detail is wrong. If the answer is "a customer loses money" or "we look careless to a regulator", check every line.

Step two: mark the claims you can test

Read the answer once, slowly, and underline anything that could be proved wrong:

• Names of people, firms, places and products
• Figures, prices and totals
• Dates and deadlines
• Quotes, links and named sources
• Statements about what a rule, law or policy says

Opinions and wording choices don't need this. Facts do.

Step three: find the original for each claim

Go to the place the fact would normally live. For a price, that's the seller's own page. For a rule, it's the agency that publishes the rule. For a figure about your own business, it's your records.

Search for the claim yourself rather than asking the same tool to confirm it. In the New York case where lawyers filed fake cases, the tool had assured them that the cases "indeed exist".² Even so, the court could not find a few of those cases.² A tool vouching for itself proves nothing.

Step four: compare word for word

Don't settle for "close enough". Check that the number, the name spelling, the date and the condition all match the original. Small changes matter: "may" can become "will", and "some" can become "all".

If a claim names a source, open that source and find the claim in it. If you can't find it, treat the claim as unconfirmed. The page on quotes and sources goes into this.

Step five: fix, cut or flag

For each claim you tested, do one of three things:

• Fix it using the original.
• Cut it if you can't confirm it.
• Flag it, and ask a person who knows, if it matters and you're unsure.

Cutting is often the right call. A shorter page with confirmed facts beats a fuller one with a hidden error.

Step six: write down what you checked

A two-line note is enough: what you checked, where, and who did it. If a customer or boss asks later, you'll have an answer. If you spot a recurring problem, such as a tool that keeps getting your prices wrong, you'll see it.

When the answer comes from a chatbot on your own site

The same applies, and more strongly. A Canadian tribunal ruled that the airline was fully responsible for the information its chatbot gave, because the chatbot was part of its website.³ That case is covered in When an AI Answer Needs a Second Person.

For the details most likely to be wrong, read about the small details that slip through.