Check an AI claim against its original source

You ask an AI something, and its answer includes a figure, date or rule you want to use. It may even include a link. The answer could still be wrong, outdated or broader than the source. An AI can sound just as confident about information unsupported by any source, and it may invent a plausible-looking citation. A few minutes of checking will tell you whether the original source supports the claim. Then you can decide whether to use it, qualify it or leave it out.

Four classifications for a claim

After checking, a claim falls into one of these categories:

• Supported: the source says the same thing at the same scope.
• Partly supported: the source supports the core of the claim, but the claim is broader than the source, refers to a different time, or omits an important condition or step.
• Contradicted: the source says something different, such as giving a different figure or the opposite rule.
• Unverified: you cannot find a source or passage that settles the question.

The classification guides what you do next. You can use a supported claim. Rewrite a partly supported claim to match the source. Correct or leave out a contradicted claim. Mark an unverified claim as unresolved or leave it out.

Check a claim in five steps

1. 1

   Choose one claim

   Choose a claim you plan to rely on: a figure, date, price, rule or name. Write it as a sentence of its own.

2. 2

   Find the original source

   An original source is the page or publication issued by the source of the information, such as a government agency’s guidance, a service provider’s pricing page, a study or a statistical release. A link from an AI is a lead. A search-result summary or another site reporting the information is not the original source.

3. 3

   Find the passage relevant to the claim

   Find the paragraph or line in the source that states the information. Also check who published the page and when it was updated.

4. 4

   Compare the wording

   Compare the figure, date and scope. Words such as “everyone,” “always” and “free” can make a claim broader than what the source says.

5. 5

   Classify and decide

   Choose a classification for the claim, then write one sentence stating what you will do with it and what remains unresolved.

If the answer is long, you can ask the AI to separate out the claims. You still need to open the source yourself.

From your previous answer, pick up to three claims that include a figure, date, price or rule. Write each as its own sentence and preserve the wording of your answer, including words such as everyone, always and free. Do not assess whether the claims are true.

Example: the source supports only part of a claim

You ask whether an image-editing app costs anything. The answer says: “The app is free for all users, and its pricing page confirms this.” You open the pricing page yourself. It says the basic version is free, while the AI features require a monthly subscription.

• Claim: the app is free for all users.
• Source passage: the basic version is free; the AI features are part of a subscription.
• Classification: partly supported, because “free for everyone” is broader than what the source says.
• Decision: I will write, “The app’s basic version is free, but the AI features require a subscription.”
• Unresolved: prices can change, so I will note the date I checked.

Which claims should you check?

• The claim includes a figure, date, legal provision, study or person’s name.
• The information can change, such as a price, rule, service feature or schedule.
• Your decision affects health, money, rights, safety or another person.
• The AI provides a source you cannot find or one that does not say what the answer claims.

If the issue concerns your health, money or rights and the source does not settle it, ask an expert. Brainstorming and writing tone do not need a source.

Common pitfalls

• The link looks trustworthy, but the page says something else. Open the page and find the relevant passage; the address or title alone proves nothing.
• You are reading another site’s summary. Find the organization that originally published the information and check it there.
• You do not notice that the page is out of date. Check its publication or update date and look for the latest version if the information can change.
• The claim is broader than the source. When the answer says “everyone,” “always” or “never,” check whether the source says the same and narrow the claim to match its scope.

How to tell the check is complete

• The claim is written as one precise sentence.
• You opened the original source yourself and found the passage relevant to the claim.
• You classified the claim based on the source passage.
• You recorded what you will do with the claim and what remains unresolved.