Foundation · 6 min
What you will learn
- Why a confident wrong answer costs more than an obviously wrong one
- How to give every commercially important fact an owner
- How to separate what the system states from what it writes
- What to test, what to measure and when to re-test
Fluent is not factual
Nothing in the output signals doubt. A missing specification comes back as a complete sentence in the same tone as a verified one. A blank field looks like a gap and invites a question. An invented value looks like an answer and ends the conversation.
Small details decide purchases. Compatibility, material, dimensions, stock, delivery date and return eligibility are exactly the facts a shopper acts on. An error in any of them does not produce a confused customer. It produces a returned parcel.
And the claim belongs to your store. A shopper told an item fit, ships Thursday or can be returned has reasonable grounds to expect it, whatever system said. That is why this is an accuracy problem with commercial cost, not a writing-quality problem.
Give every important fact an owner
The core fix is architectural rather than editorial. Decide which system is authoritative for each kind of fact, then make the AI layer fetch from that system instead of reconstructing the answer from what it generally knows.
| The fact | Where the truth lives | When the AI should check it |
|---|---|---|
| Specifications, materials, compatibility | Your product record | Every time it is stated |
| Price and active promotions | Your commerce system | At the moment of the answer |
| Stock and delivery estimate | Inventory and fulfillment | Live, never from a cached copy |
| Returns, shipping, warranty rules | One written policy source | Every time, with no paraphrasing of a paraphrase |
| Order and customer status | Your order system | Live, and never reconstructed from the chat |
Policies deserve particular attention because they are the easiest to get wrong invisibly. If the rule exists in three places with different wording, the system will find one of them, and it may not be the current one.
Separate the facts from the writing
Keep retrieved fields and generated prose as different things. The system’s job in the sentence is to arrange facts it was handed, not to supply them. When a field is missing, the sentence should be missing a clause, not filled with a plausible one.
Draw a hard line around a few claim types that should never be generated: safety and medical claims, guarantees, legal rights, and any superlative the data does not support. “Meets the constraints you gave” is checkable. “The best option for you” is not, and it is the kind of sentence that gets quoted back to you.
Then give the system a sanctioned way out. An assistant with no acceptable way to say it does not know will produce something rather than nothing, because that is what completing a sentence means.
Verify where the consequence rises
Review effort should follow the cost of being wrong, not apply evenly. A general explanation of how a product category works can be checked by sampling. A live stock statement cannot, because it is wrong the moment it is stale.
So recheck anything that moves at the point it is used rather than at the start of the conversation and let the commerce system make the final call on anything transactional. The same rule governs the assistant pages in this series, and it is the one that prevents most customer-facing failures.
Test it like a catalog, not like a chatbot
Build a test set from things that actually break: real merchant and shopper questions, products with thin or contradictory attributes, facts that recently changed, policy edge cases and the failures you have already seen. Demonstration questions tell you nothing, because the system was never going to fail those.
Track four numbers rather than a general impression of quality.
- Factual error rate, checked against the source rather than against plausibility
- Unsupported-claim rate, where the answer may be right but nothing backed it
- Correction rate, how often a reviewer or shopper has to fix the output
- Escalations and complaints traced back to a wrong answer
Re-test after any change to the model, the prompt, the retrieval setup or the catalog. The last one is the one teams forget, because adding products does not feel like changing the AI.
Key takeaways
- The problem is confidence without grounding, not poor writing.
- Every commercially important fact needs a named source the system retrieves.
- A missing field should produce a missing clause, never a plausible one.
- Recheck anything that moves at the point of use and let the commerce system decide.
- Test on what breaks, measure factual error, and re-test after catalog changes too.