Blog · AI agents and workflows

How to Write a Knowledge Base Your AI Agent Can Actually Use

Why agents miss answers that are sitting in your documents, the five files every business agent needs, the writing rules that make each entry findable, and a test to prove it works.

By Dijae Dadula · November 16, 2026 · 9 min read

Dark paper-cut illustration of a stack of documents with a green tab and a check mark, linked by a dashed green path to a friendly robot and a chat bubble
Picture this · illustrative

A customer asks a shop's AI agent how much delivery to Cavite costs. The answer is in the knowledge base, on page six of a long document, in a table under the heading “Rates”. The agent replies that it is not sure and offers to connect a person. The owner opens the document, sees the fee right there, and wonders why the agent “can't read”.

An AI-ready knowledge base is a set of short, single-topic entries that each make sense on their own, so the agent can find the right one and quote it accurately.

Knowledge base the documents your agent answers fromChunk a small piece of a document the system stores and searchesRetrieval the search step that picks which chunks the agent seesEntry one topic, written to stand aloneSource of truth the one place a fact lives

Why does an AI agent miss answers that are in your documents?

Short answer: the agent never reads your whole document. It searches small pieces of it, and a piece that does not say what it is about is hard to find and easy to misread.

Most AI agents answer with retrieval-augmented generation (RAG): before replying, the system searches your files and passes the best-matching passages to the model. To make that search work, documents are split into chunks. OpenAI's file search, for example, splits files into chunks of 800 tokens with a 400-token overlap by default. A long document becomes dozens of separate pieces.

The problem is what a piece loses when it is cut out. Anthropic's research on retrieval gives the classic example: a chunk that says “The company's revenue grew by 3% over the previous quarter” does not say which company or which quarter. The same happens with business documents. A table row that says “₱150 · 2 to 3 days” means nothing once it is separated from the heading that said “Delivery to Cavite”.

The same fact written two ways Two note cards side by side. The left card, labelled hard to find, holds a bare table row: 150 pesos, 2 to 3 days, with a handwritten note asking delivery to where. The right card, labelled easy to find, holds a full sentence: delivery to Cavite costs 150 pesos and takes 2 to 3 working days, updated November 2026, with the word Cavite highlighted. A dashed arrow runs from the left card to the right card. The amounts are an illustrative example. SAME FACT, TWO WAYS HARD TO FIND ₱150 · 2 to 3 days delivery to where? for what order? EASY TO FIND Delivery to Cavite costs ₱150 and takes 2 to 3 working days. Updated November 2026 Illustrative amounts. The right card still makes sense if it is the only thing the agent reads.
Sketch 1. A fact that names its own subject survives being cut into chunks; a bare table row does not.

Anthropic measured how much this matters. Adding a short line of context to each chunk before storing it cut failed retrievals by 35%, and by 49% when combined with keyword search, on its test sets. You do not need to build that system yourself. You get much of the same effect by writing entries that carry their own context from the start.

What files should a business knowledge base have?

Short answer: five, each with one job: business facts, products or services, policies, FAQs, and handoff rules.

A common mistake is uploading everything: brochures, old price lists, chat exports, the website copied page by page. More documents do not make a smarter agent. They make more places for two versions of the same fact to disagree. Start with five files and put each fact in exactly one of them, the “one fact, one home” rule from grounding an AI agent in your business knowledge.

FileWhat goes in itChanges how often
Business factsName, locations, hours, contact channels, service areas, payment methodsRarely
Products or servicesOne entry per item: what it is, who it is for, price, options, what is not includedWith every price or menu change
PoliciesDelivery, returns, refunds, cancellations, warranties, booking rules, each as its own entryWhen the policy changes
FAQsThe real questions customers ask, in their words, with full-sentence answersMonthly, from new questions in chat
Handoff rulesWhat the agent must not answer, and who takes overWhen the team changes

The last file connects to your agent's instructions. How to write those is in AI agent instructions for any business.

Myth

The more documents you upload, the more the agent knows.

Reality

Every extra document is another chance for an outdated or conflicting version to be retrieved. A small, clean set beats a large, messy one.

How do you write an entry an AI agent can use?

Short answer: one topic per entry, the subject named in the first sentence, full sentences instead of yes or no, and exact numbers with a date.

Intercom's guidance for its Fin AI agent lines up closely with what the retrieval research shows. Its help centre asks for single-topic sections, full sentences instead of bare “yes” or “no” answers, precise numbers and limits, numbered lists for steps, and written text alongside every image. Put together, that gives eight rules:

  1. One topic per entry. “Returns” and “exchanges” are two entries if the rules differ.
  2. Name the subject in the first sentence. Write “Delivery to Cavite costs...” rather than “It costs...”. Headings do not always travel with the chunk.
  3. Answer in full sentences. “Yes, we deliver to Cebu by courier in 5 to 7 working days” beats “Yes”.
  4. State numbers exactly. Prices, fees, limits, days and hours, with the currency and unit.
  5. Write the conditions. Who it applies to, minimum order, cut-off times, exceptions.
  6. Introduce every table. One sentence above it saying what the table covers.
  7. Never put a fact only in an image. A price list as a photo is invisible to most agents.
  8. Date every entry and name an owner. The agent and your team both need to know how fresh it is.
Dijae's rule

Write every entry so it still makes sense if it is the only thing the agent reads.

What does a good entry look like?

Short answer: a short title phrased like the customer's question, a one-sentence answer, the details and conditions, and a footer with the date and owner.

Copy these two templates and replace the brackets. Keep each entry short; if it runs past a few paragraphs, it is probably two topics.

ENTRY: [Customer question, e.g. How much is delivery to Cavite?]
ANSWER: [One full sentence that names the subject, e.g. Delivery to
Cavite costs PHP 150 and takes 2 to 3 working days.]
DETAILS:
- Applies to: [who or which orders]
- Conditions: [minimum order, cut-off time, exceptions]
- Not included: [what this does not cover]
IF UNSURE: [what the agent should do, e.g. hand off to the team]
UPDATED: [date] | OWNER: [name or role]
POLICY: [Name, e.g. Refunds for online orders]
SUMMARY: [One sentence a customer could act on.]
STEPS:
1. [First thing the customer does]
2. [What happens next, with the time it takes]
3. [How they get the refund or replacement]
LIMITS: [time window, condition of item, exclusions, exact numbers]
THE AGENT MAY: [explain the policy, share the link, collect details]
THE AGENT MAY NOT: [approve exceptions, promise refunds] -> hand off
UPDATED: [date] | OWNER: [name or role] | SOURCE: [link to the public policy page]

The “may not” line matters as much as the facts. A refund rule the agent invents is a promise your business may have to keep, as covered in the Air Canada chatbot ruling.

How do you keep the knowledge base accurate?

Short answer: give every entry an owner and a review date, retire expired content on the day it expires, and add new entries from the questions the agent could not answer.

The monthly knowledge base loop A dashed loop with four stops: read the questions the agent handed off or got wrong, fix or add the entry in its one home, retest the question set, then retire expired promos and old prices. The fix step is highlighted. A handwritten note says once a month, plus every price change. THE MONTHLY LOOP 1 · Readmisseshandoffs,wrong answers 2 · Fix the entry 3 · Retestthe question set 4 · Retire old content once a month + every price change
Sketch 2. The knowledge base improves from real misses: each handoff or wrong answer points to an entry to fix or add.

Three habits keep it honest. First, put an end date on every promo the day you add it, and delete it on that date. Second, treat a price change as a knowledge base change: the same person who updates the price list updates the entry. Third, read the conversations your agent handed off. Each one is either a missing entry, an unclear entry, or a question the agent should never answer. The weekly side of this is in AI agent monitoring after launch.

What can you set up today?

  1. Collect the 20 questions customers ask most, from your chat inbox, comments and calls.
  2. Create the five files and move every fact you have into exactly one of them. Delete duplicates.
  3. Rewrite each entry with the template: question as title, one-sentence answer, conditions, date and owner.
  4. Replace any price list or schedule that exists only as an image with text.
  5. Load the files into your agent and ask it all 20 questions, then compare each answer with its entry.

How you will know it works (acceptance criteria for the 20-question test):

The table below lists the checks to run after loading the files, and what a pass looks like.

CheckPass when
AccuracyAt least 18 of 20 answers match their entry on every number, date and condition
No inventionZero answers contain a price, fee or policy that is not in the knowledge base
Uncovered questionsQuestions with no entry are handed off, not guessed
FreshnessEvery entry has a date from the last 90 days or a confirmed review
RetestThe 20 questions are rerun after every price, policy or agent change

An honest limitation: a well-written knowledge base raises the odds of a correct answer, but it does not guarantee one. How well retrieval works also depends on the platform's search, the model and your agent's instructions, which you may not control. That is why the test set and a human handoff stay in place even after the agent passes.

Frequently asked questions

What is a knowledge base for an AI agent?

It is the set of documents an AI agent searches before answering, such as business facts, product and service details, policies and FAQs. The agent quotes from it instead of relying on what the model learned in training.

Should a chatbot knowledge base be one big document or many small ones?

Many small, single-topic entries usually work better. Systems split documents into chunks before searching, so an entry that names its own subject and stands alone is easier to find and harder to misread.

Can I just upload my website to the AI agent?

You can, but website pages mix several topics, repeat facts and often hold prices in images. Use them as raw material, then rewrite the facts into short entries with one home each.

How often should I update the knowledge base?

Update it the same day a price, promo or policy changes, and review it once a month using the questions the agent handed off or answered wrongly.

Get your agent's knowledge right

Want a knowledge base and AI agent set up so customers get accurate answers from day one? Book a free AI Automation Blueprint call.

Prefer to learn it step by step? Join the free weekly newsletter.

References

  1. Anthropic: Introducing Contextual Retrieval, September 19, 2024 (chunk context problem; 35% and 49% fewer failed retrievals).
  2. Intercom: Fin AI agent content best practices. Checked November 2026.
  3. OpenAI Java SDK reference: StaticFileChunkingStrategy (default chunk size 800 tokens, overlap 400).
  4. Lewis et al.: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, 2020.

Keep reading

Get one practical idea like this every week

Join the free newsletter on AI agents, automation and turning your skills into income. Prefer video? Watch the walkthroughs on YouTube.

Subscribe free
Book a free call

Want a system like this built for your business? Book a free AI Automation Blueprint call, or see the done-for-you services at my consultancy, IRD Digital Marketing Services.

DIJAE DADULA

Dijae Dadula · AI automation and systems specialist · Cavite, Philippines · Founder of IRD Digital Marketing Services

Some links on this site are affiliate links. If you buy through them, I may earn a commission at no extra cost to you.

© 2026 Dijae Dadula. All rights reserved.