What is RAG? AI answers grounded in company documents

What is RAG, and when should you use it?

Retrieval-augmented generation puts a search step in front of a language model: it finds relevant passages in your own documents and asks the model to answer from those passages. Use it when answers must come from text that changes, is too large for a prompt, or has to be cited — not when a database query, a rule or a short prompt would answer more reliably.

The two halves, and why the split matters

Retrieval is search engineering: parse, split, embed, index, rank. Generation is the model writing an answer from whatever retrieval handed it. They fail differently, and conflating them is why pilots stall.

When an answer is wrong, first ask whether the right passage was retrieved. If it was not, prompt editing cannot help; the chunking, index or query is wrong. If it was, the problem sits in the prompt or the answer format. Measuring the stages separately turns a vague complaint into a specific repair.

What you actually build

Chunking

Split documents into passages small enough to retrieve precisely and large enough to make sense alone. Split along structure — headings, clauses, table rows, question-and-answer pairs — not at a fixed character count, which cuts tables in half and separates a condition from its exception. Keep a link to the parent section, and store metadata with each chunk: source, date, department, permission tag.

Embeddings and the index

An embedding model turns text into vectors, and the same model must embed both documents and queries. Store the model name and version with the index: changing the model means re-embedding everything, which is a planned operation, not a surprise. For Thai and English in one corpus, choose a multilingual model and test both. Add keyword search alongside vectors: part numbers, names and rare Thai terms are where pure embeddings blur.

Retrieval and ranking

Retrieve more candidates than you will use, filter by date, department and permission, then rerank. Deduplicate near-identical passages, which otherwise crowd out the useful paragraph. The filters matter as much as the ranking: a correct answer from a document the user may not see is a security incident.

Generation and citation

Constrain the answer to the retrieved passages, require a citation for each claim, and allow a refusal when the passages do not contain the answer. Refusal is a feature: a system that always answers cannot be trusted on the questions that matter. Show the cited passage, not only the document title, so the reader can check the claim in one click.

Evaluation: what to measure before launch

| What you measure | Question it answers | How to get it |

Build the test set from real questions, including awkward and unanswerable ones. A set of friendly questions produces a demo; a set with ambiguous and unanswerable cases produces a system.

When not to use RAG

• The corpus is small. If the whole policy fits in the prompt and changes twice a year, retrieval adds moving parts for no gain.
• The answer is a database lookup. Stock levels, invoice status and balances belong in SQL, not in a vector index.
• The question is an aggregate. "How many contracts expire this quarter" needs a query over every record; retrieval returns a sample and the model will generalise from it.
• The rules must be deterministic. Eligibility and pricing should be code, with the model explaining the outcome rather than deciding it.
• The documents are wrong or unowned. Retrieval is faithful. It will find the outdated policy and quote it confidently.
• Permissions cannot be expressed per document. Fix the access model before indexing, not after.

Four common mistakes

Chunking by character count only. It is fast to build and a reliable cause of missing context. Split on structure, keep parent links, and read a sample chunk before indexing a million.

No unanswerable questions in the test set. Teams tune until the demo answers everything, then find in production that confidence and correctness differ.

One index for everyone. Access control bolted on after launch leaks documents. Filter at retrieval, and test with a user who should not see one.

Tuning the prompt while retrieval is broken. If recall is poor, prompt work is cosmetic. Measure retrieval first, fix it, then work on tone and format.

FAQ

Do we need to fine-tune a model as well?

Usually not for a first version. Fine-tuning changes how a model writes, not what it knows. Start with retrieval and evaluation, and fine-tune when the failure is format or tone, not missing knowledge.

How current can the answers be?

As current as the index. A nightly sync suits policies and contracts; for values that change hourly, fetch them live instead of embedding them. Show the document date in the answer so the user can judge it.

Does RAG work with Thai documents?

Yes, with two adjustments. Thai is written without spaces between words, so chunking and keyword search need a segmenter, and the test set has to contain real Thai questions. Mixed Thai-English documents belong in evaluation, not in a later surprise.

Next step

If you have a document collection and a recurring question, we can tell you whether retrieval is the right tool. Start at seansolution.tech/contact, or write to [email protected], call +66 94-865-9978, or message us on LINE at lin.ee/WQlPJc5.