You will be able to split documents into chunks, store their embeddings with metadata, and retrieve the closest matches.
Daniel is building a tool that answers questions from his organisation's policy documents: a handbook, a leave policy and several claims policies. Once he had learned about embeddings, he turned each whole policy document into one vector, and search got worse.
One vector for a whole document blurs every topic in it into one average meaning. Even a correct whole-document match causes trouble. A question about childcare leave matched Daniel's 40-page leave policy (an invented example). The match was right, but he would then have to send all 40 pages to the model, and avoiding that was the reason for retrieval in the first place. You have to cut documents into smaller pieces before embedding them, and store each piece so it can be found again and traced back to its source.
A chunk is a piece of a document small enough to be about one thing, usually a few paragraphs. Each chunk gets its own embedding, and search returns chunks rather than whole documents.
Chunk size is a trade-off. Chunks that are too small, such as single sentences, lose context. For example, a sentence saying a rule applies only to confirmed staff means nothing without the paragraph before it. Chunks that are too large blur several topics into one vector again. A few paragraphs is a sensible starting size, and you can adjust it after testing.
Where you can, split along the document's own structure. Headings, sections and numbered clauses make natural boundaries, because the author has already grouped related text under them. Daniel's handbook has clear section headings, so he made one chunk per section and split very long sections into a few paragraphs at a time.
Add overlap between neighbouring chunks. This means repeating the last part of one chunk at the start of the next. Without overlap, an answer that crosses a boundary gets cut in half, for example a rule in one chunk and its exception in the next, and search may return only one of them. A sentence or two of overlap is usually enough.
You don't need to write the splitting code yourself. Most document libraries can split by headings, or by length with overlap, so ask a coding assistant to use one.
Each stored chunk needs its text, its vector and its metadata, meaning the labels that describe where it came from. Useful metadata includes the file name, the page number or section heading, the date the document was issued, and anything you might want to filter by later, such as department.
Metadata has two jobs. First, it lets answers cite sources that staff can check, such as "Claims Policy, section 4.2, page 7". Lesson 3.4 relies on this. Second, it lets you filter before searching. When Daniel added the 2025 claims policy, he marked the 2024 version as superseded in its metadata and excluded it from searches, so the tool never answers from the old rule.
You need somewhere to store vectors and search them by similarity. There are two common choices:
A vector database, which is a database built to store vectors and search them by similarity. Several exist. Some are hosted and some you run yourself, and most have free tiers that are enough for a practice project. A database extension that adds vector search to a database you may already use. The best-known is pgvector, which adds a vector column type and similarity search to PostgreSQL. If your app already uses PostgreSQL, keeping the vectors there means one less service to run.
For a few hundred chunks, either choice is fine. For practice, even a simple file loaded into memory will work.
Whichever you choose, the search call is the same. You pass in the question's vector, ask for the top few matches, and get back the chunk text with its metadata. If you fetch too few results, you may miss the answer. If you fetch many more, you fill the prompt with noise and add cost. Daniel fetches the top five chunks per search, which is a reasonable default.
Embeddings struggle with text whose meaning depends on the exact characters, such as form numbers, invoice references, people's names and specific dollar figures. To an embedding model, HR-07 and HR-08 look almost identical, but to staff they are different forms. Plain keyword search handles exact matches like these well.
Many retrieval systems run vector search and keyword search together and combine the results. This is called hybrid search. "Where do I find form HR-07?" is answered by the keyword match. "What can I claim after working late?" is answered by the vector match, which works on meaning. Many vector databases offer keyword search, and so does PostgreSQL itself, so hybrid search rarely needs another tool.
Test both approaches on your own questions before you choose. Daniel ran vector-only search and hybrid search on his questions. Vector search alone missed every question that named a form number. With hybrid search, those results came back first.
Now try it on your own documents. In the activity below, you chunk one document, store it with metadata, and look at what comes back for each of your five questions from lesson 3.1. Split the document along its headings, sections or clauses, with a sentence or two of overlap. Embed each chunk separately, and store the text, vector and metadata together. Compare vector-only and hybrid search on your questions, then adjust your chunk size based on what comes back.
Split one of your documents into chunks, store them with metadata, and print the top three matches for each of your five questions.
Junxiong-WFG Organisation is an authorised representative of AIA Financial Advisers Private Limited (Reg. No. 201715016G).