How your agent knows things it was never trained on
A model doesn’t know your return policy. Retrieval finds the right passages of your own content and hands them to the model with the question. Here is how Thaliq does it, step by step.
A language model knows a staggering amount about the world and nothing about your business. It has never seen your return policy, your shipping zones, or that you take Yape. Ask it directly and it will do the most dangerous thing a support agent can do: guess, confidently.
The fix isn’t fine-tuning. It’s retrieval: bring the relevant piece of your knowledge into the conversation, and let the model answer with it in front of it, not from memory.
Retrieval in four moves
- Chunk your content into small passages. Thaliq cuts at the end of a paragraph, then at the end of a sentence, and only then at a hard limit: at most 1,200 characters per chunk, with 150 characters of overlap between neighbors.
- Embed each chunk as a vector, a point in meaning-space. Thaliq uses Voyage AI’s
voyage-3and stores the vectors in PostgreSQL, with pgvector. - Search that space with the question. Before the agent’s first model call of a run, Thaliq embeds the end user’s last message and keeps the nearest chunks above a similarity threshold, within a budget of tokens. The defaults: at most 8 chunks, at least 0.2 similar, 1,500 tokens.
- Ground the answer. The chunks reach the model as reference material, under a heading that marks them as data and not as instructions, each tagged with its source and title.
What you give it
A KnowledgeBase is declared in a file, like the rest of your agents. Its sources are web pages (an https address) or text and Markdown that you write in the file:
apiVersion: thaliq.com/v1alpha1kind: KnowledgeBasemetadata: name: kb-public-sitespec: description: Public pages of the site. sources: - name: about url: https://www.example.com/about - name: booking-faq format: markdown text: | ## How do I book a session? Choose a specialist, then a free slot.Apply it, and the command tells you how each source ended:
$ thaliq apply -f kb/ --waitKnowledgeBase/kb-public-site created (version 1); waiting for the ingestion of 2 sourcesKnowledgeBase/kb-public-site source about ready: 4 chunks (voyage/voyage-3)KnowledgeBase/kb-public-site source booking-faq ready: 1 chunk (voyage/voyage-3)applied 1 resourceAn agent names the KnowledgeBase as its long-term memory (memory.longTerm.collection), or a retrieval node of its graph searches it where you put it. topK, similarityThreshold and maxContextTokens are settings of either, for when the defaults don’t fit.
Grounded, not guessed
Retrieval only returns what is similar enough. Ask something the knowledge base can’t answer and nothing comes back above the threshold; an agent whose instructions say so can then ask a clarifying question or hand the conversation to a person instead of inventing an answer.
And you can check what the agent was given. Every search is a retrieval span in the run’s trace in Lens: the chunks it returned, with their document, position and score, and how many tokens it kept. When an answer looks wrong, open the run and see what the model saw before you blame the model.
Retrieval quality is the product
Everyone can stuff text into a prompt. The hard, boring wins are in the retrieval: how the text is cut, which embedding model reads it, how many chunks get through and how similar they must be, and what happens when a page changes. Write each source as short paragraphs separated by empty lines, each about one thing, so that a chunk holds a whole answer. Nothing refreshes on a schedule: when a page changes, thaliq refresh knowledgebase kb-public-site fetches it again, and a page whose text did not change isn’t embedded again.
Know what you are sending
- A KnowledgeBase is for your organization’s own content. It sits outside conversation retention and outside an end user’s erasure, so it must hold no end user’s personal data.
- The text of your sources, and your users’ questions, go to Voyage AI to be embedded. That applies only to organizations that use knowledge bases.
- Today a source is a web page or a text you write in the file. There is no file upload.
Your agent doesn’t need to be trained on your business. It just needs to be able to look it up.
Jhosetp Chino
We write about how we build Thaliq: the decisions, what went wrong and what we changed. From Lima.