Chunking is the process of breaking a long document into smaller pieces before an AI stores or searches it. The size of those pieces, and how much they overlap, quietly decides how good the AI’s answers will be later. For agencies, it is one of the most common and least visible reasons an AI tool gives vague answers about your own material.
Also known as text chunking, document chunking
An AI does not read your 90-page brand guide all at once. Before it can search a document, the document gets cut into smaller pieces, and each piece is stored separately. When you later ask a question, the system pulls a handful of those pieces and writes its answer from them.
How the cutting happens matters more than it sounds. Cut the pieces too large and each one is noisy and unfocused, so retrieval drags in irrelevant text. Cut them too small and a single idea gets split across two pieces, so the system grabs half of it and misses the rest. Good chunking keeps related ideas together and adds a little overlap so nothing important falls through the cracks.
You rarely control chunking directly, but it shapes every answer an AI gives you about your own content.
It explains vague answers. When a tool returns mush about your brand guidelines, the model is usually fine. The way your document was sliced before it ever reached the model is the problem.
It affects every document-based tool. Custom GPTs, knowledge bases, and chat-with-your-files features all rely on chunking under the hood. Knowing it exists changes how you diagnose a weak result.
It is a fair question for vendors. When a vendor pitches an AI tool trained on your materials, how they handle chunking is a revealing thing to ask about.
An agency loads its creative playbook into a custom assistant so the team can ask quick questions during pitches. It keeps returning incomplete answers about the agency’s naming conventions, which span several pages. A closer look shows the document was chunked in a way that split each convention from its examples. Re-chunking the document so each rule and its examples stay together fixes the answers overnight, with no change to the underlying model.
The workshop covers how agencies set up AI tools on their own materials, including the unglamorous details, like chunking, that decide whether the answers are usable.