Choosing a chunking strategy

A chunk is the unit that gets embedded, retrieved, and shown to the model, so it must be self-contained enough to be understood alone and specific enough to be matched precisely. Fixed-size splitting ignores document structure and cuts tables, lists, and definitions in half. I start from structure: split on headings and paragraphs, keep tables intact, keep code blocks intact, and only then apply a size cap with recursive splitting. See Ingestion: Parsing & Chunking.

Each chunk carries metadata (document, section path, version, date) and often a prefix with the section title so an isolated chunk still says what it is about. For long-form content I use parent-child: embed small chunks for precise matching, return the parent section for generation. Overlap of 10–20% helps with boundary sentences but does not fix bad boundaries.

It is working if retrieval evals say so: build a golden set of questions with the passage that answers them, then measure recall@k and chunk-contains-answer rate across chunk sizes and strategies. The right size depends on the corpus and the question style; 200 tokens for FAQ-like content, larger for narrative reasoning. See RAG Evaluation.