What is chunking and why does it matter?

Chunking is the process of splitting documents into smaller passages before embedding them. Chunking matters because retrieval quality depends heavily on chunk size and boundaries. Chunks that are too small lose context, while chunks that are too large dilute relevance and may exceed embedding model token limits. Good chunking is one of the highest-leverage decisions in a RAG system.