How do you choose the right chunk size for RAG?

Start with 512 tokens and adjust based on your domain and query patterns. Short chunks of 128 to 256 tokens give precise retrieval but often miss surrounding context, while long chunks of 1024 or more preserve context but dilute the embedding signal and waste context window space. Technical documentation and legal text benefit from larger chunks to retain definitions and clauses, while FAQ or chat-style content works with smaller chunks. Always measure recall and answer quality on real queries before committing to a size.