Prompt Engineering Patterns: Tree of Thoughts, Graph of Thoughts, Self-Refine, Reflexion

Tree of Thoughts is an advanced prompting pattern that explores multiple reasoning paths in parallel, evaluating each and pursuing the most promising ones. Unlike linear chain-of-thought, ToT branches at each step, considers alternatives, and can backtrack. It is especially useful for problems with large search spaces like puzzles, planning, and creative writing.

ToT generates multiple candidate next steps at each point, evaluates each candidate using either a heuristic or another LLM call, prunes weak branches, and explores the best ones. The process continues until a final answer is reached or a depth limit is hit. ToT is more expensive than CoT because it makes many LLM calls, but it dramatically improves performance on hard reasoning tasks.

Use ToT when the problem benefits from exploring alternatives, when the first approach often fails and backtracking helps, when the solution space is large and structured, or when accuracy matters more than cost and latency. For everyday tasks where one correct answer is obvious, simple CoT is faster and cheaper.

Graph of Thoughts, or GoT, generalizes Tree of Thoughts by allowing arbitrary graph structures instead of pure trees. Thoughts can be combined, refined, or referenced from multiple branches. GoT enables more flexible reasoning patterns like merging partial solutions, comparing alternatives, and reusing intermediate results. It outperforms both CoT and ToT on tasks where solutions have shared subcomponents.

Self-Refine is an iterative pattern where the model generates an initial answer, then critiques its own output, then produces an improved version based on the critique. The cycle repeats until a stopping criterion is met. Self-Refine works because LLMs are often better at recognizing problems in existing text than at producing perfect output on the first try.

CoT produces reasoning before a single final answer. Self-Refine produces an answer first, then improves it through self-criticism cycles. CoT is one-shot reasoning; Self-Refine is iterative refinement. They can be combined: use CoT for the initial answer, then Self-Refine to iterate on it. The combination usually outperforms either alone.

Reflexion is a pattern where an agent reflects on its past failures, generates verbal feedback about what went wrong, and uses that feedback in subsequent attempts. Unlike Self-Refine which iterates within a single task, Reflexion accumulates lessons across many attempts. It enables LLMs to improve at a task through trial and error without weight updates.

Self-Refine improves a single output by critiquing and rewriting it. Reflexion improves an agent's overall strategy by reflecting on what went wrong across multiple attempts. Self-Refine is local and within-task; Reflexion is global and across-task. Both rely on the model's ability to critique itself, which works because evaluation is easier than generation.

Implement self-critique by asking the model to first answer a question, then evaluate its own answer against criteria like accuracy, completeness, and clarity, then produce an improved version addressing any issues found. The pattern works best when the criteria are specific and when the model has room in its context to see both the original and the critique. ---