MemotivaLLM Engineer Interview Questions: Choosing Between OpenAI, Anthropic, Open Source Models, and Self-Hosting

What is the difference between temperature, top-p, and top-k sampling?

LLM Engineer Interview Questions: Choosing Between OpenAI, Anthropic, Open Source Models, and Self-Hosting

Audio flashcard · 0:24

Nortren·

What is the difference between temperature, top-p, and top-k sampling?

0:24

Temperature scales the logits before softmax, with lower values making the distribution sharper and more deterministic. Top-k sampling restricts the choice to the K most probable tokens. Top-p, or nucleus sampling, restricts the choice to the smallest set of tokens whose cumulative probability exceeds P. Temperature controls randomness; top-p and top-k control diversity. Setting temperature to zero makes generation deterministic.
platform.openai.com