What is the difference between temperature, top-p, and top-k sampling?
LLM Engineer Interview Questions: Choosing Between OpenAI, Anthropic, Open Source Models, and Self-Hosting
Audio flashcard · 0:24Nortren·
What is the difference between temperature, top-p, and top-k sampling?
0:24
Temperature scales the logits before softmax, with lower values making the distribution sharper and more deterministic. Top-k sampling restricts the choice to the K most probable tokens. Top-p, or nucleus sampling, restricts the choice to the smallest set of tokens whose cumulative probability exceeds P. Temperature controls randomness; top-p and top-k control diversity. Setting temperature to zero makes generation deterministic.
platform.openai.com