Top-p / Top-k Sampling
Top-k and top-p are paired decoding-time strategies that restrict which tokens a model may sample from. Top-k keeps only the k highest-probability tokens at each step. Top-p, also called nucleus sampling, keeps the smallest set of tokens whose cumulative probability exceeds p. Both truncate the long tail of unlikely tokens, but top-p adapts the cutoff dynamically to the shape of the probability distribution.
Definition
Top-k and top-p are paired decoding-time strategies that restrict which tokens a model may sample from. Top-k keeps only the k highest-probability tokens at each step. Top-p, also called nucleus sampling, keeps the smallest set of tokens whose cumulative probability exceeds p. Both truncate the long tail of unlikely tokens, but top-p adapts the cutoff dynamically to the shape of the probability distribution.
Examples
- Setting top-k=40 to limit a code generator to the 40 most likely next tokens.
- Using top-p=0.9 in a chatbot to allow varied phrasing while excluding rare tokens.
- Combining temperature=0.7 with top-p=0.95 for creative writing with controlled drift.
Related terms
No credit card required
Frequently asked
- What is Top-p / Top-k Sampling?
- Top-k and top-p are paired decoding-time strategies that restrict which tokens a model may sample from. Top-k keeps only the k highest-probability tokens at each step. Top-p, also called nucleus sampling, keeps the smallest set of tokens whose cumulative probability exceeds p. Both truncate the long tail of unlikely tokens, but top-p adapts the cutoff dynamically to the shape of the probability distribution.
- How is Top-p / Top-k Sampling used in practice?
- Top-p / Top-k Sampling is commonly used in scenarios such as: Setting top-k=40 to limit a code generator to the 40 most likely next tokens.; Using top-p=0.9 in a chatbot to allow varied phrasing while excluding rare tokens.; Combining temperature=0.7 with top-p=0.95 for creative writing with controlled drift..
- How does yno.ai support Top-p / Top-k Sampling?
- yno.ai supports Top-p / Top-k Sampling as part of its agentic AI platform — combining multiple frontier models with extensible tool use.
- What other concepts are related to Top-p / Top-k Sampling?
- Top-p / Top-k Sampling relates to Temperature (Sampling), Inference, Tokenization. Each links to its own glossary page below.