How a model picks its next word
A text model generates an answer one piece at a time. After reading the prompt and the text it has already written, it scores possible next pieces—often parts of words, not whole words. Those scores are converted into probabilities. A common method, called sampling, uses the probabilities to make a weighted random choice: a likely continuation is favored, but a less likely one can still be selected.
That choice becomes part of the text the model sees on its next step. A small variation early in a sentence can therefore lead to a different sequence of words, examples, or ideas later on, even though the prompt has not changed.
What temperature changes
Temperature adjusts the balance among the candidate probabilities before the choice is made. Lower settings make the distribution more concentrated, so the leading option is more likely to win. Higher settings spread probability more broadly, giving less likely options more opportunity. At the limiting low end, generation can behave like repeatedly choosing the top-ranked option rather than sampling among alternatives.
This is a change in the model’s selection process, not a new source of knowledge. A more varied continuation may be useful when exploring possibilities, while a more concentrated choice can make outputs more consistent. But unusual wording is not the same as a better answer: the model can wander into a weaker or less coherent continuation, and repeated runs can differ even when they start from the same prompt.
