What is temperature in AI language models?

Temperature controls how random or deterministic a model's output is - low values (0–0.3) give focused, consistent answers; high values (0.7–1.5) give more varied, creative ones.

AILLM

Overview

When an LLM generates text, it produces a probability distribution over all possible next tokens. Temperature is a parameter that reshapes that distribution before sampling. At temperature 0, the model always picks the highest-probability token (greedy, deterministic). As temperature increases, lower-probability tokens get more weight, making output more varied and surprising.

Key Points

•Temperature 0 → deterministic (greedy decoding): always picks the most likely next token.
•Temperature 1.0 → sample from the raw probability distribution without modification.
•Temperature > 1.0 → flattens the distribution, making unlikely tokens more probable (more random, often incoherent above ~1.5).
•Low temperature (0–0.3): fact extraction, classification, JSON output, code generation - consistency matters.
•High temperature (0.7–1.0): creative writing, brainstorming, generating diverse options.
•`top_p` (nucleus sampling) is an alternative: sample only from the smallest set of tokens whose probabilities sum to `p`. Many practitioners set `temperature=1` and vary `top_p` instead.

Examples

How temperature changes output for the same prompt

typescript

const prompt = 'Write a one-sentence tagline for a coffee shop.';

// Low temperature - consistent, safe, predictable
const focused = await client.chat.completions.create({
  model: 'gpt-4o',
  temperature: 0.1,
  messages: [{ role: 'user', content: prompt }],
});
// "Fuel your day with every cup."  (similar every time)

// High temperature - varied, creative, sometimes weird
const creative = await client.chat.completions.create({
  model: 'gpt-4o',
  temperature: 1.2,
  messages: [{ role: 'user', content: prompt }],
});
// "Where the beans know your secrets."  (different each time)

Recommended temperature settings by task

typescript

const TEMP = {
  // Extracting structured data - must be consistent
  dataExtraction:     0.0,
  classification:     0.0,
  codeGeneration:     0.1,

  // Q&A - mostly deterministic but slight variation is OK
  factualQA:          0.2,
  documentSummary:    0.3,

  // Creative tasks - variation is desirable
  emailDrafting:      0.7,
  brainstorming:      0.9,
  creativeWriting:    1.0,
  ideaGeneration:     1.1,
} as const;

// Usage
const answer = await client.chat.completions.create({
  model: 'gpt-4o',
  temperature: TEMP.codeGeneration,
  messages: [{ role: 'user', content: userMessage }],
});

top_p (nucleus sampling) as an alternative control

typescript

// top_p: only consider tokens whose cumulative probability >= p
// top_p=0.1 → only the very top tokens (like low temperature)
// top_p=0.9 → wide vocabulary but cuts off the long tail of unlikely tokens
// top_p=1.0 → all tokens considered (no truncation)

const response = await client.chat.completions.create({
  model: 'gpt-4o',
  temperature: 1,    // leave temperature at 1
  top_p: 0.95,       // control diversity via top_p instead
  messages: [{ role: 'user', content: 'Write a haiku about databases.' }],
});

// OpenAI recommends not changing both temperature and top_p simultaneously

Pro Tip

For production APIs that extract structured data or generate code, use `temperature: 0` and pair it with `response_format: { type: "json_object" }`. This combination is the most reliable for getting parseable, consistent output. For chat applications, `temperature: 0.7` gives natural-feeling variety without going off the rails.