Local LLMs 204: Why the Same Model Is Not Always the Same

Sun, 31 May 2026 00:00:00 +0000

You may see the same model name in Ollama, Hugging Face, LM Studio, OpenRouter, a benchmark table, and a coding tool. Then you try it in two places and it behaves differently.

This is normal. “Same model” often hides several layers of difference.

Mental model: name, checkpoint, format, runtime, provider

A model name is only the label at the top of the stack. Behavior comes from the whole stack:

Layer	Example difference
Family	Qwen, Llama, Gemma, Mistral, DeepSeek, Phi
Checkpoint	Base, instruct, coder, reasoning, distilled, updated release
File format	GGUF, safetensors, runtime-specific package
Quantization	Q4, Q5, Q8, full precision
Runtime	Ollama, llama.cpp, vLLM, MLX, transformers
Prompt template	ChatML, model-specific instruct format, custom system prompt
Provider	Local machine, OpenRouter route, direct vendor API
Parameters	Temperature, top_p, max tokens, reasoning settings, stop sequences

The "Same Model" Stack

A model name is just the top layer.

Model Serving on Mohammad Movahedi

Local LLMs 204: Why the Same Model Is Not Always the Same

Mental model: name, checkpoint, format, runtime, provider

The "Same Model" Stack