A local LLM setup can feel messy because several tools appear at once: Python environments, command-line utilities, model downloaders, model runtimes, chat interfaces, and APIs. The trick is to separate the toolchain into jobs.
For a beginner, four ideas are enough to get oriented: uv, uvx, Ollama, and model files.
Mental model: tools install things, runtimes run models
uv helps manage Python projects, Python versions, dependencies, and Python command-line tools. Ollama runs language models and exposes them through a local API. A model file contains the model weights and metadata that the runtime needs.
These are different jobs:
| Tool or object | Job |
|---|---|
uv | Manage Python projects, dependencies, Python versions, and tools |
uvx | Run a Python CLI tool in an isolated temporary environment |
| Ollama | Download, manage, run, and serve LLMs locally |
| Model file | The actual model weights and configuration |
| Chat UI or coding agent | The application that sends prompts to the model |
The Toolchain Divide
Click a tool to see its primary job
Python Management
Local Models
What uv is for
uv is a fast Python package and project manager from Astral. It can create and synchronize project environments, manage dependencies, install Python versions, and run commands inside a project environment.
If you are building scripts around local LLMs, uv helps avoid the old “which Python environment is this using?” problem. A project can declare dependencies in pyproject.toml, and uv run can execute commands in the project environment.
Official docs to verify while drafting:
What uvx is for
uvx is an alias for uv tool run. It runs a Python command-line tool in an isolated environment without permanently installing it into your project. Astral’s docs describe it as a convenience alias for running tools.
This is useful when you want to try a CLI once, inspect a package, or run a helper without polluting your project dependencies.
For example, conceptually:
uvx some-tool --help
The important difference:
| Command pattern | Best for |
|---|---|
uv run ... | Running commands that need your project environment |
uvx ... | Running a standalone tool in an isolated disposable environment |
uv tool install ... | Installing a CLI you want available repeatedly on your PATH |
Official docs to verify while drafting:
What Ollama is for
Ollama is a beginner-friendly local LLM runtime. It can download models, keep a local model library, run chat-style prompts, and expose an API. Ollama’s local API is served by default at:
http://localhost:11434/api
Ollama also documents OpenAI-compatible endpoints, including /v1/chat/completions, which matters because many LLM tools already know how to talk to OpenAI-style APIs. If a local runtime exposes a compatible API, you can often point existing clients at your local server by changing the base URL and model name.
Official docs to verify while drafting:
Model files are not all the same
When people say “download a model,” they may mean several different things:
| Form | Common use |
|---|---|
| GGUF | Popular with llama.cpp-based runtimes and many local desktop workflows |
| safetensors | Common on Hugging Face for PyTorch/transformers-style workflows |
| Ollama model name | A runtime-managed reference such as gemma3 or qwen3 |
| Provider model ID | A hosted endpoint name such as an OpenRouter model slug |
The model family name is not enough. You need the exact variant, file format, quantization, context support, license, and runtime compatibility.
- Frantar et al. (2023), "GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers": foundational work on model file formats and quantization techniques that power local inference runtimes.
- Touvron et al. (2023), "Llama 2: Open Foundation and Fine-Tuned Chat Models": documents model variants, file formats, and deployment considerations relevant to understanding model distribution.
- Beurer-Kellner et al. (2023), "Efficient Serving of Large Language Models via Adaptive Structure Sharing": explores how runtimes like Ollama optimize model loading and serving to reduce resource overhead.
A beginner setup workflow
- Install one runtime first, such as Ollama.
- Run one small model and make sure it responds.
- Learn the local API address and model name.
- Add Python tooling only when you need scripts, evaluation, notebooks, or custom clients.
- Use
uvfor repeatable projects anduvxfor one-off Python CLI tools. - Record exact model names, versions, and commands in your notes.
What to verify because tools change
Do not copy install commands blindly from random posts. For any CLI, verify the official project page, package publisher, repository, release history, and install method. This matters especially for newer tools around local models, including model fit or model recommendation utilities.
The practical lesson is simple: use uv to keep Python sane, use uvx for disposable CLI experiments, use Ollama or another runtime to serve models, and keep model files conceptually separate from the apps that call them.