# Getting a Local Model Running


Before any Python script can talk to a model, the model itself has to be running and listening for requests. This is the part that happens entirely outside your code — in the terminal, one time (or once per model). Here's what's actually happening at each step, and why.

## Step 1: Install Ollama itself

```bash
curl -fsSL https://ollama.com/install.sh | sh
```

(On Windows/Mac, you'd download the installer from ollama.com instead — this command is Linux/Chromebook-Crostini specific.)

**What this actually installs:** Ollama isn't a model — it's a **model server and manager**. Think of it like Docker, but for AI models instead of containers: it handles downloading, storing, loading into memory, and serving models over a local API. Without it, you'd have to manually handle model file formats, quantization, and inference code yourself — Ollama abstracts all of that into one command-line tool.

## Step 2: Start the server

```bash
ollama serve
```

**Why this is a separate step from installing:** Installing puts the program on your disk; `serve` actually starts it running and listening on a port (`11434` by default). This is what your Python script's `requests.post("http://localhost:11434/...")` is talking to — without this running, there's nothing on the other end of that request, and you'd get a connection-refused error.

**Why it often starts automatically:** Most installers set Ollama up as a background service, so after a reboot you usually don't need to run this manually — it's already listening. You only need it explicitly if you check and find it's *not* running (which is exactly what caused one of your earlier errors).

## Step 3: Pull a model

```bash
ollama pull qwen2.5:0.5b
```

**What's actually being downloaded:** A model "pull" fetches a large binary file — the model's **weights** (the billions of numbers that encode everything it learned during training) — from Ollama's model registry, similar to how `docker pull` fetches a container image. For a 0.5B parameter model, expect roughly 300-400MB; a 7B model is several gigabytes.

**Why the tag (**`:0.5b`**) matters so much:** The same base model name often has multiple size variants hosted under it. `ollama pull qwen2.5` with no tag grabs a default size — which might be far bigger than your hardware can handle. Always specify the exact variant, especially on a 4GB machine.

**Where it's stored:** Downloaded models live in `~/.ollama/models` on disk, cached permanently until you delete them — pulling the same model again later is instant.

## Step 4: Confirm it's actually there

```bash
ollama list
```

**Why check before writing any code:** This shows you the exact model name/tag string as Ollama sees it. Copy this *exactly* into your Python script's `MODEL` variable — a typo here (missing the tag, wrong capitalization) causes a 404 from Ollama's API, not a helpful error message.

## Step 5: Sanity-check it works, before involving Python at all

```bash
ollama run qwen2.5:0.5b
```

**Why test in the terminal first:** This drops you into a direct chat with the model, no code involved. If this works — you type something, you get a coherent reply — you've confirmed the model downloaded correctly and your hardware can actually run it. This isolates "is the model itself working" from "is my Python script working," which matters a lot when debugging later — if this terminal test fails, the problem is in Ollama/the model, not your code.

**What "loading" actually means here:** The first message after starting `ollama run` is often slower than the rest — that's the model's weights being read from disk into RAM. Subsequent messages in the same session are faster because it's already loaded.

Type `/bye` or `Ctrl+D` to exit this test chat.

## Step 6: Now point your script at it

Once steps 1-5 all check out, your Python script (from the last post) just needs the exact model string from `ollama list`, and it'll work — because all your script does is send HTTP requests to a server that's already confirmed to be running with that exact model loaded.

## Why the general pattern applies beyond Ollama

Every local-model runner — Ollama, LM Studio, llama.cpp — follows this same shape: **install the runner → download the weights → start a server → confirm it responds → point your code at it.** The specific commands differ, but the mental model doesn't. Once you understand *why* each of these five steps exists, switching tools later is mostly just learning new command names for the same underlying steps.

* * *