M13: Open-source & local models (breadth module)
Every AI call so far went over the internet to a company's servers, on your dime, with your data leaving your machine. Today you flip that: download an open model and run it on your own computer: no key, no internet, no bill, full privacy. You'll feel the trade-off too (a laptop isn't a data center), which is exactly the "how to choose a model" judgment from M0, made real.
Today's win: an open model running locally that you chat with from the terminal and from your own Python, and you can say when local beats hosted, and why.
Today you will
- Run an open model locally with Ollama (no API key, offline, free)
- Call it from Python via its local API: the same shape as M4's hosted call, pointed at your machine
- Understand closed vs open and local vs hosted trade-offs (capability, cost, speed, privacy), and survey Hugging Face & LM Studio
Run of show (~50 min)
| Time | What we do |
|---|---|
| 0:00 | Hook + the win we're chasing |
| 0:05 | The one idea: a local model is a model running on your computer (full read in notes.md) |
| 0:10 | Lab Part A: install Ollama, pull a small model, chat in the terminal |
| 0:30 | Lab Part B: call the local model from Python; compare to hosted |
| 0:45 | Show: post your local model's reply (note: no key!) |
| 0:50 | Wrap + the landscape (Hugging Face, LM Studio) |
If you get stuck
- New install: Ollama (see the guide) +
pip install requests. No API key: local models don't need one. - Use a small model (
llama3.2,gemma2:2b,qwen2.5:0.5b), big ones are slow on a laptop. If a reply crawls, switch to a smaller one. Nothing here can harm your computer. - "Couldn't reach localhost:11434" just means Ollama isn't running, start the app /
ollama serve.
Optional challenge
Run the same prompt through your local model and your hosted Claude app from M4. Where does the local model hold up, and where does the frontier model clearly win? Write one sentence on which you'd pick for: a private journaling app, and a customer-facing assistant. (That's model selection, M0, with real evidence.)