DS DevShelfHub Projects · AI tools
Articles / Local Models in 2026: Gemma 4, Ollama, and OpenClaw for Real Agent Work

AI Engineering

Local Models in 2026: Gemma 4, Ollama, and OpenClaw for AI Agents

By DevShelfHub

The 2026 refresh on running local AI models with OpenClaw — sizing hardware for Mac unified memory vs Nvidia VRAM, why Gemma 4 and Qwen 3.6 finally cross the tool-calling bar, the openclaw configure flow, and the hybrid pattern that pairs local defaults with cloud fallback.

Local Models in 2026: Gemma 4, Ollama, and OpenClaw for AI Agents

Introduction

Local models took a real step up in early 2026. Gemma 4 and Qwen 3.6 hit quality levels that finally make “run the agent on your own machine” a serious option, not just a privacy talking point. A mid-tier Mac or a single decent Nvidia card can now host an agent that calls tools, runs OpenClaw, and gets real work done — without ever sending a token to OpenAI or Anthropic.

This guide is the 2026 refresh of the local-model pattern: how to size hardware for Mac and Windows, which models are actually worth running, the exact Ollama + OpenClaw setup, and how to combine local models with cloud models in a hybrid workflow. If you read the earlier OpenClaw-plus-Ollama write-up, treat this as the updated version with the new wave of models and the workflow patterns that fall out of them.

📚 Table of contents

  • What changed in early 2026
  • Sizing hardware: Mac unified memory vs Nvidia VRAM
  • Picking a model that supports tool calling
  • Installing Ollama (or upgrading)
  • Pulling Gemma 4 at the right size
  • Wiring Ollama into OpenClaw with openclaw configure
  • Hybrid setup: local default, cloud for hard problems
  • Switching models mid-session
  • Best practices
  • Common mistakes
  • Frequently asked questions

📈 What changed in early 2026

Two months made a real difference. Gemma 4 (Google’s open-weights release) and Qwen 3.6 both shipped with proper tool-calling support, thinking modes, and quality that competes with cloud mid-tier models on real tasks. That last detail matters: previous local models could chat, but they couldn’t reliably drive an agent that calls tools.

Why this unlocks OpenClaw

  • OpenClaw runs an agentic loop — the model has to call tools, not just respond
  • Tool calls require structured output and reliable function-format adherence
  • Gemma 4 and Qwen 3.6 hit this bar consistently on consumer hardware
  • That means real workflows — not just demos — can now run locally

💻 Sizing hardware: Mac vs Windows

The bottleneck for local inference is fast memory. Mac and Windows handle this very differently, and the wrong reading of your specs leads to picking a model that won’t fit.

🍎 Mac (M-series)

Apple silicon uses unified memory — the CPU and GPU share the same RAM. Check About this Mac → Memory. Whatever number is there is the ceiling for both your OS and any model you run. A 32GB Mac comfortably runs ~20GB models. An 8GB Mac is limited to the smallest models.

🪟 Windows / Linux (Nvidia)

System RAM doesn’t matter much; what matters is VRAM on your GPU. A 4090 has 24GB. An 8GB card is much more limited. Check VRAM in Task Manager → Performance → GPU or nvidia-smi.

Common sizing pitfalls

  • Macs older than 5–6 years: skip local models entirely — you’ll spend more time waiting than working
  • Windows with an iGPU only: same advice
  • Picking a model that’s slightly larger than your RAM/VRAM: it’ll run but at unusable speed
  • Forgetting OS overhead: leave a few GB for the system

🧩 Picking a model that supports tool calling

OpenClaw needs tool calling. A chat-only model is not enough. Check the model card on ollama.com for explicit tool, thinking, or function-calling support.

Strong choices in 2026

  • Gemma 4 — available at 2B, 4B, 8B, 12B, 26B, 31B. Pick the largest size that fits with headroom.
  • Qwen 3.6 — slightly larger than Gemma 4, strong on multilingual and reasoning
  • Llama 3.3 Instruct — reliable workhorse
  • DeepSeek R1 distilled variants — for reasoning-heavy tasks

Recommended default: Gemma 4 at the size your hardware can carry comfortably. For most 16GB Macs and 8GB Nvidia cards that’s the 4B–9B band. For 32GB+ machines, push to 26B or 31B for noticeably better responses.

⚙️ Installing or upgrading Ollama

Even if you have Ollama installed, run the install command again. The newer models require a newer Ollama runtime; older versions will fail to pull or run them.

  1. Visit ollama.com and copy the platform-specific install command.
  2. Run it in a terminal (or command prompt on Windows).
  3. Verify with ollama — if you see an interactive help screen, it’s installed.
  4. Hit escape to exit the help screen.
  5. If ollama isn’t found, close and reopen the terminal.

⬇️ Pulling Gemma 4 at the right size

Browse ollama.com/library/gemma3 (or the Gemma 4 page when available) and check the sizes. Pick one where the on-disk size is comfortably smaller than your available memory.

  1. Pick the size: e.g. gemma4:9b for 16GB hardware.
  2. Run ollama pull gemma4:9b (or your chosen tag).
  3. Wait for the download (a few minutes on a good connection).
  4. List installed models with ollama list.
  5. Quick smoke test: ollama run gemma4:9b — type a message, hit enter, watch it respond. /exit when done.

🔌 Wiring Ollama into OpenClaw

Once Ollama is serving the model, OpenClaw needs to know about it. The configure command walks you through it.

  1. Run openclaw configure.
  2. Select Model.
  3. Scroll down to Ollama, select Local only (don’t use Ollama Cloud here).
  4. Leave the base URL as default (http://localhost:11434).
  5. Pick the models you want enabled — check Gemma 4 and any others from ollama list.
  6. Confirm and exit.
  7. Run openclaw gateway restart so the new model registers.

📌 If OpenClaw can’t reach Ollama (“can’t reach server” error), run ollama serve in a separate terminal, or launch the Ollama desktop app so the lama icon sits in your menu bar / system tray.

🔀 Hybrid setup: local default, cloud for hard problems

Local models are good. Frontier cloud models are still better at hard reasoning. The pragmatic setup uses both.

The hybrid pattern

  • Set Gemma 4 as your default model in OpenClaw config
  • Also configure an OpenAI key and an Anthropic key for fallback
  • Use local for daily work: drafting, summarizing, structured outputs, scheduled jobs
  • Switch to cloud for: heavy multi-step reasoning, novel coding problems, anything that needs frontier capability
  • You save 80–95% of cloud bills without giving up the option when it matters

🔁 Switching models mid-session

Two ways to switch:

  • Type /model to see the list, then /model gemma4 to switch
  • Ask in plain English: “Switch to Gemma 4” — OpenClaw exposes a tool for this and will switch directly

✅ Best practices

  • Pick the biggest model your hardware can comfortably run — leave headroom for the OS
  • Always pick a tool-calling model for OpenClaw, not a chat-only one
  • Run hybrid: local default, cloud fallback. Best of both
  • Configure model preferences in CLAUDE.md/agent.md so OpenClaw knows when to use which
  • Use a VPS with a real Nvidia GPU if your local machine is underpowered
  • Keep Ollama updated — new models often need new runtime versions
  • Measure tokens/sec on your hardware before committing to a model size

❌ Common mistakes

  • Picking a model larger than your RAM/VRAM and wondering why it crawls
  • Reading system RAM on Windows when you should be reading VRAM
  • Choosing a chat-only model and getting tool-call failures in OpenClaw
  • Forgetting to run openclaw gateway restart after adding a model
  • Running everything locally when a 2× latency budget is fine for cloud anyway
  • Skipping the hybrid setup and giving up frontier capability when you need it
  • Forgetting to update Ollama and getting cryptic model-load failures

Conclusion

Running OpenClaw on a local model crossed from “cool demo” into “practical default” in early 2026. Gemma 4 and Qwen 3.6 made the tool-calling reliable; Apple silicon and consumer Nvidia cards made the hardware accessible. Spend an afternoon getting the right size of model running, wire it into OpenClaw, set a cloud fallback, and you have a stack that does most of your work for electricity prices and brings out frontier models only when they earn it.

Privacy and cost savings come along for free. The bigger win is just having the choice — you pick the model that fits the job instead of paying premium rates for everything.

Related reading

Local Models in 2026: Gemma 4, Ollama, and OpenClaw for Real Agent Work FAQ

Is Gemma 4 actually better than Llama 3.x?

For tool-calling and structured-output tasks at the same parameter count, generally yes. For pure chat quality, they’re close. Test both on your workload if it matters — both ship in Ollama and you can swap in minutes.

Can I run this on a VPS instead of locally?

Yes — provided the VPS has a real Nvidia GPU with enough VRAM. CPU-only VPS plans will run models but slowly. Hostinger, Runpod, and others rent GPU-equipped VMs for reasonable rates.

Why not use Ollama Cloud?

Ollama Cloud is fine for convenience, but it defeats the privacy and cost argument. If you’re going cloud, you might as well use OpenAI or Anthropic with stronger frontier capability for similar money.

How fast is Gemma 4 9B on a 32GB Mac?

Roughly 25–40 tokens/sec depending on M-series generation. Comfortably faster than reading speed, faster than most cloud APIs for short prompts.

Can the agent still use MCP servers?

Yes. MCP and tool-calling are protocol-level — they work with any model in OpenClaw as long as the model supports tool calls. That’s why “tool-calling” is the must-have spec.

What about Apple’s MLX or LM Studio instead of Ollama?

Both work and are sometimes faster on Apple silicon. Ollama’s edge is the OpenAI-compatible API and broad agent-tool integration. If you’re purely chatting, MLX is great; for OpenClaw, Ollama is the easiest path.

Do I still need an OpenAI / Anthropic key?

Not for the local-only setup — that’s the appeal. For hybrid, yes, you keep them as fallback. Plenty of users go fully local and never look back.