Skip to content
← Projects
SOFTWAREAI

The Ultimate OpenClaw Setup: Pi Orchestration + Local AI Offloading

1 March 2026

There is a massive trend right now in the self-hosted AI community: "Buy an M4 Mac Mini to run your local LLMs and agent frameworks."

Apple's Unified Memory architecture is great if you only own a laptop and need 64GB of VRAM to run massive 70B parameter models. But for the vast majority of developers, buying a $2,000 Mac Mini just to run agents is absolute overkill.

If you aren't strictly mandated to run local models, and you simply run your agents on a fresh, sandboxed Raspberry Pi that only has access to the specific files it needs, you are almost always better off using vastly superior paid cloud models (like Claude 3.5 Sonnet or Gemini Pro).

However, if you do require absolute privacy for sensitive data parsing, and you already own a high-end Windows PC with a dedicated NVIDIA GPU (like an RTX 4080 Super or RTX 4090), you already own a vastly superior inference engine for the 7B to 32B class of models. NVIDIA's CUDA cores will crush Apple Silicon in raw generation speed every single time.

The only problem? You don't want to leave your 850W gaming PC running 24/7 just to answer a basic Telegram message.

The solution is an Orchestrated Offloading Architecture. Here is exactly how I built a robust, secure, and cost-effective OpenClaw ecosystem that runs 24/7 on a Raspberry Pi 5, but dynamically taps into my massive PC GPU when serious localized AI processing is required. (Note: To see how I keep my power bill low by remotely cold-booting my PC only when I need it, check out my PC Remote Control Build).

🏗️ The Architecture

1. The Orchestration Brain (Raspberry Pi 5)

Instead of running OpenClaw directly on Windows, the entire agent framework (OpenClaw Gateway, memory databases, Telegram API connections, and chron jobs) runs on a Raspberry Pi 5 (4GB).

  • Cost: ~$150 AUD.
  • Power Draw: ~5 Watts.
  • Uptime: 24/7/365.

This means my Telegram bots (like my main assistant, Ronnie, and my Bambeso dropshipping agent, Mia) are always online, always listening, and always able to run lightweight Python scripts (like checking emails or scraping web data) without waking up the main PC.

2. The Storage Layer (NVMe SSD Hat)

Agents generate a massive amount of vector database reads/writes for long-term memory. Running OpenClaw on a standard MicroSD card will inevitably lead to database corruption or a dead SD card within months. I installed a PCIe NVMe SSD Hat on the Pi 5, bypassing the SD card entirely. The SQLite databases, embedding vectors, and workspace files all live on ultra-fast, highly durable enterprise flash storage.

3. The Heavy Lifter (Windows + RTX 4080 Super)

When I need to run private, hyper-sensitive data (like medical parsing for the Matavex project) or when I want to use completely free intelligence, I don't use paid cloud APIs.

I run Ollama locally on my Windows machine (via WSL2). I host models like qwen3.5:27b and gemma4:26b.

The magic happens in the .openclaw.json configuration file on the Pi. I added a custom ollama provider, pointing the Pi directly to my Windows PC's IP address:

"providers": {
  "ollama": {
    "baseUrl": "http://10.191.30.82:11434/v1",
    "apiKey": "ollama-local",
    "api": "ollama"
  }
}

Now, when the Pi needs to run a local model, it fires the API request across the network to the sleeping PC. (I use a custom hardware relay to boot the PC remotely if it's turned off).

🔒 Network Security & Hardening

Exposing local LLM APIs across a home network, especially when building web scrapers and execution agents, requires strict security boundaries.

1. The IoT VLAN Isolation

The Pi orchestration server is isolated. It can communicate with the necessary web APIs (Telegram, Shopify), but its inbound access is strictly locked down.

2. Ollama Firewall Pinning

By default, Ollama only listens to localhost (127.0.0.1) for security reasons. To allow the Pi to access it without opening my PC to the entire network:

  1. I set the Windows environment variable OLLAMA_HOST=0.0.0.0 to allow external connections.
  2. Crucially, I immediately configured the Windows Firewall to block all inbound traffic on port 11434, except for the exact, static IP address of my Raspberry Pi. No other device on the network can access the models.

3. OpenClaw Sandbox & Permissions

OpenClaw is immensely powerful. To prevent a hallucinating local model from deleting critical files:

  • Every agent has its own isolated workspace folder.
  • The global openclaw.json config is locked to strict 600 Linux permissions (chmod 600), meaning no rogue script or sub-agent can modify the master configuration or steal API keys.
  • If I assign a small, experimental local model (like an 8B model) to an agent, I explicitly strip its web access (group:web) or force "sandbox": { "mode": "all" } in its config to prevent unpredictable behavior.

🧠 True Long-Term Memory (RAG Tuning)

OpenClaw natively supports long-term memory retrieval, but out of the box, it relies on standard text search. To give my agents "Iron Mountain" recall spanning months of conversations, I fundamentally altered the memory storage mechanism.

Instead of paying Google or OpenAI to generate embedding vectors for every single conversation (which is expensive and terrible for privacy), I shifted the embedding workload entirely to my local PC.

In the OpenClaw configuration, I instructed the system to use a specialized, lightweight embedding model (nomic-embed-text) running on my local Ollama server:

"memorySearch": {
  "provider": "openai",
  "model": "nomic-embed-text",
  "baseUrl": "http://10.191.30.82:11434/v1/",
  "apiKey": "ollama"
}

I also enabled MMR re-ranking (lambda 0.7) and Temporal Decay (30 days). This means when I ask the agent a question, it doesn't just pull the most "keyword-relevant" memory; it mathematically balances relevance against recency, ensuring the agent prioritizes things we discussed yesterday over things we discussed three months ago, while keeping all data completely private and free.

🚀 The Result

I built a highly secure, completely private, 24/7 AI orchestration platform for about $250 in Raspberry Pi parts. It uses practically zero electricity while idle, but instantly wields the absolute raw power of a massive NVIDIA GPU the second it needs to do heavy lifting.

You don't need a Mac Mini. You just need to orchestrate the hardware you already have.