Introduction
OpenClaw is a self-hosted AI assistant framework that runs on your own hardware. It gives you an AI agent that can manage your files, execute commands, browse the web, and integrate with your tools — all without relying on a cloud service. The appeal of running it locally is obvious: privacy, control, no subscription fees. But it’s also a pretty ambitious project, which is why I’m documenting the setup here. It might save you some time.
The Setup
I run OpenClaw inside a Docker container on a TrueNAS server. The container includes:
- A Node.js-based gateway (running on port 18789)
- A WebSocket proxy for the chat UI (port 8888)
- Access to local tools: filesystem, cron scheduling, browser automation, messaging
The Hardware
The server has an NVIDIA GeForce RTX 3090 with 24GB of VRAM, which I use for local LLM inference. The main assistant model runs through llama.cpp, and I use Ollama for embeddings and reranking. The 3090 is old enough that it’s affordable, but powerful enough that it handles most tasks well. It’s a good starting point for local AI.
The Challenge
The biggest issue I ran into early on was VRAM contention between the persistent llama-server and QMD embeddings. They can’t both run on GPU at the same time, and when they collide the server crashes. The workaround is straightforward: set QMD_FORCE_CPU=1 to run embeddings on CPU, or stop the server temporarily while embeddings need to run. It’s not elegant, but it works.
What Comes Next
This blog will document the ongoing setup and troubleshooting. Expect posts about WordPress integration, QMD search indexing, Home Assistant integration, and whatever technical rabbit holes I fall down along the way. If you’re running something similar, feel free to follow along or reach out with your own experience.
Leave a Reply