How to Run Local AI Models with OpenClaw

Guide to running local LLMs with OpenClaw.

This guide will enable you to use open LLMs locally with OpenClaw by connecting it to Unsloth. OpenClaw is an open-source AI agent interface that connects to a model to run tasks across your project.

OpenClaw is able to works with any local model by connecting through Unsloth’s OpenAI-compatible API: including DeepSeek, Qwen, Gemma, and more. OpenClaw acts as the client, while Unsloth loads and serves models via a local API.

After setup, OpenClaw will run against your local model through Unsloth, letting you use it directly as an AI agent.

Connecting to OpenClawQuickstart

In this tutorial, we’ll use unsloth/Qwen3.6-27B-GGUF in Unsloth and access it through OpenClaw. Prefer a different model? Swap in any other model by loading it in Unsloth and updating the configuration.

Installing OpenClaw

macOS, Linux, WSL

Install OpenClaw using the official installer:

curl -fsSL https://openclaw.ai/install.sh | bash

This sets up OpenClaw and guides you through initial setup.

Windows (PowerShell)

⚡ Quickstart

After installing OpenClaw, we'll need to install Unsloth Studio to enable OpenClaw to serve and run inference of local models.

1. Install or update Unsloth Studio. Earlier versions don't expose the external API. See Installation.

2. Launch Unsloth. Note the port it starts on is usually 8000 or 8888. You'll see it in the terminal output and in the browser URL (http://localhost:PORT).

3. Load a model. Click New Chat, pick or search a model (GGUF), and wait for it to finish loading.

4. Connect OpenClaw. Run unsloth start openclaw. It mints an API key, writes the config, and launches OpenClaw against your loaded model.

⚡ Connect with unsloth start

The fastest way to point OpenClaw at your local model is the unsloth start command. With Unsloth running and a model loaded, run:

unsloth start openclaw

This mints an API key for you, writes the unsloth provider into ~/.openclaw/openclaw.json, sets it as the default model, and launches OpenClaw. You don't need to create a key or edit any config by hand.

OpenClaw uses a separate gateway daemon. If it isn't already running, start it in another terminal with openclaw gateway, then run the connect command again.

By default it uses the model already loaded in Unsloth. To load and use a specific model, pass --model:

unsloth start openclaw --model unsloth/Qwen3.6-27B-GGUF

Connecting to Unsloth on another machine? Create a key (below) and pass it with --api-key, then point UNSLOTH_STUDIO_URL at the server. The steps below set the same thing up manually, if you'd rather not use unsloth start or need a custom config.

🔑 Creating an API key

Keys are created from Unsloth → Settings → API Keys.

1. Open the sidebar, click your Unsloth avatar at the bottom-left.

2. Go to Settings → API Keys.

3. Enter a friendly name (e.g. claude-code-macbook).

4. (Optional) Set an expiry.

5. Click Create.

6. Copy the key immediately. Unsloth stores only a hash and you won't be able to view it again.

All keys start with the sk-unsloth- prefix. Revoke a key from the same page at any time. Requests made with a revoked key will fail with 401 Unauthorized.

Connecting to OpenClaw

OpenClaw reads its config from ~/.openclaw/openclaw.json. Add (or merge) a models block with a unsloth provider pointing at Unsloth's Anthropic Messages API.

~/.openclaw/openclaw.json

{
  "models": {
    "mode": "merge",
    "providers": {
      "unsloth": {
        "baseUrl": "http://localhost:8888",
        "apiKey": "sk-unsloth-xxxxxxxxxxxx",
        "api": "anthropic-messages",
        "models": [
          {
            "id": "unsloth/Qwen3.6-27B-GGUF",
            "name": "unsloth/Qwen3.6-27B-GGUF"
          }
        ],
        "authHeader": true
      }
    }
  }
}

Notes:

• baseUrl is the Studio origin with no path. OpenClaw talks to Studio over the Anthropic Messages API, and the Anthropic SDK appends /v1/messages itself, so do not add /v1 here (a trailing /v1 would send requests to /v1/v1/messages).

• api: "anthropic-messages" tells OpenClaw to talk to Unsloth's /v1/messages endpoint.

• authHeader: true sends your key as Authorization: Bearer ….

• Set each model's id and name to the name you chose when loading the model in Unsloth.

• If you're running Unsloth on a remote machine, replace localhost:8888 with that machine's address (e.g. http://10.0.0.42:8888).

Optional: configure model behavior

OpenClaw connects through the model running in Unsloth. Runtime settings can be configured when starting the server.

# Configure default generation behavior (--disable-tools passes OpenClaw's own tools through)
unsloth run \
  --model unsloth/gemma-4-26B-A4B-it-GGUF \
  --disable-tools \
  --reasoning off \
  --temp 0.6

Use --disable-tools when driving OpenClaw (or any external coding agent). By default Unsloth Studio runs its own server-side tools, which swallows the agent's tool calls, so OpenClaw answers but never edits files. --disable-tools switches to passthrough, so OpenClaw's own tools are used.

Use --reasoning off to turn thinking off, or --reasoning on to turn it on for models that support reasoning.

# Allow connections from other devices
unsloth run \
  --model unsloth/gemma-4-26B-A4B-it-GGUF \
  -H 0.0.0.0 \
  -p 8888

This starts the server on 0.0.0.0:8888, allowing other devices on your local network to connect.

For more advanced runtime configuration, see the main API tuning section.