How to Run Local AI Models with OpenClaw

This guide will show how to use open LLMs locally with OpenClaw by connecting it to Unsloth. OpenClaw is an open-source AI agent interface that connects to a model to run tasks across your project. OpenClaw is able to works with any local model by connecting through Unsloth’s OpenAI-compatible API: including DeepSeek, Qwen, Gemma, and more.

OpenClaw acts as the client, while Unsloth loads and serves models via a local API.After setup, OpenClaw will run against your local model through Unsloth, letting you use it directly as an AI agent. In this tutorial, we'll use Qwen3.6.

Connecting to OpenClawQuickstart

Installing OpenClaw

⚡ Quickstart

After installing OpenClaw, we'll need to install Unsloth Desktop to enable OpenClaw to serve and run inference of local models.

1

Download Unsloth

The easiest way to get started is by installing the Unsloth Desktop app. It supports MacOS, Linux, Windows, NVIDIA, AMD, Intel and CPU setups.

Download Unsloth

• Download for macOS

• Download for Windows

• Download for Linux

Or, if you prefer manual installation:

MacOS, Linux, WSL:

Copy

curl -fsSL https://unsloth.ai/install.sh | sh

Windows PowerShell:

Copy

irm https://unsloth.ai/install.ps1 | iex

2

Install

1. Open the Unsloth installer (.dmg, .exe files)

2. Drag Unsloth to Applications for Mac or complete setup for Windows.

3. Launch the app and wait for installation to complete

3

Choose a model

Open 'Select model' dropdown ontop or 'Model hub' tab, choose a model and a quantization that fits your device, then download it. Once it finishes, start chatting - no setup required.

4

Unsloth is now ready

To start chatting, type a message and press Enter.

Connect OpenClaw. Run unsloth start openclaw to launch OpenClaw with the loaded Unsloth model in a separate managed environment. Your normal OpenClaw configuration is left unchanged.

⚙️ Launch OpenClaw with unsloth start

OpenClaw can connect to a model already running in Unsloth Studio, or start one automatically when Unsloth is not running.

Connect to a running Unsloth instance

Once a model is loaded in Unsloth Studio, run:

Unsloth launches OpenClaw in local TUI mode with the Unsloth provider, model, and context length configured inside a separate managed environment. Your normal OpenClaw setup is left unchanged.

Once OpenClaw opens, give it a task such as:

Start a model automatically

If Unsloth is not already running, pass a model ID:

To launch OpenClaw directly with a model, run:

Unsloth automatically selects the correct parameters to use but you can still change it.

Unsloth starts the model, launches OpenClaw, and stops the temporary server when you exit. To keep your OpenClaw state and return to the same session later, add --persist and choose a session key:

Continue later using the same session key:

See the complete unsloth start reference for named sessions, model loading, remote Unsloth servers, and advanced options.

The rest of this guide covers the manual OpenClaw provider setup.

Manual OpenClaw provider setup

🔑 Creating an API key

Keys are created from Unsloth → Settings → API Keys.

1. Open the sidebar, click your Unsloth avatar at the bottom-left.

2. Go to Settings → API Keys.

3. Enter a friendly name (e.g. claude-code-macbook).

4. (Optional) Set an expiry.

5. Click Create.

6. Copy the key immediately. Unsloth stores only a hash and you won't be able to view it again.

All keys start with the sk-unsloth- prefix. Revoke a key from the same page at any time. Requests made with a revoked key will fail with 401 Unauthorized.

Connecting to OpenClaw

OpenClaw reads its config from ~/.openclaw/openclaw.json. Add (or merge) a models block with a unsloth provider pointing at Unsloth's Anthropic Messages API.

Notes:

• baseUrl is the Unsloth origin with no path. OpenClaw talks to Unsloth over the Anthropic Messages API, and the Anthropic SDK appends /v1/messages itself, so do not add /v1 here (a trailing /v1 would send requests to /v1/v1/messages).

• api: "anthropic-messages" tells OpenClaw to talk to Unsloth's /v1/messages endpoint.

• authHeader: true sends your key as Authorization: Bearer ….

• Set each model's id and name to the name you chose when loading the model in Unsloth.

• If you're running Unsloth on a remote machine, replace localhost:8888 with that machine's address (e.g. http://10.0.0.42:8888).

Optional: configure model behavior

OpenClaw connects through the model running in Unsloth. Runtime settings can be configured when starting the server.

Use --reasoning off to turn thinking off, or --reasoning on to turn it on for models that support reasoning.

This starts the server on 0.0.0.0:8888, allowing other devices on your local network to connect.

For more advanced runtime configuration, see the main API tuning section.