Run Coding Agents with Local LLMs using Unsloth Start
Unsloth lets you connect Claude Code, Codex, Hermes, OpenCode, Pi, and other coding agents to a local model via the unsloth start command. The entire workflow can run offline on your own hardware. Unsloth automatically configures the endpoint, API key, provider, model, and context length for each launch, so you can use your preferred agent without modifying them. This guide will show you how to launch models from the command line for offline use, and connect to Unsloth.
Quickstart
First, make sure you have Unsloth installed. Then open Unsloth, load a model, go to your project folder, and run the command in terminal:
unsloth start claudeYou can replace claude with any agent below:
Claude Code
unsloth start claude
OpenAI Codex
unsloth start codex
Hermes Agent
unsloth start hermes
OpenClaw
unsloth start openclaw
OpenCode
unsloth start opencode
Unsloth uses temporary or session-scoped provider configuration. It does not add an Unsloth provider to the agent's normal configuration files.
Codex currently requires a GGUF model served through the llama-server backend.
Agent Guides
Load a model from the command line
You can select and load a model while launching the agent:
The :UD-Q4_K_XL suffix selects the GGUF quant.
An explicit --gguf-variant overrides the quant written after the model name.
On the default local address, passing --model lets unsloth start start a temporary server when Unsloth Studio is not already running. The temporary server stops when the agent exits. If Unsloth is already running, the command connects to it and leaves it running.
See below for examples of agents being connected to a local LLM:
Connect to a remote Unsloth server
Set the Unsloth URL and API key before launching an agent:
You can also pass the key with --api-key. For a verified local Unsloth server, unsloth start creates or reuses the API key automatically.
Options
--model, -m
Select a model. Without it, use the first model reported by Unsloth.
--api-key
Supply a Unsloth API key. You can also set UNSLOTH_API_KEY.
--launch / --no-launch
Launch the agent or print the generated environment and command.
--serve / --no-serve
Allow or prevent automatic local server startup.
--gguf-variant
Select a GGUF quantization variant.
--context-length, --max-seq-length
Set the requested context length when loading the model.
--load-in-4bit / --no-load-in-4bit
Control 4-bit loading for non-GGUF Hugging Face models.
--tensor-parallel / --no-tensor-parallel
Enable or disable tensor-parallel GGUF loading for multi-GPU systems.
--persist / --no-persist
Keep the Unsloth-managed agent home where applicable.
--yolo
Use the selected agent's non-prompting or trust mode.
-h, --help
Show command help.
Load options are used when Unsloth needs to load or reconcile the requested model.
Pass normal commands to the agent
Arguments that are not Unsloth options are passed to the selected agent:
Use the agent's own help command for its complete list of native options.
Sessions and --persist
--persist keeps managed storage. It does not resume a conversation by itself; also pass the agent's normal resume command.
Agent
Do you need --persist?
Resume example
Claude Code
No. Claude uses its normal session store.
unsloth start claude --continue
OpenAI Codex
Yes. Its managed Codex home is temporary by default.
unsloth start codex --persist resume --last
OpenClaw
Yes. Managed config, workspace and sessions are temporary by default.
Use --persist with the same native session ID.
OpenCode
No. OpenCode uses its normal session store.
unsloth start opencode run --continue "Continue"
Hermes Agent
Yes. Its managed Hermes home is temporary by default.
unsloth start hermes --persist --continue --oneshot "Continue"
Pi Coding Agent
Yes. Its managed Pi home is temporary by default.
unsloth start pi --persist --continue
For Codex, OpenClaw, Hermes and Pi, use --persist from the first launch and again when returning to the session.
Use a stable session ID for a named session:
Print the launch command without running it
This prints the generated environment and command. The output can contain connection credentials, so do not publish it in logs or screenshots.
Permission-bypass mode
--yolo maps to the selected agent's trust or non-prompting mode. It can reduce approval prompts and allow the agent to run commands without asking first.
Only use it in an environment where unrestricted agent actions are acceptable.
Common issues
No Unsloth server was found
Start Unsloth and load a model, or pass `--model` so Unsloth can start a temporary local server.
Last updated
Was this helpful?