Connect API Providers & Model Servers to Unsloth
Learn how to run models from OpenAI, Anthropic, Ollama, llama.cpp, vLLM, and other providers through a single local UI interface with Unsloth, an open-source repo for running and training LLMs. You can also connect your ChatGPT/Codex subscription directly to Unsloth.
Whether your model is running through Unsloth’s inference or your own remote OpenAI-compatible endpoint, you can give it access to Unsloth’s full suite of tools, including web search, code execution, deep research, and more.
Once connected, you can run models with code execution, tool-calling, image generation, and other features in the same Unsloth chat interface used for both local and cloud models.
Unsloth uniquely supports prompt caching (to save you many tokens without accuracy degradation) while preserving access to provider-native capabilities, such as OpenAI’s built-in web search and code execution.
Connections
Connections fall into two groups: hosted API providers that run models for you, and model servers that you run or control.
Cloud Providers - Hosted APIs that use an account API key:
Model Servers - Inference servers running locally, on your network, or on your remote machine:
Quickstart
To run an external provider's model, add an API key and select which models Unsloth should show. In this example, we’ll use OpenAI. The same setup works for Anthropic, and other providers.
Setup Unsloth
The easiest way to get started is by installing the Unsloth Desktop app. It supports MacOS, Linux, Windows, NVIDIA, AMD, Intel and CPU setups.
Or, if you prefer manual installation:
MacOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | shWindows PowerShell:
irm https://unsloth.ai/install.ps1 | iexSee here for more detailed instructions
Configure Connections
Next, connect your provider to Unsloth.
Open Settings → Connections, then click Add Connection.
Select the provider you want to add, then paste the API key you copied earlier.
Click Reload Models to refresh the list with models available to your account.
Choose the models you want to enable, then hit save.
Connect a Model Server
Use this flow for llama.cpp, vLLM, and Ollama. Start or locate the server you want to connect.
Start llama-server with the model you want to serve:
This exposes an API endpoint at: http://localhost:8080/v1
To require an API key, add:
Start the vLLM server with the model you want to serve:
To require an API key, add:
This exposes an API endpoint at: http://localhost:8000/v1
Start Ollama, then pull the model you want to use:
This exposes an API endpoint at: http://localhost:11434/v1
Now we can connect the model server. Open Settings → Connections, then click Add Provider.
Select llama.cpp, vLLM, or Ollama then Paste the server Base URL.
llama.cpp example:
http://localhost:8080/v1Ollama example:
http://localhost:11434/v1
Click Load Models to fetch available model IDs, or enter model IDs manually if your server does not expose /models.
Then, after you click Add Provider, The models you enabled will now appear under External in the Select Model dropdown.
Code Execution
When enabled, supported OpenAI and Anthropic models can run code in a provider sandbox to solve problems, analyse data, and work with files. Anthropic models use Claude’s provider-side Code execution tool. OpenAI uses reusable containers, which you can create, delete, and select from Code Execution settings. Select the same container in a new thread to continue with its files and state.
Prompt Caching
Prompt caching reduces latency and cost when requests reuse the same long prefix. It is supported for compatible providers and servers, including OpenAI, Anthropic, and llama.cpp. Use the Prompt caching setting in the side panel to control caching behaviour for supported connections.
For llama.cpp, prompt caching is enabled by default and can be disabled when starting llama-server with:
Web Search & Thinking
Provider-side web search is available for supported models from OpenAI, Anthropic, OpenRouter, Mistral, Gemini, and Kimi. The Think control adapts to the selected model: some models use an on/off toggle, while reasoning-effort models use model specific thinking levels.
Image Generation
Just like GPT and Gemini, Unsloth also supports image generation. You can directly edit an image by clicking the “Edit Image” button and entering a new prompt to refine or regenerate it. Images are generated automatically when requested, but you can toggle this behavior off. A download button is also available, allowing you to save the image in its original full resolution.
Troubleshooting
If a provider fails to connect, check that the API key belongs to the selected provider and has access to the model you chose.
If a model does not appear after clicking Reload Models, it may not be available for your account. You can still use Unsloth’s default model list or choose another model.
Last updated
Was this helpful?