How to Run Local AI Models in VS Code
How to Run Local LLMs with VS Code
This guide shows you how to connect a local model running in Unsloth to VS Code’s Copilot Chat. Unsloth loads and serves the model, while VS Code provides the chat interface and coding tools.
You can chat about your code, ask questions, and use Agent mode to create and edit files with a model that supports tool calling.
Install VS Code
Download VS Code
Open the VS Code download page and download the macOS version for your Mac. Choose Apple silicon, Intel, or Universal if you’re unsure.
Download VS Code
Open the VS Code download page and download the Windows User Installer for your device. Most PCs use x64.
Download VS Code
Open the VS Code download page and choose the package for your distribution:
Ubuntu or Debian:
.debFedora or RHEL:
.rpm
⚡ Quickstart
After installing VS Code, we'll need to install Unsloth Desktop to enable VS Code to serve and run inference of local models.
Download Unsloth
The easiest way to get started is by installing the Unsloth Desktop app. It supports MacOS, Linux, Windows, NVIDIA, AMD, Intel and CPU setups.
Or, if you prefer manual installation:
MacOS, Linux, WSL:
curl -fsSL https://unsloth.ai/install.sh | shWindows:
irm https://unsloth.ai/install.ps1 | iexThe Unsloth Docker image unsloth/unsloth is available on Docker. Read guide.
Install Unsloth Desktop:
brew install --cask unslothUpdate Unsloth Desktop:
brew upgrade --cask unslothInstall Unsloth Desktop:
winget install --id UnslothAI.Unsloth -eUpdate Unsloth Desktop:
winget upgrade --id UnslothAI.Unsloth -e🔌Connect your model
With a model loaded in Unsloth, open a terminal and run:
unsloth start vscodeThis adds the model Unsloth is currently serving to VS Code’s model picker. It configures the connection; open VS Code separately if it isn’t already running.
You can also load a model and configure VS Code in one command:
unsloth start vscode --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL --context-length 32768Prefer another model? Replace the model ID with your choice. You can also pass a local GGUF file:
unsloth start vscode --model "/path/to/model.gguf" --context-length 32768The command sets up the connection automatically, keeps your other models, and backs up your VS Code model settings.
It also creates an API key named VS Code. To revoke it, open Settings in Unsloth and select API.
Use your local model in VS Code
Open a project folder in VS Code, or an empty folder to try it out.
Open Copilot Chat, click the model picker below the chat input, and select your model under Unsloth. Choose Agent to let it create and edit files, or Ask to chat about your code.

Agent mode requires tool-calling support. We recommend at least 32K context.
Let’s build a simple game. Try this prompt:
Review any tool permissions before approving them. Once the agent creates index.html, open it in your browser and play.
Want to change something? Ask it to add difficulty levels or a high-score tracker.
Keep your model loaded and the Unsloth server running while you work.

If your model doesn’t appear, check Manage Models in the model picker. You can also refresh VS Code by running Developer: Reload Window from the Command Palette.
Switch models
Load a different model in Unsloth, then run:
Select the new model under Unsloth in Copilot Chat.
Troubleshooting
My model doesn’t appear
Open Manage Models in the model picker and look for Unsloth. If it’s missing, check that GitHub Copilot Chat is enabled, then run unsloth start vscode again.
If it still doesn’t appear, open the Command Palette and run Developer: Reload Window.
My model appears in Ask but not Agent mode
Agent mode requires tool-calling support. Use Ask with this model, or switch to one whose chat template supports tools.
I’m getting context-length errors
Agent instructions and tools take up part of the model’s context window. We recommend at least 32K context for Agent mode.
Replace the path below with your GGUF file and run:
Reload VS Code if the model details haven’t updated. It may display 32,768 tokens as 33K.
Unsloth is running on a remote machine
In an SSH or container session without a VS Code user directory, the command prints the configuration to add locally and connection instructions.
Follow the printed instructions to make the server reachable from your computer, then add the configuration to your local VS Code installation.
If the model doesn’t appear in a Remote-SSH, WSL, or dev container window, try a local VS Code window connected to the remote Unsloth server.
For more model-loading options and remote connection settings, see the Unsloth Start guide.
Last updated
Was this helpful?



