For the complete documentation index, see llms.txt. This page is also available as Markdown.

How to Run Local AI Models in VS Code

How to Run Local LLMs with VS Code

This guide shows you how to connect a local model running in Unsloth to VS Code’s Copilot Chat. Unsloth loads and serves the model, while VS Code provides the chat interface and coding tools.

You can chat about your code, ask questions, and use Agent mode to create and edit files with a model that supports tool calling.

Install VS Code

1

Download VS Code

Open the VS Code download page and download the macOS version for your Mac. Choose Apple silicon, Intel, or Universal if you’re unsure.

2

Setup VS Code

Open the downloaded .zip file, then drag Visual Studio Code.app into your Applications folder.

Then to open, go to Applications and double-click Visual Studio Code.

1

Download VS Code

Open the VS Code download page and download the Windows User Installer for your device. Most PCs use x64.

2

Setup VS Code

Open the downloaded .exe file and follow the installer prompts.

Then to open, go to the Start menu, search for Visual Studio Code, and launch it.

If you had a terminal open during installation, close and reopen it to pick up the updated PATH.

1

Download VS Code

Open the VS Code download page and choose the package for your distribution:

  • Ubuntu or Debian: .deb

  • Fedora or RHEL: .rpm

2

Setup VS Code

Open a terminal in the folder containing your download and run:

Ubuntu or Debian:

sudo apt install ./code_*.deb

Fedora or RHEL:

sudo dnf install ./code-*.rpm

Then to open, go to your application launcher, search for Visual Studio Code, and launch it.

⚡ Quickstart

After installing VS Code, we'll need to install Unsloth Desktop to enable VS Code to serve and run inference of local models.

1

Download Unsloth

The easiest way to get started is by installing the Unsloth Desktop app. It supports MacOS, Linux, Windows, NVIDIA, AMD, Intel and CPU setups.

Download Unsloth

Or, if you prefer manual installation:

MacOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows:

irm https://unsloth.ai/install.ps1 | iex

The Unsloth Docker image unsloth/unsloth is available on Docker. Read guide.

Install Unsloth Desktop:

brew install --cask unsloth

Update Unsloth Desktop:

brew upgrade --cask unsloth

Install Unsloth Desktop:

winget install --id UnslothAI.Unsloth -e

Update Unsloth Desktop:

winget upgrade --id UnslothAI.Unsloth -e
2

Install

  1. Open the Unsloth installer (.dmg, .exe files)

  2. Drag Unsloth to Applications for Mac or complete setup for Windows.

  3. Launch the app and wait for installation to complete

3

Choose a model

Open 'Select model' dropdown ontop or 'Model hub' tab, choose a model and a quantization that fits your device, then download it. Once it finishes, start chatting - no setup required.

4

Unsloth is now ready

With your model loaded in Unsloth, run this in your terminal:

unsloth start vscode

Open Copilot Chat in VS Code and select your model under Unsloth in the model picker. Send a message to check the connection.

Keep Unsloth running while you use the model.

🔌Connect your model

With a model loaded in Unsloth, open a terminal and run:

unsloth start vscode

This adds the model Unsloth is currently serving to VS Code’s model picker. It configures the connection; open VS Code separately if it isn’t already running.

You can also load a model and configure VS Code in one command:

unsloth start vscode --model unsloth/Qwen3.8-27B-GGUF:UD-Q4_K_XL --context-length 32768

Prefer another model? Replace the model ID with your choice. You can also pass a local GGUF file:

unsloth start vscode --model "/path/to/model.gguf" --context-length 32768

The command sets up the connection automatically, keeps your other models, and backs up your VS Code model settings.

It also creates an API key named VS Code. To revoke it, open Settings in Unsloth and select API.

Use your local model in VS Code

Open a project folder in VS Code, or an empty folder to try it out.

Open Copilot Chat, click the model picker below the chat input, and select your model under Unsloth. Choose Agent to let it create and edit files, or Ask to chat about your code.

Agent mode requires tool-calling support. We recommend at least 32K context.

Let’s build a simple game. Try this prompt:

Review any tool permissions before approving them. Once the agent creates index.html, open it in your browser and play.

Want to change something? Ask it to add difficulty levels or a high-score tracker.

Keep your model loaded and the Unsloth server running while you work.

If your model doesn’t appear, check Manage Models in the model picker. You can also refresh VS Code by running Developer: Reload Window from the Command Palette.

Switch models

Load a different model in Unsloth, then run:

Select the new model under Unsloth in Copilot Chat.

Troubleshooting

My model doesn’t appear

Open Manage Models in the model picker and look for Unsloth. If it’s missing, check that GitHub Copilot Chat is enabled, then run unsloth start vscode again.

If it still doesn’t appear, open the Command Palette and run Developer: Reload Window.

My model appears in Ask but not Agent mode

Agent mode requires tool-calling support. Use Ask with this model, or switch to one whose chat template supports tools.

I’m getting context-length errors

Agent instructions and tools take up part of the model’s context window. We recommend at least 32K context for Agent mode.

Replace the path below with your GGUF file and run:

Reload VS Code if the model details haven’t updated. It may display 32,768 tokens as 33K.

Unsloth is running on a remote machine

In an SSH or container session without a VS Code user directory, the command prints the configuration to add locally and connection instructions.

Follow the printed instructions to make the server reachable from your computer, then add the configuration to your local VS Code installation.

If the model doesn’t appear in a Remote-SSH, WSL, or dev container window, try a local VS Code window connected to the remote Unsloth server.

For more model-loading options and remote connection settings, see the Unsloth Start guide.

Last updated

Was this helpful?