Deploying models to LM Studio

Saving models to GGUF so you can run and deploy them to LM Studio

You can run and deploy your fine-tuned LLM directly in LM Studio. LM Studio enables easy running and deployment of GGUF models (llama.cpp format).

You can use our LM Studio notebook or follow the instructions below:

Export your Unsloth fine-tuned model to .gguf
Import / download the GGUF into LM Studio
Load it in Chat (or run it behind an OpenAI-compatible local API)

1) Export to GGUF (from Unsloth)

If you already exported a .gguf, skip to Importing into LM Studio.

# Save locally (creates GGUF artifacts in the folder)
model.save_pretrained_gguf("my_model_gguf", tokenizer, quantization_method = "q4_k_m")
# model.save_pretrained_gguf("my_model_gguf", tokenizer, quantization_method = "q8_0")
# model.save_pretrained_gguf("my_model_gguf", tokenizer, quantization_method = "f16")

# Or push GGUF to the Hugging Face Hub
model.push_to_hub_gguf("hf_username/my_model_gguf", tokenizer, quantization_method = "q4_k_m")

q4_k_m is usually the default for local runs.

q8_0 is the optimum for near full precision quality.

f16 is largest / slowest, but original unquantized precision.

2) Import the GGUF into LM Studio

LM Studio provides a CLI called lms that can import a local .gguf into LM Studio’s models folder.

Import a GGUF file:

lms import /path/to/model.gguf

Keep the original file (copy instead of move):

lms import /path/to/model.gguf --copy

Click for more customizable private settings

Keep the model where it is (symlink):

This is helpful for large models stored on a dedicated drive.

lms import /path/to/model.gguf --symbolic-link

Skip prompts and choose the target namespace yourself:

lms import /path/to/model.gguf --user-repo my-user/my-finetuned-models

Dry-run (shows what will happen):

lms import /path/to/model.gguf --dry-run

After importing, the model should appear in LM Studio under My Models.

If you pushed your GGUF repo to Hugging Face, you can download it directly from within LM Studio.

Option A: Use LM Studio’s in-app downloader

Open LM Studio
Go to the Discover tab
Search for hf_username/repo_name (or paste the Hugging Face URL)
Download the quant you want (e.g. Q4_K_M)

Option B: Use the CLI downloader

# Download from HF by repo name
lms get hf_username/my_model_gguf

# Pick a quantization with @
lms get hf_username/my_model_gguf@Q4_K_M

If you don’t want to use the CLI, you can place the .gguf file into LM Studio’s expected model directory structure.

LM Studio expects models to look like this:

~/.lmstudio/models/
└── publisher/
    └── model/
        └── model-file.gguf

Example:

~/.lmstudio/models/
└── my-name/
    └── my-finetune/
        └── my-finetune-Q4_K_M.gguf

Then open LM Studio and check My Models.

Tip: You can manage / verify your models directory from the My Models tab in LM Studio.

3) Load and chat in LM Studio

Open LM Studio → Chat
Open the model loader
Select your imported model
(Optional) adjust load settings (GPU offload, context length, etc.)
Chat normally in the UI

4) Serve your fine-tuned model as a local API (OpenAI-compatible)

LM Studio can serve your loaded model behind an OpenAI-compatible API (handy for apps like Open WebUI, custom agents, scripts, etc.).

Load your model in LM Studio
Go to the Developer tab
Start the local server
Use the shown base URL (default is typically http://localhost:1234/v1)

1) List available models

lms ls

2) Load your model (optional flags)

lms load <model-identifier> --gpu=auto --context-length=8192

Notes:

--gpu=1.0 means “try to offload 100% to GPU”
You can set a stable identifier:

lms load <model-identifier> --identifier="my-finetuned-model"

3) Start the server

lms server start --port 1234

Quick test: list models

curl http://localhost:1234/v1/models

Python example (OpenAI SDK):

from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:1234/v1",
    api_key="lm-studio",  # LM Studio may not require a real key; this is a common placeholder
)

resp = client.chat.completions.create(
    model="model-identifier-from-lm-studio",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Hello! What did I fine-tune you to do?"},
    ],
    temperature=0.7, # adjust temperature according to your model needs
)

print(resp.choices[0].message.content)

cURL example (chat completions):

curl http://localhost:1234/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "model-identifier-from-lm-studio",
    "messages": [
      {"role": "user", "content": "Say this is a test!"}
    ],
    "temperature": 0.7 # adjust temperature according to your model needs
  }'

Debugging tip: If you’re troubleshooting formatting/templates, you can inspect the raw prompt LM Studio sends to the model by running: lms log stream

Troubleshooting

Model runs in Unsloth, but LM Studio output is gibberish / repeats

This is almost always a prompt template / chat template mismatch.

LM Studio will auto-detect the prompt template from the GGUF metadata when possible, but custom or incorrectly-tagged models may need a manual override.

Fix:

Go to My Models → click the gear ⚙️ next to your model
Find Prompt Template and set it to match the template you trained with
Alternatively, in the Chat sidebar: enable the Prompt Template box (you can force it to always show)

LM Studio doesn’t show my model in “My Models”

Prefer lms import /path/to/model.gguf
Or confirm the file is in the correct folder structure: ~/.lmstudio/models/publisher/model/model-file.gguf

OOM / slow performance

Use a smaller quant (ex: Q4_K_M)
Reduce context length
Adjust GPU offload (LM Studio “Per-model defaults” / load settings)

More resources

LM Studio + Unsloth blog post (FunctionGemma walkthrough):
LM Studio Import Models docs
LM Studio Prompt Template docs
LM Studio OpenAI-compatible API docs

Last updated 4 months ago

Was this helpful?