Muse Glimmer - How to Run Locally

Learn how to run the new Muse Glimmer 30B model from Meta.

Muse Glimmer is Meta’s new open-weight 30B parameter dense vision model, designed for local agentic and coding workflows. It is the first open model from Meta Superintelligence Labs, and is released under the Apache 2.0 license. This guide covers running Muse Glimmer 30B with Unsloth Dynamic quants for maximum performance.

Muse Glimmer 30B runs locally on 18GB RAM/VRAM setups, including Mac and GPU/CPU systems. You can run or fine-tune Muse Glimmer with Unsloth. We collaborated with Meta and Hugging Face on llama.cpp inference implementation. Thank you Meta for providing Unsloth with day-zero support.

Run Muse Glimmer Tutorials Fine-tune Muse Glimmer

Usage Guide

Muse Glimmer can plan multi-step tasks, execute sequential tool calls, recover from failures, adapt as conditions change, use runtime memory, and resume work across long-running sessions when state is persisted. This persistence comes from the agent harness, not the model. With vision support, Muse Glimmer is ideal for multimodal workflows.

Hardware requirements

Muse Glimmer 30B uses about 58 GB for full precision BF16 weights. Unsloth Dynamic quants tries to recover as much accuracy as possible through quantization allowing Muse Glimmer 30B to fix on smaller devices like RTX 5090s and so on.

Table: Muse Glimmer 30B Inference GGUF recommended hardware requirements (units = total memory: RAM + VRAM, or unified memory).

2-bit

3-bit

4-bit

6-bit

8-bit

12-14 GB

14-15 GB

17 GB

20-22 GB

34 GB

Detailed requirements table:

Quantization

Recommended RAM/VRAM:

Hardware examples

2-bit (UD-Q2_K_XL)

12-14+ GB

RTX 4080

3-bit (UD-Q3_K_XL)

14-15 GB+

RTX 4090

4-bit (UD-Q4_K_XL, NVFP4)

17 GB+

Mac 32GB

6-bit (UD-Q6_K_XL)

20-22 GB+

RTX 5090, Mac 48GB

8-bit (UD-Q8_K_XL)

34 GB+

Mac 128GB, DGX Spark

BF16 (full precision)

58 GB+

Mac 128GB, DGX Spark

As a rule of thumb, your total available memory should at least exceed the size of the quantized model you download. If it does not, llama.cpp can still run using partial RAM / disk offload, but generation will be slower. You will also need more compute, depending on the context window you use.

Recommended Settings

It is recommended to use Meta's default Muse Glimmer parameters:

temperature = 1.0
top_p = 0.95
top_k = 64

Maximum context length: 131,072 (default) up to 262,144

Thinking Settings

Muse Glimmer supports controllable reasoning efforts including:

low
medium
high
xhigh

Run Muse Glimmer Tutorials

Because quantized Muse Glimmer 30B comes in several sizes, the recommended starting point for the models is Dynamic 4-bit (UD . Muse Glimmer 30B GGUFs.

🦥 Unsloth Guide 🦙 Llama.cpp Guide

🦥 Unsloth Guide

Muse Glimmer 30B can now be run and fine-tuned in Unsloth Desktop, our new open-source desktop app for local AI. Unsloth lets you run models locally on MacOS, Windows, Linux and:

Search, download, run GGUFs and safetensor models
Self-healing tool calling + web search
Code execution (Python, Bash)
Automatic inference parameter tuning (temp, top-p, etc.)
Fast CPU + GPU inference via llama.cpp
Train LLMs 2x faster with 70% less VRAM

Install Unsloth

Unsloth runs on macOS, Windows, and Linux.

Download Unsloth

Or, for manual installation:

MacOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows PowerShell:

irm https://unsloth.ai/install.ps1 | iex

Launch Unsloth and Search and download Muse Glimmer

On first launch go to the Model hub tab and search for Muse Glimmer in the search bar and download your desired model and quant.

Run Muse Glimmer

Inference parameters should be auto-set when using Unsloth Desktop, however you can still change it manually. You can also edit the context length, chat template and other settings. You can run GGUFs and MLX files.

For more information, you can view our Unsloth inference guide.

Serve Muse Glimmer with Unsloth API

You can use unsloth run command and serve Muse Glimmer via an API using llama-server runtime flags, including context sizing, GPU layers, threading, sampling, networking, and tool configuration. For more info see our API docs or unsloth start.

unsloth run --model unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL

Unsloth is now ready

You can also do many other things with Muse Glimmer via Unsloth Desktop like:

Connect tools: Claude Code, Codex, web search, MCP and more
Train models: Fine-tune text, diffusion, embedding, and more
Generate media: Create and train images, video, TTS locally

🦙 Llama.cpp Guide

For this guide we will be utilizing Dynamic 4-bit for Muse Glimmer 30B. See: Muse Glimmer 30B collection

Obtain the latest llama.cpp on GitHub here. You can follow the build instructions below as well. Change -DGGML_CUDA=ON to -DGGML_CUDA=OFF if you don't have a GPU or just want CPU inference. For Apple Mac / Metal devices, set -DGGML_CUDA=OFF then continue as usual - Metal support is on by default.

apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

If you want to use llama.cpp directly to load models, you can follow commands below, according to each model. UD-Q4_K_XL is the quantization type. You can also download via Hugging Face (step 3). This is similar to ollama run . Use export LLAMA_CACHE="folder" to force llama.cpp to save to a specific location. There is no need to set context length as llama.cpp automatically uses the exact amount required.

export LLAMA_CACHE="unsloth/Muse-Glimmer-30B-GGUF"
./llama.cpp/llama-cli \
    -hf unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 64

You can also download the model manually as well via the code below (after installing pip install huggingface_hub). You can choose UD-Q4_K_XL or other quantized versions like Q8_0 . If downloads get stuck, see: Hugging Face Hub, XET debugging

hf download unsloth/Muse-Glimmer-30B-GGUF \
    --local-dir unsloth/Muse-Glimmer-30B-GGUF \
    --include "*mmproj-BF16*" \
    --include "*UD-Q4_K_XL*" # Use "*UD-Q2_K_XL*" for Dynamic 2bit

Then run the model in conversation mode (with vision mmproj-F16):

./llama.cpp/llama-cli \
    --model unsloth/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
    --mmproj unsloth/Muse-Glimmer-30B-GGUF/mmproj-BF16.gguf \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 64

Llama-server deployment

To deploy Muse Glimmer 30B on llama-server, use:

./llama.cpp/llama-server \
    --model unsloth/Muse-Glimmer-30B-GGUF/Muse-Glimmer-30B-UD-Q4_K_XL.gguf \
    --mmproj unsloth/Muse-Glimmer-30B-GGUF/mmproj-BF16.gguf \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 64 \
    --alias "unsloth/Muse-Glimmer-30B-GGUF" \
    --port 8001

🔧 Fine-tune

You can now fine-tune Meta’s Muse Glimmer-30B with Unsloth on a 24GB card! Muse Glimmer is a 30B parameter multimodal agentic model optimized for local deployment.

We provide multiple Kaggle notebooks which provide 30 hours for free with 2x Tesla T4 GPUs!

Muse Glimmer Vision Kaggle