> For the complete documentation index, see [llms.txt](https://unsloth.ai/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://unsloth.ai/docs/get-started/unsloth-model-catalog.md). # Unsloth Model Catalog Unsloth LLMs directory for all Unsloth [Dynamic](https://docs.unsloth.ai/basics/unsloth-dynamic-2.0-ggufs) GGUF, 4-bit, NVFP4 models on Hugging Face. QwenGemmaDeepSeekLlamaMistralGLM **GGUFs** let you run models in tools like [**Unsloth Desktop**](/docs/new/studio.md)✨ and llama.cpp.\ Use **Instruct (4-bit)** safetensors for inference or fine-tuning via Unsloth. #### **New & recommended models:** | Model | Variant | GGUF | 4-bit | | ---------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [**Muse Glimmer**](/docs/models/deepseek-v4.md) **(new)** | 30B | [link](https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF) | [link](https://huggingface.co/unsloth/Muse-Glimmer-30B-NVFP4) | | [**DeepSeek-V4**](/docs/models/deepseek-v4.md) **(new)** | Flash-0731 | [link](/docs/new/studio.md) | [link](https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731) | | **Kimi** | [**K3**](/docs/models/kimi-k3.md) | [link](https://huggingface.co/unsloth/Kimi-K3-GGUF) | [link](https://huggingface.co/unsloth/Kimi-K3) | | [**Qwen3.6**](/docs/models/qwen3.6.md) | 27B | [link](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF) • [MTP](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF) | [NVFP4](https://huggingface.co/unsloth/Qwen3.6-27B-NVFP4) | | | 35B-A3B | [link](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF) • [MTP](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF) | [NVFP4](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4) | | [**Gemma 4**](/docs/models/gemma-4.md) | [QAT](/docs/models/gemma-4/qat.md) | [link](https://huggingface.co/collections/unsloth/gemma-4-qat) | [link](https://huggingface.co/collections/unsloth/gemma-4-qat) | | | 12B | [link](https://huggingface.co/unsloth/gemma-4-12b-it-GGUF) | [NVFP4](https://huggingface.co/unsloth/gemma-4-12b-it-NVFP4) | | | 26B-A4B | [link](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF) | [NVFP4](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-NVFP4) | | | 31B | [link](https://huggingface.co/unsloth/gemma-4-31B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-4-31B-it-unsloth-bnb-4bit) • [NVFP4](https://huggingface.co/unsloth/gemma-4-31B-it-NVFP4) | | | E4B | [link](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-4-E4B-it-unsloth-bnb-4bit) • [NVFP4](https://huggingface.co/unsloth/gemma-4-E4B-it-NVFP4) | | | E2B | [link](https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-4-E2B-it-unsloth-bnb-4bit) • [NVFP4](https://huggingface.co/unsloth/gemma-4-E2B-it-NVFP4) | | [**DiffusionGemma**](/docs/models/diffusiongemma.md) | 26B-A4B | [link](https://huggingface.co/unsloth/diffusiongemma-26B-A4B-it-GGUF) | — | | **Kimi** | [K2.7-Code](/docs/models/kimi-k2.7-code.md) | [link](https://huggingface.co/unsloth/Kimi-K2.7-Code-GGUF) | — | | | [**K2.6**](/docs/models/kimi-k2.6.md) | [link](https://huggingface.co/unsloth/Kimi-K2.6-GGUF) | — | | [**NVIDIA Nemotron 3**](/docs/models/nemotron-3-nano-omni.md) | Nano-Omni-30B-A3B | [link](https://huggingface.co/unsloth/Nemotron-3-Nano-30B-A3B-GGUF) | — | | [**Qwen3.5**](https://github.com/unslothai/docs/blob/main/models/qwen3.5) | 35B-A3B | [link](https://huggingface.co/unsloth/Qwen3.5-35B-A3B-GGUF) | — | | | 27B | [link](https://huggingface.co/unsloth/Qwen3.5-27B-GGUF) | — | | | 122B-A10B | [link](https://huggingface.co/unsloth/Qwen3.5-122B-A10B-GGUF) | — | | | 0.8B | [link](https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF) | — | | | 2B | [link](https://huggingface.co/unsloth/Qwen3.5-2B-GGUF) | — | | | 4B | [link](https://huggingface.co/unsloth/Qwen3.5-4B-GGUF) | — | | | 9B | [link](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF) | — | | | 397B-A17B | [link](https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF) | — | | **Qwen3** | [Coder-Next](/docs/models/qwen3-coder-next.md) | [link](https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF) | — | | NVIDIA Nemotron 3 | [Super-120B-A12B](/docs/models/nemotron-3/nemotron-3-super.md) | [link](https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-GGUF) | [link](https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4) | | | [Nano-4B](/docs/models/nemotron-3.md) | [link](https://huggingface.co/unsloth/NVIDIA-Nemotron-3-Nano-4B-GGUF) | — | | **GLM** | [4.7-Flash](/docs/models/tutorials/glm-4.7-flash.md) | [link](https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF) | — | | | [5](/docs/models/tutorials/glm-5.md) | [link](https://huggingface.co/unsloth/GLM-5-GGUF) | — | | **Kimi** | [K2.5](/docs/models/tutorials/kimi-k2.5.md) | [link](https://huggingface.co/unsloth/Kimi-K2.5-GGUF) | — | | [**gpt-oss**](/docs/models/gpt-oss-how-to-run-and-fine-tune.md) | 120B | [link](https://huggingface.co/unsloth/gpt-oss-120b-GGUF) | [link](https://huggingface.co/unsloth/gpt-oss-120b-unsloth-bnb-4bit) | | | 20B | [link](https://huggingface.co/unsloth/gpt-oss-20b-GGUF) | [link](https://huggingface.co/unsloth/gpt-oss-20b-unsloth-bnb-4bit) | | **MiniMax** | [M2.5](/docs/models/tutorials/minimax-m25.md) | [link](https://huggingface.co/unsloth/MiniMax-M2.5-GGUF) | — | | NVIDIA [Nemotron 3](/docs/models/nemotron-3.md) | 30B | [link](https://huggingface.co/unsloth/Nemotron-3-Nano-30B-A3B-GGUF) | — | | [**Qwen-Image**](/docs/models/tutorials/qwen-image-2512.md) | 2512 | [link](https://huggingface.co/unsloth/Qwen-Image-2512-GGUF) | — | | | Edit-2511 | [link](https://huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF) | — | | [**Ministral 3**](/docs/models/tutorials/ministral-3.md) | 3B | [Instruct](https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-GGUF) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-3B-Reasoning-2512-GGUF) | [Instruct](https://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-unsloth-bnb-4bit) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-3B-Reasoning-2512-GGUF) | | | 8B | [Instruct](https://huggingface.co/unsloth/Ministral-3-8B-Instruct-2512-GGUF) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-8B-Reasoning-2512-GGUF) | [Instruct](https://huggingface.co/unsloth/Ministral-3-8B-Instruct-2512-unsloth-bnb-4bit) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-8B-Reasoning-2512-unsloth-bnb-4bit) | | | 14B | [Instruct](https://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-GGUF) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF) | [Instruct](https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-unsloth-bnb-4bit) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-unsloth-bnb-4bit) | | [**Devstral 2**](/docs/models/tutorials/devstral-2.md) | 24B | [link](https://huggingface.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF) | — | | | 123B | [link](https://huggingface.co/unsloth/Devstral-2-123B-Instruct-2512-GGUF) | — | | **Mistral Large 3** | 675B | [link](https://huggingface.co/unsloth/Mistral-Large-3-675B-Instruct-2512-GGUF) | [link](https://huggingface.co/unsloth/Mistral-Large-3-675B-Instruct-2512-NVFP4) | | [**Qwen3-Next**](/docs/models/tutorials/qwen3-next.md) | 80B-A3B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-Next-80B-A3B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-Next-80B-A3B-Instruct-bnb-4bit/) | | | 80B-A3B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-Next-80B-A3B-Thinking-GGUF) | — | | [**Qwen3-VL**](/docs/models/tutorials/qwen3-how-to-run-and-fine-tune/qwen3-vl-how-to-run-and-fine-tune.md) | 2B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-unsloth-bnb-4bit) | | | 2B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Thinking-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Thinking-unsloth-bnb-4bit) | | | 4B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Instruct-unsloth-bnb-4bit) | | | 4B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Thinking-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Thinking-unsloth-bnb-4bit) | | | 8B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit) | | | 8B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Thinking-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Thinking-unsloth-bnb-4bit) | | | 30B-A3B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-30B-A3B-Instruct-GGUF) | — | | | 30B-A3B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-30B-A3B-Thinking-GGUF) | — | | | 32B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-32B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-32B-Instruct-unsloth-bnb-4bit) | | | 32B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-32B-Thinking-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-32B-Thinking-unsloth-bnb-4bit) | | | 235B-A22B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-235B-A22B-Instruct-GGUF) | — | | | 235B-A22B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-235B-A22B-Thinking-GGUF) | — | | [**Qwen3-2507**](/docs/models/tutorials/qwen3-next.md) | 30B-A3B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF) | — | | | 30B-A3B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-Thinking-2507-GGUF) | — | | | 235B-A22B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF/) | — | | [**Qwen3-Coder**](/docs/models/tutorials/qwen3-coder-how-to-run-locally.md) | 30B-A3B | [link](https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF) | — | | [**GLM**](/docs/models/tutorials/glm-4.6-how-to-run-locally.md) | 4.7 | [link](https://huggingface.co/unsloth/GLM-4.7-GGUF) | — | | | 4.6V-Flash | [link](https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF) | — | | [**DeepSeek-V3.1**](/docs/models/tutorials/deepseek-v3.1-how-to-run-locally.md) | Terminus | [link](https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF) | — | | | V3.1 | [link](https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF) | — | #### **DeepSeek models:** | Model | Variant | GGUF | Instruct (4-bit) | | ----------------- | ---------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- | | **DeepSeek-V3.1** | Terminus | [link](https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF) | | | | V3.1 | [link](https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF) | | | **DeepSeek-V3** | V3-0324 | [link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF) | — | | | V3 | [link](https://huggingface.co/unsloth/DeepSeek-V3-GGUF) | — | | **DeepSeek-R1** | R1-0528 | [link](https://huggingface.co/unsloth/DeepSeek-R1-0528-GGUF) | — | | | R1-0528-Qwen3-8B | [link](https://huggingface.co/unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-0528-Qwen3-8B-unsloth-bnb-4bit) | | | R1 | [link](https://huggingface.co/unsloth/DeepSeek-R1-GGUF) | — | | | R1 Zero | [link](https://huggingface.co/unsloth/DeepSeek-R1-Zero-GGUF) | — | | | Distill Llama 3 8 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit) | | | Distill Llama 3.3 70 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-70B-bnb-4bit) | | | Distill Qwen 2.5 1.5 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-1.5B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-1.5B-unsloth-bnb-4bit) | | | Distill Qwen 2.5 7 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-unsloth-bnb-4bit) | | | Distill Qwen 2.5 14 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-unsloth-bnb-4bit) | | | Distill Qwen 2.5 32 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-32B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-32B-bnb-4bit) | #### **Llama models:** | Model | Variant | GGUF | 4-bit | | ----------------------------------------------- | ------------------- | ------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- | | [**Muse Glimmer**](/docs/models/deepseek-v4.md) | 30B | [link](https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF) | [link](https://huggingface.co/unsloth/Muse-Glimmer-30B-NVFP4) | | **Llama 4** | Scout 17 B-16 E | [link](https://huggingface.co/unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Llama-4-Scout-17B-16E-Instruct-unsloth-bnb-4bit) | | | Maverick 17 B-128 E | [link](https://huggingface.co/unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF) | — | | **Llama 3.3** | 70 B | [link](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct-bnb-4bit) | | **Llama 3.2** | 1 B | [link](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit) | | | 3 B | [link](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-bnb-4bit) | | | 11 B Vision | — | [link](https://huggingface.co/unsloth/Llama-3.2-11B-Vision-Instruct-unsloth-bnb-4bit) | | | 90 B Vision | — | [link](https://huggingface.co/unsloth/Llama-3.2-90B-Vision-Instruct-bnb-4bit) | | **Llama 3.1** | 8 B | [link](https://huggingface.co/unsloth/Llama-3.1-8B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit) | | | 70 B | — | [link](https://huggingface.co/unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit) | | | 405 B | — | [link](https://huggingface.co/unsloth/Meta-Llama-3.1-405B-Instruct-bnb-4bit) | | **Llama 3** | 8 B | — | [link](https://huggingface.co/unsloth/llama-3-8b-Instruct-bnb-4bit) | | | 70 B | — | [link](https://huggingface.co/unsloth/llama-3-70b-bnb-4bit) | | **Llama 2** | 7 B | — | [link](https://huggingface.co/unsloth/llama-2-7b-chat-bnb-4bit) | | | 13 B | — | [link](https://huggingface.co/unsloth/llama-2-13b-bnb-4bit) | | **CodeLlama** | 7 B | — | [link](https://huggingface.co/unsloth/codellama-7b-bnb-4bit) | | | 13 B | — | [link](https://huggingface.co/unsloth/codellama-13b-bnb-4bit) | | | 34 B | — | [link](https://huggingface.co/unsloth/codellama-34b-bnb-4bit) | #### **Gemma models:** | Model | Variant | GGUF | Instruct (4-bit) | | ----------------- | ------------- | ----------------------------------------------------------------- | ---------------------------------------------------------------------------- | | **Gemma 4** | E2B | [link](https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-4-E2B-it-unsloth-bnb-4bit) | | | E4B | [link](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-4-E4B-it-unsloth-bnb-4bit) | | | 26B-A4B | [link](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF) | — | | | 31B | [link](https://huggingface.co/unsloth/gemma-4-31B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-4-31B-it-unsloth-bnb-4bit) | | **FunctionGemma** | 270M | [link](https://huggingface.co/unsloth/functiongemma-270m-it-GGUF) | — | | **Gemma 3n** | E2B | ​[link](https://huggingface.co/unsloth/gemma-3n-E2B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-3n-E2B-it-unsloth-bnb-4bit) | | | E4B | [link](https://huggingface.co/unsloth/gemma-3n-E4B-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit) | | **Gemma 3** | 270M | [link](https://huggingface.co/unsloth/gemma-3-270m-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-3-270m-it) | | | 1 B | [link](https://huggingface.co/unsloth/gemma-3-1b-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-3-1b-it-unsloth-bnb-4bit) | | | 4 B | [link](https://huggingface.co/unsloth/gemma-3-4b-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-3-4b-it-unsloth-bnb-4bit) | | | 12 B | [link](https://huggingface.co/unsloth/gemma-3-12b-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-3-12b-it-unsloth-bnb-4bit) | | | 27 B | [link](https://huggingface.co/unsloth/gemma-3-27b-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-3-27b-it-unsloth-bnb-4bit) | | **MedGemma** | 4 B (vision) | [link](https://huggingface.co/unsloth/medgemma-4b-it-GGUF) | [link](https://huggingface.co/unsloth/medgemma-4b-it-unsloth-bnb-4bit) | | | 27 B (vision) | [link](https://huggingface.co/unsloth/medgemma-27b-it-GGUF) | [link](https://huggingface.co/unsloth/medgemma-27b-text-it-unsloth-bnb-4bit) | | **Gemma 2** | 2 B | [link](https://huggingface.co/unsloth/gemma-2-it-GGUF) | [link](https://huggingface.co/unsloth/gemma-2-2b-it-bnb-4bit) | | | 9 B | — | [link](https://huggingface.co/unsloth/gemma-2-9b-it-bnb-4bit) | | | 27 B | — | [link](https://huggingface.co/unsloth/gemma-2-27b-it-bnb-4bit) | #### **Qwen models:** | Model | Variant | GGUF | Instruct (4-bit) | | ---------------------------------------------------------------------------------------------------------- | ---------------------------------------------- | ---------------------------------------------------------------------------- | ------------------------------------------------------------------------------- | | [**Qwen3.6**](/docs/models/qwen3.6.md) | 27B | [link](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF) | — | | | 35B-A3B | [link](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF) | — | | [**Qwen3.5**](https://github.com/unslothai/docs/blob/main/models/qwen3.5) | 35B-A3B | [link](https://huggingface.co/unsloth/Qwen3.5-35B-A3B-GGUF) | — | | | 27B | [link](https://huggingface.co/unsloth/Qwen3.5-27B-GGUF) | — | | | 122B-A10B | [link](https://huggingface.co/unsloth/Qwen3.5-122B-A10B-GGUF) | — | | | 0.8B | [link](https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF) | — | | | 2B | [link](https://huggingface.co/unsloth/Qwen3.5-2B-GGUF) | — | | | 4B | [link](https://huggingface.co/unsloth/Qwen3.5-4B-GGUF) | — | | | 9B | [link](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF) | — | | | 397B-A17B | [link](https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF) | — | | **Qwen3** | [Coder-Next](/docs/models/qwen3-coder-next.md) | [link](https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF) | — | | [**Qwen-Image**](/docs/models/tutorials/qwen-image-2512.md) | 2512 | [link](https://huggingface.co/unsloth/Qwen-Image-2512-GGUF) | — | | | Edit-2511 | [link](https://huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF) | — | | [**Qwen3-VL**](/docs/models/tutorials/qwen3-how-to-run-and-fine-tune/qwen3-vl-how-to-run-and-fine-tune.md) | 2B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-unsloth-bnb-4bit) | | | 2B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Thinking-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Thinking-unsloth-bnb-4bit) | | | 4B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Instruct-unsloth-bnb-4bit) | | | 4B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Thinking-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Thinking-unsloth-bnb-4bit) | | | 8B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit) | | | 8B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Thinking-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Thinking-unsloth-bnb-4bit) | | **Qwen3-Coder** | 30B-A3B | [link](https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF) | — | | | 480B-A35B | [link](https://huggingface.co/unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF) | — | | [**Qwen3-2507**](/docs/models/tutorials/qwen3-next.md) | 30B-A3B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF) | — | | | 30B-A3B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-Thinking-2507-GGUF) | — | | | 235B-A22B-Thinking | [link](https://huggingface.co/unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF/) | — | | | 235B-A22B-Instruct | [link](https://huggingface.co/unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF/) | — | | **Qwen 3** | 0.6 B | [link](https://huggingface.co/unsloth/Qwen3-0.6B-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-0.6B-unsloth-bnb-4bit) | | | 1.7 B | [link](https://huggingface.co/unsloth/Qwen3-1.7B-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-1.7B-unsloth-bnb-4bit) | | | 4 B | [link](https://huggingface.co/unsloth/Qwen3-4B-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-4B-unsloth-bnb-4bit) | | | 8 B | [link](https://huggingface.co/unsloth/Qwen3-8B-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-8B-unsloth-bnb-4bit) | | | 14 B | [link](https://huggingface.co/unsloth/Qwen3-14B-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-14B-unsloth-bnb-4bit) | | | 30 B-A3B | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-bnb-4bit) | | | 32 B | [link](https://huggingface.co/unsloth/Qwen3-32B-GGUF) | [link](https://huggingface.co/unsloth/Qwen3-32B-unsloth-bnb-4bit) | | | 235 B-A22B | [link](https://huggingface.co/unsloth/Qwen3-235B-A22B-GGUF) | — | | **Qwen 2.5 Omni** | 3 B | [link](https://huggingface.co/unsloth/Qwen2.5-Omni-3B-GGUF) | — | | | 7 B | [link](https://huggingface.co/unsloth/Qwen2.5-Omni-7B-GGUF) | — | | **Qwen 2.5 VL** | 3 B | [link](https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-unsloth-bnb-4bit) | | | 7 B | [link](https://huggingface.co/unsloth/Qwen2.5-VL-7B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-VL-7B-Instruct-unsloth-bnb-4bit) | | | 32 B | [link](https://huggingface.co/unsloth/Qwen2.5-VL-32B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-VL-32B-Instruct-unsloth-bnb-4bit) | | | 72 B | [link](https://huggingface.co/unsloth/Qwen2.5-VL-72B-Instruct-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-VL-72B-Instruct-unsloth-bnb-4bit) | | **Qwen 2.5** | 0.5 B | — | [link](https://huggingface.co/unsloth/Qwen2.5-0.5B-Instruct-bnb-4bit) | | | 1.5 B | — | [link](https://huggingface.co/unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit) | | | 3 B | — | [link](https://huggingface.co/unsloth/Qwen2.5-3B-Instruct-bnb-4bit) | | | 7 B | — | [link](https://huggingface.co/unsloth/Qwen2.5-7B-Instruct-bnb-4bit) | | | 14 B | — | [link](https://huggingface.co/unsloth/Qwen2.5-14B-Instruct-bnb-4bit) | | | 32 B | — | [link](https://huggingface.co/unsloth/Qwen2.5-32B-Instruct-bnb-4bit) | | | 72 B | — | [link](https://huggingface.co/unsloth/Qwen2.5-72B-Instruct-bnb-4bit) | | **Qwen 2.5 Coder (128 K)** | 0.5 B | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-0.5B-Instruct-128K-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-0.5B-Instruct-bnb-4bit) | | | 1.5 B | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-1.5B-Instruct-128K-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-1.5B-Instruct-bnb-4bit) | | | 3 B | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-3B-Instruct-128K-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-3B-Instruct-bnb-4bit) | | | 7 B | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct-128K-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit) | | | 14 B | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-14B-Instruct-bnb-4bit) | | | 32 B | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-32B-Instruct-128K-GGUF) | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-32B-Instruct-bnb-4bit) | | **QwQ** | 32 B | [link](https://huggingface.co/unsloth/QwQ-32B-GGUF) | [link](https://huggingface.co/unsloth/QwQ-32B-unsloth-bnb-4bit) | | **QVQ (preview)** | 72 B | — | [link](https://huggingface.co/unsloth/QVQ-72B-Preview-bnb-4bit) | | **Qwen 2 (chat)** | 1.5 B | — | [link](https://huggingface.co/unsloth/Qwen2-1.5B-Instruct-bnb-4bit) | | | 7 B | — | [link](https://huggingface.co/unsloth/Qwen2-7B-Instruct-bnb-4bit) | | | 72 B | — | [link](https://huggingface.co/unsloth/Qwen2-72B-Instruct-bnb-4bit) | | **Qwen 2 VL** | 2 B | — | [link](https://huggingface.co/unsloth/Qwen2-VL-2B-Instruct-unsloth-bnb-4bit) | | | 7 B | — | [link](https://huggingface.co/unsloth/Qwen2-VL-7B-Instruct-unsloth-bnb-4bit) | | | 72 B | — | [link](https://huggingface.co/unsloth/Qwen2-VL-72B-Instruct-bnb-4bit) | #### **GLM models:** | Model | Variant | GGUF | Instruct (4-bit) | | ------- | ---------------------------------------------------- | ---------------------------------------------------------- | ---------------- | | **GLM** | [4.7-Flash](/docs/models/tutorials/glm-4.7-flash.md) | [link](https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF) | — | | | [5](/docs/models/tutorials/glm-5.md) | [link](https://huggingface.co/unsloth/GLM-5-GGUF) | — | | | 4.6V-Flash | [link](https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF) | — | | | 4.6 | [link](https://huggingface.co/unsloth/GLM-4.6-GGUF) | — | | | 4.5-Air | [link](https://huggingface.co/unsloth/GLM-4.5-Air-GGUF) | — | #### **Mistral models:** | Model | Variant | GGUF | Instruct (4-bit) | | ----------------- | ----------------- | ------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | | **Magistral** | Small (2506) | [link](https://huggingface.co/unsloth/Magistral-Small-2506-GGUF) | [link](https://huggingface.co/unsloth/Magistral-Small-2506-unsloth-bnb-4bit) | | | Small (2509) | [link](https://huggingface.co/unsloth/Magistral-Small-2509-GGUF) | [link](https://huggingface.co/unsloth/Magistral-Small-2509-unsloth-bnb-4bit) | | | Small (2507) | [link](https://huggingface.co/unsloth/Magistral-Small-2507-GGUF) | [link](https://huggingface.co/unsloth/Magistral-Small-2507-unsloth-bnb-4bit) | | **Mistral Small** | 3.2-24 B (2506) | [link](https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF) | [link](https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bit) | | | 3.1-24 B (2503) | [link](https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF) | [link](https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bit) | | | 3-24 B (2501) | [link](https://huggingface.co/unsloth/Mistral-Small-24B-Instruct-2501-GGUF) | [link](https://huggingface.co/unsloth/Mistral-Small-24B-Instruct-2501-unsloth-bnb-4bit) | | | 2409-22 B | — | [link](https://huggingface.co/unsloth/Mistral-Small-Instruct-2409-bnb-4bit) | | **Devstral** | Small-24 B (2507) | [link](https://huggingface.co/unsloth/Devstral-Small-2507-GGUF) | [link](https://huggingface.co/unsloth/Devstral-Small-2507-unsloth-bnb-4bit) | | | Small-24 B (2505) | [link](https://huggingface.co/unsloth/Devstral-Small-2505-GGUF) | [link](https://huggingface.co/unsloth/Devstral-Small-2505-unsloth-bnb-4bit) | | **Pixtral** | 12 B (2409) | — | [link](https://huggingface.co/unsloth/Pixtral-12B-2409-bnb-4bit) | | **Mistral NeMo** | 12 B (2407) | [link](https://huggingface.co/unsloth/Mistral-Nemo-Instruct-2407-GGUF) | [link](https://huggingface.co/unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit) | | **Mistral Large** | 2407 | — | [link](https://huggingface.co/unsloth/Mistral-Large-Instruct-2407-bnb-4bit) | | **Mistral 7 B** | v0.3 | — | [link](https://huggingface.co/unsloth/mistral-7b-instruct-v0.3-bnb-4bit) | | | v0.2 | — | [link](https://huggingface.co/unsloth/mistral-7b-instruct-v0.2-bnb-4bit) | | **Mixtral** | 8 × 7 B | — | [link](https://huggingface.co/unsloth/Mixtral-8x7B-Instruct-v0.1-unsloth-bnb-4bit) | #### **Phi models:** | Model | Variant | GGUF | Instruct (4-bit) | | ----------- | ---------------- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------- | | **Phi-4** | Reasoning-plus | [link](https://huggingface.co/unsloth/Phi-4-reasoning-plus-GGUF) | [link](https://huggingface.co/unsloth/Phi-4-reasoning-plus-unsloth-bnb-4bit) | | | Reasoning | [link](https://huggingface.co/unsloth/Phi-4-reasoning-GGUF) | [link](https://huggingface.co/unsloth/phi-4-reasoning-unsloth-bnb-4bit) | | | Mini-Reasoning | [link](https://huggingface.co/unsloth/Phi-4-mini-reasoning-GGUF) | [link](https://huggingface.co/unsloth/Phi-4-mini-reasoning-unsloth-bnb-4bit) | | | Phi-4 (instruct) | [link](https://huggingface.co/unsloth/phi-4-GGUF) | [link](https://huggingface.co/unsloth/phi-4-unsloth-bnb-4bit) | | | mini (instruct) | [link](https://huggingface.co/unsloth/Phi-4-mini-instruct-GGUF) | [link](https://huggingface.co/unsloth/Phi-4-mini-instruct-unsloth-bnb-4bit) | | **Phi-3.5** | mini | — | [link](https://huggingface.co/unsloth/Phi-3.5-mini-instruct-bnb-4bit) | | **Phi-3** | mini | — | [link](https://huggingface.co/unsloth/Phi-3-mini-4k-instruct-bnb-4bit) | | | medium | — | [link](https://huggingface.co/unsloth/Phi-3-medium-4k-instruct-bnb-4bit) | #### **Other (GLM, Orpheus, Smol, Llava etc.) models:**
ModelVariantGGUFInstruct (4-bit)
GLM4.5-Airlink—
4.54.5—
4-32B-04144-32B-0414—
Grok 2270Blink—
Baidu-ERNIE4.5-21B-A3B-Thinkinglink—
HunyuanA13Blink—
Orpheus0.1-ft (3B)linklink
LLava1.5 (7 B)—link
1.6 Mistral (7 B)—link
TinyLlamaChat—link
SmolLM 2135 Mlinklink
360 Mlinklink
1.7 Blinklink
Zephyr-SFT7 B—link
Yi6 B (v1.5)—link
6 B (v1.0)—link
34 B (chat)—link
34 B (base)—link
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://unsloth.ai/docs/get-started/unsloth-model-catalog.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.