> For the complete documentation index, see [llms.txt](https://unsloth.ai/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://unsloth.ai/docs/get-started/unsloth-model-catalog.md).

# Unsloth Model Catalog

Unsloth LLMs directory for all Unsloth [Dynamic](https://docs.unsloth.ai/basics/unsloth-dynamic-2.0-ggufs) GGUF, 4-bit, NVFP4 models on Hugging Face.

<a href="#qwen-models" class="button secondary">Qwen</a><a href="https://unsloth.ai/docs/get-started/unsloth-model-catalog.md#gemma-models" class="button secondary">Gemma</a><a href="https://unsloth.ai/docs/get-started/unsloth-model-catalog.md#deepseek-models" class="button secondary">DeepSeek</a><a href="https://unsloth.ai/docs/get-started/unsloth-model-catalog.md#meta-models" class="button secondary">Meta (Llama)</a><a href="#mistral-models" class="button secondary">Mistral</a><a href="https://unsloth.ai/docs/get-started/unsloth-model-catalog#glm-models" class="button secondary">GLM</a>

**GGUFs** let you run models in tools like [**Unsloth Desktop**](https://unsloth.ai/docs/desktop.md)✨ and llama.cpp.\
Use **NVFP4**, **FP8** for fast GPU inference via Unsloth.

### **New & recommended models:**

| Model                                                                                    | Variant    | GGUF                                                                                                                 | NVFP4 • FP8 • 4-bit                                                                                                                                                                          |
| ---------------------------------------------------------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [**DeepSeek-V4.1**](https://unsloth.ai/docs/models/deepseek-v4.1-flash)                  | Flash-1017 | [link](https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF)                                                 | [FP4](https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017)                                                                                                                               |
| [**Qwen3.8**](https://unsloth.ai/docs/models/qwen3.8)                                    | 27B        | [link](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF)                                                              | [NVFP4](https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4) • [FP8](https://huggingface.co/unsloth/Qwen3.8-27B-FP8) • [BnB 4-bit](https://huggingface.co/unsloth/Qwen3.8-27B-unsloth-bnb-4bit) |
|                                                                                          | Flash-Next | [link](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)                                                       | [FP8](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-FP8)                                                                                                                                 |
|                                                                                          | 2.4T-A95B  | [link](https://huggingface.co/unsloth/Qwen3.8-2.4T-A95B-GGUF)                                                        | —                                                                                                                                                                                            |
| [**Qwen3.6**](https://unsloth.ai/docs/models/qwen3.6)                                    | 27B        | [link](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF) • [MTP](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF) | [NVFP4](https://huggingface.co/unsloth/Qwen3.6-27B-NVFP4)                                                                                                                                    |
| [**GLM-5.3**](https://unsloth.ai/docs/models/glm-5.3)                                    | Flash      | [link](https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF)                                                            | [FP8](https://huggingface.co/unsloth/GLM-5.3-Flash-FP8)                                                                                                                                      |
|                                                                                          | 5.3        | [link](https://huggingface.co/unsloth/GLM-5.3-GGUF)                                                                  | —                                                                                                                                                                                            |
| [**NVIDIA Nemotron 3.5 Bolt**](https://unsloth.ai/docs/models/nemotron-3.5-bolt)         | E4B        | [link](https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF)                                             | —                                                                                                                                                                                            |
|                                                                                          | 74B-A9B    | [link](https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF)                                         | —                                                                                                                                                                                            |
| [**Qwen-Image**](https://unsloth.ai/docs/models/qwen-image-2.1)                          | 2.1        | [link](https://huggingface.co/unsloth/Qwen-Image-2.1-GGUF)                                                           | [FP8](https://huggingface.co/unsloth/Qwen-Image-2.1-FP8)                                                                                                                                     |
| **MiniMax H3**                                                                           | Video      | [link](https://huggingface.co/unsloth/MiniMax-H3-GGUF)                                                               | [FP8](https://huggingface.co/unsloth/MiniMax-H3-FP8)                                                                                                                                         |
|                                                                                          | Image      | —                                                                                                                    | [FP8](https://huggingface.co/unsloth/MiniMax-H3-Image-FP8)                                                                                                                                   |
| [**Recursive Flint**](https://unsloth.ai/docs/models/flint.md)                           | 40B        | [link](https://huggingface.co/unsloth/flint-GGUF)                                                                    | —                                                                                                                                                                                            |
| [**DeepSeek-V4**](https://unsloth.ai/docs/models/deepseek-v4)                            | Flash      | [link](https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF)                                                        | [FP4](https://huggingface.co/unsloth/DeepSeek-V4-Flash)                                                                                                                                      |
| [**MiniMax M3.1**](https://unsloth.ai/docs/models/minimax-m3.1-flash)                    | Flash      | [link](https://huggingface.co/unsloth/MiniMax-M3.1-Flash-GGUF)                                                       | —                                                                                                                                                                                            |
| [**Kimi**](https://unsloth.ai/docs/models/kimi-k3)                                       | K3         | [link](https://huggingface.co/unsloth/Kimi-K3-GGUF)                                                                  | —                                                                                                                                                                                            |
| [**Mistral Large 4**](https://unsloth.ai/docs/models/mistral-large-4)                    | 1T-A52B    | [link](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF)                                                  | [NVFP4](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-NVFP4) • [FP8](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-FP8)                                                    |
| [**NVIDIA Nemotron 3.5 Super VL**](https://unsloth.ai/docs/models/nemotron-3.5-super-vl) | 120B-A12B  | [link](https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF)                                   | [NVFP4](https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4)                                                                                                         |
| [**Muse Spark Open**](https://unsloth.ai/docs/models/muse-spark-open)                    | 816B-A42B  | [link](https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF)                                                | [NVFP4](https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-NVFP4)                                                                                                                      |
| [**Gemma 4**](https://unsloth.ai/docs/models/gemma-4)                                    | 12B        | [link](https://huggingface.co/unsloth/gemma-4-12b-it-GGUF)                                                           | [NVFP4](https://huggingface.co/unsloth/gemma-4-12b-it-NVFP4)                                                                                                                                 |
|                                                                                          | 26B-A4B    | [link](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF)                                                       | [NVFP4](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-NVFP4)                                                                                                                             |
|                                                                                          | 31B        | [link](https://huggingface.co/unsloth/gemma-4-31B-it-GGUF)                                                           | [NVFP4](https://huggingface.co/unsloth/gemma-4-31B-it-NVFP4) • [BnB 4-bit](https://huggingface.co/unsloth/gemma-4-31B-it-unsloth-bnb-4bit)                                                   |

### **Qwen models:**

| Model                                                                                                                        | Variant                                                          | GGUF                                                                                                                         | Instruct (4-bit) • FP8                                                                                                                                                                  |
| ---------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [**Qwen3.8**](https://unsloth.ai/docs/models/qwen3.8.md)                                                                     | 27B                                                              | [link](https://huggingface.co/unsloth/Qwen3.8-27B-GGUF)                                                                      | [link](https://huggingface.co/unsloth/Qwen3.8-27B-unsloth-bnb-4bit) • [NVFP4](https://huggingface.co/unsloth/Qwen3.8-27B-NVFP4) • [FP8](https://huggingface.co/unsloth/Qwen3.8-27B-FP8) |
|                                                                                                                              | Flash-Next                                                       | [link](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-GGUF)                                                               | [FP8](https://huggingface.co/unsloth/Qwen3.8-Flash-Next-FP8)                                                                                                                            |
|                                                                                                                              | 2.4T-A95B                                                        | [link](https://huggingface.co/unsloth/Qwen3.8-2.4T-A95B-GGUF)                                                                |                                                                                                                                                                                         |
| [**Qwen3.6**](https://unsloth.ai/docs/models/qwen3.6.md)                                                                     | 27B                                                              | [link](https://huggingface.co/unsloth/Qwen3.6-27B-GGUF) • [MTP](https://huggingface.co/unsloth/Qwen3.6-27B-MTP-GGUF)         | [NVFP4](https://huggingface.co/unsloth/Qwen3.6-27B-NVFP4)                                                                                                                               |
|                                                                                                                              | 35B-A3B                                                          | [link](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-GGUF) • [MTP](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-MTP-GGUF) | [NVFP4](https://huggingface.co/unsloth/Qwen3.6-35B-A3B-NVFP4)                                                                                                                           |
| [**Qwen3.5**](https://github.com/unslothai/docs/blob/main/models/qwen3.5)                                                    | 35B-A3B                                                          | [link](https://huggingface.co/unsloth/Qwen3.5-35B-A3B-GGUF)                                                                  | —                                                                                                                                                                                       |
|                                                                                                                              | 27B                                                              | [link](https://huggingface.co/unsloth/Qwen3.5-27B-GGUF)                                                                      | —                                                                                                                                                                                       |
|                                                                                                                              | 122B-A10B                                                        | [link](https://huggingface.co/unsloth/Qwen3.5-122B-A10B-GGUF)                                                                | [NVFP4](https://huggingface.co/unsloth/Qwen3.5-122B-A10B-NVFP4)                                                                                                                         |
|                                                                                                                              | 0.8B                                                             | [link](https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF)                                                                     | —                                                                                                                                                                                       |
|                                                                                                                              | 2B                                                               | [link](https://huggingface.co/unsloth/Qwen3.5-2B-GGUF)                                                                       | —                                                                                                                                                                                       |
|                                                                                                                              | 4B                                                               | [link](https://huggingface.co/unsloth/Qwen3.5-4B-GGUF)                                                                       | —                                                                                                                                                                                       |
|                                                                                                                              | 9B                                                               | [link](https://huggingface.co/unsloth/Qwen3.5-9B-GGUF)                                                                       | —                                                                                                                                                                                       |
|                                                                                                                              | 397B-A17B                                                        | [link](https://huggingface.co/unsloth/Qwen3.5-397B-A17B-GGUF)                                                                | —                                                                                                                                                                                       |
| **Qwen3**                                                                                                                    | [Coder-Next](https://unsloth.ai/docs/models/qwen3-coder-next.md) | [link](https://huggingface.co/unsloth/Qwen3-Coder-Next-GGUF)                                                                 | —                                                                                                                                                                                       |
| [**Qwen-Image-2.1**](https://unsloth.ai/docs/models/qwen-image-2.1)                                                          | 2.1                                                              | [link](https://huggingface.co/unsloth/Qwen-Image-2.1-GGUF)                                                                   | [FP8](https://huggingface.co/unsloth/Qwen-Image-2.1-FP8)                                                                                                                                |
|                                                                                                                              | Turbo                                                            | [link](https://huggingface.co/unsloth/Qwen-Image-2.1-Turbo-GGUF)                                                             | [FP8](https://huggingface.co/unsloth/Qwen-Image-2.1-Turbo-FP8)                                                                                                                          |
| [**Qwen-Image**](https://unsloth.ai/docs/models/tutorials/qwen-image-2512.md)                                                | 2512                                                             | [link](https://huggingface.co/unsloth/Qwen-Image-2512-GGUF)                                                                  | [BnB 4-bit](https://huggingface.co/unsloth/Qwen-Image-2512-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | Edit-2511                                                        | [link](https://huggingface.co/unsloth/Qwen-Image-Edit-2511-GGUF)                                                             | —                                                                                                                                                                                       |
| [**Qwen3-VL**](https://unsloth.ai/docs/models/tutorials/qwen3-how-to-run-and-fine-tune/qwen3-vl-how-to-run-and-fine-tune.md) | 2B-Instruct                                                      | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF)                                                             | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | 2B-Thinking                                                      | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Thinking-GGUF)                                                             | [link](https://huggingface.co/unsloth/Qwen3-VL-2B-Thinking-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | 4B-Instruct                                                      | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Instruct-GGUF)                                                             | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Instruct-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | 4B-Thinking                                                      | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Thinking-GGUF)                                                             | [link](https://huggingface.co/unsloth/Qwen3-VL-4B-Thinking-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | 8B-Instruct                                                      | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-GGUF)                                                             | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | 8B-Thinking                                                      | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Thinking-GGUF)                                                             | [link](https://huggingface.co/unsloth/Qwen3-VL-8B-Thinking-unsloth-bnb-4bit)                                                                                                            |
| **Qwen3-Coder**                                                                                                              | 30B-A3B                                                          | [link](https://huggingface.co/unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF)                                                     | —                                                                                                                                                                                       |
|                                                                                                                              | 480B-A35B                                                        | [link](https://huggingface.co/unsloth/Qwen3-Coder-480B-A35B-Instruct-GGUF)                                                   | —                                                                                                                                                                                       |
| [**Qwen3-2507**](https://unsloth.ai/docs/models/tutorials/qwen3-next.md)                                                     | 30B-A3B-Instruct                                                 | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-Instruct-2507-GGUF)                                                      | —                                                                                                                                                                                       |
|                                                                                                                              | 30B-A3B-Thinking                                                 | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-Thinking-2507-GGUF)                                                      | —                                                                                                                                                                                       |
|                                                                                                                              | 235B-A22B-Thinking                                               | [link](https://huggingface.co/unsloth/Qwen3-235B-A22B-Thinking-2507-GGUF/)                                                   | —                                                                                                                                                                                       |
|                                                                                                                              | 235B-A22B-Instruct                                               | [link](https://huggingface.co/unsloth/Qwen3-235B-A22B-Instruct-2507-GGUF/)                                                   | —                                                                                                                                                                                       |
| **Qwen 3**                                                                                                                   | 0.6 B                                                            | [link](https://huggingface.co/unsloth/Qwen3-0.6B-GGUF)                                                                       | [link](https://huggingface.co/unsloth/Qwen3-0.6B-unsloth-bnb-4bit)                                                                                                                      |
|                                                                                                                              | 1.7 B                                                            | [link](https://huggingface.co/unsloth/Qwen3-1.7B-GGUF)                                                                       | [link](https://huggingface.co/unsloth/Qwen3-1.7B-unsloth-bnb-4bit)                                                                                                                      |
|                                                                                                                              | 4 B                                                              | [link](https://huggingface.co/unsloth/Qwen3-4B-GGUF)                                                                         | [link](https://huggingface.co/unsloth/Qwen3-4B-unsloth-bnb-4bit)                                                                                                                        |
|                                                                                                                              | 8 B                                                              | [link](https://huggingface.co/unsloth/Qwen3-8B-GGUF)                                                                         | [link](https://huggingface.co/unsloth/Qwen3-8B-unsloth-bnb-4bit)                                                                                                                        |
|                                                                                                                              | 14 B                                                             | [link](https://huggingface.co/unsloth/Qwen3-14B-GGUF)                                                                        | [link](https://huggingface.co/unsloth/Qwen3-14B-unsloth-bnb-4bit)                                                                                                                       |
|                                                                                                                              | 30 B-A3B                                                         | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-GGUF)                                                                    | [link](https://huggingface.co/unsloth/Qwen3-30B-A3B-bnb-4bit)                                                                                                                           |
|                                                                                                                              | 32 B                                                             | [link](https://huggingface.co/unsloth/Qwen3-32B-GGUF)                                                                        | [link](https://huggingface.co/unsloth/Qwen3-32B-unsloth-bnb-4bit)                                                                                                                       |
|                                                                                                                              | 235 B-A22B                                                       | [link](https://huggingface.co/unsloth/Qwen3-235B-A22B-GGUF)                                                                  | —                                                                                                                                                                                       |
| **Qwen 2.5 Omni**                                                                                                            | 3 B                                                              | [link](https://huggingface.co/unsloth/Qwen2.5-Omni-3B-GGUF)                                                                  | —                                                                                                                                                                                       |
|                                                                                                                              | 7 B                                                              | [link](https://huggingface.co/unsloth/Qwen2.5-Omni-7B-GGUF)                                                                  | —                                                                                                                                                                                       |
| **Qwen 2.5 VL**                                                                                                              | 3 B                                                              | [link](https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-GGUF)                                                           | [link](https://huggingface.co/unsloth/Qwen2.5-VL-3B-Instruct-unsloth-bnb-4bit)                                                                                                          |
|                                                                                                                              | 7 B                                                              | [link](https://huggingface.co/unsloth/Qwen2.5-VL-7B-Instruct-GGUF)                                                           | [link](https://huggingface.co/unsloth/Qwen2.5-VL-7B-Instruct-unsloth-bnb-4bit)                                                                                                          |
|                                                                                                                              | 32 B                                                             | [link](https://huggingface.co/unsloth/Qwen2.5-VL-32B-Instruct-GGUF)                                                          | [link](https://huggingface.co/unsloth/Qwen2.5-VL-32B-Instruct-unsloth-bnb-4bit)                                                                                                         |
|                                                                                                                              | 72 B                                                             | [link](https://huggingface.co/unsloth/Qwen2.5-VL-72B-Instruct-GGUF)                                                          | [link](https://huggingface.co/unsloth/Qwen2.5-VL-72B-Instruct-unsloth-bnb-4bit)                                                                                                         |
| **Qwen 2.5**                                                                                                                 | 0.5 B                                                            | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-0.5B-Instruct-bnb-4bit)                                                                                                                   |
|                                                                                                                              | 1.5 B                                                            | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-1.5B-Instruct-bnb-4bit)                                                                                                                   |
|                                                                                                                              | 3 B                                                              | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-3B-Instruct-bnb-4bit)                                                                                                                     |
|                                                                                                                              | 7 B                                                              | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-7B-Instruct-bnb-4bit)                                                                                                                     |
|                                                                                                                              | 14 B                                                             | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-14B-Instruct-bnb-4bit)                                                                                                                    |
|                                                                                                                              | 32 B                                                             | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-32B-Instruct-bnb-4bit)                                                                                                                    |
|                                                                                                                              | 72 B                                                             | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-72B-Instruct-bnb-4bit)                                                                                                                    |
| **Qwen 2.5 Coder (128 K)**                                                                                                   | 0.5 B                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-0.5B-Instruct-128K-GGUF)                                                 | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-0.5B-Instruct-bnb-4bit)                                                                                                             |
|                                                                                                                              | 1.5 B                                                            | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-1.5B-Instruct-128K-GGUF)                                                 | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-1.5B-Instruct-bnb-4bit)                                                                                                             |
|                                                                                                                              | 3 B                                                              | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-3B-Instruct-128K-GGUF)                                                   | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-3B-Instruct-bnb-4bit)                                                                                                               |
|                                                                                                                              | 7 B                                                              | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct-128K-GGUF)                                                   | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit)                                                                                                               |
|                                                                                                                              | 14 B                                                             | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-14B-Instruct-128K-GGUF)                                                  | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-14B-Instruct-bnb-4bit)                                                                                                              |
|                                                                                                                              | 32 B                                                             | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-32B-Instruct-128K-GGUF)                                                  | [link](https://huggingface.co/unsloth/Qwen2.5-Coder-32B-Instruct-bnb-4bit)                                                                                                              |
| **QwQ**                                                                                                                      | 32 B                                                             | [link](https://huggingface.co/unsloth/QwQ-32B-GGUF)                                                                          | [link](https://huggingface.co/unsloth/QwQ-32B-unsloth-bnb-4bit)                                                                                                                         |
| **QVQ (preview)**                                                                                                            | 72 B                                                             | —                                                                                                                            | [link](https://huggingface.co/unsloth/QVQ-72B-Preview-bnb-4bit)                                                                                                                         |
| **Qwen 2 (chat)**                                                                                                            | 1.5 B                                                            | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2-1.5B-Instruct-bnb-4bit)                                                                                                                     |
|                                                                                                                              | 7 B                                                              | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2-7B-Instruct-bnb-4bit)                                                                                                                       |
|                                                                                                                              | 72 B                                                             | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2-72B-Instruct-bnb-4bit)                                                                                                                      |
| **Qwen 2 VL**                                                                                                                | 2 B                                                              | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2-VL-2B-Instruct-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | 7 B                                                              | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2-VL-7B-Instruct-unsloth-bnb-4bit)                                                                                                            |
|                                                                                                                              | 72 B                                                             | —                                                                                                                            | [link](https://huggingface.co/unsloth/Qwen2-VL-72B-Instruct-bnb-4bit)                                                                                                                   |

### **Gemma models:**

| Model                                                                   | Variant       | GGUF                                                                  | Instruct (4-bit)                                                                                                                      |
| ----------------------------------------------------------------------- | ------------- | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- |
| **Gemma 4**                                                             | E2B           | [link](https://huggingface.co/unsloth/gemma-4-E2B-it-GGUF)            | [link](https://huggingface.co/unsloth/gemma-4-E2B-it-unsloth-bnb-4bit) • [NVFP4](https://huggingface.co/unsloth/gemma-4-E2B-it-NVFP4) |
|                                                                         | E4B           | [link](https://huggingface.co/unsloth/gemma-4-E4B-it-GGUF)            | [link](https://huggingface.co/unsloth/gemma-4-E4B-it-unsloth-bnb-4bit) • [NVFP4](https://huggingface.co/unsloth/gemma-4-E4B-it-NVFP4) |
|                                                                         | 12B           | [link](https://huggingface.co/unsloth/gemma-4-12b-it-GGUF)            | [NVFP4](https://huggingface.co/unsloth/gemma-4-12b-it-NVFP4)                                                                          |
|                                                                         | 26B-A4B       | [link](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF)        | [NVFP4](https://huggingface.co/unsloth/gemma-4-26B-A4B-it-NVFP4)                                                                      |
|                                                                         | 31B           | [link](https://huggingface.co/unsloth/gemma-4-31B-it-GGUF)            | [link](https://huggingface.co/unsloth/gemma-4-31B-it-unsloth-bnb-4bit) • [NVFP4](https://huggingface.co/unsloth/gemma-4-31B-it-NVFP4) |
|                                                                         | QAT           | [link](https://huggingface.co/collections/unsloth/gemma-4-qat)        | —                                                                                                                                     |
| [**EmbeddingGemma 2**](https://unsloth.ai/docs/models/embeddinggemma-2) | Embedding     | [link](https://huggingface.co/unsloth/embeddinggemma-2-GGUF)          | —                                                                                                                                     |
| [**DiffusionGemma**](https://unsloth.ai/docs/models/diffusiongemma)     | 26B-A4B       | [link](https://huggingface.co/unsloth/diffusiongemma-26B-A4B-it-GGUF) | —                                                                                                                                     |
| **FunctionGemma**                                                       | 270M          | [link](https://huggingface.co/unsloth/functiongemma-270m-it-GGUF)     | [BnB 4-bit](https://huggingface.co/unsloth/functiongemma-270m-it-unsloth-bnb-4bit)                                                    |
| **Gemma 3n**                                                            | E2B           | ​[link](https://huggingface.co/unsloth/gemma-3n-E2B-it-GGUF)          | [link](https://huggingface.co/unsloth/gemma-3n-E2B-it-unsloth-bnb-4bit)                                                               |
|                                                                         | E4B           | [link](https://huggingface.co/unsloth/gemma-3n-E4B-it-GGUF)           | [link](https://huggingface.co/unsloth/gemma-3n-E4B-it-unsloth-bnb-4bit)                                                               |
| **Gemma 3**                                                             | 270M          | [link](https://huggingface.co/unsloth/gemma-3-270m-it-GGUF)           | [BnB 4-bit](https://huggingface.co/unsloth/gemma-3-270m-it-unsloth-bnb-4bit)                                                          |
|                                                                         | 1 B           | [link](https://huggingface.co/unsloth/gemma-3-1b-it-GGUF)             | [link](https://huggingface.co/unsloth/gemma-3-1b-it-unsloth-bnb-4bit)                                                                 |
|                                                                         | 4 B           | [link](https://huggingface.co/unsloth/gemma-3-4b-it-GGUF)             | [link](https://huggingface.co/unsloth/gemma-3-4b-it-unsloth-bnb-4bit)                                                                 |
|                                                                         | 12 B          | [link](https://huggingface.co/unsloth/gemma-3-12b-it-GGUF)            | [link](https://huggingface.co/unsloth/gemma-3-12b-it-unsloth-bnb-4bit)                                                                |
|                                                                         | 27 B          | [link](https://huggingface.co/unsloth/gemma-3-27b-it-GGUF)            | [link](https://huggingface.co/unsloth/gemma-3-27b-it-unsloth-bnb-4bit)                                                                |
| **MedGemma**                                                            | 4 B (vision)  | [link](https://huggingface.co/unsloth/medgemma-4b-it-GGUF)            | [link](https://huggingface.co/unsloth/medgemma-4b-it-unsloth-bnb-4bit)                                                                |
|                                                                         | 27 B (vision) | [link](https://huggingface.co/unsloth/medgemma-27b-it-GGUF)           | [link](https://huggingface.co/unsloth/medgemma-27b-text-it-unsloth-bnb-4bit)                                                          |
| **Gemma 2**                                                             | 2 B           | [link](https://huggingface.co/unsloth/gemma-2-it-GGUF)                | [link](https://huggingface.co/unsloth/gemma-2-2b-it-bnb-4bit)                                                                         |
|                                                                         | 9 B           | —                                                                     | [link](https://huggingface.co/unsloth/gemma-2-9b-it-bnb-4bit)                                                                         |
|                                                                         | 27 B          | —                                                                     | [link](https://huggingface.co/unsloth/gemma-2-27b-it-bnb-4bit)                                                                        |

### **DeepSeek models:**

| Model                                                                   | Variant                | GGUF                                                                      | Instruct (4-bit) • FP8                                                                |
| ----------------------------------------------------------------------- | ---------------------- | ------------------------------------------------------------------------- | ------------------------------------------------------------------------------------- |
| [**DeepSeek-V4.1**](https://unsloth.ai/docs/models/deepseek-v4.1-flash) | Flash-1017             | [link](https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF)      | [FP8/FP4](https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017)                    |
| [**DeepSeek-V4**](https://unsloth.ai/docs/models/deepseek-v4)           | Flash-Vision-Exp       | [link](https://huggingface.co/unsloth/DeepSeek-V4-Flash-Vision-Exp-GGUF)  | —                                                                                     |
|                                                                         | Pro-0813               | [link](https://huggingface.co/unsloth/DeepSeek-V4-Pro-0813-GGUF)          | [FP8/FP4](https://huggingface.co/unsloth/DeepSeek-V4-Pro-0813)                        |
|                                                                         | Flash-0731             | [link](https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731-GGUF)        | [FP8/FP4](https://huggingface.co/unsloth/DeepSeek-V4-Flash-0731)                      |
|                                                                         | Flash                  | [link](https://huggingface.co/unsloth/DeepSeek-V4-Flash-GGUF)             | [FP8/FP4](https://huggingface.co/unsloth/DeepSeek-V4-Flash)                           |
| **DeepSeek-V3.1**                                                       | Terminus               | [link](https://huggingface.co/unsloth/DeepSeek-V3.1-Terminus-GGUF)        |                                                                                       |
|                                                                         | V3.1                   | [link](https://huggingface.co/unsloth/DeepSeek-V3.1-GGUF)                 |                                                                                       |
| **DeepSeek-V3**                                                         | V3-0324                | [link](https://huggingface.co/unsloth/DeepSeek-V3-0324-GGUF)              | —                                                                                     |
|                                                                         | V3                     | [link](https://huggingface.co/unsloth/DeepSeek-V3-GGUF)                   | —                                                                                     |
| **DeepSeek-R1**                                                         | R1-0528                | [link](https://huggingface.co/unsloth/DeepSeek-R1-0528-GGUF)              | —                                                                                     |
|                                                                         | R1-0528-Qwen3-8B       | [link](https://huggingface.co/unsloth/DeepSeek-R1-0528-Qwen3-8B-GGUF)     | [link](https://huggingface.co/unsloth/DeepSeek-R1-0528-Qwen3-8B-unsloth-bnb-4bit)     |
|                                                                         | R1                     | [link](https://huggingface.co/unsloth/DeepSeek-R1-GGUF)                   | —                                                                                     |
|                                                                         | R1 Zero                | [link](https://huggingface.co/unsloth/DeepSeek-R1-Zero-GGUF)              | —                                                                                     |
|                                                                         | Distill Llama 3 8 B    | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF)  | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-unsloth-bnb-4bit)  |
|                                                                         | Distill Llama 3.3 70 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-70B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-70B-bnb-4bit)         |
|                                                                         | Distill Qwen 2.5 1.5 B | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-1.5B-GGUF) | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-1.5B-unsloth-bnb-4bit) |
|                                                                         | Distill Qwen 2.5 7 B   | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-GGUF)   | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-7B-unsloth-bnb-4bit)   |
|                                                                         | Distill Qwen 2.5 14 B  | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-GGUF)  | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-14B-unsloth-bnb-4bit)  |
|                                                                         | Distill Qwen 2.5 32 B  | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-32B-GGUF)  | [link](https://huggingface.co/unsloth/DeepSeek-R1-Distill-Qwen-32B-bnb-4bit)          |

### **Meta models:**

| Model                                                                 | Variant             | GGUF                                                                           | 4-bit                                                                                  |
| --------------------------------------------------------------------- | ------------------- | ------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- |
| [**Muse Spark Open**](https://unsloth.ai/docs/models/muse-spark-open) | 816B-A42B           | [link](https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-GGUF)          | [NVFP4](https://huggingface.co/unsloth/Muse-Spark-Open-816B-A42B-NVFP4)                |
| [**Muse Glimmer**](https://unsloth.ai/docs/models/deepseek-v4.md)     | 30B                 | [link](https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF)                   | [link](https://huggingface.co/unsloth/Muse-Glimmer-30B-unsloth-bnb-4bit)               |
| **Llama 4**                                                           | Scout 17 B-16 E     | [link](https://huggingface.co/unsloth/Llama-4-Scout-17B-16E-Instruct-GGUF)     | [link](https://huggingface.co/unsloth/Llama-4-Scout-17B-16E-Instruct-unsloth-bnb-4bit) |
|                                                                       | Maverick 17 B-128 E | [link](https://huggingface.co/unsloth/Llama-4-Maverick-17B-128E-Instruct-GGUF) | —                                                                                      |
| **Llama 3.3**                                                         | 70 B                | [link](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct-GGUF)             | [link](https://huggingface.co/unsloth/Llama-3.3-70B-Instruct-bnb-4bit)                 |
| **Llama 3.2**                                                         | 1 B                 | [link](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-GGUF)              | [link](https://huggingface.co/unsloth/Llama-3.2-1B-Instruct-bnb-4bit)                  |
|                                                                       | 3 B                 | [link](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-GGUF)              | [link](https://huggingface.co/unsloth/Llama-3.2-3B-Instruct-bnb-4bit)                  |
|                                                                       | 11 B Vision         | —                                                                              | [link](https://huggingface.co/unsloth/Llama-3.2-11B-Vision-Instruct-unsloth-bnb-4bit)  |
|                                                                       | 90 B Vision         | —                                                                              | [link](https://huggingface.co/unsloth/Llama-3.2-90B-Vision-Instruct-bnb-4bit)          |
| **Llama 3.1**                                                         | 8 B                 | [link](https://huggingface.co/unsloth/Llama-3.1-8B-Instruct-GGUF)              | [link](https://huggingface.co/unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit)             |
|                                                                       | 70 B                | —                                                                              | [link](https://huggingface.co/unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit)            |
|                                                                       | 405 B               | —                                                                              | [link](https://huggingface.co/unsloth/Meta-Llama-3.1-405B-Instruct-bnb-4bit)           |
| **Llama 3**                                                           | 8 B                 | —                                                                              | [link](https://huggingface.co/unsloth/llama-3-8b-Instruct-bnb-4bit)                    |
|                                                                       | 70 B                | —                                                                              | [link](https://huggingface.co/unsloth/llama-3-70b-bnb-4bit)                            |
| **Llama 2**                                                           | 7 B                 | —                                                                              | [link](https://huggingface.co/unsloth/llama-2-7b-chat-bnb-4bit)                        |
|                                                                       | 13 B                | —                                                                              | [link](https://huggingface.co/unsloth/llama-2-13b-bnb-4bit)                            |
| **CodeLlama**                                                         | 7 B                 | —                                                                              | [link](https://huggingface.co/unsloth/codellama-7b-bnb-4bit)                           |
|                                                                       | 13 B                | —                                                                              | [link](https://huggingface.co/unsloth/codellama-13b-bnb-4bit)                          |
|                                                                       | 34 B                | —                                                                              | [link](https://huggingface.co/unsloth/codellama-34b-bnb-4bit)                          |

### **GLM models:**

| Model     | Variant                                                                | GGUF                                                       | Instruct (4-bit) • FP8                                      |
| --------- | ---------------------------------------------------------------------- | ---------------------------------------------------------- | ----------------------------------------------------------- |
| **GLM-5** | [5.3-Flash](https://unsloth.ai/docs/models/glm-5.3-flash.md)           | [link](https://huggingface.co/unsloth/GLM-5.3-Flash-GGUF)  | [FP8](https://huggingface.co/unsloth/GLM-5.3-Flash-FP8)     |
|           | [5.3](https://unsloth.ai/docs/models/glm-5.3.md)                       | [link](https://huggingface.co/unsloth/GLM-5.3-GGUF)        |                                                             |
|           | 5.2                                                                    | [link](https://huggingface.co/unsloth/GLM-5.2-GGUF)        | —                                                           |
|           | 5.1                                                                    | [link](https://huggingface.co/unsloth/GLM-5.1-GGUF)        | —                                                           |
| **GLM**   | [4.7-Flash](https://unsloth.ai/docs/models/tutorials/glm-4.7-flash.md) | [link](https://huggingface.co/unsloth/GLM-4.7-Flash-GGUF)  | [NVFP4](https://huggingface.co/unsloth/GLM-4.7-Flash-NVFP4) |
|           | [5](https://unsloth.ai/docs/models/tutorials/glm-5.md)                 | [link](https://huggingface.co/unsloth/GLM-5-GGUF)          | —                                                           |
|           | 4.7                                                                    | [link](https://huggingface.co/unsloth/GLM-4.7-GGUF)        | —                                                           |
|           | 4.6V-Flash                                                             | [link](https://huggingface.co/unsloth/GLM-4.6V-Flash-GGUF) | —                                                           |
|           | 4.6                                                                    | [link](https://huggingface.co/unsloth/GLM-4.6-GGUF)        | —                                                           |
|           | 4.5-Air                                                                | [link](https://huggingface.co/unsloth/GLM-4.5-Air-GGUF)    | —                                                           |

### **Mistral models:**

| Model                                                                   | Variant           | GGUF                                                                                                                                                            | Instruct (4-bit) • FP8                                                                                                                                                                  |
| ----------------------------------------------------------------------- | ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [**Mistral Large 4**](https://unsloth.ai/docs/models/mistral-large-4)   | 1T-A52B           | [link](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-GGUF)                                                                                             | [NVFP4](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-NVFP4) • [FP8](https://huggingface.co/unsloth/Mistral-Large-4-1T-A52B-FP8)                                               |
| [**Mistral Medium 3.5**](https://unsloth.ai/docs/models/mistral-3.5)    | 128B              | [link](https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF)                                                                                             | —                                                                                                                                                                                       |
| **Mistral Small 4**                                                     | 119B              | [link](https://huggingface.co/unsloth/Mistral-Small-4-119B-2603-GGUF)                                                                                           | —                                                                                                                                                                                       |
| **Mistral Large 3**                                                     | 675B              | [link](https://huggingface.co/unsloth/Mistral-Large-3-675B-Instruct-2512-GGUF)                                                                                  | —                                                                                                                                                                                       |
| [**Devstral 2**](https://unsloth.ai/docs/models/tutorials/devstral-2)   | 123B              | [link](https://huggingface.co/unsloth/Devstral-2-123B-Instruct-2512-GGUF)                                                                                       | —                                                                                                                                                                                       |
|                                                                         | 24B               | [link](https://huggingface.co/unsloth/Devstral-Small-2-24B-Instruct-2512-GGUF)                                                                                  | —                                                                                                                                                                                       |
| [**Ministral 3**](https://unsloth.ai/docs/models/tutorials/ministral-3) | 3B                | [Instruct](https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-GGUF) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-3B-Reasoning-2512-GGUF)   | [Instruct](https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-unsloth-bnb-4bit) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-3B-Reasoning-2512-unsloth-bnb-4bit)   |
|                                                                         | 8B                | [Instruct](https://huggingface.co/unsloth/Ministral-3-8B-Instruct-2512-GGUF) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-8B-Reasoning-2512-GGUF)   | [Instruct](https://huggingface.co/unsloth/Ministral-3-8B-Instruct-2512-unsloth-bnb-4bit) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-8B-Reasoning-2512-unsloth-bnb-4bit)   |
|                                                                         | 14B               | [Instruct](https://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-GGUF) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-GGUF) | [Instruct](https://huggingface.co/unsloth/Ministral-3-14B-Instruct-2512-unsloth-bnb-4bit) • [Reasoning](https://huggingface.co/unsloth/Ministral-3-14B-Reasoning-2512-unsloth-bnb-4bit) |
| **Magistral**                                                           | Small (2506)      | [link](https://huggingface.co/unsloth/Magistral-Small-2506-GGUF)                                                                                                | [link](https://huggingface.co/unsloth/Magistral-Small-2506-unsloth-bnb-4bit)                                                                                                            |
|                                                                         | Small (2509)      | [link](https://huggingface.co/unsloth/Magistral-Small-2509-GGUF)                                                                                                | [link](https://huggingface.co/unsloth/Magistral-Small-2509-unsloth-bnb-4bit)                                                                                                            |
|                                                                         | Small (2507)      | [link](https://huggingface.co/unsloth/Magistral-Small-2507-GGUF)                                                                                                | [link](https://huggingface.co/unsloth/Magistral-Small-2507-unsloth-bnb-4bit)                                                                                                            |
| **Mistral Small**                                                       | 3.2-24 B (2506)   | [link](https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-GGUF)                                                                                 | [link](https://huggingface.co/unsloth/Mistral-Small-3.2-24B-Instruct-2506-unsloth-bnb-4bit)                                                                                             |
|                                                                         | 3.1-24 B (2503)   | [link](https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-GGUF)                                                                                 | [link](https://huggingface.co/unsloth/Mistral-Small-3.1-24B-Instruct-2503-unsloth-bnb-4bit)                                                                                             |
|                                                                         | 3-24 B (2501)     | [link](https://huggingface.co/unsloth/Mistral-Small-24B-Instruct-2501-GGUF)                                                                                     | [link](https://huggingface.co/unsloth/Mistral-Small-24B-Instruct-2501-unsloth-bnb-4bit)                                                                                                 |
|                                                                         | 2409-22 B         | —                                                                                                                                                               | [link](https://huggingface.co/unsloth/Mistral-Small-Instruct-2409-bnb-4bit)                                                                                                             |
| **Devstral**                                                            | Small-24 B (2507) | [link](https://huggingface.co/unsloth/Devstral-Small-2507-GGUF)                                                                                                 | [link](https://huggingface.co/unsloth/Devstral-Small-2507-unsloth-bnb-4bit)                                                                                                             |
|                                                                         | Small-24 B (2505) | [link](https://huggingface.co/unsloth/Devstral-Small-2505-GGUF)                                                                                                 | [link](https://huggingface.co/unsloth/Devstral-Small-2505-unsloth-bnb-4bit)                                                                                                             |
| **Pixtral**                                                             | 12 B (2409)       | —                                                                                                                                                               | [link](https://huggingface.co/unsloth/Pixtral-12B-2409-bnb-4bit)                                                                                                                        |
| **Mistral NeMo**                                                        | 12 B (2407)       | [link](https://huggingface.co/unsloth/Mistral-Nemo-Instruct-2407-GGUF)                                                                                          | [link](https://huggingface.co/unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit)                                                                                                              |
| **Mistral Large**                                                       | 2407              | —                                                                                                                                                               | [link](https://huggingface.co/unsloth/Mistral-Large-Instruct-2407-bnb-4bit)                                                                                                             |
| **Mistral 7 B**                                                         | v0.3              | —                                                                                                                                                               | [link](https://huggingface.co/unsloth/mistral-7b-instruct-v0.3-bnb-4bit)                                                                                                                |
|                                                                         | v0.2              | —                                                                                                                                                               | [link](https://huggingface.co/unsloth/mistral-7b-instruct-v0.2-bnb-4bit)                                                                                                                |
| **Mixtral**                                                             | 8 × 7 B           | —                                                                                                                                                               | [link](https://huggingface.co/unsloth/Mixtral-8x7B-Instruct-v0.1-unsloth-bnb-4bit)                                                                                                      |

### **Phi models:**

| Model       | Variant          | GGUF                                                             | Instruct (4-bit)                                                             |
| ----------- | ---------------- | ---------------------------------------------------------------- | ---------------------------------------------------------------------------- |
| **Phi-4**   | Reasoning-plus   | [link](https://huggingface.co/unsloth/Phi-4-reasoning-plus-GGUF) | [link](https://huggingface.co/unsloth/Phi-4-reasoning-plus-unsloth-bnb-4bit) |
|             | Reasoning        | [link](https://huggingface.co/unsloth/Phi-4-reasoning-GGUF)      | [link](https://huggingface.co/unsloth/phi-4-reasoning-unsloth-bnb-4bit)      |
|             | Mini-Reasoning   | [link](https://huggingface.co/unsloth/Phi-4-mini-reasoning-GGUF) | [link](https://huggingface.co/unsloth/Phi-4-mini-reasoning-unsloth-bnb-4bit) |
|             | Phi-4 (instruct) | [link](https://huggingface.co/unsloth/phi-4-GGUF)                | [link](https://huggingface.co/unsloth/phi-4-unsloth-bnb-4bit)                |
|             | mini (instruct)  | [link](https://huggingface.co/unsloth/Phi-4-mini-instruct-GGUF)  | [link](https://huggingface.co/unsloth/Phi-4-mini-instruct-unsloth-bnb-4bit)  |
| **Phi-3.5** | mini             | —                                                                | [link](https://huggingface.co/unsloth/Phi-3.5-mini-instruct-bnb-4bit)        |
| **Phi-3**   | mini             | —                                                                | [link](https://huggingface.co/unsloth/Phi-3-mini-4k-instruct-bnb-4bit)       |
|             | medium           | —                                                                | [link](https://huggingface.co/unsloth/Phi-3-medium-4k-instruct-bnb-4bit)     |

### **Other (GLM, Orpheus, Smol, Llava etc.) models:**

<table><thead><tr><th>Model</th><th>Variant</th><th width="167">GGUF</th><th>Instruct (4-bit)</th></tr></thead><tbody><tr><td><strong>Kimi K3.1</strong></td><td>Pro</td><td>—</td><td><a href="https://huggingface.co/unsloth/Kimi-K3.1-Pro">MXFP4</a></td></tr><tr><td><a href="https://unsloth.ai/docs/models/kimi-k3"><strong>Kimi K3</strong></a></td><td>K3</td><td><a href="https://huggingface.co/unsloth/Kimi-K3-GGUF">link</a></td><td>—</td></tr><tr><td><a href="https://unsloth.ai/docs/models/minimax-m3.1-flash"><strong>MiniMax M3.1</strong></a></td><td>Flash</td><td><a href="https://huggingface.co/unsloth/MiniMax-M3.1-Flash-GGUF">link</a></td><td>—</td></tr><tr><td><a href="https://unsloth.ai/docs/models/minimax-m3"><strong>MiniMax M3</strong></a></td><td>M3</td><td><a href="https://huggingface.co/unsloth/MiniMax-M3-GGUF">link</a></td><td>—</td></tr><tr><td><a href="https://unsloth.ai/docs/models/nemotron-3.5-bolt"><strong>NVIDIA Nemotron 3.5 Bolt</strong></a></td><td>E4B</td><td><a href="https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-E4B-GGUF">link</a></td><td>—</td></tr><tr><td></td><td>74B-A9B</td><td><a href="https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Bolt-74B-A9B-GGUF">link</a></td><td>—</td></tr><tr><td><a href="https://unsloth.ai/docs/models/nemotron-3.5-super-vl"><strong>NVIDIA Nemotron 3.5 Super VL</strong></a></td><td>120B-A12B</td><td><a href="https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-GGUF">link</a></td><td><a href="https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4">NVFP4</a></td></tr><tr><td><a href="https://unsloth.ai/docs/models/nemotron-3.5"><strong>NVIDIA Nemotron 3.5 Lightning</strong></a></td><td>30B-A3B</td><td><a href="https://huggingface.co/unsloth/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF">link</a></td><td>—</td></tr><tr><td>GLM</td><td>4.5-Air</td><td><a href="https://huggingface.co/unsloth/GLM-4.5-Air-GGUF">link</a></td><td>—</td></tr><tr><td></td><td>4.5</td><td><a href="https://huggingface.co/unsloth/GLM-4.5-GGUF">4.5</a></td><td>—</td></tr><tr><td></td><td>4-32B-0414</td><td><a href="https://huggingface.co/unsloth/GLM-4-32B-0414-GGUF">4-32B-0414</a></td><td><a href="https://huggingface.co/unsloth/GLM-4-32B-0414-unsloth-bnb-4bit">BnB 4-bit</a></td></tr><tr><td><strong>Grok 2</strong></td><td>270B</td><td><a href="https://huggingface.co/unsloth/grok-2-GGUF">link</a></td><td>—</td></tr><tr><td><strong>Baidu-ERNIE</strong></td><td>4.5-21B-A3B-Thinking</td><td><a href="https://huggingface.co/unsloth/ERNIE-4.5-21B-A3B-Thinking-GGUF">link</a></td><td>—</td></tr><tr><td>Hunyuan</td><td>A13B</td><td><a href="https://huggingface.co/unsloth/Hunyuan-A13B-Instruct-GGUF">link</a></td><td>—</td></tr><tr><td>Orpheus</td><td>0.1-ft (3B)</td><td><a href="https://huggingface.co/unsloth/orpheus-3b-0.1-ft-GGUF">link</a></td><td><a href="https://huggingface.co/unsloth/orpheus-3b-0.1-ft-unsloth-bnb-4bit">link</a></td></tr><tr><td><strong>LLava</strong></td><td>1.5 (7 B)</td><td>—</td><td><a href="https://huggingface.co/unsloth/llava-1.5-7b-hf-bnb-4bit">link</a></td></tr><tr><td></td><td>1.6 Mistral (7 B)</td><td>—</td><td><a href="https://huggingface.co/unsloth/llava-v1.6-mistral-7b-hf-bnb-4bit">link</a></td></tr><tr><td><strong>TinyLlama</strong></td><td>Chat</td><td>—</td><td><a href="https://huggingface.co/unsloth/tinyllama-chat-bnb-4bit">link</a></td></tr><tr><td><strong>SmolLM 2</strong></td><td>135 M</td><td><a href="https://huggingface.co/unsloth/SmolLM2-135M-Instruct-GGUF">link</a></td><td><a href="https://huggingface.co/unsloth/SmolLM2-135M-Instruct-bnb-4bit">link</a></td></tr><tr><td></td><td>360 M</td><td><a href="https://huggingface.co/unsloth/SmolLM2-360M-Instruct-GGUF">link</a></td><td><a href="https://huggingface.co/unsloth/SmolLM2-360M-Instruct-bnb-4bit">link</a></td></tr><tr><td></td><td>1.7 B</td><td><a href="https://huggingface.co/unsloth/SmolLM2-1.7B-Instruct-GGUF">link</a></td><td><a href="https://huggingface.co/unsloth/SmolLM2-1.7B-Instruct-bnb-4bit">link</a></td></tr><tr><td><strong>Zephyr-SFT</strong></td><td>7 B</td><td>—</td><td><a href="https://huggingface.co/unsloth/zephyr-sft-bnb-4bit">link</a></td></tr><tr><td><strong>Yi</strong></td><td>6 B (v1.5)</td><td>—</td><td><a href="https://huggingface.co/unsloth/Yi-1.5-6B-bnb-4bit">link</a></td></tr><tr><td></td><td>6 B (v1.0)</td><td>—</td><td><a href="https://huggingface.co/unsloth/yi-6b-bnb-4bit">link</a></td></tr><tr><td></td><td>34 B (chat)</td><td>—</td><td><a href="https://huggingface.co/unsloth/yi-34b-chat-bnb-4bit">link</a></td></tr><tr><td></td><td>34 B (base)</td><td>—</td><td><a href="https://huggingface.co/unsloth/yi-34b-bnb-4bit">link</a></td></tr></tbody></table>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://unsloth.ai/docs/get-started/unsloth-model-catalog.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
