> For the complete documentation index, see [llms.txt](https://unsloth.ai/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://unsloth.ai/docs/models/deepseek-v4.1-flash.md).

# DeepSeek-V4.1-Flash: Run Locally Guide

Run DeepSeek-V4.1-Flash-1017 locally with Unsloth GGUFs and Engram n-gram offloading.

**DeepSeek-V4.1-Flash-1017** is the new update to V4.1-Flash, making it the **strongest** open model to date for coding, agentic tasks and more. With 1M context, its **552B** backbone + **196B Engram** parameters support SSD offloading to reduce RAM/VRAM needs. This guide shows how to run DeepSeek-V4.1 locally with [Unsloth Dynamic](/docs/basics/dynamic-3.0-ggufs.md) GGUFs.

Unsloth + llama.cpp enables \~**3x faster** DeepSeek-V4.1 inference than other CPU/VRAM/ngram repos. Our **250.4 GB** 1-bit `UD-IQ1_M` quant achieves **88.2% top-1 agreement** against original DeepSeek and runs locally on 128GB+ RAM setups. `UD-Q8_K_XL` (lossless) is **508.4 GB**, while `UD-Q4_K_XL` (near lossless) is **500.7 GB**. GGUF: [**DeepSeek-V4.1-Flash-1017-GGUF**](https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF)

<a href="/pages/P265CJSPfOXTtJ9qxh3j#run-deepseek-v4.1-flash-1017-tutorials" class="button primary">Run DeepSeek-V4.1 Tutorials</a><a href="/docs/models/deepseek-v4.1-flash.md#quantization-analysis" class="button secondary">Quant Analysis</a>

### ⚙️ Usage Guide

**DeepSeek-V4.1-Flash-1017** pairs MoE with **Engram conditional memory**: n-grams of up to four tokens access a few table rows per token, so the table need not stay in GPU memory. It's best to use Unsloth for inference as it enables up to **3x faster** DeepSeek-V4.1 inference than other CPU/VRAM/ngram-focused repos by optimizing ontop of llama.cpp.

1-bit `UD-IQ1_S` will work on 128GB RAM setups like a Macbook, DGX Spark or RTX Spark while 3-bit `UD-IQ3_XXS` will work on 256GB RAM setups like a Mac Studio.

Like the PLE / n-gram layers in our [Qwen3.8-Flash-Next](/docs/models/qwen3.8-next.md) guide, Engram’s 110.6–202.8 GB table can stay on SSD via mmap. Rows load on demand into the OS cache, saving RAM/VRAM while retaining the full disk requirement.

#### **DeepSeek-V4.1-Flash-1017 Requirements:**

**Table: Hardware requirements** (RAM + VRAM, or unified memory; Engram on SSD).

| 1-bit      | 2-bit      | 3-bit      | 4-bit  | 8-bit  |
| ---------- | ---------- | ---------- | ------ | ------ |
| 128–145 GB | 176–205 GB | 256–290 GB | 305 GB | 315 GB |

Estimates allow roughly 5–10 GB above non-Engram weights. Runtime and context can require more.

**n-gram storage:** Lookup formats and weight sizes appear below. Rest GB is the absolutely minimum amount of RAM/VRAM that is needed.

| Quant        | N-gram format / GB | Rest GB |
| ------------ | ------------------ | ------- |
| UD-IQ1\_S    | IQ4\_NL / 110.6    | 122.4   |
| UD-IQ1\_M    | IQ4\_NL / 110.6    | 139.8   |
| UD-IQ2\_XXS  | Q5\_K / 135.2      | 165.3   |
| UD-Q2\_K\_XL | Q5\_K / 135.2      | 199.6   |
| UD-IQ3\_XXS  | Q6\_K / 161.3      | 233.5   |
| UD-IQ3\_S    | Q6\_K / 161.3      | 262.3   |
| UD-Q3\_K\_XL | MXFP8 / 202.8      | 282.3   |
| UD-Q4\_K\_XL | MXFP8 / 202.8      | 297.9   |
| UD-Q8\_K\_XL | MXFP8 / 202.8      | 305.6   |

### Recommended Settings

DeepSeek recommends temperature = 1.0, top-p = 1.0 and disabling top-k and min-p:

| Parameter              | Setting            |
| ---------------------- | ------------------ |
| `temperature`          | `1.0`              |
| `top_p`                | `1.0`              |
| Maximum context window | `1,048,576` tokens |

DeepSeek recommends allowing **at least 256K output tokens** for its long-generation setup.

### 📊 Quantization Analysis

<figure><img src="https://3215535692-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FZbUjhpI7wJTjf3pM1uwu%2Fsize_vs_top1(1).png?alt=media&amp;token=4e133008-8026-4e93-baa7-d7f2e2b81ecc" alt=""><figcaption></figcaption></figure>

Results compare against DeepSeek’s official reference on wiki1100 and long17k. Lower KL means closer next-token probabilities; higher top-1 means more matching predictions.

DeepSeek mixes **MXFP4**, FP8 and BF16. Q4 and Q8 retain **MXFP4** experts and the same Engram table; their labels do not describe every tensor’s precision. Q4 stores many non-expert weights in Q8\_0; Q8 uses BF16.

We also show when we change the engram precision from IQ4\_NL to Q5\_K to Q6\_K and MXFP8 and we provide **standalone engrams which allow you to mix n match.**

| Quant        | Total size | wiki1100 KL / top-1 | long17k KL / top-1 |
| ------------ | ---------- | ------------------- | ------------------ |
| UD-IQ1\_S    | 233.0 GB   | 0.457 / 82.8%       | 0.364 / 85.1%      |
| UD-IQ1\_M    | 250.4 GB   | 0.314 / 87.1%       | 0.241 / 88.2%      |
| UD-IQ2\_XXS  | 300.5 GB   | 0.203 / 90.5%       | 0.142 / 91.2%      |
| UD-Q2\_K\_XL | 334.8 GB   | 0.138 / 91.5%       | 0.0879 / 93.3%     |
| UD-IQ3\_XXS  | 394.7 GB   | 0.0625 / 94.6%      | 0.043 / 95.1%      |
| UD-IQ3\_S    | 423.6 GB   | 0.0316 / 96.0%      | 0.0195 / 96.8%     |
| UD-Q3\_K\_XL | 485.1 GB   | 0.0204 / 96.5%      | 0.0114 / 97.4%     |
| UD-Q4\_K\_XL | 500.7 GB   | 9.00e-03 / 98.2%    | 4.46e-03 / 98.5%   |
| UD-Q8\_K\_XL | 508.4 GB   | 4.76e-03 / 98.5%    | 2.81e-03 / 98.7%   |

**Choosing a quant:** IQ1\_S saves about 54% disk space versus Q8. IQ1\_M adds 17.4 GB and improves top-1 agreement by 4.3 points on wiki1100 and 3.1 on long17k. IQ3 variants reach 94.6–96.8% agreement.

#### **Thinking effort**

Thinking is enabled by default, with `reasoning_effort=75` (“high”). You can set an integer from **1 to 100**. Higher values ask the model to reason more thoroughly; they are not a token limit. The included template also accepts `low` = 50, `medium` = 62, `high` = 75 and `max` = 100.

For example, add one of these options to your launch command:

```bash
# Default thinking effort
--chat-template-kwargs '{"reasoning_effort":75}'

# Maximum thinking effort
--chat-template-kwargs '{"reasoning_effort":100}'

# Disable thinking
--chat-template-kwargs '{"enable_thinking":false}'
```

## Run **DeepSeek-V4.1-Flash-1017** Tutorials:

For this tutorial, we use `UD-Q2_K_XL`. Unsloth automatically selects the near optimal DeepSeek-V4.1 offloading, ngram etc. configurations ensuring your setup gets the near best speed possible.

### 🦥 Unsloth Guide

[Unsloth Desktop](https://unsloth.ai/docs/desktop) is our open-source UI for local AI on macOS, Windows and Linux. You can search and download GGUFs, configure CPU and GPU offloading, adjust inference settings and connect tools from one app.

{% stepper %}
{% step %}

#### Install Unsloth

The easiest way to get started is by downloading the [Unsloth Desktop app](/docs/desktop.md). Works on [macOS](/docs/get-started/install/mac.md), [Windows](/docs/get-started/install/windows-installation.md), and [Linux](/docs/get-started/install/linux.md).

<a href="https://unsloth.ai/download" class="button primary" data-icon="down-to-bracket">Download Unsloth</a>

* <i class="fa-apple">:apple:</i> [Download for macOS](https://unsloth.ai/download/mac)
* <i class="fa-windows">:windows:</i> [Download for Windows](https://unsloth.ai/download/windows)
* <i class="fa-linux">:linux:</i> [Download for Linux](https://unsloth.ai/download/linux)

Or, if you prefer to install manually:

{% tabs %}
{% tab title="Native install" %}
**MacOS, Linux, WSL:**

```bash
curl -fsSL https://unsloth.ai/install.sh | sh
```

**Windows:**

```powershell
irm https://unsloth.ai/install.ps1 | iex
```

{% endtab %}

{% tab title="Docker" %}
The [Unsloth Docker image](https://hub.docker.com/r/unsloth/unsloth) `unsloth/unsloth` is available on Docker. [Read guide.](https://unsloth.ai/docs/get-started/install/docker)
{% endtab %}

{% tab title="Homebrew" %}
Install Unsloth Desktop:

{% code collapsedlinecount="10" %}

```bash
brew install --cask unsloth
```

{% endcode %}

Update Unsloth Desktop:

```bash
brew upgrade --cask unsloth
```

{% endtab %}

{% tab title="WinGet" %}
Install Unsloth Desktop:

```powershell
winget install --id UnslothAI.Unsloth -e
```

Update Unsloth Desktop:

```powershell
winget upgrade --id UnslothAI.Unsloth -e
```

{% endtab %}
{% endtabs %}
{% endstep %}

{% step %}

#### Search and download DeepSeek-V4.1-Flash-1017

Go to [Unsloth Chat](/docs/new/studio/chat.md) or Model hub, search for **DeepSeek-V4.1-Flash-1017** and select [unsloth/DeepSeek-V4.1-Flash-1017-GGUF](https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF). Download your desired quant using the sizes and memory requirements above.
{% endstep %}

{% step %}

#### Run DeepSeek-V4.1-Flash-1017

Thinking is enabled by default with `75`, you can change it in our toggle. Unsloth automatically selects the near optimal DeepSeek-V4.1 offloading, ngram etc. configurations ensuring your setup gets the near best speed possible.

For more information, you can view our [Unsloth inference guide](/docs/new/studio/chat.md).

With UD-IQ1\_S, the 233.0 GB download includes a 110.6 GB Engram n-gram table and 122.4 GB of other weights. Use Engram on-demand loading to keep the lookup table on SSD, with additional RAM/VRAM for cached rows, context and runtime buffers. Follow the Usage Guide above for memory requirements.
{% endstep %}

{% step %}

#### Serve DeepSeek-V4.1-Flash-1017 with Unsloth API

You can use `unsloth run` command and serve DeepSeek-V4.1-Flash-1017 via an API using `llama-server` runtime flags, including context sizing, GPU layers, threading, sampling, networking, and tool configuration. For more info see our [API docs](/docs/basics/api.md) or [unsloth start](/docs/integrations/unsloth-start.md).

{% code overflow="wrap" %}

```bash
unsloth run --model unsloth/DeepSeek-V4.1-Flash-1017-GGUF:UD-Q2_K_XL \
    --temp 1.0 \
    --top-p 1.0 \
    --min-p 0.0 \
    --chat-template-kwargs '{"reasoning_effort":75}'
```

{% endcode %}

Use a runtime with DeepSeek-V4.1-1017 and Engram on-demand loading support. Apply the SSD-offload and memory settings from the Usage Guide when keeping the n-gram table on disk.
{% endstep %}

{% step %}

#### Unsloth is now ready

You can connect DeepSeek-V4.1-Flash-1017 to your tools through Unsloth Desktop. Unsloth also supports the following workflows:

* **Connect tools:** [Claude Code](/docs/basics/claude-code.md), [Codex](/docs/basics/codex.md), [web search](/docs/new/studio/chat.md#advanced-web-search), [MCP](/docs/basics/mcp.md) and more
* **Train models:** Fine-tune other supported text, diffusion, [embedding](/docs/basics/embedding-finetuning.md), and more
* **Generate media:** Create and train [images](/docs/basics/diffusion-image.md), video, [TTS](/docs/basics/text-to-speech-tts-fine-tuning.md) locally

<div align="center"><figure><img src="https://3215535692-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FYShiiub5gs5pCQLX6d4U%2FScreenshot%202026-08-15%20at%2012.18.24%E2%80%AFAM.png?alt=media&amp;token=d6996797-a144-4113-b4c3-155cca6c39a0" alt="Unsloth Desktop Agents settings interface example" width="563"><figcaption><p>Unsloth Desktop’s Agents settings (interface example). Select DeepSeek-V4.1-Flash-1017 and your downloaded quant when connecting a tool.</p></figcaption></figure></div>
{% endstep %}
{% endstepper %}

### 🦙 Run DeepSeek-V4.1-Flash in llama.cpp

{% stepper %}
{% step %}
We need to use our specific llama.cpp PR [here](https://github.com/ggml-org/llama.cpp/pull/30817). You can follow the build instructions below as well. Change `-DGGML_CUDA=ON` to `-DGGML_CUDA=OFF` if you don't have a GPU or just want CPU inference. **For Apple Mac / Metal devices**, set `-DGGML_CUDA=OFF` then continue as usual - Metal support is on by default.

```bash
apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev git -y
git clone https://github.com/ggml-org/llama.cpp
git -C llama.cpp fetch origin pull/30817/head:deepseek41
git -C llama.cpp checkout deepseek41
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp
```

{% endstep %}

{% step %}
To run the model, you can do:

```bash
pip install -U "huggingface_hub[cli]"
hf download unsloth/DeepSeek-V4.1-Flash-1017-GGUF \
    --local-dir unsloth/DeepSeek-V4.1-Flash-1017-GGUF \
    --include "*UD-Q2_K_XL*"
```

Keep all seven shards in the same folder and point `--model` at the first shard. llama.cpp loads the remaining shards automatically.
{% endstep %}

{% step %}
Then to run it:

```bash
./llama.cpp/llama-cli \
    --model unsloth/DeepSeek-V4.1-Flash-1017-GGUF/UD-Q2_K_XL/DeepSeek-V4.1-Flash-1017-UD-Q2_K_XL-00001-of-00007.gguf \
    --temp 1.0 \
    --top-p 1.0 \
    --top-k 0 \
    --min-p 0.0 \
    --chat-template-kwargs '{"reasoning_effort":75}' \
    --load-mode mmap \
    --lazy-mode on \
    --override-tensor "per_layer_token_embd.weight=CPU"
```

This example loads the Engram n-gram table from SSD on demand. Keep `--load-mode mmap` and `--lazy-mode on` enabled for SSD offloading.
{% endstep %}
{% endstepper %}

### ⚡**DSpark** - Speculative Decoding

DeepSeek-V4.1 includes **DSpark**, a speculative decoding architecture that proposes multiple tokens before the main model verifies them. It is separate from the Engram n-gram lookup table: **Engram supplies learned memory; DSpark accelerates decoding**. The similarly named n-gram speculative decoding option is also a different feature.

For DSpark, pair the main model with a matching **V4.1-1017 drafter** and a runtime with DSpark support. Keep the drafter and main checkpoint versions matched, and reserve extra memory for the drafter. The CLI and server examples in this guide use ordinary decoding.

**DSpark enables inference for DeepSeek-V4.1-Flash-GGUF to be around 1.3 to 1.5x faster for decoding!**

We provide a Q8\_0 DSpark module at 8GB, and a full BF16 at 8.5GB.

### :checkered\_flag:Mix n matching the engram

You can also change the engram precision for DeepSeek-V4.1 to whatever you desire.

| Engram Precision | Size (GB) |
| ---------------- | --------- |
| MXFP8 (original) | 202.8 GB  |
| Q6\_K            | 161.3 GB  |
| Q5\_K            | 135.2 GB  |
| IQ4\_NL          | 110.6 GB  |

For example after downloading the dynamic 1-bit quant IQ1\_S which uses by default the IQ4\_NL engram, you can switch it out for the MXFP8 one by only downloading the MXFP8 engram at <https://huggingface.co/unsloth/DeepSeek-V4.1-Flash-1017-GGUF/tree/main/engram> called `DeepSeek-V4.1-Flash-1017-engram-MXFP8.gguf_file` To do this, follow the script below:

```bash
# 1. get the table + the script
hf download unsloth/DeepSeek-V4.1-Flash-1017-GGUF \
  --include "engram/*MXFP8*" "engram/set_split.py" --local-dir .

# 2. stamp it as shard 5 of 5 (UD-IQ1_S's table slot)
python engram/set_split.py engram/DeepSeek-V4.1-Flash-1017-engram-MXFP8.gguf_file 5 5

# 3. keep the old table aside, then rename the new one to the table shard's name
mv UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00005-of-00005.gguf UD-IQ1_S/engram-IQ4_NL.bak
mv engram/DeepSeek-V4.1-Flash-1017-engram-MXFP8.gguf_file \
   UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00005-of-00005.gguf

# 4. run as usual
llama-server -m UD-IQ1_S/DeepSeek-V4.1-Flash-1017-UD-IQ1_S-00001-of-00005.gguf
```

## 📊 Benchmarks

### **GGUF KLD and Top-1% Benchmarks**

Our quants span 82.8–98.5% top-1 agreement against the official F32 reference on wiki1100 and 85.1–98.7% on long17k. UD-Q4\_K\_XL saves 7.7 GB compared with UD-Q8\_K\_XL, with an agreement gap of 0.3 percentage points on wiki1100 and 0.2 on long17k. Q8 also has lower KL divergence on both sets.

**We also alter the engram table precision from IQ4\_NL to Q5\_K, Q6\_K and MXFP8, and show how KLD gets affected**

<figure><img src="https://3215535692-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FdIFvyiUQafZ2upn6VdhJ%2Fsize_vs_kld(1).png?alt=media&amp;token=e5d14eab-9143-4d0a-b1a6-f5427d995808" alt=""><figcaption></figcaption></figure>

### **Official Benchmarks**

DeepSeek reports the following results for **V4.1-Flash-1017** compared with the previous V4.1-Flash release. These are official model results, not measurements of the GGUFs in this guide. Source: [DeepSeek’s 1017 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash-1017).

| Benchmark                   | V4.1-Flash-1017 | V4.1-Flash   |
| --------------------------- | --------------- | ------------ |
| HLE without tools           | 42.6 (44.0†)    | 36.8 (39.1†) |
| Terminal-Bench 4.0          | 42.3            | 31.2         |
| DeepSWE v1.1                | 77.0            | 74.2         |
| ProgramBench                | 29.1            | 20.3         |
| NL2Repo-Bench               | 68.2            | 65.4         |
| Automation-Bench v1.0.6     | 60.5            | 54.8         |
| Agents’ Last Exam           | 35.4            | 31.8         |
| Apex Bench v1.1             | 72.1            | 64.6         |
| Chartography (with tools)   | 83.0            | 78.9         |
| BabyVision (with tools)     | 92.5            | 89.6         |
| ZeroBench-main (with tools) | 51.0            | 49.0         |

† Text-only subset of HLE. DeepSeek uses the Claude Code harness for Terminal-Bench 4.0 and Mini-SWE-V2 for DeepSWE v1.1. Scores depend on the evaluation setup, reasoning effort and context budget.


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://unsloth.ai/docs/models/deepseek-v4.1-flash.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.