> For the complete documentation index, see [llms.txt](https://unsloth.ai/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://unsloth.ai/docs/zh/mo-xing/inkling-2-small.md).

# Inkling 2 Small - 本地运行

了解如何使用 Unsloth 和 llama.cpp 在本地运行 Thinking Machines Lab 的 Inkling 2 Small。

**Inkling 2 Small** 由 [Thinking Machines Lab](https://thinkingmachines.ai/news/inkling-2-small/) 是一款 **594B** 混合专家模型，具有 **170 亿个激活参数**。它支持文本和图像输入、文本输出，拥有 **100 万 token 的上下文窗口** ，本指南将介绍如何使用 GGUF 在本地运行该模型。Inkling 2 Small 在 DeepSWE v1.1 上的得分为 72.1%，在 SWEBench Pro v2 (Hard) 上的得分为 86.3%。感谢 Thinking Machines 提供首日访问权限。

使用 Unsloth Dynamic GGUFs 在本地运行： [**Inkling-2-Small-GGUF**](https://huggingface.co/unsloth/Inkling-2-Small-GGUF)。UD-IQ1\_S（动态 1 位）需要 171.7GB，而 NVFP4 需要 394GB，同时保留了 top-1% 精度的 89.7%，体积小 57%。

<a href="https://unsloth.ai/docs/models/inkling-2#run-inkling-2-small-tutorials" class="button primary">运行 Inkling 2 Small 教程</a><a href="https://unsloth.ai/docs/models/inkling-2#quantization-analysis" class="button secondary">量化结果</a>

#### ⚙️ 使用指南

最小的量化版本 `UD-IQ1_S`仅权重就需要 **171.7 GB 磁盘空间**。以下示例使用 2 位量化版本 `UD-IQ2_XXS` (**190.6 GB**）。请选择适合可用 **RAM + VRAM**或统一内存容量的量化版本。

{% hint style="success" %}
请为操作系统、KV 缓存、运行时缓冲区和视觉投影器预留额外内存。文件大小并不等于推理所需的总内存。建议从 32K 上下文长度开始；更长的上下文需要更多内存。
{% endhint %}

**推荐设置**

建议从以下设置开始。Thinking Machines 对 Inkling 2 Small 的评估采用了 **温度 1.0** 和 **xhigh** 推理力度。

| 温度    | Top-p | Top-k | Min-p |
| ----- | ----- | ----- | ----- |
| `1.0` | `1.0` | `0`   | `0.0` |

* **最大上下文窗口：** `1,048,576` 个 token。
* 使用 **内嵌的聊天模板**。在 llama.cpp 中，使用 `--jinja`.

**更改推理力度**

GGUF 模板默认使用 **high**。选择 `low` 以缩短推理过程，或选择 `xhigh` 适用于较难的任务，或 `none` 以请求非思考型响应。 `none` 此设置并不能严格保证完全不进行推理。

| 推理强度 | 无   | 最低  | 低   | 中   | 高   | 超高 / 最大 |
| ---- | --- | --- | --- | --- | --- | ------- |
| 值    | 0.0 | 0.1 | 0.2 | 0.7 | 0.9 | 0.99    |

### 运行 Inkling 2 Small 教程：

使用支持 Inkling 2 的 **Unsloth / llama.cpp 运行时**。支持原版 Inkling 的旧版程序可能无法识别新的 `inkling2` 架构。

#### 🦥 在 Unsloth 中运行 Inkling 2 Small

{% stepper %}
{% step %}
**安装并启动 Unsloth**

下载 [**Unsloth Desktop**](https://unsloth.ai/download) 适用于 **macOS、Windows 或 Linux**。也可以在终端中安装：

**macOS、Linux、WSL：**

```bash
curl -fsSL https://unsloth.ai/install.sh | sh
```

**Windows PowerShell：**

```powershell
irm https://unsloth.ai/install.ps1 | iex
```

{% endstep %}

{% step %}
**搜索并下载 Inkling 2 Small**

打开 **聊天** 或 **模型中心**，搜索 **Inkling-2-Small** 并选择 [**unsloth/Inkling-2-Small-GGUF**](https://huggingface.co/unsloth/Inkling-2-Small-GGUF)。下载所选量化版本。若要输入图像，还需使用对应的 `mmproj-F16.gguf` 视觉投影器（**658 MB**).
{% endstep %}

{% step %}
**运行 Inkling 2 Small**

加载模型，将上下文长度设为 **32,768 个上下文 token**，并使用上述设置。你可以在模型设置中调整上下文长度、采样和推理强度。更多信息请参阅我们的 [Unsloth 推理指南](https://unsloth.ai/docs/new/studio/chat).

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FsZTREyAjLXIj4ELXLDsz%2Fimage.png?alt=media&amp;token=130fd6b0-941e-4784-bf96-59220e55c743" alt=""><figcaption></figcaption></figure>

我们让它使用 UD-IQ1\_S 编写一个简单的 Flappy Bird 游戏，效果很好！下面也提供了它生成的代码

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FMV608A1O4RDQ25uBAko7%2Fimage.png?alt=media&amp;token=059b77ec-f05f-4642-9e53-b99accdc2243" alt="" width="188"><figcaption></figcaption></figure>

{% code overflow="wrap" expandable="true" %}

```html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no">
<title>Flappy Bird</title>
<style>
  * { margin: 0; padding: 0; box-sizing: border-box; -webkit-tap-highlight-color: transparent; }
  html, body {
    height: 100%;
    overflow: hidden;
    background: linear-gradient(#1a2a6c, #203a43, #2c5364);
    display: flex;
    justify-content: center;
    align-items: center;
    font-family: 'Courier New', monospace;
  }
  #wrap {
    position: relative;
    box-shadow: 0 0 60px rgba(0,0,0,.6), 0 0 0 6px #3a2a1a, 0 0 0 12px #57422f;
    border-radius: 4px;
    overflow: hidden;
  }
  canvas {
    display: block;
    background: #70c5ce;
    cursor: pointer;
  }
  #hint {
    position: absolute;
    bottom: 8px;
    left: 0; right: 0;
    text-align: center;
    color: rgba(255,255,255,.85);
    font-size: 12px;
    pointer-events: none;
    text-shadow: 1px 1px 2px #000;
  }
</style>
</head>
<body>
<div id="wrap">
  <canvas id="game" width="400" height="560"></canvas>
  <div id="hint">空格键 / 点击 / 轻触即可拍翅飞行 &nbsp;•&nbsp; P = 暂停</div>
</div>

<script>
(() => {
  const canvas = document.getElementById('game');
  const ctx = canvas.getContext('2d');
  const W = canvas.width, H = canvas.height;

  // ---------- 游戏常量 ----------
  const GRAVITY = 0.38;
  const FLAP = -7.2;
  const PIPE_W = 62;
  const PIPE_GAP = 145;
  const PIPE_SPEED = 2.6;
  const PIPE_SPAWN = 1550;      // 管道生成间隔（毫秒）
  const GROUND_H = 72;
  const BIRD_X = 105;
  const BIRD_R = 14;

  // ---------- 状态 ----------
  const STATE = { MENU: 0, PLAY: 1, OVER: 2, PAUSED: 3 };
  let state = STATE.MENU;
  let bird, pipes, score, best = 0, frame, lastPipe, groundX, flash;
  let clouds = [];

  try { best = parseInt(localStorage.getItem('flappy_best') || '0', 10) || 0; } catch (e) { best = 0; }

  function reset() {
    bird = { y: H / 2 - 40, vy: 0, rot: 0, wing: 0 };
    pipes = [];
    score = 0;
    frame = 0;
    lastPipe = 0;
    groundX = 0;
    flash = 0;
    clouds = [];
    for (let i = 0; i < 5; i++) {
      clouds.push({ x: Math.random() * W, y: 30 + Math.random() * 150, s: 0.6 + Math.random() * 0.8, v: 0.3 + Math.random() * 0.3 });
    }
  }
  reset();

  // ---------- 输入 ----------
  function flap() {
    if (state === STATE.MENU) { state = STATE.PLAY; bird.vy = FLAP; return; }
    if (state === STATE.OVER) { reset(); state = STATE.MENU; return; }
    if (state === STATE.PAUSED) { state = STATE.PLAY; return; }
    bird.vy = FLAP;
  }
  window.addEventListener('keydown', e => {
    if (e.code === 'Space' || e.code === 'ArrowUp') { e.preventDefault(); flap(); }
    if (e.code === 'KeyP') {
      if (state === STATE.PLAY) state = STATE.PAUSED;
      else if (state === STATE.PAUSED) state = STATE.PLAY;
    }
  });
  canvas.addEventListener('mousedown', flap);
  canvas.addEventListener('touchstart', e => { e.preventDefault(); flap(); }, { passive: false });

  // ---------- 更新 ----------
  function update(dt) {
    frame++;
    groundX = (groundX - PIPE_SPEED) % 48;
    clouds.forEach(c => { c.x -= c.v; if (c.x < -80) { c.x = W + 40; c.y = 30 + Math.random() * 150; } });

    if (state !== STATE.PLAY) {
      if (state === STATE.MENU) {
        bird.y = H / 2 - 40 + Math.sin(frame / 22) * 10;
        bird.wing = Math.sin(frame / 6) * 0.6;
      }
      if (flash > 0) flash -= dt;
      return;
    }

    bird.vy += GRAVITY;
    bird.y += bird.vy;
    bird.rot = Math.max(-0.5, Math.min(1.35, bird.vy / 9));
    bird.wing = Math.sin(frame / 5) * 0.7;

    // 生成管道
    const now = performance.now();
    if (now - lastPipe > PIPE_SPAWN) {
      lastPipe = now;
      const gapY = 90 + Math.random() * (H - GROUND_H - 180);
      pipes.push({ x: W + 20, gapY, passed: false });
    }

    // 移动管道并计分
    for (const p of pipes) {
      p.x -= PIPE_SPEED;
      if (!p.passed && p.x + PIPE_W < BIRD_X) { p.passed = true; score++; }
    }
    pipes = pipes.filter(p => p.x + PIPE_W > -10);

    // 碰撞检测
    const groundY = H - GROUND_H;
    if (bird.y + BIRD_R >= groundY) { bird.y = groundY - BIRD_R; die(); }
    if (bird.y - BIRD_R <= 0) { bird.y = BIRD_R; bird.vy = 0; }
    for (const p of pipes) {
      if (BIRD_X + BIRD_R > p.x && BIRD_X - BIRD_R < p.x + PIPE_W) {
        if (bird.y - BIRD_R < p.gapY || bird.y + BIRD_R > p.gapY + PIPE_GAP) { die(); break; }
      }
    }
    if (flash > 0) flash -= dt;
  }

  function die() {
    if (state !== STATE.PLAY) return;
    state = STATE.OVER;
    flash = 220;
    if (score > best) {
      best = score;
      try { localStorage.setItem('flappy_best', String(best)); } catch (e) {}
    }
  }

  // ---------- 绘图辅助函数 ----------
  function drawBackground() {
    // 天空渐变
    const g = ctx.createLinearGradient(0, 0, 0, H - GROUND_H);
    g.addColorStop(0, '#4ec0ca');
    g.addColorStop(0.7, '#71d3dc');
    g.addColorStop(1, '#a8e6ec');
    ctx.fillStyle = g;
    ctx.fillRect(0, 0, W, H - GROUND_H);

    // 太阳
    ctx.fillStyle = '#fff8c9';
    ctx.beginPath(); ctx.arc(W - 70, 62, 30, 0, Math.PI * 2); ctx.fill();
    ctx.fillStyle = 'rgba(255,248,201,.35)';
    ctx.beginPath(); ctx.arc(W - 70, 62, 46, 0, Math.PI * 2); ctx.fill();

    // 云朵
    ctx.fillStyle = 'rgba(255,255,255,.9)';
    for (const c of clouds) {
      ctx.save(); ctx.translate(c.x, c.y); ctx.scale(c.s, c.s);
      ctx.beginPath();
      ctx.arc(0, 0, 14, 0, Math.PI * 2);
      ctx.arc(14, -6, 12, 0, Math.PI * 2);
      ctx.arc(28, 0, 14, 0, Math.PI * 2);
      ctx.arc(14, 5, 15, 0, Math.PI * 2);
      ctx.fill();
      ctx.restore();
    }

    // 远处的山丘
    ctx.fillStyle = '#5dbb63';
    ctx.beginPath();
    ctx.moveTo(0, H - GROUND_H);
    for (let x = 0; x <= W; x += 40) {
      ctx.lineTo(x, H - GROUND_H - 26 - Math.sin((x + 200) / 55) * 14);
    }
    ctx.lineTo(W, H - GROUND_H);
    ctx.closePath(); ctx.fill();

    // 近处的灌木
    ctx.fillStyle = '#3fa34d';
    ctx.fillRect(0, H - GROUND_H - 6, W, 8);
  }

  function drawPipe(p) {
    const topH = p.gapY, botY = p.gapY + PIPE_GAP, botH = H - GROUND_H - botY;
    const grad = ctx.createLinearGradient(p.x, 0, p.x + PIPE_W, 0);
    grad.addColorStop(0, '#5aa832');
    grad.addColorStop(0.18, '#8ee06a');
    grad.addColorStop(0.35, '#73c74f');
    grad.addColorStop(0.55, '#4f9c2c');
    grad.addColorStop(0.8, '#3d7d22');
    grad.addColorStop(1, '#2c5c17');

    function pipeBody(y, h, capAtBottom) {
      ctx.fillStyle = grad;
      ctx.fillRect(p.x, y, PIPE_W, h);
      ctx.strokeStyle = '#1e4010'; ctx.lineWidth = 2;
      ctx.strokeRect(p.x, y, PIPE_W, h);
      // 管帽
      const capH = 24, capY = capAtBottom ? y + h - capH : y;
      ctx.fillStyle = grad;
      ctx.fillRect(p.x - 4, capY, PIPE_W + 8, capH);
      ctx.strokeRect(p.x - 4, capY, PIPE_W + 8, capH);
    }
    pipeBody(0, topH, true);
    pipeBody(botY, botH, false);
  }

  function drawGround() {
    const y = H - GROUND_H;
    // 草地
    ctx.fillStyle = '#ded895';
    ctx.fillRect(0, y, W, GROUND_H);
    ctx.fillStyle = '#7ee35b';
    ctx.fillRect(0, y, W, 16);
    ctx.fillStyle = '#4cbb2e';
    for (let x = groundX; x < W + 48; x += 48) {
      ctx.beginPath();
      ctx.moveTo(x, y + 16); ctx.lineTo(x + 12, y + 2); ctx.lineTo(x + 24, y + 16);
      ctx.fill();
    }
    // 泥土条纹
    ctx.fillStyle = '#c9bd7a';
    for (let x = groundX; x < W + 48; x += 48) {
      ctx.fillRect(x + 6, y + 34, 22, 6);
      ctx.fillRect(x + 26, y + 52, 18, 6);
    }
    ctx.strokeStyle = '#3a2a1a'; ctx.lineWidth = 3;
    ctx.beginPath(); ctx.moveTo(0, y + 1.5); ctx.lineTo(W, y + 1.5); ctx.stroke();
  }

  function drawBird() {
    ctx.save();
    ctx.translate(BIRD_X, bird.y);
    ctx.rotate(bird.rot);

    // 身体
    const bg = ctx.createRadialGradient(-4, -4, 2, 0, 0, BIRD_R + 2);
    bg.addColorStop(0, '#ffe873');
    bg.addColorStop(0.6, '#ffd23f');
    bg.addColorStop(1, '#e0a92e');
    ctx.fillStyle = bg;
    ctx.beginPath(); ctx.ellipse(0, 0, BIRD_R + 2, BIRD_R, 0, 0, Math.PI * 2); ctx.fill();
    ctx.strokeStyle = '#7a5b12'; ctx.lineWidth = 2; ctx.stroke();

    // 翅膀
    ctx.save();
    ctx.translate(-4, 0);
    ctx.rotate(bird.wing);
    ctx.fillStyle = '#f4b63f';
    ctx.beginPath(); ctx.ellipse(-4, 2, 9, 6.5, 0, 0, Math.PI * 2); ctx.fill();
    ctx.strokeStyle = '#7a5b12'; ctx.stroke();
    ctx.restore();

    // 眼睛
    ctx.fillStyle = '#fff';
    ctx.beginPath(); ctx.arc(7, -4, 5.5, 0, Math.PI * 2); ctx.fill();
    ctx.strokeStyle = '#333'; ctx.lineWidth = 1.2; ctx.stroke();
    ctx.fillStyle = '#222';
    ctx.beginPath(); ctx.arc(8.5, -4, 2.6, 0, Math.PI * 2); ctx.fill();

    // 鸟喙
    ctx.fillStyle = '#ff6b35';
    ctx.beginPath();
    ctx.moveTo(BIRD_R - 1, -1);
    ctx.lineTo(BIRD_R + 9, 2.5);
    ctx.lineTo(BIRD_R - 1, 6);
    ctx.closePath(); ctx.fill();
    ctx.strokeStyle = '#a83a12'; ctx.lineWidth = 1.5; ctx.stroke();

    ctx.restore();
  }

  function drawText(text, x, y, size, color = '#fff', stroke = '#000') {
    ctx.font = `bold ${size}px 'Courier New', monospace`;
    ctx.textAlign = 'center';
    ctx.lineWidth = Math.max(2, size / 8);
    ctx.strokeStyle = stroke;
    ctx.strokeText(text, x, y);
    ctx.fillStyle = color;
    ctx.fillText(text, x, y);
  }

  function drawMedal(cx, cy, s) {
    const colors = ['#cd7f32', '#c0c0c0', '#ffd700', '#b9f2ff'];
    const idx = score >= 40 ? 3 : score >= 30 ? 2 : score >= 20 ? 1 : 0;
    ctx.fillStyle = colors[idx];
    ctx.beginPath(); ctx.arc(cx, cy, 20 * s, 0, Math.PI * 2); ctx.fill();
    ctx.strokeStyle = '#5a4a1a'; ctx.lineWidth = 2; ctx.stroke();
    ctx.fillStyle = 'rgba(255,255,255,.5)';
    ctx.beginPath(); ctx.arc(cx - 5 * s, cy - 5 * s, 7 * s, 0, Math.PI * 2); ctx.fill();
    drawText(['铜牌','银牌','金牌','白金'][idx], cx, cy + 34 * s, 10, '#fff');
  }

  function draw() {
    ctx.clearRect(0, 0, W, H);
    drawBackground();
    pipes.forEach(drawPipe);
    drawGround();
    drawBird();

    if (state === STATE.PLAY || state === STATE.PAUSED) {
      drawText(String(score), W / 2, 78, 52);
    }

    if (state === STATE.MENU) {
      drawText('FLAPPY', W / 2, 130, 56, '#ffd23f', '#5a3a00');
      drawText('BIRD', W / 2, 190, 56, '#ffd23f', '#5a3a00');
      drawText('轻触 / 按空格键拍动翅膀', W / 2, 300, 17);
      drawText('最佳成绩：' + best, W / 2, 330, 15, '#fff');
    }

    if (state === STATE.PAUSED) {
      ctx.fillStyle = 'rgba(0,0,0,.45)'; ctx.fillRect(0, 0, W, H);
      drawText('已暂停', W / 2, H / 2 - 10, 40);
      drawText('按 P 键继续', W / 2, H / 2 + 28, 16);
    }

    if (state === STATE.OVER) {
      if (flash > 0) { ctx.fillStyle = `rgba(255,255,255,${flash / 220 * 0.8})`; ctx.fillRect(0, 0, W, H); }
      // 游戏结束面板
      ctx.fillStyle = 'rgba(0,0,0,.35)'; ctx.fillRect(0, 0, W, H);
      ctx.fillStyle = '#f5deb3';
      ctx.strokeStyle = '#5a3a1a'; ctx.lineWidth = 4;
      const px = W / 2 - 130, py = 150, pw = 260, ph = 190;
      ctx.beginPath(); ctx.roundRect(px, py, pw, ph, 10); ctx.fill(); ctx.stroke();
      drawText('游戏结束', W / 2, py + 38, 30, '#e74c3c', '#3a1000');
      drawText('得分：' + score, px + 130, py + 78, 20, '#333', 'rgba(0,0,0,0)');
      drawText('最佳：  ' + best, px + 130, py + 104, 20, '#333', 'rgba(0,0,0,0)');
      drawMedal(px + 55, py + 140, 1);
      drawText('轻触重试', px + 165, py + 148, 16, '#2c5c17', 'rgba(0,0,0,0)');
    }
  }

  // ---------- 主循环 ----------
  let last = performance.now();
  function loop(now) {
    const dt = Math.min(50, now - last);
    last = now;
    update(dt);
    draw();
    requestAnimationFrame(loop);
  }
  requestAnimationFrame(loop);
})();
</script>
</body>
</html>

```

{% endcode %}
{% endstep %}
{% endstepper %}

#### 🦙 在 llama.cpp 中运行 Inkling 2 Small

安装 [llama.cpp](https://github.com/ggml-org/llama.cpp) 并启用 Inkling 2 支持——上游现已直接支持该模型。请在 llama.cpp 目录中使用 Bash（macOS、Linux 或 WSL）运行以下命令。

在 Ubuntu / WSL 上安装构建依赖项：

```bash
sudo apt-get update
sudo apt-get install -y build-essential cmake git libssl-dev
```

对于 NVIDIA，请安装 [CUDA 工具包](https://developer.nvidia.com/cuda-downloads)。在 macOS 上，请安装 Xcode 命令行工具和 CMake；使用 `-DGGML_CUDA=OFF` （默认启用 Metal）。此外，纯 CPU 构建也请使用 `OFF` 。

```bash
apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev git -y
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp
```

**1. 下载模型和视觉投影器**

```bash
pip install -U huggingface_hub
hf download unsloth/Inkling-2-Small-GGUF \
    --include "UD-IQ2_XXS/*" "mmproj-F16.gguf" \
    --local-dir Inkling-2-Small-GGUF
```

将 **全部五个分片** 放在同一文件夹中。将第一个分片传给 llama.cpp；其余分片会自动加载。仓库设为私有期间，下载需要访问权限，并且 `hf auth login`.

**2. 启动本地聊天服务器**

{% code overflow="wrap" %}

```bash
./llama.cpp/llama-server \
    -m Inkling-2-Small-GGUF/UD-IQ2_XXS/Inkling-2-Small-UD-IQ2_XXS-00001-of-00005.gguf \
    --mmproj Inkling-2-Small-GGUF/mmproj-F16.gguf \
    --jinja -ngl 99 -c 32768 \
    --temp 1.0 --top-p 1.0 --top-k 0 --min-p 0.0 \
    --chat-template-kwargs '{"reasoning_effort":"high"}'
```

{% endcode %}

打开 [**http://localhost:8080**](http://localhost:8080) 进行聊天并上传图片。如果模型无法装入显存，请调低 `-ngl` ；或者使用 `-ngl 0` 进行 CPU 推理。仅使用文本时，请省略 `--mmproj`.

#### 量化分析

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2Fa3FEbHeane8hoZhRLuV5%2Fimage.png?alt=media&amp;token=84c3f9ed-cdcb-46d1-b715-283d47499ce7" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FHY57QxZ6VDo8haLPyusw%2Fimage.png?alt=media&amp;token=504b0604-944c-47f4-a725-b1883c1c9464" alt=""><figcaption></figcaption></figure>

以下结果来自 [Unsloth GGUF 模型卡](https://huggingface.co/unsloth/Inkling-2-Small-GGUF) 将每种量化版本与参考 GGUF 进行比较，数据集为 **64 × 4,096 个 WikiText-2 token**，使用 F16 KV 缓存。 **最高概率 token 一致率并非任务基准准确率。** 文件大小不包括视觉投影器。

| 量化格式         | 大小 (GB) | 平均 KLD ↓ | 最高概率 token 相同率 ↑ |
| ------------ | ------- | -------- | ---------------- |
| `UD-IQ1_S`   | 171.7   | 0.0922   | 89.7%            |
| `UD-IQ1_M`   | 178.5   | 0.0829   | 90.2%            |
| `UD-IQ2_XXS` | 190.6   | 0.0577   | 91.8%            |
| `UD-Q2_K_XL` | 205.0   | 0.0450   | 92.7%            |
| `UD-IQ3_XXS` | 226.0   | 0.0364   | 93.3%            |
| `UD-Q3_K_XL` | 285.1   | 0.0176   | 95.3%            |
| `UD-Q4_K_XL` | 354.2   | 0.0029   | 98.0%            |
| `UD-Q8_K_XL` | 386.5   | 0.0027   | 98.1%            |
| `BF16`       | 484.4   |          |                  |

**BF16** 是无损转换，可保留原始 **第 0–39 层中的 NVFP4 专家权重** ，并以 BF16 存储其他权重。 **UD-Q8\_K\_XL** 保留相同的 NVFP4 专家，并将其他权重量化为 8 位。


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://unsloth.ai/docs/zh/mo-xing/inkling-2-small.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
