{% endstep %}
{% endstepper %}
### 🦙 Llama.cpp 指南
{% stepper %}
{% step %}
获取特定的 `llama.cpp` PR 上 [**GitHub 上获取**](https://github.com/ggml-org/llama.cpp/pull/24523)。你也可以按照下面的构建说明进行操作。将 `-DGGML_CUDA=ON` 改为 `-DGGML_CUDA=OFF` 如果你没有 GPU，或者只想使用 CPU 推理。 **对于 Apple Mac / Metal 设备**，设置 `-DGGML_CUDA=OFF` 然后按常规继续——Metal 支持默认开启。
```bash
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch origin pull/24523/head:minimax-m3
git checkout minimax-m3
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-cli llama-server
```
{% endstep %}
{% step %}
你现在可以使用 `llama.cpp` 直接加载和下载模型，就像 `ollama run`。首先，选择你想要的量化类型，例如 `Q2_K_XL`。同时使用 `export LLAMA_CACHE="folder"` 来强制 `llama.cpp` 保存到指定位置。注意，此下载过程可能非常慢，因此最好使用下一节中的手动下载方式。
```bash
export LLAMA_CACHE="unsloth/MiniMax-M3-GGUF"
./build/bin/llama-cli \\
-hf unsloth/MiniMax-M3-GGUF:UD-IQ1_M \\
--temp 1.0 \\
--top-p 0.95 \\
--top-k 40
```
{% hint style="info" %}
注意：目前尚不支持 MiniMax Sparse Attention，因此推理会回退到稠密注意力。
{% endhint %}
{% endstep %}
{% step %}
如果你想手动下载模型，我们可以通过下面的代码下载模型（在安装 `pip install huggingface_hub`）。如果下载卡住，请参见： [Hugging Face Hub、XET 调试](/docs/zh/ji-chu-zhi-shi/troubleshooting-and-faqs/hugging-face-hub-xet-debugging.md)
```bash
hf download unsloth/MiniMax-M3-GGUF \\
--local-dir unsloth/MiniMax-M3-GGUF \\
--include "*UD-IQ1_M*" # 4 位请使用 "*UD-IQ4_XS*"
```
{% endstep %}
{% step %}
你可以修改 `--threads 32` CPU 线程数， `--ctx-size 32768` 用于上下文长度， `--n-gpu-layers 2` 用于 GPU 卸载的层数。如果你的 GPU 内存不足，请尝试调整它。如果你只进行 CPU 推理，也请将其移除。记住，MSA 目前尚不支持，所以请保持 `--ctx-size` 适度——在非常长的上下文中使用稠密注意力会占用大量内存。
{% code overflow="wrap" %}
```bash
./build/bin/llama-cli \\
--model unsloth/MiniMax-M3-GGUF/UD-IQ1_M/MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf \\
--temp 1.0 \\
--top-p 0.95 \\
--top-k 40
```
{% endcode %}
{% endstep %}
{% endstepper %}
## 📊 基准测试
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://unsloth.ai/docs/zh/mo-xing/minimax-m3.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
