{% endstep %}
{% endstepper %}
### 🦙 Llama.cpp 指南
{% stepper %}
{% step %}
获取特定的 `llama.cpp` PR 在 [**GitHub 这里**](https://github.com/ggml-org/llama.cpp/pull/24523)。你也可以按照下面的构建说明进行操作。将 `-DGGML_CUDA=ON` 更改为 `-DGGML_CUDA=OFF` 如果你没有 GPU，或者只想进行 CPU 推理。 **对于 Apple Mac / Metal 设备**，设置 `-DGGML_CUDA=OFF` 然后像往常一样继续——Metal 支持默认开启。
```bash
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch origin pull/24523/head:minimax-m3
git checkout minimax-m3
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-cli llama-server
```
{% endstep %}
{% step %}
你现在可以使用 `llama.cpp` 直接加载和下载模型，就像 `ollama run`。首先，选择你想要的量化类型，例如 `Q2_K_XL`。同时使用 `export LLAMA_CACHE="folder"` 以强制 `llama.cpp` 保存到特定位置。请注意，此下载过程可能非常慢，因此最好使用下一节中的手动下载流程。
```bash
export LLAMA_CACHE="unsloth/MiniMax-M3-GGUF"
./build/bin/llama-cli \\
-hf unsloth/MiniMax-M3-GGUF:UD-IQ1_M \\
--temp 1.0 \\
--top-p 0.95 \
--top-k 40
```
{% hint style="info" %}
注意：MiniMax Sparse Attention 目前尚不受支持，因此推理会回退到稠密注意力。
{% endhint %}
{% endstep %}
{% step %}
如果你想手动下载模型，我们可以在安装 `pip install huggingface_hub`之后通过下面的代码下载模型。如果下载卡住，请参见： [Hugging Face Hub、XET 调试](/docs/zh/ji-chu-zhi-shi/troubleshooting-and-faqs/hugging-face-hub-xet-debugging.md)
```bash
hf download unsloth/MiniMax-M3-GGUF \\
--local-dir unsloth/MiniMax-M3-GGUF \\
--include "*UD-IQ1_M*" # 4 位请使用 "*UD-IQ4_XS*"
```
{% endstep %}
{% step %}
你可以编辑 `--threads 32` 来设置 CPU 线程数， `--ctx-size 32768` 来设置上下文长度， `--n-gpu-layers 2` 用于 GPU 卸载时可卸载的层数。若你的 GPU 显存不足，请尝试调整它。如果你只进行 CPU 推理，也要把它移除。记住 MSA 目前尚不受支持，所以请保持 `--ctx-size` 保持适中——在超长上下文下，稠密注意力会占用大量内存。
{% code overflow="wrap" %}
```bash
./build/bin/llama-cli \\
--model unsloth/MiniMax-M3-GGUF/UD-IQ1_M/MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf \\
--temp 1.0 \\
--top-p 0.95 \
--top-k 40
```
{% endcode %}
{% endstep %}
{% endstepper %}
## 📊 基准测试
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://unsloth.ai/docs/zh/mo-xing/minimax-m3.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.