{% endstep %}
{% endstepper %}
### 🦙 Llama.cpp 指南
{% stepper %}
{% step %}
获取指定的 `llama.cpp` PR 于 [**GitHub 这里**](https://github.com/ggml-org/llama.cpp/pull/24523)。你也可以按照下面的构建说明操作。将 `-DGGML_CUDA=ON` 改为 `-DGGML_CUDA=OFF` 如果你没有 GPU，或者只想进行 CPU 推理。 **对于 Apple Mac / Metal 设备**，设置 `-DGGML_CUDA=OFF` 然后按常规继续——Metal 支持默认开启。
```bash
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch origin pull/24523/head:minimax-m3
git checkout minimax-m3
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-cli llama-server
```
{% endstep %}
{% step %}
你现在可以使用 `llama.cpp` 直接加载和下载模型，就像 `ollama run`。首先，选择你想要的量化类型，例如 `Q2_K_XL`。还可以使用 `export LLAMA_CACHE="folder"` 来强制 `llama.cpp` 来保存到指定位置。注意，此下载过程可能非常慢，因此最好使用下一节中的手动下载方式。
```bash
export LLAMA_CACHE="unsloth/MiniMax-M3-GGUF"
./build/bin/llama-cli \\
-hf unsloth/MiniMax-M3-GGUF:UD-IQ1_M \\
--temp 1.0 \
--top-p 0.95 \\
--top-k 40
```
{% hint style="info" %}
注意：MiniMax Sparse Attention 目前尚不支持，因此推理会回退到稠密注意力。
{% endhint %}
{% endstep %}
{% step %}
如果你想手动下载模型，我们可以通过下面的代码下载模型（在安装 `pip install huggingface_hub`）之后。如果下载卡住，请查看： [Hugging Face Hub，XET 调试](/docs/zh/ji-chu/troubleshooting-and-faqs/hugging-face-hub-xet-debugging.md)
```bash
hf download unsloth/MiniMax-M3-GGUF \\
--local-dir unsloth/MiniMax-M3-GGUF \\
--include "*UD-IQ1_M*" # 4 位请使用 "*UD-IQ4_XS*"
```
{% endstep %}
{% step %}
你可以修改 `--threads 32` 以设置 CPU 线程数， `--ctx-size 32768` 以设置上下文长度， `--n-gpu-layers 2` 用于 GPU 卸载的层数。若 GPU 显存不足，请尝试调整它。如果你只有 CPU 推理，也请将其移除。请记住 MSA 仍不受支持，所以请保持 `--ctx-size` 适度——在非常长的上下文中，稠密注意力会消耗大量内存。
{% code overflow="wrap" %}
```bash
./build/bin/llama-cli \\
--model unsloth/MiniMax-M3-GGUF/UD-IQ1_M/MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf \\
--temp 1.0 \
--top-p 0.95 \\
--top-k 40
```
{% endcode %}
{% endstep %}
{% endstepper %}
## 📊 基准测试
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://unsloth.ai/docs/zh/mo-xing/minimax-m3.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.