> For the complete documentation index, see [llms.txt](https://unsloth.ai/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://unsloth.ai/docs/zh/ji-chu-zhi-shi/unsloth-benchmarks.md).

# Unsloth 基准测试

Unsloth 记录的 NVIDIA GPU 基准测试。

* 如需更详细的基准测试，请阅读我们的 [Llama 3.3 博客](https://unsloth.ai/blog/llama3-3).
* Unsloth 的基准测试也由以下机构进行 [🤗Hugging Face](https://huggingface.co/blog/unsloth-trl).

{% hint style="warning" %}
如果你的速度起初看起来较慢，很可能是因为 `torch.compile` 通常需要约 5 分钟（或更久）来预热并完成编译。请确保你测量的是吞吐量 **在** 它完全加载后，因为在更长的运行中，Unsloth 应该会快得多。
{% endhint %}

在 H100 和 [Blackwell](/docs/zh/bo-ke/fine-tuning-llms-with-blackwell-rtx-50-series-and-unsloth.md) GPU 上测试。我们使用 Alpaca 数据集、批量大小为 2、梯度累积步数为 4、rank = 32 进行测试，并在所有线性层（q、k、v、o、gate、up、down）上应用了 QLoRA：

<table data-full-width="false"><thead><tr><th>模型</th><th>VRAM</th><th>🦥Unsloth 速度</th><th>🦥VRAM 降低</th><th>🦥更长上下文</th><th>😊Hugging Face + FA2</th></tr></thead><tbody><tr><td>Llama 3.3（70B）</td><td>80GB</td><td>2倍</td><td>>75%</td><td>长 13 倍</td><td>1倍</td></tr><tr><td>Llama 3.1（8B）</td><td>80GB</td><td>2倍</td><td>>70%</td><td>长 12 倍</td><td>1倍</td></tr></tbody></table>

## 上下文长度基准测试

{% hint style="info" %}
由于我们的 [梯度检查点](https://unsloth.ai/blog/long-context) 算法 + Apple 的 CCE 算法！
{% endhint %}

### **Llama 3.1（8B）最大上下文长度**

我们在批量大小为 1 的情况下，对 Llama 3.1（8B）Instruct 进行了测试，并在所有线性层（Q、K、V、O、gate、up 和 down）上以 rank = 32 进行了 4bit QLoRA。我们将所有序列填充到某个最大序列长度，以模拟长上下文微调工作负载。

| GPU 显存 | 🦥Unsloth 上下文长度 | Hugging Face + FA2 |
| ------ | --------------- | ------------------ |
| 8 GB   | 2,972           | OOM                |
| 12 GB  | 21,848          | 932                |
| 16 GB  | 40,724          | 2,551              |
| 24 GB  | 78,475          | 5,789              |
| 40 GB  | 153,977         | 12,264             |
| 48 GB  | 191,728         | 15,502             |
| 80 GB  | 342,733         | 28,454             |

### **Llama 3.3（70B）最大上下文长度**

我们在一块 80GB A100 上对 Llama 3.3（70B）Instruct 进行了测试，并在所有线性层（Q、K、V、O、gate、up 和 down）上以 rank = 32 进行了 4bit QLoRA，批量大小为 1。我们将所有序列填充到某个最大序列长度，以模拟长上下文微调工作负载。

| GPU 显存 | 🦥Unsloth 上下文长度 | Hugging Face + FA2 |
| ------ | --------------- | ------------------ |
| 48 GB  | 12,106          | OOM                |
| 80 GB  | 89,389          | 6,916              |


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://unsloth.ai/docs/zh/ji-chu-zhi-shi/unsloth-benchmarks.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.