第一张图（上图）绘制了 `yahma/alpaca-cleaned` 替换为 `max_length = 2048`上的进度，Unsloth 新版本带打包 + 内核（酒红色）与 Unsloth 旧版本（灰色）对比。两者都训练于 `max_steps = 500`，但我们把横轴绘制为实际时间。注意，在相同步数（以及只多一点实际时间）内，打包情况下我们几乎训练了一个 epoch 的 40%，而未打包情况下连 5% 的 epoch 都不到。
同样地，第二张图（上图）绘制了相同运行的损失，这次横轴是训练步数。注意，损失在规模和趋势上都匹配，但打包情况下的损失波动更小，因为模型在每个训练步骤看到的 token 更多。
### :sparkles:如何启用打包？
**先更新 Unsloth，默认就会启用无填充**！所以所有训练都会立刻快 1.1 到 2 倍，至少减少 30% 的内存占用，而且损失曲线指标没有任何变化！
{% code overflow="wrap" %}
```bash
pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth
pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth_zoo
```
{% endcode %}
我们还通过 Xformers 支持 Flash Attention 3，也支持 SDPA、Flash Attention 2，而且这适用于旧 GPU（Tesla T4、RTX 2080）以及新的 GPU，如 H100、B200 等！样本打包适用于 *无论选择哪种注意力后端或模型家族*，所以可以享受之前这些快速注意力实现带来的相同加速！
如果你想启用显式打包，那么添加 `packing = True` 即可实现最多 5 倍更快的训练！
{% hint style="warning" %}
注意 `packing=True` 会改变训练损失，并会让数据集行数被截断，因为多个短序列会被打包成 1 个序列。你可能会看到数据集中的示例数量减少。
如果不想得到不同的训练损失数值，只需设置 `packing=False` ，我们就会启用自动无填充，这本身就已经能让训练更快！
{% endhint %}
```python
from unsloth import FastLanguageModel
from trl import SFTTrainer, SFTConfig
model, tokenizer = FastLanguageModel.from_pretrained(
“unsloth/Qwen3-14B”，
)
trainer = SFTTrainer(
model = model,
processing_class = tokenizer,
train_dataset = dataset,
args = SFTConfig(
per_device_train_batch_size = 1,
max_length = 4096,
…,
packing = True, # 需要启用样本打包！
),
)
trainer.train()
```
我们所有的 notebook 都会自动变得更快（无需做任何事情）。见 [Unsloth 笔记本](/docs/zh/kuai-su-kai-shi/unsloth-notebooks.md)
{% columns %}
{% column %}
Qwen3 14B 更快：
{% embed url="" %}
{% endcolumn %}
{% column %}
Llama 3.1 对话更快：
{% embed url="" %}
{% endcolumn %}
{% endcolumns %}
谢谢你！如果你感兴趣，请查看我们的 [500K Context Training](/docs/zh/bo-ke/500k-context-length-fine-tuning.md) 博客， [内存高效的 RL](/docs/zh/kuai-su-kai-shi/reinforcement-learning-rl-guide/memory-efficient-rl.md) 博客，以及 [Long Context gpt-oss](/docs/zh/mo-xing/gpt-oss-how-to-run-and-fine-tune/long-context-gpt-oss-training.md) 博客，了解更多关于内核和性能提升的主题！
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://unsloth.ai/docs/zh/bo-ke/3x-faster-training-packing.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
