> For the complete documentation index, see [llms.txt](https://unsloth.ai/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://unsloth.ai/docs/zh/kuai-su-kai-shi/reinforcement-learning-rl-guide/advanced-rl-documentation/fp16-vs-bf16-for-rl.md). # RL 中的 FP16 与 BF16 通过 FP16 击败训练-推理不匹配 https\://arxiv.org/pdf/2510.26788 表明使用 float16 比 bfloat16 更好 ### Float16 与 Bfloat16 有一篇题为“**通过 FP16 解决训练-推理不匹配**" 展示了在进行强化学习时，使用 float16 精度在很多情况下会比使用 bfloat16 好得多。

事实上，生成越长，使用 bfloat16 时情况就越糟：

我们做了一项调查，并且 **确实发现 float16 更稳定** 相比 bfloat16，其梯度范数要小得多，见和 {% columns %} {% column width="50%" %}

{% endcolumn %} {% column width="50%" %}

{% endcolumn %} {% endcolumns %} ### :exploding\_head:A100 级联注意力 Bug 根据和，较旧版本的 vLLM（0.11.0 之前）在 A100 及类似 GPU 上的注意力机制存在故障。请更新 vLLM！如果我们检测到较旧的 vLLM 版本，在 Unsloth 强化学习期间我们也会默认禁用级联注意力。

不同的硬件也会改变结果，其中更新、更昂贵的 GPU 在推理端和训练端之间的 KL 差异更小：

### :fire:在 Unsloth RL 中使用 float16 要在 Unsloth GRPO 和 RL 中使用 float16 精度，你只需要设置 `dtype = torch.float16` 剩下的就交给我们！ {% code overflow="wrap" %} ```python from unsloth import FastLanguageModel import torch max_seq_length = 2048 # 可为更长的推理轨迹提高 lora_rank = 32 # 更大的 rank = 更聪明，但更慢 model, tokenizer = FastLanguageModel.from_pretrained( model_name = "unsloth/Qwen3-4B-Base", max_seq_length = max_seq_length, load_in_4bit = False, # LoRA 16bit 时为 False fast_inference = True, # 启用 vLLM 快速推理 max_lora_rank = lora_rank, gpu_memory_utilization = 0.9, # 如果显存不足则调低 dtype = torch.float16, # 使用 torch.float16、torch.bfloat16 ) ``` {% endcode %} --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://unsloth.ai/docs/zh/kuai-su-kai-shi/reinforcement-learning-rl-guide/advanced-rl-documentation/fp16-vs-bf16-for-rl.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.