然后我们加载 LoRA 并测试它。我们的推理模型要好得多——但并不总是正确，因为我们只训练了大约一个小时；如果我们延长序列长度并训练更久，它会更好！
你可以按照我们的 [这里的指南](https://unsloth.ai/docs/zh/kuai-su-kai-shi/reinforcement-learning-rl-guide/pages/e722e86e330786e5915445b91f900d9f9e0ba067#id-7.-running--saving-the-model).
将模型保存为 GGUF、Ollama 等格式。如果你仍然没有得到任何推理过程，可能是训练步数太少，或者你的奖励函数/验证器不够理想。
{% endstep %}
{% step %}
#### 保存你的模型
我们有多种方式来保存你的微调模型，但我们将重点介绍最简单、最流行的方法，你可以在这里阅读更多关于 [这里](/docs/zh/ji-chu-zhi-shi/inference-and-deployment.md)
**以 16 位精度保存**
你可以使用以下命令以 16 位精度保存模型：
```python
# 保存为 16 位精度
model.save_pretrained_merged("model", tokenizer, save_method="merged_16bit")
```
**推送到 Hugging Face Hub**
为了分享你的模型，我们将使用 `push_to_hub_merged` 方法将其推送到 Hugging Face Hub。这样可以将模型保存为多种量化格式。
```python
# 推送到 Hugging Face Hub（需要令牌）
model.push_to_hub_merged(
"your-username/model-name", tokenizer, save_method="merged_16bit", token="your-token"
)
```
**以 GGUF 格式保存用于 llama.cpp**
Unsloth 也支持保存为 **GGUF 格式**，使其兼容 **llama.cpp** 和 **Ollama**.
```python
model.push_to_hub_gguf(
"your-username/model-name",
tokenizer,
quantization_method=["q4_k_m", "q8_0", "q5_k_m"],
token="your-token",
)
```
一旦保存为 GGUF 格式，模型就可以使用 **llama.cpp** 在轻量级环境中轻松部署，或者用于其他推理引擎。
{% endstep %}
{% endstepper %}
## 视频教程
以下是由一些出色的 YouTuber 制作的视频教程，我们认为他们非常棒！
{% embed url="" %}
{% columns %}
{% column width="50%" %}
{% embed url="" %}
适合学习如何准备你的数据集，以及强化学习 + GRPO 基础知识背后的解释
{% endembed %}
{% embed url="" %}
{% endcolumn %}
{% column width="50%" %}
{% embed url="" %}
{% embed url="" %}
在你自己的设备上进行本地 GRPO
{% endembed %}
{% endcolumn %}
{% endcolumns %}
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.
Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:
```
GET https://unsloth.ai/docs/zh/kuai-su-kai-shi/reinforcement-learning-rl-guide/tutorial-train-your-own-reasoning-model-with-grpo.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
%20(1)%20(1).png?alt=media)
