> For the complete documentation index, see [llms.txt](https://unsloth.ai/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://unsloth.ai/docs/zh/ji-cheng/jiang-python-sdk-lian-jie-dao-unsloth.md).

# 将 Python SDK 连接到 Unsloth

指南：使用官方 OpenAI 或 Anthropic SDK 从 Python 调用 Unsloth 的本地 API，包括流式输出、视觉、函数调用，以及 Unsloth 内置的服务器端工具。

Unsloth 在同一个基础 URL 上提供三种与 OpenAI 兼容的协议：Chat Completions、Responses 和 Anthropic Messages，因此主流 Python SDK 都可以使用。你只需修改客户端上的 `base_url` 和 `api_key`  ；其他功能（流式传输、工具调用、视觉、结构化输出）的行为都与 SDK 文档中的说明一致。本页介绍大多数开发者最先使用的两种 SDK：官方的 **OpenAI Python SDK** 和官方的 **Anthropic Python SDK**.

{% hint style="info" %}
如果不确定该使用哪个 URL、密钥或模型名称，请先阅读 API 概览。该指南会介绍如何启动服务、加载模型，以及创建 `sk-unsloth-…` 密钥。
{% endhint %}

### 🔑 前置条件

运行下面任何代码片段之前，你需要准备：

* **在本地运行 Unsloth** 并加载一个模型（请留意端口：通常是 `8000` 或 `8888`).
* **一个 `sk-unsloth-…` API 密钥** 在 **设置 → API**.
* **一个模型名称。** Unsloth 中 GGUF 模型的名称（例如 `qwen-local`, `unsloth/Qwen3.6-27B-GGUF`）。如果忘记了，可以运行：

  ```bash
  curl http://localhost:8888/v1/models -H "Authorization: Bearer sk-unsloth-…"
  ```

  并复制 `id` 字段。

将密钥设为环境变量，这样就不必把它粘贴到代码中：

```bash
export UNSLOTH_STUDIO_AUTH_TOKEN=sk-unsloth-xxxxxxxxxxxx
```

#### 🤖 OpenAI SDK

Unsloth 的 `/v1/chat/completions` 端点可直接替代 OpenAI Python SDK 的对应端点。客户端会像对待其他任何兼容 OpenAI 的服务提供商一样使用 Unsloth。

**1. 安装 SDK：**

```bash
pip install openai
```

**2. 创建一个** 连接到 Unsloth 的客户端：

```python
import os
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8888/v1",              # your unsloth port + /v1
    api_key=os.environ["UNSLOTH_STUDIO_AUTH_TOKEN"],     # your sk-unsloth-… key
)
```

#### 基础聊天补全

```python
response = client.chat.completions.create(
    model="default",                               # the name you gave the model in unsloth or default
    messages=[
        {"role": "user", "content": "Give me two facts about Paris"}
    ],
)
print(response.choices[0].message.content)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FZkxVYS7fcvwXZ8AxvaBF%2FScreenshot%202026-04-22%20at%202.38.44%E2%80%AFAM.png?alt=media&amp;token=06dece4e-dade-4c2d-bcdb-7c946cdd1290" alt=""><figcaption></figcaption></figure>

#### 流式传输

将 `stream=True` ，并遍历返回的生成器：

```python
stream = client.chat.completions.create(
    model="qwen-local",
    messages=[{"role": "user", "content": "Write a haiku about locally-run LLMs."}],
    stream=True,
)
for chunk in stream:
    if chunk.choices:
        delta = chunk.choices[0].delta.content
        if delta:
            print(delta, end="", flush=True)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2F0eItNT5zanSB8rXY4QHn%2Fopenai_streaming.png?alt=media&amp;token=2922bbf3-69da-409f-b954-76b6259233f8" alt=""><figcaption></figcaption></figure>

#### 图像（视觉）

将图像作为 `image_url` 内容部分附加。Unsloth 接受 HTTP(S) URL 或 `data:` base64 URI：

```python
import base64
from pathlib import Path

img_b64 = base64.b64encode(Path("test.jpg").read_bytes()).decode()

response = client.chat.completions.create(
    model="default",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image_url",
                    "image_url": {"url": f"data:image/jpeg;base64,{img_b64}"},
                },
                {"type": "text", "text": "What's in this image?"},
            ],
        }
    ],
)
print(response.choices[0].message.content)
```

{% hint style="info" %}
加载的模型必须支持多模态。如果加载的是纯文本模型，视觉请求在结构上会成功，但模型无法“看到”图像。
{% endhint %}

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FTNCX4qEMj8zFSAsOJw2Y%2Fopenai_sdk_vision_test.png?alt=media&amp;token=caa22501-66fc-4e3c-b4a0-636ebe5af4bf" alt=""><figcaption></figcaption></figure>

#### 函数调用（OpenAI 工具）

传入 OpenAI 风格的 `tools` 以及（可选的） `tool_choice` ，Unsloth 会将它们转发到后端。客户端负责执行每个工具调用，并在下一轮对话中返回结果：

```python
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a city",
            "parameters": {
                "type": "object",
                "properties": {
                    "city": {"type": "string", "description": "City name, e.g. 'Paris'"},
                },
                "required": ["city"],
            },
        },
    }
]

response = client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "What's the weather in Perth right now?"}],
    tools=tools,
    tool_choice="auto",
)

tool_call = response.choices[0].message.tool_calls[0]
print(tool_call.function.name, tool_call.function.arguments)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2F8nzgxHsN8J23B6duQ3Zv%2Fopenai_sdk_function_calling.png?alt=media&amp;token=3551592d-31b2-4fa6-94a9-6a9f0974c21a" alt=""><figcaption></figcaption></figure>

#### Unsloth 服务端工具（简写）

除了 OpenAI 风格的客户端工具外，Unsloth 还可以在服务端执行 **Python**, **终端** （Shell 命令）和 **网络搜索** ，并自动将结果流式传回。通过以下参数启用： `extra_body` ，这样这些字段就会直接传递给 Unsloth：

```python
stream = client.chat.completions.create(
    model="default",
    messages=[{"role": "user", "content": "What is 123 * 456? Use Python to compute it."}],
    stream=True,
    extra_body={
        "enable_tools": True,
        "enabled_tools": ["python", "web_search"],
        "session_id": "my-session",
    },
)
for chunk in stream:
    if chunk.choices:
        delta = chunk.choices[0].delta.content
        if delta:
            print(delta, end="", flush=True)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FvdZuLwIh7LT0C8y7AWaR%2FScreenshot%202026-04-22%20at%203.01.06%E2%80%AFAM.png?alt=media&amp;token=35ff3039-d941-406b-b3fc-5e1e564c7651" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FnW60ezYCOJQmTjd52bnu%2FScreenshot%202026-04-22%20at%203.05.23%E2%80%AFAM.png?alt=media&amp;token=f7125db4-cb94-46f7-974d-abe58ec348a7" alt=""><figcaption></figcaption></figure>

session\_id `session_id` 是可选项。使用它可以在多次调用之间保留工具状态（例如 Python 内核）。

{% hint style="info" %}
`enabled_tools` 目前支持 `"python"`, `"terminal"`和 `"web_search"`。工具结果会以 `tool_result` 事件的形式流式传回，以便模型在下一轮中查看。
{% endhint %}

**列出模型**

```python
models = client.models.list()
for m in models.data:
    print(m.id)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2F6GKdrximwyplotWu3u5s%2FScreenshot%202026-04-22%20at%203.06.57%E2%80%AFAM.png?alt=media&amp;token=48d17278-e4b0-4437-a426-5f12c0c45b1e" alt=""><figcaption></figcaption></figure>

#### 🧠 Anthropic SDK

Unsloth 的 `/v1/messages` 端点可直接替代 Anthropic Python SDK。

**1. 安装 SDK：**

```bash
pip install anthropic
```

**2. 创建一个** 连接到 Unsloth 的客户端：

{% code overflow="wrap" %}

```python
import os
from anthropic import Anthropic

client = Anthropic(
    base_url="http://localhost:8888", # your unsloth port (no /v1 here - the SDK adds it)
    api_key="dummy", # a random none empty value
    default_headers={"Authorization": f"Bearer {os.environ['UNSLOTH_STUDIO_AUTH_TOKEN']}"} # your sk-unsloth-… key
)
```

{% endcode %}

#### 基本消息

```python
message = client.messages.create(
    model="default",
    max_tokens=1024,
    messages=[
        {"role": "user", "content": "Say hello in three languages."}
    ],
)
print(message.content[0].text)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2Fc4yQCiRHjebKomV4H2Eh%2FScreenshot%202026-04-22%20at%202.29.04%E2%80%AFAM.png?alt=media&amp;token=4ad983da-83fd-48c0-8fa8-1fbe92fcfd40" alt=""><figcaption></figcaption></figure>

#### 流式传输

SDK 提供了一个上下文管理器，可生成文本增量：

```python
with client.messages.stream(
    model="default",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Explain LoRA in two sentences."}],
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
```

#### 图像（视觉）

Anthropic 风格的图像内容使用 `source` 块，其中包含 base64 数据：

```python
import base64
from pathlib import Path

img_b64 = base64.standard_b64encode(Path("photo.jpg").read_bytes()).decode()

message = client.messages.create(
    model="default",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": "image/jpeg",
                        "data": img_b64,
                    },
                },
                {"type": "text", "text": "What's in this image?"},
            ],
        }
    ],
)
print(message.content[0].text)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FKI75sgbWxmo8BNXQEP6s%2Fanthropic_sdk_vision_test.png?alt=media&amp;token=f72c2dd6-b811-41a7-82bb-a2a6f30513ac" alt=""><figcaption></figcaption></figure>

#### 工具调用（Anthropic 工具）

传入 Anthropic 风格的 `tools` 以及 `input_schema` ，Unsloth 会原生转发它们：

```python
tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "input_schema": {
            "type": "object",
            "properties": {
                "city": {"type": "string", "description": "城市名称，例如 'Tokyo'"},
            },
            "required": ["city"],
        },
    }
]

message = client.messages.create(
    model="default",
    max_tokens=1024,
    tools=tools,
    tool_choice={"type": "auto"},
    messages=[{"role": "user", "content": "What's the weather in Tokyo?"}],
)

for block in message.content:
    if block.type == "tool_use":
        print(block.name, block.input)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FWnWPZdZOtV2o1o5FIzlD%2Fanthropic_tool_use_get_weather_tokyo.png?alt=media&amp;token=39587916-f4c0-45cf-8e59-11245b3432b8" alt=""><figcaption></figcaption></figure>

#### Unsloth 服务端工具（简写）

相同的 `enable_tools` / `enabled_tools` / `session_id` 简写方式也适用于 `/v1/messages` 通过 `extra_body`:

```python
with client.messages.stream(
    model="default",
    max_tokens=1024,
    messages=[{"role": "user", "content": "Search for Python 3.13 features and summarize."}],
    extra_body={
        "enable_tools": True,
        "enabled_tools": ["web_search", "python"],
        "session_id": "my-session",
    },
) as stream:
    for text in stream.text_stream:
        print(text, end="", flush=True)
```

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2F2wA6aleUHPJN1gAB3ClL%2Fserver_side_tool_call_backend.png?alt=media&amp;token=7440befb-d517-473a-a431-c6b4cb413ca5" alt=""><figcaption></figcaption></figure>

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FwcIerVboFjEZPkI2vU0L%2Fserver_side_tool_call_output.png?alt=media&amp;token=33800495-70d1-4a5f-9173-cde144abecb3" alt=""><figcaption></figcaption></figure>

Unsloth 会为模型查看每个工具调用的输出生成自定义 `tool_result` SSE 事件。Anthropic SDK 会在其事件流中原样传递这些事件。

#### JSON 解码（`response_format`)

Unsloth 通过 `response_format`支持 OpenAI 风格的结构化输出。传入 JSON Schema 后，模型会受到约束，只能生成与其匹配的 JSON。

````python
import json
import os
import re
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8888/v1",
    api_key=os.environ["UNSLOTH_STUDIO_AUTH_TOKEN"],
)

response = client.chat.completions.create(
    model="default",
    stream=False,
    temperature=0.0,
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": "Pick a country: Japan, Egypt, or Peru. Explain why in one sentence.",
        },
    ],
    response_format={
        "type": "json_schema",
        "json_schema": {
            "name": "country_pick",
            "schema": {
                "type": "object",
                "properties": {
                    "country": {"type": "string", "enum": ["Japan", "Egypt", "Peru"]},
                    "reason":  {"type": "string"},
                },
                "required": ["country", "reason"],
                "additionalProperties": False,
            },
            "strict": True,
        },
    },
)

raw = response.choices[0].message.content
# Strip the markdown fence Gemma 4 wraps around the JSON, then parse.
cleaned = re.sub(r"^```(?:json)?\s*", "", raw)
cleaned = re.sub(r"\s*```$", "", cleaned)
parsed = json.loads(cleaned)

print(json.dumps(parsed, indent=2))
print()
print("country:", parsed["country"])
print("reason :", parsed["reason"])
````

这个 `strict: True` 标志会让 Unsloth 在解码期间强制遵循该架构，而不是依赖模型自行遵守。 `additionalProperties: False` 和 `required` 的作用与标准 JSON Schema 中相同。

终端输出大致如下：

<figure><img src="https://2657992854-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FxhOjnexMCB3dmuQFQ2Zq%2Fuploads%2FmhPxaVA6sxkyF3nFf51f%2FScreenshot%202026-05-05%20at%201.46.44%E2%80%AFPM.png?alt=media&amp;token=50b1600d-9e5a-4d22-819a-274c6bfbf8f7" alt=""><figcaption></figcaption></figure>

### 🧪 选择 SDK

两种 SDK 都能与 Unsloth 配合使用。选择哪一种取决于你技术栈的其他部分：

* 如果你的代码已经依赖 OpenAI Python 包、想使用 OpenAI 风格的 **OpenAI SDK** ， `tools` / `tool_choice`，或者计划调用 Responses API，请使用
* 如果你的代码已经依赖 Anthropic 包、偏好 Anthropic 的 **Anthropic SDK** ， `input_schema` 工具格式，或者想使用 Anthropic 原生的流式事件类型，请使用。

你可以在同一个项目中同时使用两者。Unsloth 在同一端口上提供这两种 SDK 的服务，因此一个 `sk-unsloth-…` 密钥即可通过两者的身份验证。

### ❔ 故障排除

**`401 Unauthorized`**  如果 `UNSLOTH_STUDIO_AUTH_TOKEN` 环境变量未设置或密钥有误。请重新导出该变量，并使用以下命令确认： `echo $UNSLOTH_STUDIO_AUTH_TOKEN`.

**`404 Not Found` （来自 OpenAI SDK）** 请检查 `base_url` 是否以 `/v1`结尾。OpenAI SDK 会原样将端点路径追加到基础 URL 后面。

**`404 Not Found` （来自 Anthropic SDK）** 请检查 `base_url` 是否 **不** 以 `/v1`结尾。Anthropic SDK 会自行添加 `/v1/messages` 。

**`extra_body` 字段未传到 Unsloth** 请确保使用的是较新版本的 `openai` / `anthropic` SDK。旧版本会静默丢弃未知字段。使用以下命令升级： `pip install -U openai anthropic`.

**流式输出“卡住”，然后一次性全部输出** 包装输出的组件在缓冲数据。在脚本中， `print(..., flush=True)`；在笔记本中通常没问题；如果经过代理，请在代理上禁用响应缓冲。

有关端点级问题（端口错误、模型无法加载、连接中断等），请参阅 API 概览页面。

### 可选：设置服务器默认值

使用 Python SDK 连接之前，可以通过 `unsloth run` 命令配置服务器的默认行为。

```bash
# 使用自定义默认值启动服务器
unsloth run \
  --model unsloth/Qwen3-1.7B-GGUF \
  --reasoning off \
  --temp 0.6 \
  -p 8888
```

使用 `--reasoning off` 关闭思考功能；对于支持推理的模型，使用 `--reasoning on` 开启该功能。

```bash
# 在本地网络上公开 API
unsloth run \
  --model unsloth/Qwen3-1.7B-GGUF \
  -H 0.0.0.0 \
  -p 8888
```

这会在 `0.0.0.0:8888`上启动服务器，让本地网络中的其他设备也能连接。

当请求未指定自己的生成参数时，这些设置将作为服务器默认值。

请求级别的值，例如 `temperature`, `top_p`, `max_tokens`，以及 `stream` 仍可覆盖该请求的默认值。


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://unsloth.ai/docs/zh/ji-cheng/jiang-python-sdk-lian-jie-dao-unsloth.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.
