Fine-tuning LLMs on AMD GPUs with Unsloth Guide

One-line installer

To install Unsloth on AMD, the easiest way is to download our Desktop app.

Download for WindowsDownload for Linux For manual installation:

Linux :

curl -fsSL https://unsloth.ai/install.sh | sh

Windows (PowerShell):

irm https://unsloth.ai/install.ps1 | iex

The manual steps below are for users who want to install the Unsloth Python library on AMD with the required dependencies.

Make a new isolated environment (Optional)

The steps below are for Linux. On Windows, use the Desktop app or the PowerShell installer above: they install AMD's Windows build of PyTorch for you.

To not break any system packages, make an isolated environment. Reminder to check what Python version you have! It might be python3, python3.12, python3.13 etc. Python 3.10 to 3.13 works.

apt update && apt install python3.13-venv -y
python3.13 -m venv unsloth_env
source unsloth_env/bin/activate
pip install uv

Install PyTorch

PyPI only has the NVIDIA (CUDA) build of PyTorch, so install the ROCm build first from the PyTorch index that matches your ROCm version. Check it with amd-smi version (the ROCm version: line), then:

uv pip install "torch<2.15" torchvision torchaudio \
    --index-url https://download.pytorch.org/whl/rocm7.2 --upgrade --force-reinstall

Change rocm7.2 to your ROCm version. Available index tags are rocm6.0 to rocm6.4 and rocm7.0 to rocm7.2. ROCm 6.5 to 6.9 uses rocm6.4, and ROCm 7.3 or newer uses rocm7.2. ROCm 6.0 or newer is required.

Or let this command pick the tag for you:

ROCM_VER="$(amd-smi version 2>/dev/null | sed -n 's/.*ROCm version: \([0-9]*\.[0-9]*\).*/\1/p' | head -n 1)"
[ -n "$ROCM_VER" ] || ROCM_VER="$(cut -d. -f1,2 /opt/rocm/.info/version 2>/dev/null)"
case "$ROCM_VER" in
    6.[0-4]|7.[0-2]) ROCM_TAG="rocm$ROCM_VER" ;;
    6.*) ROCM_TAG="rocm6.4" ;;
    *) ROCM_TAG="rocm7.2" ;;
esac
echo "ROCm ${ROCM_VER:-unknown}: using $ROCM_TAG"
uv pip install "torch<2.15" torchvision torchaudio \
    --index-url "https://download.pytorch.org/whl/$ROCM_TAG" --upgrade --force-reinstall

Install Unsloth

Unsloth on PyPI depends on torch, so a plain install can swap the ROCm PyTorch you just installed for the CUDA one from PyPI. To prevent that, pin your installed PyTorch and install with uv pip (not plain pip):

uv pip freeze | grep -E '^(torch|torchvision|torchaudio)==' > torch-pins.txt
echo "triton ; sys_platform == 'never'" >> torch-pins.txt
echo "xformers ; sys_platform == 'never'" >> torch-pins.txt
uv pip install "unsloth[amd]" --overrides torch-pins.txt

torch-pins.txt should list a torch==...+rocm... line. The last two lines keep PyPI's triton from being installed over the Triton that came with your PyTorch, and skip PyPI's xformers, which is built for NVIDIA GPUs only.

Updating Unsloth

Upgrade only Unsloth, with the same pins:

uv pip install --upgrade-package unsloth --upgrade-package unsloth-zoo "unsloth[amd]" --overrides torch-pins.txt

Do not run uv pip install -U unsloth or --force-reinstall without the pins: both resolve PyTorch again from PyPI and replace your ROCm build with the CUDA one. If that already happened, import unsloth reports an AMD GPU but no usable HIP accelerator: rerun the PyTorch step above, then this one.

Start fine-tuning with Unsloth!

And that's it. Try some examples in our Unsloth Notebooks page!

You can view our dedicated fine-tuning or reinforcement learning guides. Heres a brief example as well:

1. Set environment variables

export HF_HUB_DISABLE_XET=1            # Fixes HuggingFace download issues on AMD

2. Load and configure model

from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    model_name = "unsloth/gemma-4-26b-a4b-it",
    max_seq_length = 2048,
    load_in_4bit = True,
)

model = FastModel.get_peft_model(
    model,
    r = 16,
    lora_alpha = 16,
    target_modules = ["q_proj", "k_proj", "v_proj", "o_proj"],
)

3. Train

from trl import SFTTrainer, SFTConfig

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    train_dataset = dataset,
    formatting_func = formatting_func,
    args = SFTConfig(
        per_device_train_batch_size = 1,
        gradient_accumulation_steps = 4,
        max_steps = 60,
        output_dir = "outputs",
        report_to = "none",
    ),
)

trainer_stats = trainer.train()

Note: Flash Attention 2 is not installed on AMD GPUs by default, so Unsloth uses PyTorch's built-in SDPA attention instead. The warning can be safely ignored.