Gemma 4 Fine-tuning Guide

Train Gemma 4 by Google with Unsloth.

You can now fine-tune Google's Gemma 4 E2B, E4B, 26B-A4B and 31B with Unsloth. Support includes all vision, text, audio and RL fine-tuning.

Fine-tune Gemma 4 via our free Google Colab notebooks:

Gemma-4-E2B (Vision)

E2B (Text)

E2B (Audio)

E2B (Inference)

If you want to preserve reasoning ability, you can mix reasoning-style examples with direct answers (keep a minimum of 75% reasoning). Otherwise you can emit it fully.
Full fine-tuning (FFT) works as well. It will use 4x more VRAM.
Gemma 4 is powerful for multilingual fine-tuning as it supports 140 languages.
After fine-tuning, you can export to GGUF (for llama.cpp/Unsloth/Ollama/etc.)

If you’re on an older version (or fine-tuning locally), update first:

Unsloth Studio:

unsloth studio update

Unsloth code-based:

pip install --upgrade --force-reinstall --no-cache-dir unsloth unsloth_zoo

Quickstart

🦥 Unsloth Studio Guide

Gemma 4 can be run and fine-tuned in Unsloth Studio, our new open-source web UI for local AI. With Unsloth Studio, you can run models locally on MacOS, Windows, Linux and:

Train LLMs 2x faster with 70% less VRAM
Search, download, run GGUFs and safetensor models
Self-healing tool calling + web search
Code execution (Python, Bash)
Automatic inference parameter tuning (temp, top-p, etc.)
Fast CPU + GPU inference via llama.cpp

Install Unsloth

Run in your terminal:

MacOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows PowerShell:

irm https://unsloth.ai/install.ps1 | iex

Installation will be quick and take approx 1-2 mins.

Launch Unsloth

MacOS, Linux, WSL and Windows:

unsloth studio -H 0.0.0.0 -p 8888

Then open http://localhost:8888 in your browser.

Train Gemma 4

On first launch you will need to create a password to secure your account and sign in again later. You’ll then see a brief onboarding wizard to choose a model, dataset, and basic settings. You can skip it at any time.

Search for Gemma 4 in the search bar and select your desired model and dataset. Next, adjust your hyperparameters, context length as desired.

Monitor training progress

After you click start training, you will be able to monitor and observe the training progress of the model. The training loss should be steadily decreasing. Once done, the model will be automatically saved.

Export your fine-tuned model

Once done, Unsloth Studio allows you to export the model to GGUF, safetensor etc formats.

🦥 Unsloth Core (code-based) Guide

Below is a minimal SFT recipe (works for “text-only” fine-tuning). See also our vision fine-tuning section.

If you'd like to do GRPO, it works in Unsloth if you disable fast vLLM inference and use Unsloth inference instead. Follow our Vision RL notebook examples.

from unsloth import FastLanguageModel
import torch
from datasets import load_dataset
from trl import SFTTrainer, SFTConfig

max_seq_length = 2048  # start small; scale up after it works

# Example dataset (replace with yours). Needs a "text" column.
url = "https://huggingface.co/datasets/laion/OIG/resolve/main/unified_chip2.jsonl"
dataset = load_dataset("json", data_files={"train": url}, split="train")

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "google/gemma-4-E4b-it",
    max_seq_length = max_seq_length,
    load_in_4bit = False,     # MoE QLoRA not recommended, dense 27B is fine
    load_in_16bit = True,     # bf16/16-bit LoRA
    full_finetuning = False,
)

model = FastLanguageModel.get_peft_model(
    model,
    r = 16,
    target_modules = [
        "q_proj", "k_proj", "v_proj", "o_proj",
        "gate_proj", "up_proj", "down_proj",
    ],
    lora_alpha = 16,
    lora_dropout = 0,
    bias = "none",
    # "unsloth" checkpointing is intended for very long context + lower VRAM
    use_gradient_checkpointing = "unsloth",
    random_state = 3407,
    max_seq_length = max_seq_length,
)

trainer = SFTTrainer(
    model = model,
    train_dataset = dataset,
    tokenizer = tokenizer,
    args = SFTConfig(
        max_seq_length = max_seq_length,
        per_device_train_batch_size = 1,
        gradient_accumulation_steps = 4,
        warmup_steps = 10,
        max_steps = 100,
        logging_steps = 1,
        output_dir = "outputs_qwen35",
        optim = "adamw_8bit",
        seed = 3407,
        dataset_num_proc = 1,
    ),
)

trainer.train()

If you OOM:

Drop per_device_train_batch_size to 1 and/or reduce max_seq_length.
Keep use_gradient_checkpointing="unsloth" on (it’s designed to reduce VRAM use and extend context length).

Loader example for MoE (bf16 LoRA):

import os
import torch
from unsloth import FastModel

model, tokenizer = FastModel.from_pretrained(
    model_name = "unsloth/Gemma-4-26B-A4B-it",
    max_seq_length = 2048,
    load_in_4bit = False,     # MoE QLoRA not recommended, dense 27B is fine
    load_in_16bit = True,     # bf16/16-bit LoRA
    full_finetuning = False,
)

Once loaded, you’ll attach LoRA adapters and train similarly to the SFT example above.

MoE fine-tuning (26B-A4B)

The 26B-A4B model is the speed / quality middle ground in the Gemma 4 lineup. Since it is an MoE model with only a subset of parameters active per token, a conservative fine-tuning approach is:

use LoRA rather than full fine-tuning
prefer 16-bit / bf16 LoRA if memory allows
start with shorter contexts and smaller ranks first
scale up only after the pipeline is stable

If your goal is the highest quality and you have more memory, use 31B instead.

Multimodal fine-tuning (E2B / E4B)

Because E2B and E4B support image and audio, they are the main Gemma 4 variants for multimodal fine-tuning.

load the multimodal model with FastVisionModel
keep finetune_vision_layers = False first
fine-tune only the language, attention, and MLP layers
enable vision or audio layers later if your task needs it

Gemma 4 Multimodal LoRA example:

from unsloth import FastVisionModel
from unsloth.trainer import UnslothVisionDataCollator
from trl import SFTTrainer, SFTConfig

model, tokenizer = FastVisionModel.from_pretrained(
    model_name = "google-gemma-4-31b-it",
    max_seq_length = 2048,
    load_in_4bit = False,
    use_gradient_checkpointing = "unsloth",
)

model = FastVisionModel.get_peft_model(
    model,
    finetune_vision_layers     = False,  # Start text-only if VRAM is limited
    finetune_language_layers   = True,
    finetune_attention_modules = True,
    finetune_mlp_modules       = True,
    r = 16,
    lora_alpha = 16,
    lora_dropout = 0,
    bias = "none",
    target_modules = "all-linear",
)

trainer = SFTTrainer(
    model = model,
    tokenizer = tokenizer,
    data_collator = UnslothVisionDataCollator(model, tokenizer),
    train_dataset = dataset,
    args = SFTConfig(
        per_device_train_batch_size = 1,
        gradient_accumulation_steps = 4,
        learning_rate = 2e-4,
        output_dir = "outputs_gemma4_mm",
    ),
)

trainer.train()

Image example format

Remember: for Gemma 4 multimodal prompts, put the image before the text instruction.

{
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "image", "image": "path_or_image_object"},
        {"type": "text", "text": "Extract all text from this receipt. Return line items, total, merchant, and date as JSON."}
      ]
    },
    {
      "role": "assistant",
      "content": [
        {"type": "text", "text": "{\"merchant\": \"Example Store\", \"total\": \"19.99\"}"}
      ]
    }
  ]
}

Audio example format

Audio is for E2B / E4B only. Keep clips short and task-specific.

{
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "audio", "audio": "path_or_audio_object"},
        {"type": "text", "text": "Transcribe the following speech segment in English into English text. Only output the transcription."}
      ]
    },
    {
      "role": "assistant",
      "content": [
        {"type": "text", "text": "Hello everyone and welcome back."}
      ]
    }
  ]
}

Saving / export fine-tuned model

You can view our specific inference / deployment guides for Unsloth Studio, llama.cpp, vLLM, llama-server, Ollama or SGLang.

Save to GGUF

Unsloth supports saving directly to GGUF:

model.save_pretrained_gguf("directory", tokenizer, quantization_method = "q4_k_m")
model.save_pretrained_gguf("directory", tokenizer, quantization_method = "q8_0")
model.save_pretrained_gguf("directory", tokenizer, quantization_method = "f16")

Or push GGUFs to Hugging Face:

model.push_to_hub_gguf("hf_username/directory", tokenizer, quantization_method = "q4_k_m")
model.push_to_hub_gguf("hf_username/directory", tokenizer, quantization_method = "q8_0")

If the exported model behaves worse in another runtime, Unsloth flags the most common cause: wrong chat template / EOS token at inference time (you must use the same chat template you trained with).

For more details read our inference guides:

🖥️Inference & Deployment GGUF & llama.cpp

Model Export vLLM

Gemma 4 data best practices

Gemma 4 has a few formatting details you need to keep in mind.

1. Use standard chat roles

Gemma 4 uses the standard:

system
user
assistant

This means your SFT dataset should be written in regular chat format rather than older Gemma-specific role formats.

2. Thinking mode is explicit

To enable thinking mode, put <|think|> at the start of the system prompt.

Thinking enabled:

<|think|>
You are a careful coding assistant. Explain your answer clearly.

Thinking disabled:

You are a careful coding assistant. Explain your answer clearly.

If you want to preserve thinking-style behavior during SFT:

keep the format consistent
decide whether you want to train on visible thought blocks or on final answers only
do not mix multiple incompatible thought formats in the same dataset

For most production assistants, the simplest setup is to fine-tune on the final visible answer only.

3. Multi-turn rule

For multi-turn conversations, only keep the final visible answer in the conversation history. Do not feed earlier thought blocks back into later turns.

4. Multimodal content should come first

For Gemma 4 multimodal prompts, put:

image before text
audio before text
video frames before text

This should be reflected in your training data too.

PreviousGemma 4 NextQwen3.5

Last updated 4 hours ago

Was this helpful?

hashtagQuickstart

hashtag🦥 Unsloth Studio Guide

hashtagInstall Unsloth

hashtagLaunch Unsloth

hashtagTrain Gemma 4

hashtagMonitor training progress

hashtagExport your fine-tuned model

hashtag🦥 Unsloth Core (code-based) Guide

hashtagMoE fine-tuning (26B-A4B)

hashtagMultimodal fine-tuning (E2B / E4B)

hashtagGemma 4 Multimodal LoRA example:

hashtagImage example format

hashtagAudio example format

hashtagSaving / export fine-tuned model

hashtagSave to GGUF

hashtagGemma 4 data best practices

hashtag1. Use standard chat roles

hashtag2. Thinking mode is explicit

hashtag3. Multi-turn rule

hashtag4. Multimodal content should come first

Quickstart

🦥 Unsloth Studio Guide

Install Unsloth

Launch Unsloth

Train Gemma 4

Monitor training progress

Export your fine-tuned model

🦥 Unsloth Core (code-based) Guide

MoE fine-tuning (26B-A4B)

Multimodal fine-tuning (E2B / E4B)

Gemma 4 Multimodal LoRA example:

Image example format

Audio example format

Saving / export fine-tuned model

Save to GGUF

Gemma 4 data best practices

1. Use standard chat roles

2. Thinking mode is explicit

3. Multi-turn rule

4. Multimodal content should come first