How to Fine-tune LLMs with Unsloth & Docker

Local training can be complex due to dependency hell or breaking environments. Unsloth’s Docker image can bypass these issues. No setup is needed: pull and run the image and start training.

• Unsloth official Docker image: unsloth/unsloth

Why Use Unsloth & Docker?

Unsloth’s Docker image is stable, up-to-date and works in supported setups like Windows.

• Fully contained dependencies keep your system clean. Runs safely without root.

• Use locally or on any platform with pre-installed notebooks.

⚡ Step-by-Step Tutorial

1

Install Docker and NVIDIA Container Toolkit.

Install Docker via Linux or Desktop (other). Then install NVIDIA Container Toolkit:

Copy

export NVIDIA_CONTAINER_TOOLKIT_VERSION=1.17.8-1
sudo apt-get update && sudo apt-get install -y \
  nvidia-container-toolkit=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  nvidia-container-toolkit-base=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container-tools=${NVIDIA_CONTAINER_TOOLKIT_VERSION} \
  libnvidia-container1=${NVIDIA_CONTAINER_TOOLKIT_VERSION}

2

Run the container.

unsloth/unsloth is Unsloth's only Docker image. For Blackwell and 50-series GPUs, use this same image - no separate image needed. If using DGX Spark, you'll need to follow our DGX guide.

Copy

docker run -d -e JUPYTER_PASSWORD="mypassword" \
  -p 8888:8888 -p 2222:22 \
  -v $(pwd)/work:/workspace/work \
  --gpus all \
  unsloth/unsloth

3

Access Jupyter Lab

Go to http://localhost:8888 and open Unsloth.

Access the unsloth-notebooks tabs to see Unsloth notebooks.

4

Start training with Unsloth

If you're new, follow our step-by-step Fine-tuning Guide, RL Guide or just save/copy any of our premade notebooks.

📂 Container Structure

• /workspace/work/ — Your mounted work directory

• /workspace/unsloth-notebooks/ — Example fine-tuning notebooks

• /home/unsloth/ — User home directory

📖 Usage Example

Full Example

Setting up SSH Key

If you don't have an SSH key pair:

⚙️ Advanced Settings

• Jupyter Lab: -p 8000:8888

• SSH access: -p 2222:22

🔒 Security Notes

• Container runs as non-root unsloth user by default

• Use USER_PASSWORD for sudo operations inside container

• SSH access requires public key authentication