| 80GB GPU | Inference Engine (50%) | Training Engine (50%) |
|---|---|---|
| Model Weights | 16GB | 16GB |
| KV Cache | 24GB | |
| Activations, Gradients, Optimizer States | 24GB |
Multi-purpose
64GB space
| KV Cache | Activations, Gradients, Optimizer States | To enable this, simply add the below to all RL / GRPO training runs before any Unsloth import: ```python import os os.environ["UNSLOTH_VLLM_STANDBY"] = "1" ``` ## 🧪Performance Experiments Here you will find out how we benchmarked memory usage and context length for GRPO. Note that we do **2 generations per prompt because for GRPO to work**, we need at least 2 generations for which to calculate the sample mean and variance. **Without 2 generations, the standard deviation of one sample is 0**. This causes the advantages which uses this: (reward - mean)/std **to be undefined**. $$ Z=\frac{r\_i - \mu}{\sqrt{\frac{1}{n}\sum(r\_i-\mu)^2}} \\ Z\_{n=1}=\frac{r\_1 - \mu}{\sqrt{\frac{1}{1}\sum(r\_1-\mu)^2}}=\frac{0}{0}=\text{undefined} $$ This means for GRPO specifically, a maximum context length of 6,144 for Qwen-3 32B is actually 6,144 multiplied by 2 generations ie 12,288 in length. We provide experiments for Llama-3.1 8B on both LoRA (16bit) and QLoRA (4bit) below:.png?alt=media)
| Experiments | Config | Status | GPU Memory usage | Comments |
|---|---|---|---|---|
| Runs for 40 steps/ 40 minutes | 14.5 GiB (set by vllm_gpu_util) | Enough to fit in 32K KVCache with chunk of 2-4K or say 16K KVCache + 16K chunks | |
| Runs 32 steps in 40 m | 13.8 GiB (set by…) | Approx enough to fit in ~28K KVCache with chunk of 2-4K or say 15K KVCache + 15K chunks | |
| model loads but can’t train because even batch size of 1 doesn’t fit | OOM | ||
| model loads but can’t train because even batch size of 1 doesn’t fit | OOM | ||
| Trains fine 28 steps take 39min | ~15.1GiB | any input slightly longer will result in OOM on colab | |
| Trains fine 29 steps take 40min | 13GiB but most of the time around 10-11GB | At the same config, we save 2GiB aka 15% memory here. Can be higher for longer sequences |
