MiniMax M3 - How to Run Locally

Run MiniMax M3 LLM locally on your own device!

MiniMax M3 is a new ~428B (23B active) open model for coding, agentic workflows, cowork tasks, and multimodal chat. The multimodal model has support for text, image, and video inputs, and a 1M context window. The unquantized bf16 weights are ~855GB and the 1-bit GGUF decreases this to just 128GB (-85%): MiniMax-M3 GGUF

The model performs on par with Gemini 3.1 Pro - scoring 59% on SWE-Bench Pro, 66% on Terminal-Bench 2.1, 34.8% on SWE-fficiency, and 28.8% on KernelBench Hard. Thanks MiniMax for day zero access.

MiniMax-M3 GGUFs are currently experimental. MiniMax-M3 itself is native multimodal, but the current experimental GGUF is text-only and does not support MiniMax Sparse Attention.

⚙️ Usage Guide

The smallest GGUF quant, UD-IQ1_M, uses 128GB of disk space. Because the file size does not include KV cache, context allocation, try to have at least 133GB RAM to run the model. It's recommended to use UD-IQ3_XXS which is 159GB for best results.

The 4-bit UD-IQ4_XS quant is 208GB, while UD-Q4_K_XL is 265GB. These are better suited to 256GB+ or 512GB-class systems, multi-GPU servers, or systems with CPU RAM plus GPU offload.

Table: Inference hardware requirements (units = total memory: RAM + VRAM, or unified memory)

1-bit	2-bit	3-bit	4-bit	5-bit	8-bit
133 GB	148 GB	164-200 GB	213-270 GB	325 GB	460-470 GB

For best performance, make sure your total available memory, including VRAM and system RAM, exceeds the quantized model file size by a comfortable margin.

Recommended Settings

MiniMax recommends the following parameters for best performance: temperature=1.0, top_p=0.95, top_k=40.

temperature = 1.0
top_p = 0.95
top_k = 40

• Maximum context window: 1,048,576

• Default system prompt:

You are a helpful assistant. Your name is MiniMax-M3 and was built by MiniMax.

Run MiniMax-M3 Tutorials:

For this tutorial, we will use the smallest current quant, UD-IQ1_M, because MiniMax-M3 is large. Replace UD-IQ1_M with UD-IQ4_XS, UD-Q4_K_XL, or another quant if your machine has enough memory. You can now run MiniMax-M3 in Unsloth.

Unsloth Guide

Download Unsloth

Download Unsloth

https://unsloth.ai/download

https://unsloth.ai/download

Unsloth Guide

Download Unsloth

https://unsloth.ai/download

You can now run MiniMax M3 via Unsloth ✨. Ensure you use > v0.1.463-beta or 2026.6.6.

MiniMax M3 runs in Unsloth on MacOS, Windows, and Linux. You can:

• Search, download, run GGUFs and safetensor models

• Self-healing tool calling + web search

• Code execution (Python, Bash)

• Automatic inference parameter tuning (temp, top-p, etc.)

• Fast CPU + GPU inference via llama.cpp

• Train LLMs 2x faster with 70% less VRAM

1

Install Unsloth

Ensure you use the latest v0.1.463-beta or 2026.6.6. Run in your terminal:

MacOS, Linux, WSL:

curl -fsSL https://unsloth.ai/install.sh | sh

Windows PowerShell:

irm https://unsloth.ai/install.ps1 | iex

2

Launch Unsloth

MacOS, Linux, WSL and Windows:

3

Search and download MiniMax M3

Then go to the Unsloth Chat tab and search for MiniMax M3 in the search bar and download your desired model and quant.

4

Run MiniMax M3

🦙 Llama.cpp Guide

1

Obtain the SPECIFIC llama.cpp PR on GitHub here. You can follow the build instructions below as well. Change -DGGML_CUDA=ON to -DGGML_CUDA=OFF if you don't have a GPU or just want CPU inference. For Apple Mac / Metal devices, set -DGGML_CUDA=OFF then continue as usual - Metal support is on by default.

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch origin pull/24523/head:minimax-m3
git checkout minimax-m3
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j --target llama-cli llama-server

2

You can now use llama.cpp directly to load and download models, just like ollama run. First, select the quantization type you want like Q2_K_XL. Also use export LLAMA_CACHE="folder" to force llama.cpp to save to a specific location. Note this download process might be very slow, so it's probably best to use the manual download process in the next section.

export LLAMA_CACHE="unsloth/MiniMax-M3-GGUF"
./build/bin/llama-cli \
    -hf unsloth/MiniMax-M3-GGUF:UD-IQ1_M \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 40

Note: MiniMax Sparse Attention is not supported yet, so inference falls back to dense attention.

3

If you want to download the model manually, we can download the model via the code below (after installing pip install huggingface_hub). If downloads get stuck, see: Hugging Face Hub, XET debugging

hf download unsloth/MiniMax-M3-GGUF \
    --local-dir unsloth/MiniMax-M3-GGUF \
    --include "*UD-IQ1_M*" # Use "*UD-IQ4_XS*" for 4-bit

4

You can edit --threads 32 for the number of CPU threads, --ctx-size 32768 for context length, --n-gpu-layers 2 for GPU offloading on how many layers. Try adjusting it if your GPU goes out of memory. Also remove it if you have CPU only inference. Remember MSA is not yet supported, so keep --ctx-size modest - dense attention at very long contexts will use a lot of memory.

./build/bin/llama-cli \
    --model unsloth/MiniMax-M3-GGUF/UD-IQ1_M/MiniMax-M3-UD-IQ1_M-00001-of-00004.gguf \
    --temp 1.0 \
    --top-p 0.95 \
    --top-k 40

📊 Benchmarks