---

tags:

• gguf
• quantized
• unsloth-dynamic
• imatrix

base_model:

• mfielding92/thefriend-31b-v2

---

thefriend-31b-v2 — GGUF

Unsloth Dynamic 2.0 (UD) GGUF quants of mfielding92/thefriend-31b-v2.

• UD recipes extracted from unsloth/gemma-4-31B-it-GGUF
• imatrix from the same repo
• Per-tensor quant overrides applied via stock llama.cpp's --tensor-type

Recommended quants

| Quant | Approx size | Notes |

|---|---|---|

| UD-Q4_K_XL | best quality/size ratio for most users | recommended default |

| UD-Q5_K_M | higher quality | if you have headroom |

| UD-Q3_K_XL | smaller, still very usable | tight VRAM |

| UD-Q2_K_XL | extreme compression | budget setups |

Files appear here as they finish quantizing. See the file list below.

Run with llama.cpp

./llama.cpp/llama-cli \
  --model thefriend-31b-v2-UD-Q4_K_XL.gguf \
  --temp 0.6 --top-p 0.95 --top-k 20 --min-p 0.0 \
  --ctx-size 16384 --n-gpu-layers 99