---

license: apache-2.0

base_model: Kolyadual/MIXdevAI-yandexGPT5-8B

tags:

• gguf
• llama.cpp
• text-generation

---

MIXdevAI-yandexGPT5-8B-GGUF

Full set of GGUF quantizations for Kolyadual/MIXdevAI-yandexGPT5-8B.

Files

| File | Quantization | Type |

|---|---|---|

| mixdev-ya-f16.gguf | F16 | Base Model |

| mixdev-ya-q8_0.gguf | Q8_0 | Text Model |

| mixdev-ya-q6_k.gguf | Q6_K | Text Model |

| mixdev-ya-q5_k_m.gguf | Q5_K_M | Text Model |

| mixdev-ya-q5_k_s.gguf | Q5_K_S | Text Model |

| mixdev-ya-q5_1.gguf | Q5_1 | Text Model |

| mixdev-ya-q5_0.gguf | Q5_0 | Text Model |

| mixdev-ya-q4_k_m.gguf | Q4_K_M | Text Model |

| mixdev-ya-q4_k_s.gguf | Q4_K_S | Text Model |

| mixdev-ya-q4_1.gguf | Q4_1 | Text Model |

| mixdev-ya-q4_0.gguf | Q4_0 | Text Model |

| mixdev-ya-q3_k_l.gguf | Q3_K_L | Text Model |

| mixdev-ya-q3_k_m.gguf | Q3_K_M | Text Model |

| mixdev-ya-q3_k_s.gguf | Q3_K_S | Text Model |

| mixdev-ya-q2_k.gguf | Q2_K | Text Model |

Quantization notes

• Q8_0 / F16: Almost lossless. Best quality, largest size.
• Q6_K / Q5_K_M: Excellent balance between quality and size.
• Q4_K_M: Golden standard for local inference.
• Q3_K_M / Q2_K: Noticeable quality degradation. Use only if you have severe RAM/VRAM constraints.

How to run locally

Example using llama.cpp server:

llama-server \
  -m mixdev-ya-q4_k_m.gguf \
  -ngl 999 \
  --host 0.0.0.0 --port 8080 \
  -c 32768