---

language:

• en
• fr
• de
• es
• pt
• it
• ja
• ko
• ru
• zh
• ar
• fa
• id
• ms
• ne
• pl
• ro
• sr
• sv
• tr
• uk
• vi
• hi
• bn

license: apache-2.0

new_version: EnlistedGhost/Mistral-Small-4-119B-2603-GGUF

tags:

• MistralAI
• Mistral-Small-4
• Vision
• Multi-modal
• MoE
• GGUF
• llama.cpp
• Ollama
• Quantize
• 119B
• Agentic

base_model:

• mistralai/Mistral-Small-4-119B-2603

---

<img src="https://huggingface.co/EnlistedGhost/Mistral-Small-4-119B-2603-GGUF/resolve/main/resources/Introducing%20Mistral-Small-4.png" alt="Example image" width="398" height="338">

Mistral-Small-4-119B-A6B | GGUF Edition

<br />

Coversion Details (Safetensors --> GGUF)

• Converter: Llama.cpp (Build 9888)
• Quantizer: Llama.cpp (Build 9760)

--------------------------------------------

Info<br />___

Thank You for Viewing This Release! <br />

*Model files (GGUF) are still being uploaded.<br />

This modelcard will be updated soon!

Your understanding and patience is very much appreciated.*

• Mistral-Small-4-119B-A6B is a huge model in physical storage size

and due to this: Uploads are taking longer than expected.*

<br /><br />

Technical Details<br />___________________<br />These quantized GGUF files found in this release are unique<br />in the sense that when initially converting

this model it was observed that there are native Float32 (F32) weights in the layers.<br />

Upon realizing this during the initial conversion;the model was then re-converted using an "F32" flag in Llama.cpp (Version 9888)

prior to being quantized.<br />

(This was done in order to snub quality loss compared to converting to BF16 or F16 conversion+quantize)