ed90c095d3/Mistral-7B-Instruct-v0.3-GGUF overview
MaziyarPanahi/Mistral 7B Instruct v0.3 GGUF https://huggingface.co/MaziyarPanahi/Mistral 7B Instruct v0.3 GGUF Model creator: mistralai https://huggingface.co/…
Runs locally from ~1.50 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
| File | Type | Quantization | Size | Link |
|---|---|---|---|---|
| Mistral-7B-Instruct-v0.3.IQ1_M.gguf | GGUF | GGUF | 1.64 GB | Download |
| Mistral-7B-Instruct-v0.3.IQ1_S.gguf | GGUF | GGUF | 1.50 GB | Download |
| Mistral-7B-Instruct-v0.3.IQ2_XS.gguf | GGUF | GGUF | 2.05 GB | Download |
| Mistral-7B-Instruct-v0.3.IQ3_XS.gguf | GGUF | GGUF | 2.82 GB | Download |
| Mistral-7B-Instruct-v0.3.IQ4_XS.gguf | GGUF | GGUF | 3.64 GB | Download |
| Mistral-7B-Instruct-v0.3.Q2_K.gguf | GGUF | GGUF | 2.54 GB | Download |
| Mistral-7B-Instruct-v0.3.Q3_K_L.gguf | GGUF | GGUF | 3.56 GB | Download |
| Mistral-7B-Instruct-v0.3.Q3_K_M.gguf | GGUF | GGUF | 3.28 GB | Download |
| Mistral-7B-Instruct-v0.3.Q3_K_S.gguf | GGUF | GGUF | 2.95 GB | Download |
| Mistral-7B-Instruct-v0.3.Q4_K_M.gguf | GGUF | GGUF | 4.07 GB | Download |
| Mistral-7B-Instruct-v0.3.Q4_K_S.gguf | GGUF | GGUF | 3.86 GB | Download |
| Mistral-7B-Instruct-v0.3.Q5_K_M.gguf | GGUF | GGUF | 4.78 GB | Download |
| Mistral-7B-Instruct-v0.3.Q5_K_S.gguf | GGUF | GGUF | 4.66 GB | Download |
| Mistral-7B-Instruct-v0.3.Q6_K.gguf | GGUF | GGUF | 5.54 GB | Download |
| Mistral-7B-Instruct-v0.3.Q8_0.gguf | GGUF | GGUF | 7.17 GB | Download |
| Mistral-7B-Instruct-v0.3.fp16.gguf | GGUF | GGUF | 13.50 GB | Download |
Model Details
| Model ID | ed90c095d3/Mistral-7B-Instruct-v0.3-GGUF |
|---|---|
| Author | ed90c095d3 |
| Pipeline | text-generation |
| License | — |
| Base model | mistralai/Mistral-7B-Instruct-v0.3 |
| Last modified | 2026-06-07T08:16:19.000Z |
Model README
---
tags:
- quantized
- 2-bit
- 3-bit
- 4-bit
- 5-bit
- 6-bit
- 8-bit
- GGUF
- transformers
- safetensors
- mistral
- text-generation
- conversational
- license:apache-2.0
- autotrain_compatible
- endpoints_compatible
- text-generation-inference
- region:us
- text-generation
model_name: Mistral-7B-Instruct-v0.3-GGUF
base_model: mistralai/Mistral-7B-Instruct-v0.3
inference: false
model_creator: mistralai
pipeline_tag: text-generation
quantized_by: MaziyarPanahi
---
MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF
- Model creator: mistralai
- Original model: mistralai/Mistral-7B-Instruct-v0.3
Description
MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF contains GGUF format model files for mistralai/Mistral-7B-Instruct-v0.3.
About GGUF
GGUF is a new format introduced by the llama.cpp team on August 21st 2023. It is a replacement for GGML, which is no longer supported by llama.cpp.
Here is an incomplete list of clients and libraries that are known to support GGUF:
- llama.cpp. The source project for GGUF. Offers a CLI and a server option.
- llama-cpp-python, a Python library with GPU accel, LangChain support, and OpenAI-compatible API server.
- LM Studio, an easy-to-use and powerful local GUI for Windows and macOS (Silicon), with GPU acceleration. Linux available, in beta as of 27/11/2023.
- text-generation-webui, the most widely used web UI, with many features and powerful extensions. Supports GPU acceleration.
- KoboldCpp, a fully featured web UI, with GPU accel across all platforms and GPU architectures. Especially good for story telling.
- GPT4All, a free and open source local running GUI, supporting Windows, Linux and macOS with full GPU accel.
- LoLLMS Web UI, a great web UI with many interesting and unique features, including a full model library for easy model selection.
- Faraday.dev, an attractive and easy to use character-based chat GUI for Windows and macOS (both Silicon and Intel), with GPU acceleration.
- candle, a Rust ML framework with a focus on performance, including GPU support, and ease of use.
- ctransformers, a Python library with GPU accel, LangChain support, and OpenAI-compatible AI server. Note, as of time of writing (November 27th 2023), ctransformers has not been updated in a long time and does not support many recent models.
Special thanks
🙏 Special thanks to Georgi Gerganov and the whole team working on llama.cpp for making all of this possible.
Run ed90c095d3/Mistral-7B-Instruct-v0.3-GGUF with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models