Auguments/Qwen3-4B-Instruct-2507-heretic-GGUF-BF16 overview
heretic org/Qwen3 4B Instruct 2507 heretic Am I blind, or are there no BF16 GGUFs of this? I don't trust quants if I don't know if they were made from BF16 or …
Runs locally from ~2.47 GB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).
Repository Files & Downloads
Model Details
Model README
---
base_model: heretic-org/Qwen3-4B-Instruct-2507-heretic
base_model_relation: quantized
library_name: gguf
tags:
- gguf
- quantized
license: apache-2.0
---
heretic-org/Qwen3-4B-Instruct-2507-heretic
Am I blind, or are there no BF16 GGUFs of this? I don't trust quants if I don't know if they were made from BF16 or FP16.
---
Highest quality conversion - BF16.
In this repo:
- Base model in BF16, and a custom Q4_K_M converted from BF16
Q4_K_M_custom retains more weights in Q8 or Q6_K, specifically those where Qwen3 is most prone to quality loss.
llama.cpp quant command:
./llama-quantize --tensor-type "blk\.\d+\.attn_v\.weight=q8_0" --tensor-type "blk\.\d+\.attn_k\.weight=q6_k" --tensor-type "token_embd\.weight=q8_0" qwen3-4b.bf16.gguf qwen3-4b.q4_k_m_custom.gguf Q4_K_M
Run Auguments/Qwen3-4B-Instruct-2507-heretic-GGUF-BF16 with guIDE
Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.
Source: Hugging Face · Compare models