---

license: mit

language:

- en

library_name: gguf

pipeline_tag: text-generation

tags:

- gguf

- llama.cpp

- quantized

- qwen3

- reasoning

- unsloth

- text-generation

base_model: ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth

datasets:

- ermiaazarkhalili/claude-reasoning-distillation

---

FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth-GGUF

GGUF quantizations of FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth for CPU and edge inference with llama.cpp, Ollama, LM Studio, and other GGUF runtimes.

This model is a fine-tune of FastContext-1.0-4B-SFT for reasoning distillation (chain-of-thought), trained with Unsloth on claude-reasoning-distillation.

• Full-precision model: FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth
• Base model: FastContext-1.0-4B-SFT
• Parameters: 4B

Available Quantizations

| Quant | Size | Recommended Use | File |

|-------|------|-----------------|------|

| Q2_K | 1.67 GB | Smallest, lowest quality — quick tests / very constrained devices | fastcontext-4b-sft_base-sft-claude-opus-reasoning-unsloth.q2_k.gguf |

| Q3_K_M | 2.08 GB | Small, acceptable quality | fastcontext-4b-sft_base-sft-claude-opus-reasoning-unsloth.q3_k_m.gguf |

| Q4_K_M | 2.50 GB | Recommended — best size/quality balance | fastcontext-4b-sft_base-sft-claude-opus-reasoning-unsloth.q4_k_m.gguf |

| Q5_K_M | 2.89 GB | High quality, larger | fastcontext-4b-sft_base-sft-claude-opus-reasoning-unsloth.q5_k_m.gguf |

| Q6_K | 3.31 GB | Very high quality, near-fp16 | fastcontext-4b-sft_base-sft-claude-opus-reasoning-unsloth.q6_k.gguf |

| Q8_0 | 4.28 GB | Near-lossless, largest | fastcontext-4b-sft_base-sft-claude-opus-reasoning-unsloth.q8_0.gguf |

Q4_K_M is the recommended default for most users.

Usage

Download a single quant

pip install -U "huggingface_hub[cli]"
hf download ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth-GGUF \
  --include "*q4_k_m*.gguf" --local-dir ./FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth-GGUF

llama.cpp

# build: https://github.com/ggerganov/llama.cpp
./llama-cli -m ./FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth-GGUF/fastcontext-4b-sft_base-sft-claude-opus-reasoning-unsloth.q2_k.gguf \
  -p "Solve step by step: What is the sum of the first 10 prime numbers?" -n 512

Ollama

ollama run hf.co/ermiaazarkhalili/FastContext-4B-SFT_base-SFT-Claude-Opus-Reasoning-Unsloth-GGUF:Q4_K_M "Solve step by step: What is the sum of the first 10 prime numbers?"

Training Outcome

| Metric | Value |

|--------|-------|

| SLURM Job ID | 45169147 |

| Runtime | 21m 31s |

| Final Training Loss | 0.8678 |

| Peak VRAM | 13.3 GB |

| GPU | H100 80GB HBM3 (MIG 3g.40gb) |

License

MIT — see the base model for full terms.

Acknowledgments

• Unsloth for 2x faster fine-tuning
• llama.cpp for GGUF quantization
• Base model developers (microsoft)
• Compute Canada / DRAC for HPC resources