KikoCis/FastContext-1.0-4B-longctx-imatrix-GGUF overview

---

license: mit

base_model: microsoft/FastContext-1.0-4B-SFT

base_model_relation: quantized

library_name: gguf

tags:

• gguf
• imatrix
• long-context
• 256k
• qwen3
• repository-exploration
• subagent
• coder
• agentic
• llama.cpp
• ollama

language:

• en

pipeline_tag: text-generation

---

!banner

<div style="border:2px solid currentColor; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,'Liberation Mono',monospace;">

<div style="border-bottom:1px solid currentColor; padding:6px 12px; font-size:11px; letter-spacing:3px; text-transform:uppercase; opacity:0.7; text-align:center;">KIKOCIS // LONG-CONTEXT IMATRIX GGUF // PRESERVED &amp; QUANTIZED</div>

<div style="padding:14px; display:flex; flex-wrap:wrap; align-items:center; justify-content:center; gap:18px;">

<pre style="margin:0; flex:0 0 auto; font-family:ui-monospace,'SF Mono','Cascadia Mono',Consolas,monospace; font-size:9px; line-height:1.15; letter-spacing:0;">

repo/ ┌─────────┐

├── src/ ═══════▶│ ◉ 4B │

│ ├── auth.rs ◀═══════│ scout │

│ └── db.rs └─────────┘

├── lib/ READ·GLOB·GREP

│ └── core.rs ──▶ auth.rs:41-77

└── tests/ ──▶ core.rs:102-130

256K ctx only what you need

</pre>

<div style="flex:0 1 auto; max-width:100%; text-align:center;">

<div style="font-size:23px; font-weight:800; letter-spacing:1px;">FASTCONTEXT-1.0-4B</div>

<div style="font-size:12.5px; letter-spacing:1px; opacity:0.8; margin-top:5px;"><span style="white-space:nowrap;">256K REPO-EXPLORER</span> · <span style="white-space:nowrap;">QWEN3 DENSE 4B</span> · <span style="white-space:nowrap;">LONG-CTX IMATRIX</span> · <span style="white-space:nowrap;">8.0 GB → 1.96 GB</span></div>

</div>

</div>

<table style="display:table; table-layout:fixed; width:100%; margin:0; border-collapse:collapse; font-family:ui-monospace,'SF Mono',Consolas,monospace; font-size:12px;">

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">FORMAT</div><div style="font-weight:700;">GGUF · IQ3_M / Q4_K_M</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">SIZE</div><div style="font-weight:700;">1.96 / 2.50 GB</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">ARCH</div><div style="font-weight:700;">QWEN3 DENSE · 36L</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">CONTEXT</div><div style="font-weight:700;">256K NATIVE</div></td>

</tr>

<tr>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">IMATRIX</div><div style="font-weight:700;">LONG-CONTEXT CALIB</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">RETRIEVAL @5K</div><div style="font-weight:700;">30/30 = BF16</div></td>

<td style="border-top:1px solid currentColor; border-right:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">RUNS ON</div><div style="font-weight:700;">METAL·CUDA·CPU·VULKAN</div></td>

<td style="border-top:1px solid currentColor; padding:8px 12px;"><div style="font-size:10px; letter-spacing:1px; opacity:0.6;">LICENSE</div><div style="font-weight:700;">MIT</div></td>

</tr>

</table>

</div>

> *Microsoft open-sourced it, then deleted it from HuggingFace and GitHub (verified: 404 on both). These are long-context-imatrix GGUF quants so the weights stay in your* hands — the full preserved original (bf16, 8.0 GB) is at KikoCis/FastContext-1.0-4B-SFT. Own your AI.

🔍 What FastContext is (and why it's special)

FastContext isn't a chatbot — it's a repository-exploration subagent for coding agents. Your main agent (Claude Code, Copilot, Cursor, OpenHands…) delegates file discovery to it:

1. Main agent asks: "where is auth handled?"
2. FastContext fires parallel read-only tool calls — READ / GLOB / GREP — across the repo,
3. and returns just the file paths + line ranges you need as compact, focused context.

Your expensive frontier agent stops burning tokens crawling directories. Microsoft's (now-deleted) announcement reported ~60% fewer tokens from the main agent and +5.5% on SWE-bench — their figures, not independently reproduced here.

📦 Which file should I pick?

| file | bits | size | vs original | pick this if… |

|---|---|---|---|---|

| fastcontext4b.IQ3_M.imx.gguf | ~3.3 | 1.96 GB | 4.1× smaller | tightest RAM — smallest FastContext GGUF anywhere, retrieval-validated |

| fastcontext4b.Q4_K_M.imx.gguf | ~4.5 | 2.50 GB | 3.2× smaller | the safe default — more headroom for long contexts |

_{K-quants (Q4_K_M) = solid general quants. I-quants (IQ3_M) = smaller at similar quality; they need an imatrix (we ship ours: fastcontext4b.imatrix).}

What's different vs the other FastContext GGUFs: the importance matrix here is calibrated on long, multi-thousand-token sequences (LongAlign), not the usual short generic corpus — matching the 256K regime this model was built for. For AMD Strix Halo specifically, see plunderstruck's ROCmFP4 build (different target, code-weighted imatrix).

🧮 Will it fit? (RAM/VRAM cheat-sheet)

Total ≈ weights + KV-cache (KV grows with context):

| you have | quant | context you can run |

|---|---|---|

| 4 GB | IQ3_M | ~8–16K |

| 6 GB | IQ3_M / Q4_K_M | ~32K |

| 8 GB | Q4_K_M | ~64–128K |

| 12 GB+ | Q4_K_M | up to 256K native |

🚀 How to run it

# llama.cpp — point it at your repo dump, ask for locations:
llama-cli -m fastcontext4b.Q4_K_M.imx.gguf -c 32768 \
  -p "…repo contents…\n\nWhere is authentication handled? Return file:line ranges only."

# llama-server (use it as a subagent endpoint for your main coding agent):
llama-server -m fastcontext4b.Q4_K_M.imx.gguf -c 65536 --port 8091

# Ollama (Modelfile included, 32K default):
ollama create fastcontext -f Modelfile && ollama run fastcontext

Recommended sampling: temperature 0.6, top_p 0.9, top_k 20. For pure retrieval calls, temperature 0 works well.

Subagent pattern: keep FastContext resident on a cheap local endpoint; have your main agent call it for "where is X?" queries and inject only the returned ranges into its own context.

📊 Validation — measured on these files (honest)

Needle-in-haystack retrieval (find an inserted fact inside real long documents), greedy decoding:

| model | needle retrieval @~5K ctx |

|---|---|

| original (bf16) | 30/30 |

| Q4_K_M (imx) | 30/30 |

| IQ3_M (imx) | 30/30 |

At 5K context all three — including the aggressive IQ3_M — match the original bf16 perfectly: quantization is lossless for retrieval here. Deeper long-context numbers will be added once measured on a clean harness — no placeholder claims.

• Harness: llama-server + OpenAI-compat API, temp 0, 30 tasks, haystacks built from real LongAlign documents, deterministic gold.
• Date: 2026-07-02.

⚠️ Good to know

• Strengths: repo exploration, long-document retrieval, read-only tool calling (READ/GLOB/GREP), returning compact file:line evidence.
• It's a scout, not a solver — pair it with your main coding agent; don't expect it to write the patch itself.
• The original repo is gone, so upstream docs/issues are gone with it; the harness conventions above are from the model's own announcement and community usage.

🗒️ Changelog

• 2026-07-02 v1 — IQ3_M + Q4_K_M with long-context imatrix; retrieval validated @5K (30/30 all); imatrix + Modelfile included; original preserved in the sibling repo.

📚 Credit & license

Model, weights, training: © Microsoft — FastContext-1.0-4B-SFT (MIT), sourced via the ShaunGves re-upload after the original was removed. Quantization + long-context imatrix + validation: KikoCis. MIT (same as upstream). No weights modified — faithful quantization only.