---

base_model: Cloudflare/clef-flash

license: apache-2.0

tags:

• gguf
• llama.cpp
• quantized
• clef
• cloudflare
• qwen3.5
• qwen3_5
• decision-model
• structured-output

---

Clef-Flash — GGUF quants (language tower)

Cloudflare/clef-flash quantized for llama.cpp. The source is a 9.4B-parameter model (architecture Qwen3_5ForConditionalGeneration, 32 layers, hidden size 4096, vocab 248,320), post-trained by Cloudflare from Qwen/Qwen3.5-9B and released under Apache-2.0. This repo has 30 verified files: 29 GGUF quants plus the full-precision BF16 source.

⚠️ What these files do and don't contain

Clef-Flash is a decision model: it reads a state (text, JSON, images, or video) plus a schema of typed questions and returns a probability for every allowed option of every question in a single forward pass — no free-form text generation, no output parsing. That behavior comes from a backbone plus a separate joint schema head (and optionally the vision encoder).

This repo contains the language tower only.

• The vision encoder is not included — there is no --mmproj file here, so image and video inputs are not available from these GGUFs.
• The joint schema head is not included (joint_head.safetensors is a separate component in the source release). Per-question probabilities and the Jev / SystemOne API behavior therefore cannot be reproduced from these files.

For the full decision-model behavior, use Cloudflare/clef-flash directly with transformers (5.10.2) and torch (2.11) per its README. What you get from this repo in llama.cpp is the post-trained Qwen3.5-9B language tower.

About Clef-Flash

From the Cloudflare announcement: Clef-Flash turns a state plus a schema of typed questions into decisions. Each question is one of three types — noul (true/false), choice (named options), or score (ordered options) — and the model's joint schema head reads the backbone's final hidden states, routes evidence from the state to each question, and scores all options of all questions jointly. The Clef-Flash API is compatible with Jev and SystemOne. Clef-Flash is the smaller sibling of Cloudflare/clef.

Files

All 30 files below are verified present on the Hub.

| File | Size (GB) | Notes |

|---|---|---|

| clef-flash-BF16.gguf | 16.69 | Full precision source. Every quant below is cut from this file. |

| clef-flash-Q8_0.gguf | 8.87 | Effectively lossless. Use when disk and RAM are not the constraint. |

| clef-flash-Q6_K.gguf | 6.85 | Near-lossless; the last stop before quality becomes measurable. |

| clef-flash-Q5_1.gguf | 6.33 | Legacy. Prefer Q5_K_M. |

| clef-flash-Q5_K_M.gguf | 6.02 | Very good quality, noticeably smaller than Q6_K. |

| clef-flash-Q5_K_S.gguf | 5.87 | Slightly smaller than Q5_K_M for a slight quality cost. |

| clef-flash-Q5_0.gguf | 5.87 | Legacy. Prefer Q5_K_M. |

| clef-flash-Q4_K_M.gguf | 5.24 | The usual default. Best quality-per-byte for most people. |

| clef-flash-Q4_1.gguf | 5.41 | Legacy. Prefer Q4_K_M. |

| clef-flash-IQ4_NL.gguf | 5.05 | Non-linear 4-bit; good on hardware without fast K-quant kernels. |

| clef-flash-Q4_K_S.gguf | 4.98 | A little smaller than Q4_K_M, a little worse. |

| clef-flash-Q4_0.gguf | 4.95 | Legacy round-to-nearest. Prefer Q4_K_M unless a runtime needs this. |

| clef-flash-IQ4_XS.gguf | 4.84 | Best sub-4.5bpw option; usually beats Q4_K_S at a smaller size. |

| clef-flash-Q3_K_L.gguf | 4.59 | Small, with real quality loss. Usable when RAM is tight. |

| clef-flash-Q3_K_M.gguf | 4.31 | Smaller again; noticeable degradation. |

| clef-flash-IQ3_M.gguf | 4.11 | Strong at ~3.7bpw, clearly better than Q3_K_M. |

| clef-flash-IQ3_S.gguf | 4.07 | Slightly smaller than IQ3_M. |

| clef-flash-Q3_K_S.gguf | 3.97 | Aggressive. Prefer IQ3_M at a similar size. |

| clef-flash-IQ3_XS.gguf | 3.95 | Aggressive but coherent. |

| clef-flash-IQ3_XXS.gguf | 3.67 | Very aggressive; the last coherent step down. |

| clef-flash-Q2_K.gguf | 3.56 | Very small, heavily degraded. For experimentation. |

| clef-flash-Q2_K_S.gguf | 3.44 | Smaller than Q2_K, at a further quality cost. |

| clef-flash-IQ2_M.gguf | 3.36 | The smallest size most people find usable. |

| clef-flash-IQ2_S.gguf | 3.19 | Below the usual usability line. |

| clef-flash-Q2_0.gguf | 3.14 | Extreme, group-64. Included for completeness. |

| clef-flash-IQ2_XS.gguf | 3.06 | Experimental. |

| clef-flash-IQ2_XXS.gguf | 2.89 | Experimental. |

| clef-flash-IQ1_M.gguf | 2.68 | Extreme. Expect substantial degradation. |

| clef-flash-IQ1_S.gguf | 2.55 | Extreme. Expect substantial degradation. |

| clef-flash-Q1_0.gguf | 1.83 | Extreme. Included for completeness. |

Quantized 2026-10-04. The importance matrix was computed on the full BF16 weights, not on a lower-precision surrogate.

Usage

# run directly from the Hub
llama-cli -hf NANI-Nithin/clef-flash-GGUF:Q4_K_M -p "Hello"

# or serve it
llama-server -hf NANI-Nithin/clef-flash-GGUF:Q4_K_M

# or download a single file
huggingface-cli download NANI-Nithin/clef-flash-GGUF clef-flash-Q4_K_M.gguf --local-dir .
llama-cli -m clef-flash-Q4_K_M.gguf -p "Hello"

These files are llama.cpp GGUF. They are not loaded by vLLM, SGLang or transformers the way the source repo is — for that, use Cloudflare/clef-flash.

Which file?

• Q4_K_M (5.24 GB) — the default choice; best quality-per-byte for most setups.
• Q8_0 (8.87 GB) — effectively lossless, when resources allow.
• Q6_K (6.85 GB) — near-lossless middle ground.
• IQ4_XS (4.84 GB) — the best pick under ~4.5 bits per weight.
• IQ2_M (3.36 GB) — the smallest size most people find usable.
• Everything smaller than IQ2_M is experimental or extreme; expect substantial degradation. The lowest-bit files exist for completeness only.

Results

Cloudflare publishes decision-model benchmark results in the source README and on the Decision Index leaderboard. In Cloudflare's internal run of the Decision Index 0.2.1 suite, Clef-flash posts the best score among the compared decision models in the majority of listed rows — for example BFCL 98.8 (case exact accuracy), MMLU 91.8, HellaSwag 98.6, WinoGrande 97.5, and a median request latency of 38.8 ms — with ForecastBench reported as a Brier score (10.6), where lower is better. These numbers describe the source model with its joint head on Cloudflare's own harness; the quantized files in this repo were not re-evaluated.

Licence

Apache-2.0, following the base model Qwen/Qwen3.5-9B. Clef-Flash was created by Cloudflare; these quants are an independent llama.cpp conversion of the published weights.