---

license: apache-2.0

base_model: TokenRhythm/NeoHorse-1-4B

base_model_relation: quantized

library_name: llama.cpp

pipeline_tag: text-generation

tags:

• gguf
• llama.cpp
• qwen3.5
• quantized
• text-generation
• agentic
• tool-use
• coding
• reasoning

---

NeoHorse-1-4B GGUF

Community GGUF quantizations of TokenRhythm/NeoHorse-1-4B.

<div align="center" style="background-color:#f59e0b;color:#ffffff;padding:16px 20px;border-radius:10px;line-height:1.7;">

☕ If this GGUF made your day easier, a coffee would make mine.<br>

<a href="https://ko-fi.com/ngquocvinh" style="color:#ffffff;"><strong style="color:#ffffff;">Send a coffee ☕</strong></a><br>

I build and test these releases myself. Your coffee helps keep me going.<br>

Thank you for supporting this work.

</div>

About NeoHorse-1-4B

NeoHorse-1-4B is an approximately 4B-parameter text-only causal language model from TokenRhythm, post-trained from Qwen3.5-4B for text-based agent harnesses, tool use, coding, instruction following, and conversational use. The upstream card reports a native context length of 262,144 tokens; practical context depends on runtime, memory, KV-cache settings, and workload.

See the official upstream model card for the original benchmark table, deployment notes, technical report, and model details. Vision weights are not included in the upstream text-only release.

These releases change the storage format through GGUF quantization only. No training or fine-tuning was performed here.

Fidelity measurements

The table covers every one of the 26 published GGUF files. Size (GB) uses

decimal gigabytes (1 GB = 1,000,000,000 bytes). It averages eight

chunks from each of wiki.test.raw and wiki.valid.raw, using a 4,096-token

context and the same llama.cpp runtime against the BF16 GGUF reference. The

average BF16 PPL in this evaluation was 8.635307. Lower Mean KLD, ΔPPL, and

RMS Δp, and higher Top-1 agreement, indicate closer next-token behavior to

BF16. These are fidelity measurements, not a direct percentage of capabilities

retained.

Sweet spots: choose Q8_0 or Q6_K for fidelity, Q5_K_M for the

balanced default, and Q4_K_M for a smaller practical package.

| File | Size (GB) | Mean KLD | Top-1 vs BF16 | ΔPPL | RMS Δp |

|---|---:|---:|---:|---:|---:|

| NeoHorse-1-4B-Q8_0.gguf | 4.48 | 0.006291 | 98.065% | +0.192% | 2.041% |

| NeoHorse-1-4B-Q6_K.gguf | 3.46 | 0.010910 | 96.751% | +0.831% | 2.881% |

| NeoHorse-1-4B-Q6_K_F16.gguf | 4.21 | 0.010624 | 96.697% | +0.595% | 2.736% |

| NeoHorse-1-4B-Q5_K_M.gguf | 3.07 | 0.032008 | 94.498% | +1.706% | 4.631% |

| NeoHorse-1-4B-Q4_K_M.gguf | 2.71 | 0.055031 | 91.909% | +1.339% | 5.920% |

| NeoHorse-1-4B-Q3_K_M.gguf | 2.26 | 0.160972 | 85.173% | +11.466% | 10.610% |

| NeoHorse-1-4B-Q2_K.gguf | 1.92 | 0.348854 | 76.007% | +12.720% | 16.212% |

| NeoHorse-1-4B-IQ2_XS.gguf | 1.63 | 0.517311 | 70.662% | +32.407% | 20.489% |

| NeoHorse-1-4B-IQ1_M.gguf | 1.43 | 1.303395 | 54.895% | +168.381% | 32.950% |

| NeoHorse-1-4B-Q1_0.gguf | 1.04 | 12.339880 | 0.406% | +18400566.679% | 60.432% |

| NeoHorse-1-4B-Q5_K_S.gguf | 2.99 | 0.033239 | 94.324% | +1.094% | 4.638% |

| NeoHorse-1-4B-Q4_K_S.gguf | 2.56 | 0.059554 | 91.276% | +0.938% | 6.339% |

| NeoHorse-1-4B-Q3_K_L.gguf | 2.42 | 0.151958 | 85.775% | +10.301% | 10.352% |

| NeoHorse-1-4B-Q3_K_S.gguf | 2.07 | 0.214466 | 81.864% | +16.238% | 12.637% |

| NeoHorse-1-4B-Q2_K_S.gguf | 1.85 | 0.420451 | 74.035% | +18.702% | 17.861% |

| NeoHorse-1-4B-IQ4_NL.gguf | 2.61 | 0.065933 | 91.054% | +5.454% | 6.901% |

| NeoHorse-1-4B-IQ4_XS.gguf | 2.51 | 0.065569 | 90.874% | +5.479% | 6.969% |

| NeoHorse-1-4B-IQ3_M.gguf | 2.16 | 0.141647 | 85.371% | +7.065% | 10.366% |

| NeoHorse-1-4B-IQ3_S.gguf | 2.14 | 0.141352 | 85.296% | +5.683% | 10.218% |

| NeoHorse-1-4B-IQ3_XXS.gguf | 1.90 | 0.209927 | 81.678% | +6.872% | 12.586% |

| NeoHorse-1-4B-IQ2_M.gguf | 1.74 | 0.370630 | 75.507% | +13.970% | 17.041% |

| NeoHorse-1-4B-IQ2_S.gguf | 1.65 | 0.473452 | 71.806% | +29.192% | 19.599% |

| NeoHorse-1-4B-IQ2_XXS.gguf | 1.54 | 0.711936 | 65.844% | +53.226% | 24.370% |

| NeoHorse-1-4B-IQ1_S.gguf | 1.36 | 1.857279 | 46.657% | +356.917% | 39.251% |

| NeoHorse-1-4B-TQ2_0.gguf | 1.46 | 12.427480 | 0.034% | +14183266.448% | 61.065% |

| NeoHorse-1-4B-TQ1_0.gguf | 1.29 | 12.427480 | 0.034% | +14183266.448% | 61.065% |

The fixed hold-out corpus is disjoint from the calibration inputs. See the

ladder quality summary,

supplemental quality summary,

and the reproducibility manifest for the exact

split, hashes, and runtime profile.

Behavior and runtime validation

BF16 baseline probes covered English, Vietnamese, Chinese, coding, plain JSON,

and the upstream-style tool-call text format. Those probes generated usable

responses for the tested prompts. The strict llama.cpp --json-schema probe

failed during runtime sampler initialization and is recorded as a runtime

compatibility limitation, not counted as a model capability result. The

tool-call probe tested serialization format only; it was not an end-to-end tool

executor test.

BF16 and every published quantization loaded and generated non-empty output on

one NVIDIA A10M using the same GPU-0 profile. Low-bit files can pass this

compatibility gate while still producing poor responses; the fidelity table is

why Q3 and lower are marked experimental.

Quick start

For the balanced starting point:

./llama-cli \
  -m NeoHorse-1-4B-Q5_K_M.gguf \
  --jinja \
  --reasoning off \
  -p 'Answer briefly in English: What is GGUF and why is it useful for running language models locally?' \
  -n 128 -c 4096 -ngl 99

The GGUF contains the converted chat template. Reduce -ngl or the context

size if the available VRAM is smaller.

Reproducibility and validation

The locked upstream revision is

56f0584bb40578a2c33b1b40a08ccd17243ad710. The BF16 source GGUF SHA256 is

10aae5451e1c0b6089b25f68cfe32cd32a4bdcb92e30008e083e6ba351f3cbd8; the

NeoHorse-specific combined imatrix SHA256 is

8c66b45b73826b0b00fdd973d308d8f81f7db0f8a3b97cf93bc7813d722d6515.

The package includes SHA256SUMS.txt, the

quantization manifest, source and

calibration manifests, the runtime version, and the copied upstream chat

template. Detailed conversion, quantization, smoke, behavior, and fidelity

logs remain local under reports/neohorse-1-4b/ and are not uploaded.

License and attribution

NeoHorse-1-4B is released upstream under the Apache License 2.0. The upstream

license and attribution notices are included in LICENSE.

These are community GGUF quantizations, not an official TokenRhythm release or endorsement.

Supplemental variants

The nine-file ladder above is complete. These additional llama.cpp

formats are built directly from the same locked NeoHorse-1-4B BF16

source and model-specific imatrix. A row is marked published only

after runtime smoke, hold-out fidelity, SHA256, and Hugging Face

metadata verification pass. IQ1_S, TQ2_0, and TQ1_0 are

published for compatibility/experimentation only and are not

recommended for normal use because their fidelity drift is severe.

Q6_K_F16 is a hybrid variant: quantized tensors use Q6_K while

token embeddings remain F16, so it is larger than standard Q6_K.

A failed candidate would remain in the status ledger but be withheld.

See the supplemental status ledger and supplemental fidelity summary.

<!-- SUPPLEMENTAL_VARIANTS_BEGIN -->

| File | Status | Size (bytes) |

|---|---|---:|

| NeoHorse-1-4B-Q5_K_S.gguf | published | 2990036064 |

| NeoHorse-1-4B-Q4_K_S.gguf | published | 2563888224 |

| NeoHorse-1-4B-Q3_K_L.gguf | published | 2421316704 |

| NeoHorse-1-4B-Q3_K_S.gguf | published | 2069879904 |

| NeoHorse-1-4B-Q2_K_S.gguf | published | 1852392544 |

| NeoHorse-1-4B-IQ4_NL.gguf | published | 2609435744 |

| NeoHorse-1-4B-IQ4_XS.gguf | published | 2514285664 |

| NeoHorse-1-4B-IQ3_M.gguf | published | 2163186784 |

| NeoHorse-1-4B-IQ3_S.gguf | published | 2139511904 |

| NeoHorse-1-4B-IQ3_XXS.gguf | published | 1904493664 |

| NeoHorse-1-4B-IQ2_M.gguf | published | 1741554784 |

| NeoHorse-1-4B-IQ2_S.gguf | published | 1651975264 |

| NeoHorse-1-4B-IQ2_XXS.gguf | published | 1538393184 |

| NeoHorse-1-4B-IQ1_S.gguf | published | 1359234144 |

| NeoHorse-1-4B-TQ2_0.gguf | published | 1456442464 |

| NeoHorse-1-4B-TQ1_0.gguf | published | 1289141344 |

| NeoHorse-1-4B-Q6_K_F16.gguf | published | 4213982304 |

<!-- SUPPLEMENTAL_VARIANTS_END -->