---

license: apache-2.0

base_model: Nanbeige/Nanbeige4.2-3B

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- llama.cpp

- nanbeige

---

Nanbeige4.2-3B GGUF

Unofficial GGUF conversions of

Nanbeige/Nanbeige4.2-3B,

pinned to source revision

451ed48c3273ecef7ea8faaa43c31ce529763bb1.

Nanbeige4.2-3B is not an ordinary Llama stack. It reuses 22 physical decoder

layers for two loops, applies the shared final RMSNorm at the loop boundary,

and needs 44 logical KV-cache slots. These files therefore use the custom GGUF

architecture name nanbeige and require the included llama.cpp patch.

Files

| File | Size | SHA-256 |

|---|---:|---|

| Nanbeige4.2-3B-Q4_K_M.gguf | 2,574,807,840 bytes | 99c7bfb88907f7eee0a04c4314f1c46bca391819478d8cb90b3e164f09576489 |

| Nanbeige4.2-3B-Q8_0.gguf | 4,434,787,104 bytes | 32b60cd0fb3da4d8a3c01d0ca7d0461818c29c81ffb42f8fc71a94141bd803c6 |

| llama.cpp-nanbeige.patch | 10,137 bytes | 1177badb8f8f6228052d2f67a9eabe81f2ce69f3ea0b92cf7b1ea2352204284a |

The source checkpoint contains 4,169,800,704 physical parameters. The Q4_K_M

file is about 4.93 bits per weight; Q8_0 is about 8.51 bits per weight.

Runtime

Apply the patch to the exact tested llama.cpp revision:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout cf512566dc89d7a2bdefd43f6d4b85a1c18e93a7
git apply /path/to/llama.cpp-nanbeige.patch
cmake -B build
cmake --build build --config Release

Then run either quant:

./build/bin/llama-cli \
  -m Nanbeige4.2-3B-Q4_K_M.gguf \
  -c 4096 -n 256 \
  -p "Hello"

Stock llama.cpp builds that do not include Nanbeige support will reject the

nanbeige architecture. The patch adds conversion metadata, tensor loading,

loop-boundary normalization, and 44-layer cache handling; it does not replace

model weights.

Validation

Built and tested on Windows with llama.cpp

cf512566dc89d7a2bdefd43f6d4b85a1c18e93a7 and an AMD Ryzen 9 7950X3D.

• Both files loaded as architecture nanbeige, block count 44, loop count 2.
• Q4_K_M: prompt benchmark 92.77 tok/s; generation benchmark 15.67 tok/s.
• Q8_0: prompt benchmark 53.95 tok/s; generation benchmark 9.37 tok/s.
• Deterministic eight-token generation completed for both files.
• Interactive Q8_0 sample measured 68.6 tok/s prompt and 10.2 tok/s generation.

These are CPU smoke/throughput tests, not quality benchmarks. The original

model advertises a 262,144-token context, but local validation used a short

512-token context.

License

The base model is Apache-2.0 licensed. Review the upstream model card for its

full terms and intended-use information.