---

license: other

license_name: inkling

license_link: https://huggingface.co/unsloth/inkling-GGUF

base_model:

- unsloth/inkling-GGUF

tags:

- imatrix

- inkling

- gguf

- calibration

---

Inkling imatrix (BF16-direct)

An importance matrix for TML's Inkling (975B MoE, 41B active), computed directly on the full BF16 weights. To our knowledge this is the first published imatrix for Inkling: the architecture is not yet merged into mainline llama.cpp, so standard quantization pipelines cannot produce one.

This repo contains ONLY the imatrix calibration file. There are no model weights here and no quantized builds are promised. If our VRAM-fit quantization experiments produce a build worth shipping, it will land in a separate repo aimed at memory-constrained rigs in the ~192 GB class (2x RTX PRO 6000, 2x DGX Spark and similar).

How it was made

| | |

|---|---|

| Source weights | BF16 GGUF (unsloth/inkling-GGUF, BF16/, 41 shards, ~1.8 TiB) |

| Collection | BF16-direct (not collected on a quantized model) |

| Calibration corpus | wikitext-2-raw train, full corpus |

| Chunks | 6,400 x 512 ctx = ~3.28M tokens |

| Build | llama.cpp PR #25731 (inkling architecture), CUDA build, CPU-streamed inference |

| Hardware | 2x RTX PRO 6000 Blackwell workstation, weights streamed from PCIe 5.0 NVMe |

| Wall time | ~10.6 hours |

BF16-direct collection matters at extreme quantization targets: importance statistics gathered from an already heavily quantized model are noticeably noisier, and this file is intended for 1-2 bpw work where that difference is largest.

Usage

llama-quantize --imatrix inkling-bf16.imatrix \
  inkling-BF16-00001-of-00041.gguf inkling-IQ1_S.gguf IQ1_S

The file uses the GGUF container format written by current llama-imatrix. A build with the inkling architecture (PR #25731) is required for any end-to-end use with this model.

Notes

• Calibration text is English wikitext only. For heavily multilingual or code-dominant targets you may want to blend your own corpus.
• Made by satgeze. Feedback and results welcome in discussions.