---

license: apache-2.0

base_model: thinkingmachines/Inkling-Small

base_model_relation: quantized

library_name: gguf

pipeline_tag: image-text-to-text

tags:

• inkling
• gguf
• mixed-quantization
• multimodal
• dgx-spark

---

Inkling-Small — intermediate GGUF artifacts

Intermediate artifacts for Inkling-Small Mixed Quant.

Independent conversion from thinkingmachines/Inkling-Small revision

8cc5877b44d343f88b92086aa1fb72897950f06a.

> MTP-BF16 sidecar verified with multimodal MQ85GB serving (2026-09-10): Paired with the separate MQ85GB main artifact, ds4-dfm-rs on one DGX Spark serves text, PNG/JPEG image and 16 kHz WAV audio input with text output and checks this eight-layer draft stack on and off over HTTP. The Q8_0 main artifact in this repository is a structurally verified conversion and is not qualified for native serving. HTTP evidence runtime: ds4-dfm-rs@4c32556, PR #28.

The MTP-BF16 sidecar is used with the separate MQ85GB main artifact in

ds4-dfm-rs's DGX Spark integration. All eight draft layers

have short CLI and serial HTTP MTP on/off checks. This does not qualify the

262.704 GiB Q8_0 main artifact for native serving. The separate

GB10 performance report (PR #30) applies to MQ85GB with

MTP off; it does not establish Q8_0-main throughput or an MTP speedup.

No llama.cpp compatibility or long-context result is claimed.

| Artifact | Tensors | Shards | Size |

|---|---:|---:|---:|

| Q8_0 reference, protected BF16 ingress/attention | 888 | 20 | 262.704 GiB |

| MTP BF16 sidecar, all 8 dense draft layers | 160 | 1 | 4.183 GiB |

The Q8_0 reference uses Q8_0 for embedding/head, dense MLP matrices, shared

experts and all routed experts. Attention, vision/audio, norms, convolutions,

router weights/bias/scales preserve their original BF16/F32 bytes.

It is a Q8 matrix reference with native anchors, not a uniformly Q8 model.

It was quantized directly from the original BF16 checkpoint, without imatrix.

MTP is kept separately and is not required for ordinary decoding.

general.architecture = inkling, inkling.tensor_layout = source-interleaved-v1.

Every tensor keeps its original checkpoint name, shape and row order.

w13 rows are [gate0, up0, gate1, up1, ...]; experts are never removed,

merged or reordered. All 42 decoder layers and 256 experts per sparse layer

are present. Full tokenizer/config/processor JSON and chat template are embedded

in the first shard and also supplied as sidecars. This is a custom explicit

contract, not an existing llama.cpp tensor-name convention.

Each artifact directory contains tensor-plan.json, audit.json and

SHA256SUMS. Audits check all tensor names/shapes/types/byte counts, native

payload bit equality, sampled Q8 dequantization error and every file hash.

These structural checks do not establish model-output quality or runtime support.

hf download Baekpica/Inkling-Small-GGUF --include 'Q8_0/*' --local-dir ./inkling
cd inkling/Q8_0
sha256sum -c SHA256SUMS

Source: Thinking Machines Inkling-Small, Apache-2.0.

GGML quantizers: Baekpica/ds4-dfm-rs,

pinned 0bf398f459987c611293aa889772c80e272f045d.

Support my work

I work on making large language models practical on hardware they were never really designed to fit on — through mixed quantization, inference optimization, custom kernels, and serving experiments.

While much of the development happens on local hardware, calibration, profiling, and large-scale validation often require expensive on-demand GPUs.

Contributions help pay for that compute, storage, and testing infrastructure, so I can keep experimenting and publishing the results openly.

<a href="https://www.buymeacoffee.com/baekpica" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me a Coffee" style="height: 60px !important;width: 217px !important;"></a> <a href="https://github.com/sponsors/Baekpica" target="_blank"><img src="https://img.shields.io/badge/Sponsor-EA4AAA?style=for-the-badge&logo=githubsponsors&logoColor=white" alt="Sponsor Baekpica on GitHub" style="height: 60px !important;width: 217px !important;"></a>