---

license: other

license_name: iquest-q1

license_link: LICENSE

base_model: IQuestLab/IQuest-Q1

language:

• en
• zh

tags:

• gguf
• mixed-quantization
• iquest-q1
• moe
• ds4-dfm-rs
• recursive-mtp

pipeline_tag: text-generation

---

IQuest-Q1 Mixed Quant GGUF

Mixed GGML quantization of IQuestLab/IQuest-Q1, preserving every main-model expert, learned attention sink and the separate recursive MTP weights. The minimum matrix precision is IQ2_XXS. No IQ1 or ternary tensors are used.

Source revision: 5c21b0630586ef77d38cff1094b8cf37a417fcd8. All 190 source files passed hf cache verify. The checkpoint contains 88 main layers with 256 experts in each of layers 1–87, top-8 routing, and one recursive MTP layer. Its native context limit is 524,288 tokens.

GGUF tensor structure and file hashes are verified. The original main/MTP quality comparisons and B300 runtime gates remain recorded below. Fresh DGX Spark checks pass bounded serial/MTP parity through main and MTP ring wraps, exact bank-state restoration and thinking-mode HTTP reuse/restart with canonical shared weights. Some warm/cold reasoning text differs; the strict full-message parity gate remains failed. Embedded MTP is slower than ordinary decode in the matched benchmark. Serial/continuous API checks and two concurrent requests pass within the recorded scope. Whole-model native source quality, cross-width output parity and a full 512K-token workload remain unqualified.

Historical initial runtime host fixes: f94697c5. See the dated Spark report and compact evidence.

GB10 runtime optimization — 2026-10-02

Three prefill and three dedicated decode rounds are retained at runtime commit 2a43171a. The final 8K cold-KV result is:

| Phase | Median | Range, tokens/s |

| --- | ---: | ---: |

| Prefill | 132.72 | 132.51–132.80 |

| Ordinary decode | 6.77 | 6.77–6.78 |

Each measured worker starts with an empty session: 8,192 prompt tokens, capacity 16,384, chunk 128, 32 EOS-suppressed greedy outputs, MTP off. Six fresh ABBAAB processes share one canonical weight owner; weights and OS caches are not claimed cold. The table reports the three final-path samples. Observed clocks are 2190–2197 MHz under the unchanged user policy. The synthetic inventory prompt measures throughput, not answer quality. See the full report and evidence.

The tiled prefill round changes arithmetic: full prefill logits differ (RMS 1.1791, maximum 4.90625), and 31/32 forced greedy choices agree. Four bounded answer pairs pass, including an 8676-token retrieval; this does not establish corpus-wide quality equivalence. The other five rounds preserve their recorded full logits, tokens and native state. The default tiled path adds 24,576 B common scratch; diagnostic fallbacks are documented in the runtime report.

Final native MTP/SWA physical-wrap and bank/disk-state gates pass. Thinking-mode Chat, Responses and Anthropic buffered/SSE replies, generated tool continuations, sampling fallback and two concurrent streams pass. MTP-off/on HTTP reuse and disk restart preserve all five visible arithmetic answers. Strict cold-versus-reused full-message parity still fails three cases per mode because reasoning wording/completion counts differ; cross-width/source-quality/512K and Agent-speed claims remain unqualified. These gates establish correctness within their scope, not MTP acceleration.

The earlier Spark qualification and 2K performance numbers below are historical and use a different workload. Model weights, quantization and licenses are unchanged.

Recipe

| Tensor group | Base type |

| --- | --- |

| Main routed gate / up / down | IQ2_XXS |

| Measured sensitive routed projections | IQ2_XS |

| Main attention Q / O and first dense FFN | Q5_K |

| Main attention K / V and token embedding | Q6_K |

| Output head | Q8_0 |

| Router, norms and learned sink keys | F32 |

| Recursive MTP experts | Q4_K |

| Remaining MTP matrices | Q8_0 |

The release contains 6 GGUF shards, 1,254 tensors and 87.9952 GB / 81.9520 GiB of GGUF files. All main and MTP weights are included. The 15 measured IQ2_XS promotions are recorded in release-manifest.json; all other routed main projections use IQ2_XXS.

The original expert fc tensor is split exactly into its first 1,536 gate rows and last 1,536 up rows. Expert IDs and down-projection layouts are unchanged. MTP embedding, output head and target final norm are byte-identical to the main tensors; the integrated GGUF shares them through explicit metadata. No expert pruning or weight reconstruction is applied.

Calibration and validation

Calibration uses the source BF16 weights on Runpod B300 GPUs through an independently reconstructed graph. The corpus is rendered with the original IQuest template and re-tokenized with its native tokenizer. It measures actual top-8 routes, router logits and expert-specific squared activation moments using FP64 sums and observed counts. Down importance is measured after SwiGLU and before routing weights. Missing routes are never replaced with synthetic observations.

The retained short-context calibration uses device negative-power RoPE frequencies and combined-sink SDPA. Bounded two-layer comparisons measured their numerical effects separately: RoPE changes 36 of 1,934 top-8 route sets (1.8614%), and combined versus two-stage sink attention changes 63 of 1,934 (3.2575%). The latter comparison includes SDPA/LSE backend arithmetic. These are reference approximations, and the full coverage report records their audited provenance. Held-out quality evaluation uses official CPU frequency bits and ordinary BF16 attention followed by FP32 sink correction and BF16 storage; its PyTorch kernels are independently reconstructed rather than bit-exact official fused kernels.

The publication gate was completed main/MTP calibration coverage plus the full source-derived GGUF tensor and checksum audit. The public upload was verified before the following comparisons, as requested. All six GGUF shards retain their original hashes.

The disjoint held-out set contains 32 sequences and 59,267 native tokens across English, Korean, Chinese, code, math, tools and longer text. Both independent PyTorch paths use BF16 hidden states and BF16 K/V; the quantized path decodes the actual audited GGML blocks to BF16. This measures weight quantization and does not simulate native MMQ activation packing or Q8 KV.

| Comparison | BF16 reference PPL | Quantized PPL | Top-1 agreement with reference |

| --- | ---: | ---: | ---: |

| All 88 main layers, 59,235 labels | 10.9370 | 10.5446 | 78.9128% |

| One-step MTP + shared embedding/head, source main feedback, 59,203 labels | 12.1867 | 12.2658 | 97.1488% |

| One-step MTP + shared tensors, quantized main feedback, 59,203 labels | 12.1867 | 11.3426 | 79.9875% |

Main-model KL(reference || quantized) is 0.666466 and the maximum absolute logit difference is 83.1061. These are measured statistics on this limited set; lower average perplexity does not establish a general quality improvement. MTP measurements are teacher-forced one-step results, not recursive acceptance or speed measurements. See main report, MTP report and their main/MTP execution receipts.

The independent native prefix comparison uses three checkpoint token IDs through two layers, including Q8 KV. Embeddings and all three top-8 expert selections/order match; layer-one hidden cosine is 0.99995631. Its reported arithmetic and KV-byte differences reflect native MMQ activation packing and reductions; it does not establish whole-model native perplexity. See the stage comparison.

With the complete public checkpoint, the final native harness verifies MTP versus serial logits/cache continuation and snapshot/disk restoration. Greedy serial and continuous HTTP lanes produce identical text for the same short request; each records 16 draft proposals and six accepted proposals. These gates use draft count 3, confidence margin 0, a 13-token prompt and 12-token output. The dedicated output parser also passed official-source vectors and an independent 96-case valid-grammar streaming review. See post-publication validation for exact scope, source identity and remaining qualifications.

DGX-Spark memory

The model has 25 full-attention and 63 sliding-attention layers. The sliding window is 4,096; MTP uses a 512-token window. At 524,288 tokens, the raw BF16 main KV payload is 50.9844 GiB, so this weight budget requires a smaller KV representation.

Q8_0 KV with bounded sliding-window storage is approximately 27.0865 GiB. With the actual GGUF files, the sum is approximately 109.0385 GiB before runtime workspace, allocator overhead and the operating system. The B300 gate at capacity 524,288 measured 29,090,663,072 bytes of owned bank tensors and 117,207,341,760 total owned device bytes (109.158 GiB), including canonical weights, scratch, kernel partials and graphs. It generated only the short prompt/output above; it used explicit lazy device residency and zero derived weight buffers. File metadata, CUDA driver overhead and the operating system are additional.

Fresh Spark runs use one VMM owner with 91 canonical ranges: 81.934 GiB logical tensor payload and 82.05 GiB allocated/imported weights. Workers report zero owned canonical, derived or registered-host weight bytes. Matching-model derived IPC records reject before any import; disable all owner repacks. The MTP feedback buffer is bounded; a full-context 524288 x 3072 hidden-state history is unnecessary.

At context 8,192 and chunk 128, the planned first bank plus session workspace is 1.026 GiB; each additional bank is 0.955 GiB. Eight lazy partial-reuse checkpoints add up to 4.192 GiB. At context 524,288, the first-bank quote becomes 27.174 GiB and each additional bank 27.103 GiB. These quotes exclude common CUDA/MMQ scratch, allocator overhead and host/driver headroom. Disk KV persists checkpoints; active banks still require GPU memory. The 512K quote and historical B300 allocation do not establish Spark fit or long-context correctness.

Runtime contract

This is the explicit iquest_q1 contract for Baekpica/ds4-dfm-rs. IQuest uses partial NeoX RoPE over the first 32 dimensions, different full/sliding RoPE bases, Q/K RMS normalization, query-dependent learned zero-value sink keys, and normalized-input residuals in main layers 1–87. Generic Llama-family aliases do not implement this graph.

The official template and complete tokenizer JSON are included. The Rust host preserves the source Jinja, distinct developer role, NFC normalization, ordered split rules, special tokens and BPE merges. IQuest XML tool calls become typed JSON arguments in Chat Completions, Responses and Anthropic Messages; string properties preserve text. Large integer arguments retain exact decimal serialization. Runtime revision f94697c5 honors explicit sampling parameters in thinking mode and preserves reasoning-bearing history for cache retirement. Keep thinking enabled for normal use; the recorded HTTP reuse checks use reasoning_effort=high and max_tokens=256. Do not force reasoning_effort=none as the default.

Embedded recursive MTP needs no sidecar. Its feedback uses normalized target hidden state and the next token embedding at the original target position. Greedy verification runs sequential target steps and restores rejected target/MTP writes. Sampling uses ordinary target decoding. Start serving with --mtp-mode off; separately enable correctness checks with --mtp-mode on --mtp-draft 3 --mtp-margin 0. Drafts 2–7 are supported; external drafters reject. ds4-bench --mtp-draft 1 uses ordinary decode, while drafts 2–7 select embedded speculation. A fresh 2048-prompt/32-output ABBA comparison measured ordinary decode at 3.39 tokens/s and draft3 at 2.08–2.09 tokens/s, with identical frontier logits and output tokens. MTP is slower on this workload.

Rust-host Spark candidate

The tested serving shape is context 8,192, two banks, native/scheduler chunk 128 and disk KV. Thinking-mode reuse/restart checks pass; qualification remains bounded by the limitations below. Build the Rust hosts with make cuda-spark and the owner with make CUDA_ARCH=sm_121 ds4_weight_server. Keep all shards and the official template/tokenizer files together. Start the owner first and retain it while workers run:

IQUEST_MODEL=/absolute/path/IQuest-Q1-MQ-IQ2_XXS-00001-of-00006.gguf
IQUEST_RUN="$PWD/scratch/iquest-serving"
mkdir -p "$IQUEST_RUN"
./ds4_weight_server --base "$IQUEST_MODEL" --backend vmm --scope base \
  --reserve-gb 28 --no-repack-iq2-aligned --no-repack-q8-aligned \
  --no-repack-q2k-aligned --manifest "$IQUEST_RUN/weights.ipc"

In another terminal, set the same variables and wait for owner broker/manifest readiness before starting the worker:

DS4_CUDA_WEIGHT_IPC_MANIFEST="$IQUEST_RUN/weights.ipc" \
DS4_CUDA_WEIGHT_IPC_SCOPE=base \
./ds4-server --cuda -m "$IQUEST_MODEL" --model-id iquest-q1 \
  --host 127.0.0.1 --port 8002 -c 8192 --max-seqs 2 \
  --native-chunk 128 --prefill-chunk 128 --prefill-chunk-live 128 \
  --prefix-reuse partial --mtp-mode off \
  --kv-disk-dir "$IQUEST_RUN/disk-kv" --kv-disk-space 16G \
  --kv-cache-min-tokens 1024 --print-plan --no-update-check

Fresh serial checks compare each accepted MTP prefix with ordinary target execution from the same snapshot:

| Fixture | Context / chunk | Prompt + output | Draft | Accepted / proposed |

| --- | --- | ---: | ---: | ---: |

| Short text | 512 / 4 | 13 + 12 | 3 | 6 / 15 |

| MTP ring wrap | 8192 / 128 | 517 + 32 | 7 | 19 / 74 |

| Main SWA ring wrap | 8192 / 128 | 4220 + 32 | 7 | 18 / 92 |

All rounds have exact full-vocabulary logits and committed target/MTP payloads. The physical 519-row MTP and 4,223-row main SWA rings wrap. Truncated native payloads reject and invalidate the checkpoint; restoring the valid payload recovers exact state and logits. Census faults remain zero. This is in-process snapshot recovery, not HTTP worker-restart qualification. Diagnostic timings include readback and establish no throughput result.

The real vocabulary passes 534 tokenizer cases and four rendered chat vectors against the pinned source. A separate three-token, two-layer actual-weight prefix has 21 finite float stages, exact embeddings/routes and final hidden mean cosine 0.999956309 against the archived decoded-weight reference. This bounded comparison does not establish whole-model native quality.

Six native bank checks preserve exact payloads beyond the physical main ring: source preservation, full/partial fork, matched-schedule replay, in-place rewind and disk restore. Different cold-prefill widths still produce numerical differences: two audited KV layers first differ at position 4352, and the long disabled-thinking append changes EOS selection. Exact state copies do not establish cross-width output parity.

The initial v6 thinking campaign exposed two host defects: a legacy override changed explicit temperature0 to1 and disabled greedy MTP; retirement dropped the reasoning-bearing cache key. After f94697c5, seed, warm append/edit/branch and disk restart pass with MTP on and off. All five warm/cold replies preserve strict visible answers 4/5/6/8/9, stop finishes and nonempty reasoning. Entire messages match in four of five MTP-on cases and three of five MTP-off cases; the remaining differences are reasoning wording. The strict whole-message cold gate remains failed. Earlier disabled-thinking formatting and long warm/cold output failures remain recorded.

At the same serving shape, serial and continuous lanes each pass six buffered/SSE reasoning checks across Chat Completions, Responses and Anthropic Messages. Both lanes pass Chat/Responses generated-tool continuation with typed arguments 19,23 and result 42, plus temperature0.7 thinking through ordinary sampling with no speculative counter increase. Two actual SSE requests interleave generation in the continuous lane and return exact 1..24 sequences with nonempty reasoning; fault counters remain unchanged. Eight disabled-thinking API checks fail their strict answer format by adding explanations, also with unchanged fault counts.

The matched benchmark uses four fresh workers in ordinary/MTP/MTP/ordinary order. Prefill is 62.16–62.20 tokens/s ordinary and 60.27–60.28 with MTP; decode is 3.39 versus 2.08–2.09 tokens/s. All 160000 frontier logits and 32 EOS-suppressed output tokens match. These timings exclude initialization and snapshot/logit I/O and establish no Agent acceleration.

Serving qualification fields remain unset because they cannot express the thinking-mode and prefill-width limitations. The recorded checks establish no general context/bank limit, deterministic full-message parity or whole-model native source-quality qualification.

Files

| Shard | Bytes | SHA256 |

| --- | ---: | --- |

| IQuest-Q1-MQ-IQ2_XXS-00001-of-00006.gguf | 17,106,294,176 | 59cfbc372f2e8988a960626508e62ec49a373efbba4848e122d6cc96288b332d |

| IQuest-Q1-MQ-IQ2_XXS-00002-of-00006.gguf | 16,952,852,736 | 68db6ed9aacb347f41f732c048220905c9336793bcc5a95a6ee59590f6d7ed36 |

| IQuest-Q1-MQ-IQ2_XXS-00003-of-00006.gguf | 16,872,983,520 | c2e686e4396d57e33bc9f1662e535b4e910033b95d6875b07c12ff81e34b1342 |

| IQuest-Q1-MQ-IQ2_XXS-00004-of-00006.gguf | 16,942,136,256 | 1eaccda174c4505c70cf2cd4a7a4962ba60a7a29528d3fe777d25537c6934fc5 |

| IQuest-Q1-MQ-IQ2_XXS-00005-of-00006.gguf | 17,060,018,304 | 4ffacc99ab740130b9ccafaf46151c38e48e6940bde105cc393ef041ccc54c23 |

| IQuest-Q1-MQ-IQ2_XXS-00006-of-00006.gguf | 3,060,964,704 | ae3eb2281a673a4f96bedd704266ea0213fe909af8d456767bc6fdefe23ce844 |

Download all shards with hf download Baekpica/IQuest-Q1-Mixed-Quant-GGUF --include "*.gguf" --local-dir ./IQuest-Q1-MQ, then fetch the checksum list with hf download Baekpica/IQuest-Q1-Mixed-Quant-GGUF SHA256SUMS --local-dir ./IQuest-Q1-MQ. Use the first shard as the model path; the runtime resolves its companion shards. Run sha256sum -c SHA256SUMS from that directory.

License

The original modified MIT license is included as LICENSE. Its commercial-use condition requires prominently displaying IQuest-Q1 on the product or service interface. Preserve the copyright and license notice when distributing derivatives.