---

license: apache-2.0

tags: [gguf, moe, qwen3.5, 35b, uncensored, quantization, pxq, single-gpu, vision]

base_model: [ornith-35b-aeon, siq-1-35b]

---

<p align="center"><img src="banner.png" alt="PXA Fusion2-35B" width="100%"></p>

PXA Fusion2-35B — run a real 35B on a card you already have

Fusion2 is a 35B Mixture-of-Experts model (Qwen3.5-family, ~3B active, 40 layers / 256 routed

experts + 1 shared, 248k vocab, vision-capable), created by a task-vector (DELLA) merge of

Ornith-35B-Aeon (0.80) and SIQ-1-35B (0.20). It is uncensored and tool-calling reliable.

What makes this release different is the quantization: these GGUFs use PXQ, a proprietary

PXA-native quant family (not a mainline llama.cpp quant), designed to put a genuine 35B on

landfill datacenter cards (Tesla P100 / V100, 16 GB) and on consumer 11–16 GB cards — even a GTX 1080 Ti.

You run them with our companion kernel — a fork of ik_llama.cpp — linked below.

> 💛 If this saved you a GPU upgrade, consider supporting the work: https://ko-fi.com/shatteredrealms1

Pick your tier

| model | bits | file size | fits | quality (wikitext ppl) | use it when |

|---|---|---|---|---|---|

| PXQ6-MTP (flagship) | 4.27 bpw | 19.2 GB | 2× 16 GB (or 1× 24 GB) | 7.36 | best quality; +MTP speculative-decode; vision |

| PXQ6 | 4.27 bpw | 18.7 GB | 2× 16 GB / 1× 24 GB | 7.36 | flagship, no MTP |

| PXQ3 | 3.27 bpw | 14.7 GB | 1× 16 GB (ub512) | 7.44 (+1.1%) | ~flagship quality on one 16 GB card |

| PXQ2 | 2.27 bpw | 10.7 GB | 1× 11 GB (1080 Ti) | 8.39 (+14%) | smallest; a real 35B on an 11 GB card (even a 1080 Ti) |

| PXQU-16 | ~3.2 bpw mix | 14.0 GB | 1× 16 GB @ full ub2048 | 7.58 | biggest model that runs full-speed prefill on ONE 16 GB card |

| PXQU-12 | ~2.7 bpw mix | 11.6 GB | 1× 11–12 GB @ ub2048 | 7.93 | best that fits a 1080 Ti at full ub2048 |

PXQ2/3/6 vs PXQU — two ways to choose: the pure tiers (PXQ2/PXQ3/PXQ6) are single-type

quants — one uniform bit-width, "pick your quality." PXQU (PXQ-Universal) is a knapsack mix of

PXQ2/3/6 across tensors, sized to fit a specific card at full ub2048 prefill — "pick your card,

get the best that fits it." Same PXQ kernels, same speed. Why it matters: at -ub2048 on a single

16 GB card, **PXQ3 (14.7 GB) and even subgraph-incumbent IQ3_K (15.1 GB) OOM — but PXQU-16 (14.0 GB)

fits** and runs full-speed prefill on one card. See HEAD-TO-HEAD.md.

Headline: PXQ3 is essentially flagship quality — +1.1% ppl — at 14.7 GB. A 16 GB card gets

~4-bit quality. PXQ2 fits an 11 GB card — even a GTX 1080 Ti — at +14% ppl and stays coherent + uncensored.

All tiers are single-type, imatrix-calibrated quants (PXQ2 / PXQ3 / PXQ6).

Quantization damage — KL-divergence vs the bf16 source (measured)

Perplexity moves with model quality; KL-divergence isolates quant damage — how far each tier's

next-token distribution drifts from the bf16 merge source. Measured against the true bf16

reference (wikitext-2, 100 chunks; full table + method in the repo bench/ + KLD-RESULTS.md):

| tier | Mean KLD vs bf16 | Median KLD | top-1 token agreement |

|---|---|---|---|

| PXQ6 (4.27 bpw) | 0.056 | 0.031 | 89.4% |

| PXQ3 (3.27 bpw) | 0.076 | 0.040 | 87.8% |

| PXQ2 (2.27 bpw) | 0.205 | 0.109 | 80.3% |

Monotonic, and PXQ3 (3-bit) holds within ~35% of the flagship's divergence with 87.8% top-1

agreement — the low-bit tiers are honest, not lucky. bf16 reference ppl (100 ch): 7.08.

Head-to-head vs ik IQ_K quants (same weights, same imatrix, same reference)

<p align="center">

<img src="https://raw.githubusercontent.com/poisonxa16/pxq_llama/main/assets/benchmarks/pxq-vs-iqk-scorecard.png" alt="PXQ vs ik IQ_K benchmark scorecard" width="100%">

</p>

Matched-size head-to-head, PXQ vs ik_llama IQ_K — both formats measured on the same binary (our fork), same weights + imatrix + wikitext-2 KLD reference. Decode / prefill / fidelity (losses shown) / the single-16GB-card result. Full numbers below and in HEAD-TO-HEAD.md.

We quantized our own bf16 to ikawrakow's IQ_K incumbents (same imatrix) and compared at matched size

(full table: HEAD-TO-HEAD.md). Honest result: **PXQ is faster on every tier — ~15–37% faster decode

and ~16–29% faster prefill than the matched IQ_K quant on the same Pascal/Volta cards** (the fused

kernels; prefill is the biggest margin). On fidelity the crossover is ~2.7–3 bpw: at 3–4 bit ik's

IQ_K edges PXQ per byte (better non-linear codebook + higher-precision backbone), but **at ≤12 GB

(≤~2.7 bpw) PXQ wins fidelity too** — PXQU-12 beats IQ2_KL (KLD 0.161 vs 0.165) and PXQ2 beats IQ2_KS

(0.205 vs 0.284), both while faster.

The ub2048-single-card result (the regime that matters): on ONE 16 GB card at full -ub2048

prefill, IQ3_K (15.1 GB) OOMs — PXQU-16 (14.0 GB) fits, and among quants that fit, PXQ wins

outright at ≤12 GB (fidelity + speed) and trades a hair of fidelity for +20% speed at the 14 GB

ceiling. We do not claim to beat IQ_K on fidelity-per-byte at 3–4 bit — we're faster at matched

size and we fit a bigger model at full prefill speed on one card, which is what "usable tokens/sec on

a $100 GPU" actually needs.

Measured quality (fusion2 flagship, PXQ6)

Canonical hard-gauntlet, 103 items:

| dimension | score |

|---|---|

| Factual / knowledge (myth-busting) | 11/12 |

| Uncensored (edgy + edgy-hard + edgy-operational) | 56/57 complied, 0 refusals |

| Agentic-hard (bug analysis, ambiguity, conditional logic) | 9/10 |

| Coding | ~19/24 coherent solutions |

Fully uncensored, knowledgeable, coherent. (~3% of the hardest adversarial prompts trip a sampling

degeneration at temp 0.6 — lower temp or a retry resolves it.)

Agentic coding (SWE-bench)

The flagship drives a full agentic SWE loop end-to-end: a ReAct tool loop (list_dir / read_file /

grep / submit_patch) over a real shallow git checkout at the issue's base commit, graded by the

official swebench.harness.run_evaluation (prebuilt per-instance Docker images, FAIL_TO_PASS /

PASS_TO_PASS) — not a reimplemented grader.

SWE-bench Lite: 11/56 resolved = 19.6%. Of the 38 problems where it produced a non-empty patch, it

resolved 28.9%; on 18 it hit the turn limit without a patch. For a ~3B-active 35B running a

bare-bones ReAct harness on a single salvaged GPU, ~20% on real GitHub issues (officially graded) is a

genuinely strong result — no fine-tuning for SWE, no elaborate scaffold. The full-run command ships with the

kernel repo (run_swe.py --limit 300).

Speed — on every landfill card (measured decode, our kernel)

| card | ~used price | tier | decode |

|---|---|---|---|

| Tesla P100 16 GB — the bread-and-butter | ~$100 | PXQ3 (14.7 GB, ~flagship) | 55.8 t/s |

| 2× Tesla P100 | ~$200 | PXQ6 (18.7 GB, full flagship) | 55.7 t/s |

| Tesla V100 16 GB — the fast lane | ~$280 | PXQ6 | 99.1 t/s |

| GTX 1080 Ti 11 GB — the closet card | ~$150 | PXQ2 (10.7 GB) | 71.4 t/s |

**The cheapest datacenter card you can feasibly use — a ~$100 Tesla P100 — runs a flagship-quality

35B (PXQ3, +1.1% ppl) at ~56 t/s, fully GPU-resident.** That's the whole point. The 1080 Ti (it has

DP4A) actually edges the P100 at 71.4 t/s on PXQ2; the V100 is the fast lane at ~99 t/s.

• Decode is bandwidth-bound, so bit-width barely moves it — the win is size (fitting the model on the card).
• Prefill ~1920–1960 t/s @ ub2048 (V100). PXQ6-MTP: +5% decode (0.63 draft acceptance; flagship only).
• All numbers: 200-token generation, fully GPU-resident, timings.predicted_per_second, 3 runs.

How to run (you need our kernel)

These are not mainline-llama.cpp quants. Build/run with the companion fork:

https://github.com/poisonxa16/pxq_llama (PXQ2–PXQ6 support + the E16-row scale kernels; CUDA sm_60/61/70, so P100/V100/1080Ti/modern all work).

# build (see the kernel repo README), then:
LD_LIBRARY_PATH=build/bin:build/src:build/ggml/src \
PXA_PXQ6=1 PXA_PXQ2=1 PXA_PXQ3=1 \
PXA_PXQ6_KSPLIT=1 PXA_PXQ6_VECX=1 PXA_PXQ6_GUFUSE=1 PXA_PXQ6_SCATFUSE=1 PXA_PXQ6_RAGTAIL=1 \
./build/bin/llama-server -m PXA-Fusion2-35B-PXQ3.gguf \
  -c 8192 -np 1 -ngl 99 -sm layer -fa on -ctk f16 -ctv f16 -b 512 -ub 512 \
  --jinja --chat-template-kwargs '{"enable_thinking":false}' \
  --temp 1.0 --top-p 0.95 --top-k 20 --host 0.0.0.0 --port 8080

• Enable the format families with PXA_PXQ6=1 + PXA_PXQ2=1 + PXA_PXQ3=1; the PXA_PXQ6_* flags

are the (bit-exact) fast kernels.

• Vision: the flagship supports images — add --mmproj mmproj-fusion2-f16.gguf.
• MTP (flagship): add --spec-type mtp:n_max=3,p_min=0.5.
• Samplers: temp 1.0 / top_p 0.95 / top_k 20 (Ornith generation_config; the model wants temp 1.0).

What PXQ is (short version)

PXQ quantizes the MoE experts (93% of the params) with a learned codebook + E16-row scales

(a per-row fp16 anchor packed as 2 B/row over a 64-row panel, plus a 4-bit sub-scale per 16-element

block). PXQ6 = 4-bit codes (−12.6% weight error vs a plain 4-bit float); PXQ3 = 3-bit bit-plane codes;

PXQ2 = 2-bit codes. The small backbone (attention/router/embeddings) stays MXFP4 — standard

mixed-precision. Full spec + kernels in the kernel repo.

License & lineage

Released under Apache-2.0. Fusion2 is a DELLA task-vector merge of two permissively-licensed parents:

• Ornith-1.0-35B-AEON (0.80) — MIT.
• SIQ-1-35B (AlexWortega/SIQ-1-35B, 0.20) — Apache-2.0.

Both derive from Qwen3.6-35B-A3B (Apache-2.0) upstream. Apache-2.0 is the compatible umbrella for the

combined MIT + Apache-2.0 work; per Apache-2.0 §4 we retain the upstream copyright/attribution NOTICE.

The vision projector (mmproj-fusion2-f16.gguf) is from Ornith. PXQ quantization, the E16-row-scale

kernels, and validation are original work of the PXA project. Credit: Ornith (AEON), SIQ-1 (AlexWortega),

Qwen team (Qwen3.6-35B-A3B), and ikawrakow/ggml-org for the base inference engine.

Support

Built by one person on a homelab of salvaged Tesla cards. If PXQ let you run a 35B on hardware you

already own, a tip keeps the lights (and the GPUs) on: https://ko-fi.com/shatteredrealms1 🙏