GraySoft
Download guIDE →
Projects Models Compare Cloud benchmarks FAQ Download guIDE →
Model Intelligence Sheet

nerkyor/Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-GGUF overview

Qwen3.6 35B A3B DSV4Pro SFT GPT56Sol RL Agent GGUF Qwen3.6 35B A3B AgentRL GGUF evaluation overview assets/hero en gguf v5.png Project GitHub · LynnMerkyor/Lyn…

ggufqwen3.6moeagentrlimatrixlynnstylemultimodalvisionenzhlicense:apache-2.0endpoints_compatibleregion:usconversational

Runs locally from ~585.7 MB disk (4 GB VRAM class GPUs with llama.cpp / guIDE).

Downloads
9,544
Likes
5
Pipeline
—
Author
nerkyor

Repository Files & Downloads

5 GGUF files detected
Direct downloads for local inference
FileTypeQuantizationSizeLink
Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q3_16G-LynnStyle-imatrix.ggufGGUFQ3_16G13.30 GBDownload
Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q4-imatrix.ggufGGUFQ419.06 GBDownload
Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q5-imatrix.ggufGGUFQ527.60 GBDownload
Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q8_0.ggufGGUFQ8_034.37 GBDownload
mmproj-Qwen3.6-35B-A3B-Q8_0.ggufGGUFQ8_0585.7 MBDownload

Model Details

Model IDnerkyor/Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-GGUF
Authornerkyor
Pipeline—
Licenseapache-2.0
Base model—
Last modified2026-08-01T04:18:32.000Z

Model README

---

license: apache-2.0

language:

- en

- zh

frameworks:

- llama.cpp

tasks:

- text-generation

tags:

- qwen3.6

- moe

- agentrl

- gguf

- imatrix

- lynnstyle

- multimodal

- vision

---

Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-GGUF

!Qwen3.6-35B-A3B AgentRL GGUF evaluation overview

Project

<!-- LCB-NOTE-EN:START -->

> LCB note: LCB is the standard abbreviation for LiveCodeBench. It selects competitive-programming problems published after the evaluated model's training-data cutoff and grades executable solutions with test cases to reduce training-data contamination. LCB100 on this card is the fixed 100-problem coding evaluation used for all compared tiers.

<!-- LCB-NOTE-EN:END -->

<!-- REAP-35B-EN:START -->

Why the featured 16GB LynnStyle tier is not uniform Q3

The released architecture remains 40 MoE blocks × 256 routed experts per block, with top-8 experts activated per token. The 16GB tier is a mixed-precision GGUF, not a uniform 3-bit conversion: it keeps the 256-expert architecture and reserves error budget for routing, convergence, and long-chain state paths.

Benchmark-REAP routing evidence was collected by forward-only observation on 128 MMLU, 64 GPQA, and 64 LCB prompts. It records router workloads only and does not use answer labels, canonical code, or execution outcomes to choose protected layers. A separate 24-chunk, 900-entry imatrix constrains tensor error inside the explicit protection schedule.

!35B Benchmark-REAP routing concentration

!35B Benchmark-REAP expert workload heatmap

The three task families expose different hot blocks. Q3 v2 raises L8/L19/L21/L28 to Q3_K and L20/L25 to Q4_K to prioritize long-chain GPQA reasoning. These plots are routing evidence, not benchmark scores.

Actual protection schedule for the featured 16GB Q3 LynnStyle tier

| Protection bucket | Tensors / layers | Actual 16GB Q3 type | Purpose |

|---|---|---|---|

| Input/output anchors | token_embd.weight, output.weight | Q5_K | Preserve vocabulary input and final convergence |

| Routing and small structures | 40 ffn_gate_inp_shexp tensors in BF16; router inputs, norms, biases, and small SSM a/conv/dt/norm structures in F32 | BF16 / F32 | Keep routing, normalization, and state updates out of low-bit expert compression |

| Shared-expert and main SSM paths | Shared-expert/SSM paths in L0–L3 use Q5_K; the remaining main shared-expert/SSM matrices use Q4_K | Q5_K / Q4_K | Preserve the shared convergence path and linear-attention state channel used by every token |

| Front-end and full-attention anchors | MoE experts in L0–L3; full-attention q/k/v/o in L3, L7, L11, …, L39 | L0–L1 experts Q4_K; L2–L3 experts Q5_K; attention q/k/v/o Q5_K | Protect early representation, full-attention read/write, and cross-segment alignment |

| Routed experts: highest protection | L2–L3 | Q5_K, 6 tensors | Protect early input modeling |

| Routed experts: strong protection | L0–L1, L20, L25, L39 | Q4_K, 15 tensors | Protect the opening blocks, GPQA hotspots, and final convergence |

| Routed experts: reasoning hotspots | L8, L19, L21, L28 | Q3_K, 12 tensors | Add v2 protection for long-chain GPQA hotspots |

| Routed experts: shoulder protection | L9, L13, L14, L16, L17, L26, L29, L31 | IQ3_XXS, 24 tensors | Preserve shared shoulder layers across task families |

| Routed experts: regular layers | The remaining 21 layers | IQ2_S, 63 tensors | Carry most of the size reduction |

| Imatrix calibration | 24 chunks, 900 entries | Applied throughout the schedule above | Further constrain tensor error inside the explicit protection policy |

For Q3 v2: embeddings/output and full-attention q/k/v/o use Q5_K; router inputs, norms, biases, and small controls remain F32; 40 shared-expert gates remain BF16; shared-expert and SSM paths use Q5_K/Q4_K. Routed experts use Q5_K on L2–L3, Q4_K on L0–L1/L20/L25/L39, Q3_K on L8/L19/L21/L28, IQ3_XXS on eight shoulder layers, and IQ2_S on the remaining 21 layers. The actual 673-tensor distribution is F32 261, BF16 40, Q5_K 69, Q4_K 204, Q3_K 12, IQ3_XXS 24, and IQ2_S 63.

<!-- REAP-35B-EN:END -->

This repository contains LynnStyle GGUF tiers for Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent plus an optional official vision projector.

Highlights

  • Featured 16GB tier: Q3 LynnStyle v2, 14.28 GB (13.30 GiB).
  • Versus the former Q3, v2 keeps MMLU unchanged, improves GPQA by 1.01pp, and moves LCB from 61/100 to 59/100 while reducing length/empty/runtime anomalies.
  • This tier is reasoning-first. Avoid heavy code generation on Q3 v2; choose Q4 for coding.
  • An optional official Q8_0 vision projector is provided and was functionally validated with the exact Q4 file. No MTP sidecar is included. This card does not claim a strict Pareto improvement.

Training lineage and data role

| Stage | Role | How this card reports it |

|---|---|---|

| Original Qwen3.6-35B-A3B architecture | 35B-A3B MoE general-purpose base | Results from the former runner are not mixed with the current clean runner, avoiding invalid comparisons |

| DSV4Pro + GPT-5.6 Sol SFT parent | Improves reasoning, task convergence, and high-quality text behavior | Same-runner baseline: MMLU 91.0%, GPQA 79.80%, LCB 73/100, Agent100 38/100 |

| AgentRL v3 | Tool protocol, failure exit, and evidence-bound delivery | Q3/Q4 have the clean full-evaluation rows below; Q5/Q8 are additionally listed only after their own exact-file SHA, GGUF parse, and runtime gates passed |

Full evaluation

Primary evaluation settings: open thinking, maximum output of 32768 tokens, and up to 1800 seconds per problem. MMLU500, GPQA198, and LCB100 are scored from final outcomes; parse, length, timeout, runtime, and empty failures are never silently converted into correct answers.

| Version | MMLU500 | GPQA198 | LCB100 | Agent100 | File size |

|---|---:|---:|---:|---:|---:|

| DSV4Pro/SFT parent, same runner | 91.0% (455/500) | 79.80% (158/198) | 73/100 | 38/100 | — |

| Q4 LynnStyle | 90.00% (450/500) | 81.31% (161/198) | 68/100 | 33/100 | 20.47 GB |

| Q3 16G LynnStyle v2 | 90.20% (451/500) | 79.29% (157/198) | 59/100 | 32/100 | 14.28 GB |

<!-- AGENT100-V4-EN:START -->

> Complete Agent100 evaluation: Q4 and Q3 v2 each completed 100 real trajectories under the same C4, 32K-per-slot, 10-step, 8192-token-per-step, 900-second-timeout setup, followed by an independent gpt-5.6-sol (xhigh) final judge. Q4 scores 33/100 and Q3 v2 scores 32/100; false-verify counts are 12/20, protocol-error trajectory flags are 5/5, and judge errors are zero. A protocol-error flag describes the task trajectory; it is not a download, transport, or judge-service failure.

<!-- AGENT100-V4-EN:END -->

Q3 v2 is selected because MMLU stays flat and GPQA gains two correct answers over the former Q3. LCB loses two points, but average completion tokens fall 22.93%, length finishes improve from 16 to 10, empty code from 14 to 9, and runtime failures from 9 to 7. The remaining coding loss is the 13GB tradeoff, not a new generation-instability regression.

The actual Q3 16GB tradeoff

| Comparison | MMLU | GPQA | LCB |

|---|---:|---:|---:|

| Q3 v2 vs former Q3 | 0.00pp | +1.01pp | -2.00pp |

| Q3 v2 vs Q4 | +0.20pp | -2.02pp | -9.00pp |

Quantization design

The REAP figures above document the task-family routing differences that inform the LynnStyle layer/path protection schedule. Direct quant, or imatrix alone without explicit core-path protection, is not a LynnStyle tier in this repository.

Download tiers

<!-- VERIFIED-ARTIFACTS-V8-EN:START -->

> Availability: all four files below passed their own exact-file SHA, GGUF-parse, and runtime-generation gates. The clean full-evaluation table on this card remains limited to Q3/Q4; no Q5/Q8 benchmark score is inferred from an artifact gate.

| Recommended VRAM | File | Positioning |

|---:|---|---|

| 16GB | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q3_16G-LynnStyle-imatrix.gguf | Featured v2 reasoning-first tier |

| 24GB | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q4-imatrix.gguf | Balanced tier |

| 32GB | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q5-imatrix.gguf | Q5 LynnStyle artifact; SHA/GGUF/runtime verified |

| 48GB+ | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q8_0.gguf | Q8_0 artifact; SHA/GGUF/runtime verified |

<!-- VERIFIED-ARTIFACTS-V8-EN:END -->

Why the ModelScope sidebar mislabels the mixed quants

The GGUF panel in the right sidebar is generated by ModelScope's automatic scanner. It cannot fully represent a LynnStyle file that mixes Q5_K, Q4_K, Q3_K, IQ families, and preserved tensors by layer and tensor family. It may therefore label the mixed-quant Q3/Q4/Q5 files as unknown / IMATRIX; use the exact filename in the file list instead.

Open the repository's file list, download exactly one complete GGUF by the filename in the table above, and launch it with llama.cpp. Do not use the sidebar's unknown / IMATRIX entry to infer quantization type, file size, or VRAM requirements.

Actual context capacity depends on KV-cache format, backend, concurrency, and non-model VRAM use. This repository provides an optional official vision projector validated with the exact Q4 file, but no MTP sidecar.

llama.cpp launch

Featured Q3 16GB tier:

llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q3_16G-LynnStyle-imatrix.gguf \
  -ngl 999 -c 16384 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

This repository does not provide an MTP sidecar. Do not add --model-draft or treat the vision projector/any other GGUF as a draft model.

<!-- VISION-PROJECTOR-V6-EN:START -->

Q4 + official vision projector (tested)

This repository provides the optional official vision projector mmproj-Qwen3.6-35B-A3B-Q8_0.gguf (614.19 MB, SHA256 904cbf8c8e876220066ab3bf676c7efa40f3da372276fdaf8b01d2fb2a37a51d). It is neither a main model nor an MTP sidecar; pass it to llama.cpp together with the main model. Compatibility was validated only with the exact Q4 file in this repository; Q3/Q5/Q8 vision paths are not marked as tested.

The command below is the CPU-only, single-concurrency command that actually passed. It proves joint loading and functional image input; it is not a GPU TPS or VRAM claim:

CUDA_VISIBLE_DEVICES= llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q4-imatrix.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf \
  -ngl 0 --no-mmproj-offload --fit off \
  --image-min-tokens 1024 -c 3072 -np 1 -n 512 \
  --host 127.0.0.1 --port 18283 \
  --metrics --no-webui --cache-ram 0

Validated with llama.cpp b9922 (commit 230ea9d21): joint model/projector load and health passed; OCR returned exact Z8K4; the shape test returned three red circles and one blue square; the text regression returned exact TEXT PATH OK.

<!-- VISION-PROJECTOR-V6-EN:END -->

Limitations

  • Vision compatibility was smoke-tested only for the exact Q4 + official Q8_0 projector. This is not a full vision benchmark, and Q3/Q5/Q8 vision paths are not marked as validated.
  • Q3 v2 prioritizes MMLU/GPQA and scores 59/100 on LCB. For long, code-heavy work, prefer Q4.
  • The model may still declare completion too early when evidence is insufficient. Preserve tool output and independently review high-risk tasks.

---

中文

!Qwen3.6-35B-A3B AgentRL GGUF 评测总览

<!-- LCB-NOTE-ZH:START -->

> LCB 说明： LCB 是 LiveCodeBench 的标准缩写。它按照被测模型训练数据的截止日期，选取该日期之后发布的竞赛编程题，并通过执行测试用例判定代码正确性，以降低训练数据污染。本卡的 LCB100 为固定 100 题同口径编码测试。

<!-- LCB-NOTE-ZH:END -->

<!-- REAP-35B-ZH:START -->

为什么 Q3 v2 选择推理优先

35B-A3B 的实际结构是 40 个 MoE block × 每层 256 个 routed experts，单 token 激活 top-8 experts。因此，Q3 16G v2 不是把整模型统一转成一种 3-bit 格式：它保留完整的256专家架构，让专家大矩阵承担主要压缩，同时保护路由、共享路径、full-attention、SSM及任务热点层。

这份策略有两层依据：Benchmark-REAP路由观测使用MMLU 128题、GPQA 64题和LCB 64题，只记录前向router负载，用来识别三类任务各自的热点层；24块、900 entries的imatrix再约束量化误差。REAP产物不保存题目、答案、标准代码或执行结果，也不使用正确性标签决定保护层。

!35B Benchmark-REAP 路由集中度

!35B Benchmark-REAP 专家工作负载热力图

三类任务的热点并不重合。v2因此把L8/L19/L21/L28提升到Q3_K，并把L20/L25提升到Q4_K，优先换回GPQA长链推理；MMLU维持旧Q3水平。图是前向路由证据，不是测评分数。

主推 16GB Q3 LynnStyle 的实际保护表

| 保护桶 | 真实张量 / 层区 | 16GB Q3 实际类型 | 作用 |

|---|---|---|---|

| 输入输出锚点 | token_embd.weight、output.weight | Q5_K | 保住词表进入与收口 |

| 路由与小结构 | 40 个 ffn_gate_inp_shexp 为 BF16；router input、norm、bias、SSM a/conv/dt/norm 小结构为 F32 | BF16 / F32 | 路由选择、归一化和状态更新不随专家矩阵一起低比特化 |

| shared-expert 与 SSM 主路径 | L0–L3 的 shared-expert/SSM 为 Q5_K；其余 shared-expert/SSM 主矩阵为 Q4_K | Q5_K / Q4_K | 保留每个 token 都经过的共享收口路径与线性注意力状态通道 |

| 首段与 full-attention 锚点 | L0–L3 MoE 专家；以及 L3、L7、L11、…、L39 的 full-attention q/k/v/o | L0–L1 experts Q4_K；L2–L3 experts Q5_K；attention q/k/v/o Q5_K | 保护输入早期建模、full-attention 读写和跨段对齐 |

| routed experts：最高保护 | L2–L3 | Q5_K，6 tensors | 保护输入早期建模 |

| routed experts：强保护 | L0–L1、L20、L25、L39 | Q4_K，15 tensors | 保护首段、GPQA热点与最终收口 |

| routed experts：推理热点 | L8、L19、L21、L28 | Q3_K，12 tensors | v2新增的GPQA长链热点保护 |

| routed experts：肩部保护 | L9、L13、L14、L16、L17、L26、L29、L31 | IQ3_XXS，24 tensors | 保留任务族共同肩部层 |

| routed experts：普通层 | 其余21层 | IQ2_S，63 tensors | 承担主要体积压缩 |

| imatrix误差校准 | 24块、900 entries | 贯穿上述量化 | 在显式保护规则内继续控制张量误差 |

v2的673个tensor实际分布为：F32 261、BF16 40、Q5_K 69、Q4_K 204、Q3_K 12、IQ3_XXS 24、IQ2_S 63。标准GGUF把同一层的256个routed experts堆在一个张量中，所以当前公开档采用层级/路径保护，而不是在同一张量内给单个专家指定不同类型。

仅做 direct quant，或只有 imatrix 而没有这张显式保护表，都不属于本仓的 LynnStyle 档位。

<!-- REAP-35B-ZH:END -->

这是 Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent 的 LynnStyle GGUF 量化仓，并提供一个可选官方视觉投影器。全部主模型档位都使用 LynnStyle MoE 分层混合精度，而不是对所有张量统一降位。

> 先看结论

>

> - 16GB 主推档：Q3 LynnStyle v2，14.28 GB（13.30 GiB）。

> - 相对旧Q3：MMLU持平，GPQA提高1.01pp；LCB从61/100降至59/100，但截长、空代码和runtime异常均减少。

> - v2明确选择通识与重推理优先。13GB预算下的代码能力下降不再用额外量化补救；请规避用该档承担重度代码生成。

> - 本仓新增可选官方 Q8_0 视觉投影器；已与精确 Q4 文件完成真实加载、OCR、图形识别与文本回归。

> - 本仓仍不提供 MTP sidecar；不要把视觉投影器当作 draft model。

> - ModelScope 源仓已通过发布前/发布后校验并开放下载。

项目

训练谱系与数据作用

| 阶段 | 作用 | 本卡如何报告 |

|---|---|---|

| Qwen3.6-35B-A3B 原始架构 | 35B-A3B MoE 通用母体 | 旧 runner 结果不与当前 clean runner 混排，避免伪横比 |

| DSV4Pro + GPT-5.6 Sol SFT 母模 | 提升推理、任务收敛与高质量文本行为 | 作为同 runner 基线：MMLU 91.0%、GPQA 79.80%、LCB 73/100、Agent100 38/100 |

| AgentRL v3 | 工具协议、失败退出、证据绑定交付 | Q3/Q4 保留下面的 clean 全量评测；Q5/Q8 仅在各自同文件 SHA、GGUF 解析与运行门禁通过后列出 |

同口径全量评测

主评测口径：open-thinking、最大输出 32768 tokens、单题最长 1800 秒；MMLU500 / GPQA198 / LCB100 均按最终结果计分，parse/length/timeout/runtime/empty 不会被静默改判为正确。

| 版本 | MMLU500 | GPQA198 | LCB100 | Agent100 | 文件大小 |

|---|---:|---:|---:|---:|---:|

| DSV4Pro/SFT 同口径母模 | 91.0% (455/500) | 79.80% (158/198) | 73/100 | 38/100 | — |

| Q4 LynnStyle | 90.00% (450/500) | 81.31% (161/198) | 68/100 | 33/100 | 20.47 GB |

| Q3 16G LynnStyle v2 | 90.20% (451/500) | 79.29% (157/198) | 59/100 | 32/100 | 14.28 GB |

<!-- AGENT100-V4-ZH:START -->

> Agent100完整补测： Q4 与 Q3 v2 均完成100条真实轨迹，使用相同C4、每路32K、最多10步、单步最多8192 tokens与900秒超时设置，并由 gpt-5.6-sol（xhigh）按同一规则独立终判。Q4为 33/100，Q3 v2为 32/100；false-verify分别为12/20，protocol-error轨迹标记均为5，判分器errors均为0。protocol-error是任务轨迹结果，不是下载、传输或判分服务错误。

<!-- AGENT100-V4-ZH:END -->

Q3 16G 的真实取舍

| 对比 | MMLU | GPQA | LCB |

|---|---:|---:|---:|

| Q3 v2 vs 旧Q3 | 0.00pp | +1.01pp | -2.00pp |

| Q3 v2 vs Q4 | +0.20pp | -2.02pp | -9.00pp |

v2选择理由很直接：MMLU不退，GPQA多答对2题，平均思考tokens基本持平；虽然LCB少2题，但平均completion tokens下降22.93%，截长16→10、空代码14→9、runtime 9→7、timeout仍为1。也就是说，代码分数下降并不是生成异常恶化，而是13GB预算下代码能力本身的取舍。需要稳定写代码时请选择Q4；Q3 v2建议用于通识、推理和非重度代码场景。

下载档位

<!-- VERIFIED-ARTIFACTS-V8-ZH:START -->

> 当前文件： 下列四份文件均已通过各自的同文件 SHA、GGUF 解析与真实生成门禁。下方 clean 全量评测表仍只报告 Q3/Q4；不会把文件门禁推导成 Q5/Q8 的评测分数。

| 建议显存 | 文件 | 定位 |

|---:|---|---|

| 16GB | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q3_16G-LynnStyle-imatrix.gguf | v2推理优先主推档 |

| 24GB | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q4-imatrix.gguf | 平衡档 |

| 32GB | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q5-imatrix.gguf | Q5 LynnStyle；已验证 SHA/GGUF/运行 |

| 48GB+ | Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q8_0.gguf | Q8_0；已验证 SHA/GGUF/运行 |

<!-- VERIFIED-ARTIFACTS-V8-ZH:END -->

为什么右侧自动量化档位显示不准确

ModelScope右侧的GGUF量化卡片来自平台自动扫描，无法准确表达LynnStyle这种逐层、逐tensor混合精度文件：Q3/Q4/Q5同时包含Q5_K、Q4_K、Q3_K、IQ系列和保留精度张量，平台因此可能标为unknown / IMATRIX。请以文件列表中的完整文件名为准。

请进入“模型文件”页，严格按上表完整文件名下载单个GGUF，并使用下方llama.cpp命令启动；不要依据右侧unknown / IMATRIX入口判断量化类型、文件大小或显存需求。

显存档位是实用选型，不等于文件大小；实际可用上下文依赖 KV cache 格式、后端、并发和系统显存占用。

llama.cpp 启动

Q3 16G 主推档：

llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q3_16G-LynnStyle-imatrix.gguf \
  -ngl 999 -c 16384 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

本仓不提供 MTP sidecar，不要添加 --model-draft 或把视觉投影器/任何其他 GGUF 当作 draft model。

<!-- VISION-PROJECTOR-V6-ZH:START -->

Q4 + 官方视觉投影器（已实测）

本仓提供可选官方视觉投影器 mmproj-Qwen3.6-35B-A3B-Q8_0.gguf（614.19 MB，SHA256 904cbf8c8e876220066ab3bf676c7efa40f3da372276fdaf8b01d2fb2a37a51d）。它不是主模型，也不是 MTP sidecar；必须与主模型同时传给 llama.cpp。当前只对本仓精确 Q4 文件完成兼容性验收，未把 Q3/Q5/Q8 视觉能力标为已测。

以下是本次实际通过的 CPU-only 单并发命令；它用于证明联合加载与图像链路可用，不代表 GPU TPS 或显存成绩：

CUDA_VISIBLE_DEVICES= llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q4-imatrix.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf \
  -ngl 0 --no-mmproj-offload --fit off \
  --image-min-tokens 1024 -c 3072 -np 1 -n 512 \
  --host 127.0.0.1 --port 18283 \
  --metrics --no-webui --cache-ram 0

验收环境为 llama.cpp b9922（commit 230ea9d21）：联合加载与健康检查通过，OCR 精确读取 Z8K4，图形题精确识别 3 个红圆和 1 个蓝方块，纯文本回归精确输出 TEXT PATH OK。

<!-- VISION-PROJECTOR-V6-ZH:END -->

限制

  • 视觉兼容性仅对本仓精确 Q4 + 官方 Q8_0 投影器完成 smoke；这不是完整视觉基准，也不代表 Q3/Q5/Q8 视觉路径已验收。
  • Q3 v2优先保住MMLU/GPQA；LCB为59/100，不建议用于重度代码长链任务，代码优先请选择Q4。
  • 模型仍可能在证据不足时过早宣告完成；高风险任务要保留工具输出并独立复核。

Run nerkyor/Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-GGUF with guIDE

Download guIDE — the AI-native code editor with local LLM inference and 69 built-in tools.

Download guIDE → · Browse 524k+ models · Compare models

Source: Hugging Face · Compare models