---

language:

• en
• fi

license: apache-2.0

tags:

• finnish
• uncensored
• multimodal
• vision
• reasoning
• moe
• gguf
• imatrix
• mtp
• industrial-automation
• plc-st-scl
• quality-inspector
• data-surgeon
• kataguru

pipeline_tag: image-text-to-text

base_model: Qwen/Qwen3.5-35B-A3B

---

Kataguru-Sceptic-35B-A3B-v2.6-GGUF

<p align="center">

<img src="sceptic.png" alt="Kataguru Sceptic 35B-A3B v2.6" width="46%" />

<img src="inspector.png" alt="Kataguru Quality Inspector & Surgeon" width="46%" />

</p>

Kataguru-Sceptic-35B-A3B-v2.6-GGUF is the llama.cpp / LM Studio / Ollama GGUF release of Kataguru-Sceptic-35B-A3B-v2.6, a bilingual (English & Finnish) 35-billion parameter (3B active parameters per token, 254 routed experts per layer) multimodal Mixture-of-Experts (MoE) model.

Unlike the 25.44 GB NVFP4 and 27.85 GB AWQ server packages, these imatrix-calibrated 3-bit GGUF quantizations (12.0 GiB to 15.3 GiB) are specifically engineered to fit wholly inside the VRAM of a single 16 GB or 24 GB GPU.

---

1. Specialized Capabilities & Architectural Scope

1. Dual-Stage 2D Zero-Swap Epistemic Triage (System 1 Reflex):

- Evaluates candidate training samples in < 20 ms (measured 18.2 ms single-call latency on RTX 5090) using 1-token logprob scoring (max_tokens=1, top_logprobs=10) with 0 generated output tokens.

- Computes both sample quality $E[\text{sample}]$ (decision mode) and prompt learning potential $E[\text{topic}]$ (topic mode) to fast-reject trivial data while routing high-potential STEM prompts with flawed answers to the Surgeon.

1. Autonomous Self-Healing Data Surgeon:

- In surgeon mode ([MODE: SURGEON] / [TILA: KIRURGI]), rewrites flawed mathematical, physical, or software responses from first principles so that they can be verified by an external execution sandbox (e.g., isolated Python 3.12 subprocess execution with assert checks).

1. Industrial Automation & PLC/CNC Code Generation:

- Trained on curriculum mixtures covering Beckhoff TwinCAT 3 Structured Text (ST), Siemens TIA Portal SCL, Fanuc / Siemens 840D CNC G-code, and OPC UA (IEC 62541) diagnostic schemas.

1. Multimodal Visual & SCADA Diagnostic Agent (Zero-Swap Unified Pipeline):

- Why Integrated Multimodality? Hosting separate vision and coding/reasoning models either doubles VRAM consumption or introduces model-swapping latency. Integrating the vision tower (visual) directly into the 254-expert MoE backbone allows a single model instance to read an HMI alarm screenshot or wiring schematic, issue XML tool calls (qwen3_xml / qwen3_coder), and generate PLC ST/SCL diagnostic code without swapping weights.

- Hardware & Memory Scope: Running the unified multimodal pipeline on a single 16–24 GB GPU without CPU offloading requires the 3-bit quantized GGUF release (APEX-Small 12.0 GiB or APEX-Medium 13.5 GiB + mmproj 0.84 GiB). The NVFP4 (25.44 GB / 23.70 GiB) and AWQ (27.85 GB / 25.94 GiB) safetensors packages do not fit in 16 GB VRAM without CPU offloading; they target 2x 24 GB / 2x 32 GB GPUs (TP=2) or a 32 GB+ GPU.

1. Layer 40 Multi-Token Prediction (MTP) Speculative Decoding:

- Preserves the 254-expert Layer 40 MTP head (mtp.*), providing multi-token speculative decoding in both vLLM (--speculative-config) and llama.cpp (--spec-type draft-mtp, measured +12.0% tok/s on 1x RTX 5090).

1. Bilingual Finnish & Technical English Focus:

- Trained to avoid English-to-Finnish translation calques and morphological case errors in technical prose. (Note: Evaluated via internal curriculum filtering gates during data curation; a standalone public Finnish morphological benchmark dataset is not yet bundled in eval/.)

1. Epistemic Directness & ASD-STE100 Technical Style:

- Suppresses sycophantic filler and conversational preambles, following ASD-STE100 (Simplified Technical English) writing principles (short sentences, active voice, one instruction per sentence).

- Important Safety Notice: ASD-STE100 is a technical writing clarity standard; adhering to it improves human readability of comments and diagnostics but does not guarantee functional software or machinery safety. All generated code, PLC logic, and G-code must be validated in external deterministic sandboxes, vendor compilers (TwinCAT / TIA Portal), and hardware-in-the-loop safety checks before deployment.

---

2. Empirical Benchmark Comparison (Same-Backbone & Cross-Backbone Baselines)

To separate the effect of Kataguru post-training from base model differences, the table below compares Kataguru-Sceptic-35B-A3B-v2.6 (NVFP4) against:

1. Qwen3.5-35B-A3B Pre-SFT Base (cp0): The same 35B-A3B MoE backbone after initial SOMA/ARA orthogonalization before SFT curriculum training (isolates cumulative MoE post-training impact on tasks 1–11; raw log in eval/qwen35_35b_a3b_cp0_base_summary.json).
2. Qwen 3.8 27B Base (Untuned Dense): Cross-architecture dense baseline evaluated on the same 13-task harness.

All differences ($\Delta$) are reported in percentage points (pp).

| # | Benchmark (Task ID) | Capability Area & Metric | Kataguru-Sceptic-35B-A3B-v2.6 | Qwen3.5-35B-A3B Base (cp0) | $\Delta$ vs. 35B Base (pp) | Qwen 3.8 27B Base | $\Delta$ vs. 27B Base (pp) |

| :- | :--- | :--- | :---: | :---: | :---: | :---: | :---: |

| 1 | PIQA | Physical Intuition (acc_norm) | <b style="color:#0f5132;">83.46 %</b> | 81.72 % | +1.74 pp | 80.10 % | +3.36 pp |

| 2 | WinoGrande | Contextual Reasoning (acc) | <b style="color:#0f5132;">75.45 %</b> | 71.82 % | +3.63 pp | 71.10 % | +4.35 pp |

| 3 | MedQA (USMLE) | Clinical Medicine 4-opts (acc) | <b style="color:#0f5132;">85.31 %</b> | 84.10 % | +1.21 pp | 78.40 % | +6.91 pp |

| 4 | TruthfulQA MC2 | Epistemic Truthfulness (acc) | 52.47 % | <b style="color:#0f5132;">52.89 %</b> | -0.42 pp | 49.80 % | +2.67 pp |

| 5 | TruthfulQA MC1 | Single-True Factuality (acc) | <b style="color:#0f5132;">36.23 %</b> | 34.39 % | +1.84 pp | 31.40 % | +4.83 pp |

| 6 | ARC-Easy | Standard STEM (acc_norm) | <b style="color:#0f5132;">82.95 %</b> | 74.83 % | +8.12 pp | 72.98 % | +9.97 pp |

| 7 | HellaSwag | Linguistic Coherence (acc_norm) | 82.42 % | <b style="color:#0f5132;">83.30 %</b> | -0.88 pp | 74.60 % | +7.82 pp |

| 8 | OpenBookQA | Scientific Application (acc_norm) | 45.00 % | <b style="color:#0f5132;">46.00 %</b> | -1.00 pp | 44.80 % | +0.20 pp |

| 9 | ARC-Challenge | Advanced STEM (acc_norm) | <b style="color:#0f5132;">62.54 %</b> | 57.94 % | +4.60 pp | 58.62 % | +3.92 pp |

| 10 | BoolQ | Natural Language Logic (acc) | <b style="color:#0f5132;">90.37 %</b> | 85.78 % | +4.59 pp | 89.60 % | +0.77 pp |

| 11 | GSM8K Math (CoT) | Exact Match (strict / flex) | <b style="color:#0f5132;">89.39 % / 90.07 %</b> | 1.06 % / 12.36 %\* | +88.33 pp | 76.50 % | +12.89 pp |

| $\Sigma_{10}$ | 10-Task Mean (1–10) | Mean Excluding GSM8K Format Gap | <b style="color:#0f5132;">69.62 %</b> | 67.28 % | +2.34 pp | 65.14 % | +4.48 pp |

| $\Sigma_{11}$ | 11-Task Mean (1–11) | Official Overall Cognitive Mean | <b style="color:#0f5132;">71.42 %</b> | 61.26 % (62.28 % flex) | +10.16 pp | 66.17 % | +5.25 pp |

| 12 | GPQA Diamond | PhD-Level STEM (198 items, Exact Match) | <b style="color:#0f5132;">61.11 % (121/198)</b> | not evaluated | — | 45.20 % | +15.91 pp |

| 13 | IFEval (Strict) | Instruction Following (prompt / inst) | <b style="color:#0f5132;">70.98 % / 78.42 %</b> | not evaluated | — | 48.20 % / 58.40 % | +22.78 pp / +20.02 pp |

\Note on GSM8K in cp0: The pre-SFT base checkpoint lacked the #### <number> final-answer formatting convention required by lm-eval strict/flex regex extractors, causing extraction failures (1.06% strict / 12.36% flex). Therefore, both $\Sigma_{10}$ (tasks 1–10 excluding GSM8K, +2.34 pp) and $\Sigma_{11}$ (tasks 1–11, +10.16 pp) are reported above.*

Evaluation Methodology & Published Raw Logs (eval/)

• Evaluation Harness & Versions: Executed on 2x NVIDIA RTX 5090 32GB Blackwell (sm_120, TP=2, port 8003, CUDAGraphs enabled) using lm-evaluation-harness 0.4.13 (recorded in eval/benchmark_13_summary.json) for tasks 1–11 and task 13 (ifeval), and a standalone zero-shot evaluator for task 12 (gpqa_diamond).
• Complete Item-by-Item Logs Included in Repository:

- eval/benchmark_13_summary.json: Aggregated metrics, runtime configurations, and lm-eval 0.4.13 metadata for all 13 tasks.

- eval/<task>/kataguru__Kataguru-Sceptic-v2.6-Symmetric-CP100-NVFP4/samples_<task>_*.jsonl: Complete question-by-question prompts, model completions, and scores for all 12 lm-eval tasks (gsm8k, ifeval, medqa_4options, arc_challenge, arc_easy, boolq, hellaswag, openbookqa, piqa, truthfulqa_mc1, truthfulqa_mc2, winogrande).

- eval/gpqa_diamond.json: Complete item-by-item completions and extracted answers for all 198 GPQA Diamond questions (121/198 = 61.11%).

- eval/qwen35_35b_a3b_cp0_base_summary.json: Historical evaluation summary for the Qwen3.5-35B-A3B pre-SFT base (jackripper-sovereign).

---

2.1 Preliminary Functional Smoke Tests (Data Surgeon, PLC & CNC)

To verify end-to-end runtime functionality of the specialized modes, a small preliminary smoke test suite (eval/specialized_smoke_tests/benchmark_specialized_capabilities.py) was executed against the live NVFP4 endpoint:

| Smoke Test Suite | Sample Size ($N$) | Input Baseline | Kataguru-Sceptic-v2.6 Result | Scope & Verification Limitations |

| :--- | :---: | :---: | :---: | :--- |

| Autonomous Data Surgeon ([MODE: SURGEON]) | $N = 5$ flawed STEM/Python tasks | 0 / 5 pass (0.0 %) | 3 / 5 pass (60.0 %) | External Python 3.12 Sandbox: Extracted python blocks executed in an isolated subprocess (returncode == 0, all assert checks passed). Single-shot without retry. |

| Industrial PLC Syntax Smoke Test (ST / SCL) | $N = 2$ tasks | N/A | 2 / 2 pass (100.0 %) | Structural Keyword Check Only: Verifies presence of FUNCTION_BLOCK/FUNCTION, VAR..END_VAR, and CASE..END_CASE. Does not constitute formal verification or compilation in Beckhoff TwinCAT / Siemens TIA Portal. |

| CNC G-Code Smoke Test (ISO 6983) | $N = 2$ tasks | N/A | 2 / 2 pass (100.0 %) | Structural Regex Check Only: Verifies presence of modal $G0/G1$ moves, $X/Y/Z$ coordinates, and $M3/M30$ spindle/program-end codes. Does not replace CAM toolpath collision simulation. |

| 2D System 1 Epistemic Triage (decision / topic) | $N = 3$ pipeline cases | N/A | 3 / 3 routed (18.2 ms single-call) | Separates trivial prompt ($E[\text{topic}]=0.96$) from flawed STEM prompt ($E[\text{topic}]=6.73$) and golden sample ($E[\text{sample}]=8.04$) with 0 generated tokens (eval/specialized_smoke_tests/sceptic_v2_6_zero_swap_surgeon_pipeline.py). |

Caveat: The PLC ($N=2$), CNC ($N=2$), and Surgeon ($N=5$) results above are small-scale functional smoke tests demonstrating pipeline mechanics and external sandbox integration, not statistically large domain benchmarks.

---

3. GGUF Files & Single-GPU VRAM Breakdown

| File Name | Disk Size | Precision | Measured VRAM Breakdown & Target GPU |

| :--- | :---: | :---: | :--- |

| Kataguru-Sceptic-35B-A3B-v2.6-APEX-Small.gguf | 12.0 GiB | 2.92 bpw | Fits on 1x 16 GB GPU: 12.0 GiB weights + ~0.84 GiB mmproj (optional) + ~1.5 GiB KV/compute (8k–16k q8_0 KV) = ~14.3 GiB VRAM. |

| Kataguru-Sceptic-35B-A3B-v2.6-APEX-Medium.gguf | 13.5 GiB | 3.28 bpw | Recommended for 16–24 GB GPUs: 13.5 GiB weights + ~1.5 GiB KV/compute = ~15.0 GiB VRAM (text-only on 16 GB GPU, or with mmproj + mtp on 24 GB GPU). |

| Kataguru-Sceptic-35B-A3B-v2.6-1M-APEX-Medium.gguf | 13.5 GiB | 3.28 bpw | 1M Context Header Edition (qwen35moe.context_length = 1048576 in GGUF header; use -c <len> to size KV cache to available VRAM/RAM). |

| Kataguru-Sceptic-35B-A3B-v2.6-APEX-Good.gguf | 15.3 GiB | 3.71 bpw | Target: 1x 24 GB or 32 GB GPU (RTX 3090 / 4090 / 5090): 15.3 GiB weights + 0.84 GiB mmproj + 1.9 GiB mtp-Q8_0 + KV cache = ~20.5 GiB VRAM at 32k context. |

| mmproj-Kataguru-Sceptic-35B-A3B-v2.6-f16.gguf | 861 MB | FP16 | Multimodal vision projector (--mmproj). |

| mtp-Kataguru-Sceptic-35B-A3B-v2.6-Q8_0.gguf | 1.9 GiB | Q8_0 | Layer 40 254-expert MTP draft model (-md ... --spec-type draft-mtp). Measured 30.7 tok/s vs. 27.4 tok/s (+12.0%) on GPU. |

| mtp-Kataguru-Sceptic-35B-A3B-v2.6-Q4_K_M.gguf | 1.2 GiB | Q4_K_M | Compact Layer 40 MTP draft model. |

| sceptic_254.imatrix & apex_*.txt | 99 MB | — | Activation importance matrix and layer quantization recipes. |

---

4. Native 6 Operating Modes (Kataguru Master Chat Template)

The embedded chat_template.jinja recognizes both English and Finnish trigger prefixes (case-insensitive) at the start of the user prompt, or via chat_template_kwargs={"sceptic_mode": "..."}:

| Mode (sceptic_mode) | English Text Prefix | Finnish Text Prefix | Behavior & Description |

| :--- | :--- | :--- | :--- |

| chat (default) | (none) | (ei etuliitettä) | General engineering, coding, and reasoning assistant. Supports {REASON:spoon}, {REASON:einstein}, {REASON:deep}/{REASON:xhigh}, {REASON:high}, {REASON:medium}, {REASON:low}, {REASON:none}/{REASON:off}. |

| decision | [MODE: DECISION] or [SYSTEM 1 DECISION] | [LAATUORAAKKELI: SYSTEEMI 1] | System 1 logprob quality oracle (0–9). Appends the single-digit scoring prompt automatically and disables <think>. |

| topic | [MODE: TOPIC] or [SYSTEM 1 TOPIC] | [LAATUORAAKKELI: AIHE] | System 1 topic potential oracle (0–9). Scores prompt learning potential to separate STEM diamonds ($E[\text{topic}] \ge 6.0$) from trivial queries. |

| inspect | [MODE: INSPECT] or [QUALITY INSPECTOR] | [TILA: TARKISTUS] | System 2 Verdict-First JSON quality inspector ({"verdict": "ACCEPT"|"REJECT", "score": 0-10, "learning_value": 0-10}). |

| curate | [MODE: CURATE] or [DATA CURATOR] | [TILA: KURAATTORI] or [TILA: KOULUTUS] | First-Principles dataset synthesis and audit mode. |

| surgeon | [MODE: SURGEON] or [DATA SURGEON] | [TILA: KIRURGI] | Autonomous Data Surgeon: rewrites failing derivations and broken Python code blocks from first principles. |

---

5. Deployment Examples (llama.cpp)

Text Inference + Layer 40 MTP Speculative Decoding

llama-server \
  -m Kataguru-Sceptic-35B-A3B-v2.6-APEX-Medium.gguf \
  -md mtp-Kataguru-Sceptic-35B-A3B-v2.6-Q8_0.gguf \
  --spec-type draft-mtp \
  --draft-max 2 \
  -ngl 99 -ngld 99 \
  -c 32768 --port 8080

Multimodal Vision & SCADA Diagnostics

llama-server \
  -m Kataguru-Sceptic-35B-A3B-v2.6-APEX-Medium.gguf \
  --mmproj mmproj-Kataguru-Sceptic-35B-A3B-v2.6-f16.gguf \
  -ngl 99 -c 16384 --port 8080

---

---

Suomenkielinen Yhteenveto & Käyttöopas

Kataguru-Sceptic-35B-A3B-v2.6 on suomen- ja englanninkielinen 35 miljardin parametrin (3B aktiivista parametria / token, 254 reititettyä asiantuntijaa / kerros) multimodaalinen Mixture-of-Experts (MoE) -malli, joka yhdistää kuusi toimintatilaa (decision, topic, inspect, curate, surgeon, chat), visioprojektorin ja Layer 40 MTP-spekulaatiopään yhteen mallipakettiin.

Keskeiset ominaisuudet ja rajaukset

• 2D System 1 Episteeminen Pikatarkastus (decision & topic): Pisteyttää näytteen laadun ($E[\text{sample}]$) ja aiheen potentiaalin ($E[\text{topic}]$) yhden tokenin logprob-jakaumasta (< 20 ms) ilman <think>-hukkatokeneita.
• Autonominen Datan Kirurgi ([TILA: KIRURGI] / [MODE: SURGEON]): Korjaa virheelliset STEM-johdot ja kaatuvat Python-koodilohkot siten, että korjattu koodi voidaan todentaa ulkoisessa Python-suoritusportissa (alustavassa toimivuustestissä 3/5 viallisesta näytteestä läpäisi ulkoisen Python assert -ajon yhdellä yrityksellä).
• Teollisuusautomaatio & Multimodaalinen Diagnostiikka: Tukee Beckhoff TwinCAT 3 ST-, Siemens TIA Portal SCL- ja ISO CNC G-koodin generointia sekä kytkentäkaavioiden ja HMI-hälytysruutujen lukua.

- Miksi integroitu multimodaalisuus? Yhdistämällä visio suoraan MoE-runkoon vältetään kahden erillisen mallin yhtäaikainen muistirasitus tai mallien latausviiveet. Yhden 16–24 Gt näytönohjaimen käyttö ilman CPU-siirtoa edellyttää GGUF-versiota (APEX-Small 12,0 GiB tai APEX-Medium 13,5 GiB); NVFP4- (25,44 GB) ja AWQ-versiot (27,85 GB) on tarkoitettu 2x 24 Gt / 2x 32 Gt (TP=2) tai 32 Gt+ ympäristöihin.

• Suomen kieli ja ASD-STE100 -selkeysstandardi: Malli on koulutettu tuottamaan teknistä suomea ja englantia ilman käännösanglismeja sekä noudattamaan koodikommenteissa ja diagnostiikassa ilmailualan ASD-STE100 (Simplified Technical English) -selkeyssääntöjä (lyhyet virkkeet, aktiivimuoto, ei täytesanoja).

- Turvallisuushuomautus: ASD-STE100 on teknisen kirjoittamisen selkeysstandardi; se ei takaa ohjelman tai logiikan toiminnallista turvallisuutta. Kaikki tuotettu koodi sekä PLC- ja CNC-ohjelmat on aina todennettava ulkoisella suoritusportilla ja kohdejärjestelmän omalla kääntäjällä.