---

license: apache-2.0

base_model: FabricAI/Fabric1.6

library_name: transformers

pipeline_tag: image-text-to-text

datasets:

• open-thoughts/OpenThoughts3-1.2M
• open-r1/OpenR1-Math-220k
• HuggingFaceTB/smoltalk2
• NousResearch/hermes-function-calling-v1
• OpenAssistant/oasst2
• FabricAI/maple

tags:

• fabric1.6
• mixture-of-experts
• multimodal
• image-text-to-text
• video-text-to-text
• vision
• video
• reasoning
• agentic
• tool-calling
• long-context
• hybrid-attention
• transformers
• pytorch
• safetensors
• bf16

model-index:

• name: Fabric 1.6

results:

- task:

type: text-generation

name: Math and Reasoning

dataset:

type: aime-2025

name: AIME25

metrics:

- type: pass@1

value: 92.8

name: Pass@1

- task:

type: text-generation

name: Math and Reasoning

dataset:

type: aime-2026

name: AIME26

metrics:

- type: pass@1

value: 93.1

name: Pass@1

- task:

type: text-generation

name: Math and Reasoning

dataset:

type: hmmt-feb-2026

name: HMMT26

metrics:

- type: pass@1

value: 83.2

name: Pass@1

- task:

type: text-generation

name: Math and Reasoning

dataset:

type: imo-answerbench

name: IMOAB

metrics:

- type: pass@1

value: 79.2

name: Pass@1

- task:

type: text-generation

name: Math and Reasoning

dataset:

type: math-500

name: M500

metrics:

- type: accuracy

value: 84.8

name: Accuracy

- task:

type: text-generation

name: Science and Knowledge

dataset:

type: gpqa

name: GPQA

metrics:

- type: accuracy

value: 86.7

name: Accuracy

- task:

type: text-generation

name: Science and Knowledge

dataset:

type: gpqa-diamond

name: GPQA-D

metrics:

- type: accuracy

value: 84.9

name: Accuracy

- task:

type: text-generation

name: Science and Knowledge

dataset:

type: hle

name: HLE

metrics:

- type: accuracy

value: 21.4

name: Accuracy

- task:

type: text-generation

name: Science and Knowledge

dataset:

type: mmlu-pro

name: MMLU-P

metrics:

- type: accuracy

value: 85.6

name: Accuracy

- task:

type: text-generation

name: Science and Knowledge

dataset:

type: mmlu-redux

name: MMLU-R

metrics:

- type: accuracy

value: 93.5

name: Accuracy

- task:

type: text-generation

name: Science and Knowledge

dataset:

type: ceval

name: C-Eval

metrics:

- type: accuracy

value: 92.3

name: Accuracy

- task:

type: text-generation

name: Code Generation

dataset:

type: livecodebench-v6

name: LCB6

metrics:

- type: pass@1

value: 80.2

name: Pass@1

- task:

type: text-generation

name: Code Generation

dataset:

type: swe-bench-verified

name: SWEB-V

metrics:

- type: resolve-rate

value: 72.9

name: Resolve Rate

- task:

type: text-generation

name: Code Generation

dataset:

type: swe-bench-pro

name: SWEB-P

metrics:

- type: resolve-rate

value: 50.1

name: Resolve Rate

- task:

type: text-generation

name: Instruction Following

dataset:

type: ifeval

name: IFEval

metrics:

- type: instruction-level

value: 93.09

name: Instruction Level

- task:

type: text-generation

name: General Reasoning

dataset:

type: gsm8k-platinum

name: GSM8K-Pt

metrics:

- type: accuracy

value: 95.73

name: Accuracy

- task:

type: text-generation

name: Agentic Tool Use

dataset:

type: tau3-bench

name: TAU3

metrics:

- type: pass-rate

value: 67.2

name: Pass Rate

- task:

type: image-text-to-text

name: Visual Question Answering

dataset:

type: mmmu-pro

name: MMMU-P

metrics:

- type: accuracy

value: 74.10

name: Accuracy

- task:

type: image-text-to-text

name: Visual Question Answering

dataset:

type: realworldqa

name: RWQA

metrics:

- type: accuracy

value: 85.4

name: Accuracy

- task:

type: text-generation

name: Agentic Tool Use

dataset:

type: mcp-atlas

name: MCP-A

metrics:

- type: completion

value: 62.8

name: Completion

- task:

type: text-generation

name: Agentic Tool Use

dataset:

type: widesearch

name: WS

metrics:

- type: rubric-score

value: 60.3

name: Rubric Score

- task:

type: image-text-to-text

name: Visual Question Answering

dataset:

type: mathvista-mini

name: MV-mini

metrics:

- type: accuracy

value: 86.6

name: Accuracy

---

<div align="center">

<picture>

<img src="banner_fabric1.6.png" width="100%" alt="Fabric AI">

</picture>

</div>

<hr>

<div align="center" style="line-height:1">

<a href="https://huggingface.co/FabricAI" target="_blank"><img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-FabricAI-ffc107?color=e0a800&logoColor=white"/></a>

<a href="https://fabricai.co.uk" target="_blank"><img alt="Homepage" src="https://img.shields.io/badge/Homepage-Fabric%20AI-white?logo=globe&logoColor=white"/></a>

<a href="https://x.com/fabricai_uk" target="_blank"><img alt="X" src="https://img.shields.io/badge/X-%40fabricai_uk-white?logo=x&logoColor=black"/></a>

</div>

> [!Note]

> This is a GGUF repository for easy inference with llama.cpp, Ollama and LM Studio, quantized from the base model FabricAI/Fabric1.6.

Fabric 1.6

Fabric 1.6 is a 35-billion-parameter Mixture-of-Experts (MoE) reasoning model developed by Fabric AI, with approximately 3 billion parameters activated per token. It is a native multimodal, agentic model built on a hybrid Gated DeltaNet + Gated Attention architecture, with explicit chain-of-thought reasoning, a native 262,144-token context window, and built-in Multi-Token Prediction (MTP) for up to 50% faster generation.

Fabric 1.6 is designed for agentic use in harnesses such as OpenCode, Pi Agent, Hermes Agent and other OpenAI-compatible tool-calling environments, and offers the option to preserve thinking context from past messages across long multi-turn sessions.

1. Key Features

• Hybrid Architecture: Gated DeltaNet (linear attention) layers interleaved with Gated Attention layers inside a 256-expert MoE transformer — sub-quadratic scaling with full attention capacity where it matters.
• Native Long Context: 262,144 tokens natively, extensible up to 1,010,000 tokens.
• Multi-Token Prediction (MTP): predicts multiple future tokens per step for up to 50% faster generation.
• Native Multimodality: accepts text, image and video inputs within the same model.
• Explicit Reasoning: produces an internal chain of thought before answering; reasoning is exposed in a structured format that can be streamed and stored.
• Agentic by Design: reliable structured tool-calling, long-horizon task execution, and preserved thinking context across turns.

2. Model Summary

<div align="center">

<table>

<tbody>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Architecture</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">Hybrid Gated DeltaNet + Gated Attention, Mixture-of-Experts (MoE)</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Total Parameters</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">35B</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Activated Parameters</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">~3B</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Number of Layers</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">40</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Layer Layout</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">10 × (3 × (Gated DeltaNet → MoE) → 1 × (Gated Attention → MoE))</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Hidden Dimension</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">2048</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Gated DeltaNet</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">32 value heads, 16 QK heads, head dimension 128</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Gated Attention</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">16 Q heads, 2 KV heads, head dimension 256, RoPE dim 64</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Number of Experts</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">256</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Selected Experts per Token</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">8 routed + 1 shared</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Expert Intermediate Dimension</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">512</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Vocabulary Size</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">248,320</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Context Length</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">262,144 (extensible to 1,010,000)</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Multi-Token Prediction</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">1 MTP layer (up to 50% faster generation)</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Vision Encoder</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">27-layer ViT, hidden 1152, patch 16, temporal patch 2</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Modality</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">Text, Image, Video</td>

</tr>

<tr>

<td align="center" style="vertical-align: middle; text-align: center"><strong>Precision</strong></td>

<td align="center" style="vertical-align: middle; text-align: center">GGUF Q4_K_M (4.89 BPW)</td>

</tr>

</tbody>

</table>

</div>

3. Datasets Used to Train

Fabric 1.6 was developed from the Qwen3.5-35B-A3B-Base foundation through continuous pre-training followed by post-training (supervised fine-tuning and reinforcement-learning-based alignment).

Pre-training was performed primarily on a large, in-house proprietary synthetic dataset spanning code, mathematics and reasoning, complemented by open reasoning corpora:

• OpenThoughts3-1.2M — 1.2M high-quality reasoning traces across mathematics, science, coding and general problem solving.
• OpenR1-Math-220k — 225k mathematical problems with think-style solutions.

Post-training instruction data combines permissively licensed open corpora with proprietary data:

• smoltalk2 — an Apache-2.0 SFT subset (~340k examples) covering multilingual instruction following, multi-turn reasoning, tool-calling traces, system chats and table understanding.
• hermes-function-calling-v1 — structured tool-calling traces.
• oasst2 — curated, reviewed conversational chains.
• maple — a proprietary instruction and reasoning corpus developed in-house by Fabric AI (CC-BY-4.0).

In total, approximately 12 billion tokens were processed across the pre-training and post-training stages. Knowledge cutoff: July 2026.

4. Evaluation Results

Fabric 1.6 was evaluated on 22 benchmarks with greedy decoding (temperature 0).

| Category | Benchmark | Score |

|---|---|---|

| Math & Reasoning | AIME25 | 92.8 |

| | AIME26 | 93.1 |

| | HMMT26 | 83.2 |

| | IMOAB | 79.2 |

| | M500 | 84.8 |

| Science & Knowledge | GPQA | 86.7 |

| | GPQA-D | 84.9 |

| | HLE | 21.4 |

| | MMLU-P | 85.6 |

| | MMLU-R | 93.5 |

| | C-Eval | 92.3 |

| Coding | LCB6 | 80.2 |

| | SWEB-V | 72.9 |

| | SWEB-P | 50.1 |

| | IFEval | 93.09 |

| General Reasoning | GSM8K-Pt | 95.73 |

| Agentic Tools | TAU3 | 67.2 |

| | MMMU-P | 74.10 |

| | RWQA | 85.4 |

| | MCP-A | 62.8 |

| | WS | 60.3 |

| | MV-mini | 86.6 |

5. Deployment

Install llama.cpp

apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev -y
git clone https://github.com/ggml-org/llama.cpp
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first --target llama-cli llama-mtmd-cli llama-server llama-gguf-split
cp llama.cpp/build/bin/llama-* llama.cpp

Serve with llama.cpp

./llama.cpp/llama-server \
    -hf FabricAI/Fabric1.6-GGUF:Q4_K_M \
    -ngl 99 -c 8192 -fa on -np 1 \
    --spec-type draft-mtp --spec-draft-n-max 2

6. Model Usage

Fabric 1.6 always has thinking enabled and returns reasoning_content alongside the answer. The model was trained in preserved thinking history mode: for multi-turn conversations and tool calls, pass the complete assistant message returned by the API back to messages as-is — including reasoning_content and tool_calls, not just content — so that reasoning from earlier turns remains available to later ones.

7. License

The model weights are released under the Apache License 2.0.

8. Citation

If you use Fabric 1.6 in your work, please cite it as:

@misc{fabric1.6,
    title = {{Fabric1.6}: Agentic Open Model for Enterprises},
    url = {https://huggingface.co/FabricAI/Fabric1.6},
    author = {{Fabric AI}},
    month = {August},
    year = {2026}
}

9. Contact

For questions, collaborations or access requests, contact the Fabric AI research team at research@fabricai.co.uk.