---

base_model:

• Qwen/Qwen-AgentWorld-35B-A3B

tags:

• Qwen
• agent
• world
• 35b
• a3b
• moe

---

Qwen-AgentWorld-35B-A3B GGUF

<!-- polished-overview:start -->

Overview

Qwen-AgentWorld-35B-A3B-GGUF is a GGUF release for llama.cpp-compatible runtimes and local inference, published by groxaxo.

It is intended for open-source evaluation, reproducible experimentation, and compatible local or

hosted inference workflows. The wording below is deliberately limited to what can be verified

from this repository's metadata and artifacts.

At a glance

| Field | Details |

|---|---|

| Format | GGUF |

| Source / base | Qwen/Qwen-AgentWorld-35B-A3B |

| Intended task | the task described by the included configuration and documentation |

| License | the license declared in the repository files |

What is included

• *.gguf (6 files)
• Additional configuration, tokenizer, processor, or shard files (6 visible artifacts total)

Quick start

llama.cpp

Download a .gguf file that fits your available memory, then run it with a current llama.cpp

build:

llama-cli \
  -m /path/to/model.gguf \
  -p "Write a concise technical summary."

For vision or any-to-any models, download the matching multimodal projection file when one is

provided and follow the source model's modality-specific instructions.

Compatibility and responsible use

• Use a runtime that explicitly supports this format, architecture, and modality.
• Keep configuration, tokenizer, processor, projection, and weight files from the same revision together.
• Review the source model card and license before redistribution or deployment.
• Hardware needs depend on parameter count, context length, cache precision, quantization, and concurrency.
• Report reproducible issues with the runtime version, hardware, launch command, and a minimal example.

Quantization or conversion changes numerical behavior, memory use, and throughput relative to the source checkpoint; validate quality on your own workload.

Generated outputs may be inaccurate or unsuitable for a given use case. Users are responsible for

testing behavior, applying appropriate safeguards, and complying with applicable licenses and laws.

<!-- polished-overview:end -->

GGUF quantizations of Qwen/Qwen-AgentWorld-35B-A3B generated with llama.cpp.

• Architecture: qwen35moe (35B params, 256 experts / 8 active, A3B)
• Context length: 262 144
• Source: BF16 GGUF converted via convert_hf_to_gguf.py
• Importance matrix (coding.imatrix.gguf) generated from a curated coding calibration text (1 840 samples), 128 chunks, ctx 4 096.

Files

| File | Size (GiB) | BPW | Use |

| --- | --- | --- | --- |

| Qwen-AgentWorld-35B-A3B-BF16.gguf | 64.61 | 16.01 | reference / highest fidelity |

| Qwen-AgentWorld-35B-A3B-Q8_0.gguf | 34.37 | 8.52 | near-lossless, needs >24 GB VRAM with all layers offloaded |

| Qwen-AgentWorld-35B-A3B-Q6_K.gguf | 26.56 | 6.58 | very high quality, ~20 GB VRAM |

| Qwen-AgentWorld-35B-A3B-Q4_K_M.gguf | 19.71 | 4.88 | balanced quality / size, ~14 GB VRAM |

| Qwen-AgentWorld-35B-A3B-Q2_K.gguf | 12.05 | 2.99 | smallest, ~9 GB VRAM, quality trade-off |

| Qwen-AgentWorld-35B-A3B-coding.imatrix.gguf | 0.18 | — | importance matrix for finer quant recipes (Tensor-type overrides). |

Plain quantization passes (no recipe overrides) only — these avoid the

std::bad_alloc triggered by --tensor-type-file + imatrix on this

specific MoE architecture in the current llama.cpp build (8194 /

1179bfc82). The imatrix file is still provided for users who want to

mix types via --tensor-type-file.

Recommended inference

llama.cpp/build/bin/llama-server \
    -m Qwen-AgentWorld-35B-A3B-Q4_K_M.gguf \
    -ngl 999 --ctx-size 8192 -b 2048 -ub 512 -np 1 \
    --temp 0.6 --top-p 0.95 --top-k 20 \
    -fa --host 0.0.0.0 --port 8080