---

language:

• en

license: apache-2.0

base_model: Qwen/Qwen3.5-122B-A10B

library_name: llama.cpp

pipeline_tag: image-text-to-text

tags:

• qwen
• qwen3.5
• 122b
• a10b
• moe
• gguf
• q6_k
• quantized
• reasoning
• local-ai
• llama.cpp
• multimodal

---

Qwen3.5-122B-A10B - Q6_K GGUF

A Q6_K quantized GGUF release of Qwen3.5-122B-A10B,

prepared for local inference and self-hosted AI applications.

Published by oscarwest1.

Model Details

| Property | Value |

|---|---|

| Developer | Qwen Team |

| Original Model | Qwen/Qwen3.5-122B-A10B |

| Total Parameters | Approximately 122 Billion |

| Active Parameters | Approximately 10 Billion |

| Architecture | Mixture of Experts (MoE) |

| Quantization | Q6_K |

| Format | GGUF |

| Runtime | llama.cpp |

| Publisher | oscarwest1 |

| License | Apache 2.0 |

About This Release

This repository provides a Q6_K GGUF conversion of

Qwen3.5-122B-A10B for local AI inference.

Q6_K is a 6-bit-class quantization format designed

to reduce memory requirements compared with

full-precision model weights.

The goal is to retain relatively high numerical

precision while reducing model storage requirements.

Quantization is lossy and may introduce differences

compared with the original full-precision model.

No additional fine-tuning is claimed for this release.

Model Architecture

Qwen3.5-122B-A10B uses a Mixture-of-Experts

architecture with approximately 122 billion

total parameters and 10 billion active parameters.

Only a subset of the model's parameters is activated

during token processing, reducing computation

relative to an equivalently sized dense model.

The original Qwen3.5 model family supports

multimodal processing, subject to runtime support

and the availability of required components.

Intended Applications

• Advanced reasoning and logical analysis
• Programming and software development
• IT infrastructure troubleshooting
• Technical research
• Mathematical problem solving
• Long-context document analysis
• Local AI assistants
• Self-hosted AI inference

Running with llama.cpp

Interactive Chat

llama-cli -m Qwen3.5-122B-A10B-Q6_K.gguf -c 32768 -cnv

Local API Server

llama-server -m Qwen3.5-122B-A10B-Q6_K.gguf --host 127.0.0.1 --port 8080 -c 32768

A recent llama.cpp version with support for

Qwen3.5 MoE architecture is recommended.

Quantization Information

• Quantization: Q6_K
• File format: GGUF
• Runtime: llama.cpp
• Fine-tuning: None claimed
• Independent benchmarks: Not provided

Actual model quality and inference performance

depend on the hardware, runtime, context length,

and generation settings.

Hardware Considerations

This is a large Mixture-of-Experts model.

Substantial RAM or VRAM is required to load

the quantized model weights.

Additional memory is required for context

processing, KV cache and runtime overhead.

CPU inference performance depends heavily

on memory bandwidth and processor architecture.

Original Model and Credits

Original model developed by the Qwen Team:

https://huggingface.co/Qwen/Qwen3.5-122B-A10B

GGUF conversion and quantization tooling:

https://github.com/ggml-org/llama.cpp

This is an independently published quantized

release and is not an official Qwen distribution.

License

Apache License 2.0.

Please refer to the original model repository

for applicable license and attribution requirements.

---

I miss you NYZ