---

language:

• en

license: apache-2.0

library_name: llama.cpp

pipeline_tag: text-generation

tags:

• qwen
• qwen3
• moe
• gguf
• q6_k
• quantized
• reasoning
• local-ai
• llama.cpp

---

Qwen3 Next 35B - Q6_K GGUF

A Q6_K quantized GGUF release of a 35-billion-parameter

Qwen-family language model, prepared for local AI inference.

Published by oscarwest1.

Model Details

| Property | Value |

|---|---|

| Model | Qwen3 Next 35B |

| Parameters | 35 Billion class |

| Quantization | Q6_K |

| Format | GGUF |

| Runtime | llama.cpp |

| Publisher | oscarwest1 |

| License | Apache 2.0 (subject to upstream verification) |

About This Release

This repository provides a Q6_K quantized GGUF model

for local inference with compatible GGUF runtimes.

Q6_K is designed to reduce memory requirements while

preserving relatively high numerical precision.

Quantization is lossy, and output quality may differ

from the original full-precision model.

No additional fine-tuning is claimed for this release.

Intended Uses

• Advanced reasoning and problem solving
• Programming and code generation
• IT infrastructure troubleshooting
• Technical analysis
• Research and experimentation
• Private AI assistants
• Self-hosted AI services

Running with llama.cpp

Interactive Chat

llama-cli -m qwen3-next-35b-Q6_K.gguf -c 32768 -cnv

Local API Server

llama-server -m qwen3-next-35b-Q6_K.gguf --host 127.0.0.1 --port 8080 -c 32768

Quantization Information

• Quantization: Q6_K
• File format: GGUF
• Inference runtime: llama.cpp
• Independent benchmarks: Not provided

The exact upstream checkpoint and quantization

provenance should be verified before making

comparisons with official releases.

Attribution

Qwen models are developed by the Qwen Team.

Official Qwen organization:

https://huggingface.co/Qwen

GGUF tooling:

https://github.com/ggml-org/llama.cpp

This is an independently published quantized release

and is not an official Qwen distribution.

License

The applicable license and redistribution requirements

must be confirmed against the exact upstream model.

---

I miss you NYZ