---

license: apache-2.0

library_name: gguf

base_model: sandeshrajx/Qwen3.5-24B-A3B-REAP-0.32

tags:

• qwen3.5
• moe
• gguf
• iq4_nl
• reap
• pruned
• quantzhai
• quantized

---

Qwen3.5-24B-A3B-REAP-0.32 IQ4_NL GGUF

Quanter's Note: This model is the parent of every solid performing reasoning distill I've benched out of 40 or so models in the last month, this thing has some solid potential for a model that has been given a partial lobotomy, awfully impressive.

Perplexity test I devised is Extremely Tough on models, high convergence rate on alien/unusual codebases, plenty of potential for finetunes.

I felt the base model deserved some attention alongside its author and I'm glad to see people downloading it.

Every finetune of this model I benched, IMPROVED over this base model. Very impressive.

GGUF quantization of sandeshrajx/Qwen3.5-24B-A3B-REAP-0.32 — a REAP-pruned 24B total, ~3B active MoE model.

REAP (Razor Edge And Pruning, arxiv:2510.13999) is a structured pruning technique that reduces the base Qwen3.5 model while preserving capability.

Source: sandeshrajx/Qwen3.5-24B-A3B-REAP-0.32

Converted by: QuantZhai benchmark pipeline

Quantization: IQ4_NL with importance matrix (imatrix from sandeshrajx)

Model Details

| Property | Value |

|---|---|

| Architecture | Qwen3.5 MoE, REAP-pruned |

| Parameters | 24B total, ~3B active |

| Blocks | 40 |

| Experts | 175 (REAP-split), 8 active per token |

| Context length | 262144 (256K) |

| Hidden size | 3072 |

| Attention heads | 32, KV heads = 2 |

| Quantization | IQ4_NL (4.58 bpw) with imatrix |

| File size | 14.0 GB |

Benchmarks

Hardware: dual-GPU (RTX 3080 10GB + V100-SXM2 32GB)

Engine: llama.cpp with TurboQuant KV (q8_0 K / turbo3 V)

Perplexity: macvox68 code corpus, ctx=4096, stride=512

| Metric | Cold | Warm |

|---|---|---|

| PPL | 3.0205 | 1.8298 |

| TPS | 33.6 tok/s | 45.9 tok/s |

| TTFT | 1486 ms | 1090 ms |

QuantZhai Ranking

Rank #15 of 47 — combined score 63.6 (equal-weight: TPS, PPL, convergence).

Higher than many larger dense models — REAP pruning + IQ4_NL quantization is an efficient combination.

Usage

llama-cli -m Qwen3.5-24B-A3B-REAP-0.32-IQ4_NL.gguf \
  -p "Write a mergesort in Python" \
  -n 1024 -t 12 --temp 0.6 --top-p 0.95

llama-server -m Qwen3.5-24B-A3B-REAP-0.32-IQ4_NL.gguf \
  --host 0.0.0.0 --port 8080 -ngl 99 -t 12 \
  --cache-type-k q8_0 --cache-type-v turbo3

License

Apache 2.0 (this quantization).

Source model by sandeshrajx under Apache 2.0 — see arxiv:2510.13999.