---

license: apache-2.0

language:

• zh
• en

library_name: rapidspeech

tags:

• automatic-speech-recognition
• streaming
• zipformer2
• transducer
• ggml
• gguf
• code-switching

base_model:

• GilgameshWind/X-ASR-zh-en

pipeline_tag: automatic-speech-recognition

---

X-ASR zh-en streaming zipformer2 transducer — GGUF

GGUF conversions of the X-ASR zh-en

streaming zipformer2 transducer (k2-fsa / sherpa-onnx export), for use with

RapidSpeech.cpp (ggml backend,

CPU + CUDA). Mandarin–English code-switching ASR with punctuation.

Converted with tools/convert_xasr_to_gguf.py. The encoder/decoder/joiner are

fused into a single GGUF per chunk variant; streaming uses per-layer recurrent

caches mirroring the ONNX state contract.

Variants

All four chunk variants share the same architecture (6 stacks / 19 layers,

dims 192·256·512·768·512·256, vocab 5000); they differ only in the streaming

chunk size (latency vs. accuracy trade-off).

| Folder | Chunk shift | Encoder T | Latency | f16 | Q4_K |

|-----------|-------------|-----------|---------|--------|--------|

| 160ms/ | 16 frames | 29 | lowest | ~292 MB| ~85 MB |

| 480ms/ | 48 frames | 61 | low | ~292 MB| ~85 MB |

| 960ms/ | 96 frames | 109 | medium | ~292 MB| ~85 MB |

| 1920ms/ | 192 frames | 205 | highest accuracy | ~292 MB| ~86 MB |

Each folder contains -f16.gguf, -q8_0.gguf, -q4_k.gguf, -q3_k.gguf

(imatrix-calibrated), the imatrix-*.dat calibration file, and tokens.txt.

The 960 ms folder additionally ships *-iq4_xs.gguf (a lossless 4-bit IQ

build — the 960 ms variant was used for the full quant sweep below).

Convolution kernels are always kept at f16 (quantizing them hurts accuracy).

Which weight format to use

Per-weight accuracy, measured on the 960 ms variant. Accuracy is the

token edit-distance vs the f16 reference on a zh-en code-switch clip

(0 = token-exact). Sizes are the actual GGUF bytes.

| Format | Size | Edit-dist | Published | Notes |

|--------|-----:|:---------:|:---------:|-------|

| f16 | 307 MB | 0 (ref) | ✅ | reference |

| q8_0 | 165 MB | 0 — lossless | ✅ | best quality; ~1.2× faster than f16 on CPU |

| iq4_xs | 87 MB | 0 — lossless | ✅ | lossless 4-bit IQ (960 ms only) |

| q4_k | 90 MB | 3 | ✅ | near-lossless (minor casing: Monday→monday) |

| q3_k (imatrix) | 72 MB | 0 — lossless | ✅ | smallest lossless build |

The q3_k.gguf files here are imatrix-calibrated (activation-aware, AWQ):

an importance matrix collected over calibration audio protects the most important

weight channels, recovering the accuracy 3-bit quantization normally loses

(without it, q3_k scores edit-dist 4 — Monday→MD). Generate your own with

xasr-dev-test imatrix + rs-quantize --imatrix.

**Recommendation: q8_0 for lossless quality, or q3_k (imatrix) for the

smallest lossless footprint (72 MB).** Quantizing the matmul weights also

speeds up ggml CPU inference (less memory traffic + tuned vec-dot kernels).

Sub-3-bit was evaluated but is not published

A full sweep below 3-bit was run on the 960 ms variant and **deliberately

excluded** — none are useful:

| Format | Size | Edit-dist | Why excluded |

|--------|-----:|:---------:|--------------|

| q2_k (imatrix) | 59 MB | 3 | degraded — below the 3-bit floor |

| iq2_s | 58 MB | 3–4 | degraded |

| iq3_s | 86 MB | 4 | dominated (bigger than q3_k-im and worse) |

| iq2_xxs (imatrix) | 53 MB | 6 | degraded |

| iq1_s, iq2_xxs (no imatrix) | 90 MB | 3 | fake — fell back to q4_k size, not real low-bit |

| iq1_m | 45 MB | 25–58 | broken — garbage output |

3-bit + imatrix is the accuracy floor. Below it, accuracy degrades (edit-dist

3–6) and 1-bit collapses entirely.

Parity

RapidSpeech.cpp (CPU, f16) is **token-exact with sherpa-onnx (onnxruntime CPU,

fp32)** on the reference audio for all four variants. Q4_K matches to within

occasional capitalization.

Example (10 s zh-en code-switching clip):

> 昨天是 Monday，today is 礼拜二，the day after tomorrow 是星期三

Benchmark (streaming, steady-state ms/chunk, warm-up excluded)

Measured on an NVIDIA GB10 host (the original Jetson Nano gen1 target was

unavailable). RapidSpeech CUDA uses the FP32 non-tensor path (emulating the

Nano's tensor-core-less sm_53). Numbers are relative, not Nano wall-clock.

| Variant | sherpa-onnx CPU | RapidSpeech CPU | RapidSpeech CUDA |

|---------|----------------:|----------------:|-----------------:|

| 160 ms | 16.0 | 27.7 | 26.9 |

| 480 ms | 23.1 | 48.0 | 38.4 |

| 960 ms | 31.1 | 83.8 | 53.3 |

| 1920 ms | 40.9 | 169.7 | 83.0 |

All configurations run faster than real time. CUDA's speedup over CPU grows with

chunk size (1.0× → 2.0×) as larger GEMMs amortize per-chunk kernel-launch cost.

Usage

# RapidSpeech.cpp WebSocket streaming server
rs-xasr-ws-server -m 960ms/x-asr-zh-en-960ms-f16.gguf --port 6006

See RapidSpeech.cpp for build

instructions (incl. the CUDA-10.2 / sm_53 Jetson Nano path).

License

Apache-2.0, following the upstream X-ASR model.