---

license: apache-2.0

base_model: Qwen/Qwen3-ASR-0.6B

tags: [automatic-speech-recognition, gguf, rapidspeech, qwen3-asr, jetson]

language: [zh, en]

library_name: gguf

---

Qwen3-ASR-0.6B — GGUF (Jetson Nano gen1 optimized)

Single-file GGUF builds of Qwen/Qwen3-ASR-0.6B (audio encoder + Qwen3-0.6B LLM

decoder) that run with CUDA on the original Jetson Nano gen1 (Tegra X1, sm_53,

CUDA 10.2). Native Traditional Chinese + punctuation, zh/en code-switch.

Variants

| file | size | accuracy | recommended for |

|---|---|---|---|

| qwen3-asr-0.6b-f16.gguf | 1.88 GB | full (native Traditional) | reference / max accuracy |

| qwen3-asr-0.6b-q8_0.gguf | 0.96 GB | full (native Traditional) | Jetson Nano gen1 (recommended) |

| qwen3-asr-0.6b-q4_0.gguf | 0.52 GB | warning: collapses to Simplified | Ampere+ GPUs only, Simplified-OK |

Why q8_0, not q4_0, on the Jetson Nano (Maxwell sm_53)

Maxwell sm_53 has no dp4a (int8 dot-product), so ggml dequantizes low-bit

weights and the extra ALU work makes *q4_0 slower than q8_0/f16 on this GPU*

(measured ~1.7x slower). q4_0 also loses Traditional-character precision on

this 0.6B model (output collapses to Simplified). So q8_0 is the sweet spot:

half the size of f16, same speed and accuracy on Maxwell. q4_0 is only worth it on

Ampere+ (fast low-bit GEMM) where Simplified output is acceptable.

Engines (both Jetson-Nano-gen1 optimized)

Two open engines run this GGUF on the Nano gen1; both share the cuFFT batched mel

and the CUDA-10.2/sm_53 build fixes:

| engine | RTF (13.5s clip, q8_0) | notes |

|---|---|---|

| vieenrose/qwen3-asr.cpp (standalone) | ~1.1 | dedicated 2-engine design: encoder on GPU + autoregressive decode on CPU (~195 ms/tok); fastest on the Nano |

| vieenrose/RapidSpeech.cpp (integrated) | ~1.3 | Qwen3-ASR inside the multi-model RapidSpeech framework (all weights on GPU); ships the converter scripts/convert_qwen3_asr_to_gguf.py |

The standalone reaches lower RTF because it keeps encoder weights on the GPU and

LLM weights on the CPU simultaneously (batch-1 decode is faster on the A57 than on

Maxwell); RapidSpeech's unified single-weight-buffer keeps everything on GPU.

Usage

Standalone:

qwen3-asr-cli -m qwen3-asr-0.6b-q8_0.gguf -f clip_16k.wav -t 4

RapidSpeech:

rs-asr-offline -m qwen3-asr-0.6b-q8_0.gguf -w clip_16k.wav

Convert your own

From a local Qwen/Qwen3-ASR-0.6B checkpoint, using the RapidSpeech fork:

python scripts/convert_qwen3_asr_to_gguf.py --hf-dir /path/to/Qwen3-ASR-0.6B \
    --output qwen3-asr-0.6b-q8_0.gguf --quant q8_0