cstr/canary-1b-v2-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "cc-by-4.0",
    "language": [
      "bg",
      "cs",
      "da",
      "de",
      "el",
      "en",
      "es",
      "et",
      "fi",
      "fr",
      "hr",
      "hu",
      "it",
      "lt",
      "lv",
      "mt",
      "nl",
      "pl",
      "pt",
      "ro",
      "ru",
      "sk",
      "sl",
      "sv",
      "uk"
    ],
    "pipeline_tag": "automatic-speech-recognition",
    "tags": [
      "audio",
      "speech-recognition",
      "transcription",
      "translation",
      "speech-translation",
      "ggml",
      "gguf",
      "canary",
      "fastconformer",
      "multilingual"
    ],
    "library_name": "ggml",
    "base_model": "nvidia/canary-1b-v2",
    "frontmatter": {
      "license": "cc-by-4.0",
      "language": [
        "bg",
        "cs",
        "da",
        "de",
        "el",
        "en",
        "es",
        "et",
        "fi",
        "fr",
        "hr",
        "hu",
        "it",
        "lt",
        "lv",
        "mt",
        "nl",
        "pl",
        "pt",
        "ro",
        "ru",
        "sk",
        "sl",
        "sv",
        "uk"
      ],
      "pipeline_tag": "automatic-speech-recognition",
      "tags": [
        "audio",
        "speech-recognition",
        "transcription",
        "translation",
        "speech-translation",
        "ggml",
        "gguf",
        "canary",
        "fastconformer",
        "multilingual"
      ],
      "library_name": "ggml",
      "base_model": "nvidia/canary-1b-v2"
    },
    "hero_image_url": "",
    "summary": "GGUF / ggml conversions of nvidia/canary-1b-v2 for use with the canary-main CLI from **CrispStrobe/CrispASR@parakeet**. Canary 1B v2 is NVIDIA's 978 M-parameter multilingual ASR + speech translation model: This is the encoder–decoder companion to **cstr/parakeet-tdt-0.6b-v3-GGUF**, which is the same FastConformer encoder family but with a TDT decoder for ASR-only. Both share the runtime and were ported in the same fork.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: cc-by-4.0\nlanguage:\n- bg\n- cs\n- da\n- de\n- el\n- en\n- es\n- et\n- fi\n- fr\n- hr\n- hu\n- it\n- lt\n- lv\n- mt\n- nl\n- pl\n- pt\n- ro\n- ru\n- sk\n- sl\n- sv\n- uk\npipeline_tag: automatic-speech-recognition\ntags:\n- audio\n- speech-recognition\n- transcription\n- translation\n- speech-translation\n- ggml\n- gguf\n- canary\n- fastconformer\n- multilingual\nlibrary_name: ggml\nbase_model: nvidia/canary-1b-v2\n---\n\n# Canary 1B v2 — GGUF (ggml-quantised)\n\nGGUF / ggml conversions of [`nvidia/canary-1b-v2`](https://huggingface.co/nvidia/canary-1b-v2) for use with the `canary-main` CLI from **[CrispStrobe/CrispASR@parakeet](https://github.com/CrispStrobe/CrispASR/tree/parakeet)**.\n\nCanary 1B v2 is NVIDIA's 978 M-parameter multilingual ASR + speech translation model:\n\n- **25 European languages** with **explicit `source_lang` / `target_lang` task tokens** (no auto-detect ambiguity)\n- **Speech translation** in both directions: X→English (24 languages) and English→X (24 languages)\n- **7.15% avg WER** on the HuggingFace Open ASR Leaderboard (English) — competitive with Whisper-large-v3 at 1/1.6× the size\n- **CC-BY-4.0** licence\n\nThis is the encoder–decoder companion to **[`cstr/parakeet-tdt-0.6b-v3-GGUF`](https://huggingface.co/cstr/parakeet-tdt-0.6b-v3-GGUF)**, which is the same FastConformer encoder family but with a TDT decoder for ASR-only. Both share the runtime and were ported in the same fork.\n\n## Files\n\n| File | Size | Notes |\n| --- | ---: | --- |\n| `canary-1b-v2.gguf`        | 1.97 GB | F16, full precision |\n| `canary-1b-v2-q8_0.gguf`   | 1.1 GB  | Q8_0, near-lossless |\n| `canary-1b-v2-q5_0.gguf`   | 777 MB  | Q5_0 |\n| `canary-1b-v2-q4_k.gguf`   | 673 MB  | **Q4_K — recommended default** |\n\n## Quick Start\n\n```bash\n# 1. Build the runtime\ngit clone -b parakeet https://github.com/CrispStrobe/CrispASR\ncd CrispASR\ncmake -B build -DCMAKE_BUILD_TYPE=Release\ncmake --build build -j$(nproc) --target canary-main\n\n# 2. Download a quantisation\nhuggingface-cli download cstr/canary-1b-v2-GGUF \\\n    canary-1b-v2-q4_k.gguf --local-dir .\n\n# 3. ASR (English → English)\n./build/bin/canary-main \\\n    -m canary-1b-v2-q4_k.gguf \\\n    -f your-audio.wav \\\n    -sl en -tl en -t 8\n\n# 4. ASR (German → German)\n./build/bin/canary-main -m canary-1b-v2-q4_k.gguf \\\n    -f german_audio.wav -sl de -tl de\n\n# 5. Speech translation (German → English)\n./build/bin/canary-main -m canary-1b-v2-q4_k.gguf \\\n    -f german_audio.wav -sl de -tl en\n```\n\n## Verified end-to-end output\n\n**English ASR (`samples/jfk.wav`, 11 s):**\n> And so, my fellow Americans, ask not what your country can do for you, ask what you can do for your country.\n\n**German ASR (Wikimedia Commons `Amardeo_Sarma_voice_-_de.ogg`, 91 s):**\n> Ich heiße Amadeus Scharma. Ich bin 1955 in Kassel in Deutschland geboren, weitgehend in Indien aufgewachsen. Ich hatte meine Ausbildung in Ingenieurwissenschaften in Neu-Delhi und dann später auch in Darmstadt, an der Technischen Universität von Darmstadt. ...\n\n**Speech translation DE → EN (same clip):**\n> My name is Amadeo Sharma. I was born in Kassel in Germany in 1955, and I grew up largely in India. I had my education in engineering in New Delhi and then later also in Darmstadt, at the Technical University of Darmstadt. ...\n\n## Why explicit language tokens?\n\nAuto-detect language ID can misfire on accented or noisy speech. We tested parakeet (which has no `-l` flag and relies on auto-detect) on the same German clips and it picked **Russian** for Angela Merkel and **code-switched into English** on Sarma's recording. Canary's `-sl LANG` removes that whole class of failures by telling the decoder explicitly what language to expect — see [`test_german.md`](https://github.com/CrispStrobe/CrispASR/blob/parakeet/test_german.md) in the runtime repo.\n\n## Supported languages\n\n`bg cs da de el en es et fi fr hr hu it lt lv mt nl pl pt ro ru sk sl sv uk` (25 European languages).\n\nFor each pair (`source_lang`, `target_lang`):\n- `sl == tl` → ASR\n- `sl != tl` → speech translation\n\nTranslation supports any pair from 24 non-English languages → English, and English → any of 24 non-English languages.\n\n## Architecture\n\n| Component | Details |\n| --- | --- |\n| Encoder       | 32-layer FastConformer, d=1024, 8 heads, head_dim=128, FFN=4096, conv kernel=9, biases on every linear/conv |\n| Subsampling   | Conv2d dw_striding stack, 8× temporal (100 → 12.5 fps) |\n| Decoder       | 8-layer pre-LN Transformer (self-attn + cross-attn + FFN), d=1024, 8 heads, head_dim=128, FFN=4096, max_ctx=1024 |\n| Embedding     | Token (16384 × 1024) + learned positional (1024 × 1024) + LN |\n| Output head   | Linear (1024 → 16384) |\n| Vocab         | 16384 SentencePiece (NeMo CanaryBPETokenizer) |\n| Audio         | 16 kHz mono, 128 mel bins, n_fft=512, hop=160, win=400 |\n| Parameters    | ~978 M (encoder 811M + decoder 152M + head 17M) |\n\nThe mel filterbank and Hann window are baked into the GGUF (`preprocessor.fb` and `preprocessor.window`), so no recomputation at runtime. BatchNorm in the convolution module is folded into the depthwise conv weights at load time. Cross-attention K/V is pre-computed once per audio slice from the encoder output and then reused across decoder steps.\n\n## How this was made\n\n1. **Inspect** the `.nemo` tarball: 1510 tensors total — encoder (1294), `transf_decoder` (214), `log_softmax` head (2), preprocessor (2). Skipped the auxiliary `timestamps_asr_model_weights.ckpt` which is the separate Parakeet CTC model used by NeMo Forced Aligner for segment-level timestamps.\n2. **Convert** with [`models/convert-canary-to-gguf.py`](https://github.com/CrispStrobe/CrispASR/blob/parakeet/models/convert-canary-to-gguf.py): remap NeMo state-dict keys (`transf_decoder._embedding.token_embedding` → `decoder.embed`, `first_sub_layer.query_net` → `sa_q`, etc.) and write 1478 tensors as F16 (matmul) + F32 (norms / biases / mel filterbank). 1.97 GB GGUF.\n3. **C++ runtime** in [`src/canary.{h,cpp}`](https://github.com/CrispStrobe/CrispASR/blob/parakeet/src/canary.cpp): mmap the GGUF, fold BN into the depthwise conv at load time, build the encoder graph (32-layer FastConformer with biases), build the decoder graph per step (with self-attention KV cache + pre-computed cross-K/V), greedy decode with task-token prompt, detokenise via SentencePiece.\n4. **Quantise** with [`cohere-quantize`](https://github.com/CrispStrobe/CrispASR/blob/parakeet/examples/cohere-main/cohere-quantize.cpp): same llama.cpp-style quantiser used for the cohere and parakeet GGUFs in this fork.\n\n## Comparison with Parakeet TDT 0.6B v3\n\n| | parakeet-tdt-0.6b-v3 | **canary-1b-v2 (this repo)** |\n| --- | --- | --- |\n| Architecture | FastConformer + TDT (transducer) | FastConformer + Transformer (encoder–decoder) |\n| Parameters | 600M | 978M |\n| Languages | 25 (auto-detect) | 25 (**explicit `-sl` / `-tl`**) |\n| Speech translation | ❌ | ✅ X→En and En→X |\n| Word timestamps | ✅ from TDT duration head | ✗ (segment-level via aux CTC) |\n| Q4_K size | 467 MB | ~600 MB |\n| Open ASR WER (avg) | 6.34% | 7.15% |\n| Use case | fastest multilingual ASR with word stamps | best multilingual ASR + translation, language is known |\n\n## Attribution\n\n- **Original model:** [`nvidia/canary-1b-v2`](https://huggingface.co/nvidia/canary-1b-v2) (CC-BY-4.0). NVIDIA NeMo team. See the [Canary-1B-v2 & Parakeet-TDT-0.6B-v3 technical report](https://arxiv.org/abs/2509.14128).\n- **GGUF conversion + ggml runtime:** [`CrispStrobe/CrispASR@parakeet`](https://github.com/CrispStrobe/CrispASR/tree/parakeet). The decoder structure was cross-checked against NeMo's `transformer_decoders.py` and `transformer_modules.py` source.\n- **Encoder graph patterns:** shared between cohere/parakeet/canary in the same fork, originally adapted for the [CrispASR ggml branch](https://github.com/CrispStrobe/CrispASR/tree/ggml).\n\n## Related\n\n- C++ runtime: **[CrispStrobe/CrispASR@parakeet](https://github.com/CrispStrobe/CrispASR/tree/parakeet)**\n- Sister model (ASR-only, smaller, with word timestamps): [`cstr/parakeet-tdt-0.6b-v3-GGUF`](https://huggingface.co/cstr/parakeet-tdt-0.6b-v3-GGUF)\n- Sister runtime (Cohere Transcribe, lowest English WER): [`cstr/cohere-transcribe-03-2026-GGUF`](https://huggingface.co/cstr/cohere-transcribe-03-2026-GGUF)\n- ONNX INT4 (Cohere): [`cstr/cohere-transcribe-onnx-int4`](https://huggingface.co/cstr/cohere-transcribe-onnx-int4)\n- ONNX INT8 (Cohere): [`cstr/cohere-transcribe-onnx-int8`](https://huggingface.co/cstr/cohere-transcribe-onnx-int8)\n\n## License\n\nCC-BY-4.0, inherited from the base model. Use of these GGUF files must comply with the CC-BY-4.0 license including attribution.\n",
    "related_quantizations": []
  },
  "tags": [
    "ggml",
    "gguf",
    "audio",
    "speech-recognition",
    "transcription",
    "translation",
    "speech-translation",
    "canary",
    "fastconformer",
    "multilingual",
    "automatic-speech-recognition",
    "bg",
    "cs",
    "da",
    "de",
    "el",
    "en",
    "es",
    "et",
    "fi",
    "fr",
    "hr",
    "hu",
    "it",
    "lt",
    "lv",
    "mt",
    "nl",
    "pl",
    "pt",
    "ro",
    "ru",
    "sk",
    "sl",
    "sv",
    "uk",
    "arxiv:2509.14128",
    "base_model:nvidia/canary-1b-v2",
    "base_model:quantized:nvidia/canary-1b-v2",
    "license:cc-by-4.0",
    "region:us"
  ],
  "likes": 0,
  "downloads": 369,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-08T06:01:35.000Z",
  "created_at": "2026-04-07T22:20:53.000Z",
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "ggml"
}

{
  "_id": "69d58345de8b204b18786668",
  "id": "cstr/canary-1b-v2-GGUF",
  "modelId": "cstr/canary-1b-v2-GGUF",
  "sha": "b3715a517928f8f68833142c90fc5810ad583210",
  "createdAt": "2026-04-07T22:20:53.000Z",
  "lastModified": "2026-04-08T06:01:35.000Z",
  "author": "cstr",
  "downloads": 369,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "ggml",
  "siblings_count": 6
}