cstr/parakeet-tdt-0.6b-v3-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "cc-by-4.0",
    "language": [
      "bg",
      "cs",
      "da",
      "de",
      "el",
      "en",
      "es",
      "et",
      "fi",
      "fr",
      "hr",
      "hu",
      "it",
      "lt",
      "lv",
      "mt",
      "nl",
      "pl",
      "pt",
      "ro",
      "ru",
      "sk",
      "sl",
      "sv",
      "uk"
    ],
    "pipeline_tag": "automatic-speech-recognition",
    "tags": [
      "audio",
      "speech-recognition",
      "transcription",
      "ggml",
      "gguf",
      "parakeet",
      "tdt",
      "fastconformer",
      "multilingual"
    ],
    "library_name": "ggml",
    "base_model": "nvidia/parakeet-tdt-0.6b-v3",
    "frontmatter": {
      "license": "cc-by-4.0",
      "language": [
        "bg",
        "cs",
        "da",
        "de",
        "el",
        "en",
        "es",
        "et",
        "fi",
        "fr",
        "hr",
        "hu",
        "it",
        "lt",
        "lv",
        "mt",
        "nl",
        "pl",
        "pt",
        "ro",
        "ru",
        "sk",
        "sl",
        "sv",
        "uk"
      ],
      "pipeline_tag": "automatic-speech-recognition",
      "tags": [
        "audio",
        "speech-recognition",
        "transcription",
        "ggml",
        "gguf",
        "parakeet",
        "tdt",
        "fastconformer",
        "multilingual"
      ],
      "library_name": "ggml",
      "base_model": "nvidia/parakeet-tdt-0.6b-v3"
    },
    "hero_image_url": "",
    "summary": "GGUF / ggml conversions of nvidia/parakeet-tdt-0.6b-v3 for use with the parakeet-main CLI from **CrispStrobe/CrispASR@parakeet**. Parakeet TDT 0.6B v3 is NVIDIA's 600 M-parameter multilingual ASR model: This repo provides four quantisations, all converted from the same .nemo checkpoint via the streaming convert-parakeet-to-gguf.py script and quantised with cohere-quantize.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: cc-by-4.0\nlanguage:\n- bg\n- cs\n- da\n- de\n- el\n- en\n- es\n- et\n- fi\n- fr\n- hr\n- hu\n- it\n- lt\n- lv\n- mt\n- nl\n- pl\n- pt\n- ro\n- ru\n- sk\n- sl\n- sv\n- uk\npipeline_tag: automatic-speech-recognition\ntags:\n- audio\n- speech-recognition\n- transcription\n- ggml\n- gguf\n- parakeet\n- tdt\n- fastconformer\n- multilingual\nlibrary_name: ggml\nbase_model: nvidia/parakeet-tdt-0.6b-v3\n---\n\n# Parakeet TDT 0.6B v3 — GGUF (ggml-quantised)\n\nGGUF / ggml conversions of [`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) for use with the `parakeet-main` CLI from **[CrispStrobe/CrispASR@parakeet](https://github.com/CrispStrobe/CrispASR/tree/parakeet)**.\n\nParakeet TDT 0.6B v3 is NVIDIA's 600 M-parameter multilingual ASR model:\n\n- **25 European languages** with automatic language detection (no prompt prefix needed)\n- **Built-in word-level timestamps** from the TDT (Token-and-Duration Transducer) decoder — no separate CTC alignment model required\n- **6.34 % avg WER** on the HuggingFace Open ASR Leaderboard\n- **CC-BY-4.0** licence (friendlier than most ASR models)\n\nThis repo provides four quantisations, all converted from the same `.nemo` checkpoint via the streaming `convert-parakeet-to-gguf.py` script and quantised with `cohere-quantize`.\n\n## Files\n\n| File | Size | Notes |\n| --- | ---: | --- |\n| `parakeet-tdt-0.6b-v3.gguf`        | 1.26 GB | F16, full precision |\n| `parakeet-tdt-0.6b-v3-q8_0.gguf`   | 711 MB  | Q8_0, near-lossless |\n| `parakeet-tdt-0.6b-v3-q5_0.gguf`   | 516 MB  | Q5_0 |\n| `parakeet-tdt-0.6b-v3-q4_k.gguf`   | 467 MB  | **Q4_K — recommended default** |\n\nAll quantisations produce identical text on `samples/jfk.wav`:\n> And so my fellow Americans. Ask not what your country can do for you. Ask what you can do for your country.\n\n## Quick Start\n\n```bash\n# 1. Build the runtime\ngit clone -b parakeet https://github.com/CrispStrobe/CrispASR\ncd CrispASR\ncmake -B build -DCMAKE_BUILD_TYPE=Release\ncmake --build build -j$(nproc) --target parakeet-main\n\n# 2. Download a quantisation\nhuggingface-cli download cstr/parakeet-tdt-0.6b-v3-GGUF \\\n    parakeet-tdt-0.6b-v3-q4_k.gguf --local-dir .\n\n# 3. Transcribe\n./build/bin/parakeet-main \\\n    -m parakeet-tdt-0.6b-v3-q4_k.gguf \\\n    -f your-audio.wav -t 8\n```\n\n## Word-level timestamps for free\n\nPass `-v` to dump per-token timestamps from the TDT duration head. Each token spans one or more encoder frames; one frame = **80 ms**. No separate alignment model is required.\n\n```\n$ ./build/bin/parakeet-main -m parakeet-tdt-0.6b-v3-q4_k.gguf -f samples/jfk.wav -t 8 -v\n  [    0.32s →     0.64s]  ' And'\n  [    0.64s →     0.88s]  ' so'\n  [    1.04s →     1.28s]  ' my'\n  [    1.28s →     1.76s]  ' fellow'        ← f + ell + ow grouped\n  [    1.76s →     2.56s]  ' Americans'\n  [    2.96s →     3.28s]  '.'\n  [    3.28s →     3.84s]  ' Ask'\n  [    4.08s →     4.40s]  ' not'\n  [    5.28s →     5.92s]  ' what your'\n  ...\n```\n\nThis is roughly **10× tighter** than the cross-attention DTW path used for Cohere Transcribe word timestamps (~360 ms MAE), and comparable to running a separate wav2vec2 + CTC forced alignment model — but at zero extra cost.\n\n## Model architecture\n\n| Component | Details |\n| --- | --- |\n| Encoder       | 24-layer FastConformer, d=1024, 8 heads, head_dim=128, FFN=4096, conv kernel=9 |\n| Subsampling   | Conv2d dw_striding stack, 8× temporal (50 → 12.5 fps) |\n| Predictor     | 2-layer LSTM, hidden 640, embed 8193 × 640 |\n| Joint head    | enc(1024 → 640) + pred(640 → 640) → ReLU → linear(640 → 8198) |\n| Vocab         | 8192 SentencePiece tokens (multilingual) |\n| Audio         | 16 kHz mono, 128 mel bins, n_fft=512, hop=160, win=400 |\n| Parameters    | ~600 M |\n\nThe mel filterbank and Hann window are baked directly into the GGUF (`preprocessor.fb` and `preprocessor.window` from the original `.nemo` checkpoint), so there is no recomputation at runtime. BatchNorm in the convolution module is folded into the depthwise conv weights at load time.\n\n## How this was made\n\n1. The `.nemo` checkpoint was unpacked, NeMo state-dict keys were remapped to ggml-friendly names, and weights were written to GGUF F16 (matmul tensors) + F32 (norms / biases / mel filterbank). A synthetic zero `conv.dw.bias` is added per encoder layer so the runtime BN-fold pass has somewhere to write the absorbed bias shift.\n2. Quantised variants are produced by `cohere-quantize` (the same llama.cpp-style quantiser used for the Cohere Transcribe GGUFs).\n3. Inference is implemented in `src/parakeet.{h,cpp}`: the FastConformer encoder runs as a single ggml graph (BN folded out), the LSTM predictor and joint head run as manual F32 CPU loops, and the TDT greedy decode loop alternates \"advance encoder frame\" / \"emit token + advance predictor\" using the duration head's argmax.\n\n## Supported languages\n\n`bg cs da de el en es et fi fr hr hu it lt lv mt nl pl pt ro ru sk sl sv uk`\n\nThe model auto-detects the language at inference time. No prompt prefix or `-l` flag is needed.\n\n## Attribution\n\n- **Original model:** [`nvidia/parakeet-tdt-0.6b-v3`](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3) (CC-BY-4.0). NVIDIA NeMo team.\n- **GGUF conversion + ggml runtime:** [`CrispStrobe/CrispASR@parakeet`](https://github.com/CrispStrobe/CrispASR/tree/parakeet) — community contribution. Encoder graph borrows the dw_striding subsampling + Conformer block patterns from the same fork's `cohere.cpp`.\n- **Reference inference:** [`istupakov/onnx-asr`](https://github.com/istupakov/onnx-asr) was the cross-check for the joint head + TDT greedy loop.\n\n## Related\n\n- C++ runtime: **[CrispStrobe/CrispASR@parakeet](https://github.com/CrispStrobe/CrispASR/tree/parakeet)**\n- Sister repo (Cohere Transcribe): [`cstr/cohere-transcribe-03-2026-GGUF`](https://huggingface.co/cstr/cohere-transcribe-03-2026-GGUF)\n\n## License\n\nCC-BY-4.0, inherited from the base model. Use of these GGUF files must comply with the CC-BY-4.0 license including attribution.\n",
    "related_quantizations": []
  },
  "tags": [
    "ggml",
    "gguf",
    "audio",
    "speech-recognition",
    "transcription",
    "parakeet",
    "tdt",
    "fastconformer",
    "multilingual",
    "automatic-speech-recognition",
    "bg",
    "cs",
    "da",
    "de",
    "el",
    "en",
    "es",
    "et",
    "fi",
    "fr",
    "hr",
    "hu",
    "it",
    "lt",
    "lv",
    "mt",
    "nl",
    "pl",
    "pt",
    "ro",
    "ru",
    "sk",
    "sl",
    "sv",
    "uk",
    "base_model:nvidia/parakeet-tdt-0.6b-v3",
    "base_model:quantized:nvidia/parakeet-tdt-0.6b-v3",
    "license:cc-by-4.0",
    "region:us"
  ],
  "likes": 0,
  "downloads": 551,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-08T06:01:37.000Z",
  "created_at": "2026-04-07T19:15:27.000Z",
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "ggml"
}

{
  "_id": "69d557cfdeb30b8d545ed30a",
  "id": "cstr/parakeet-tdt-0.6b-v3-GGUF",
  "modelId": "cstr/parakeet-tdt-0.6b-v3-GGUF",
  "sha": "75034ed594847d19fb1d4120a9c0a9f22863530f",
  "createdAt": "2026-04-07T19:15:27.000Z",
  "lastModified": "2026-04-08T06:01:37.000Z",
  "author": "cstr",
  "downloads": 551,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "ggml",
  "siblings_count": 6
}