cstr/granite-speech-4.0-1b-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "apache-2.0",
    "base_model": "ibm-granite/granite-speech-4.0-1b",
    "language": [
      "en",
      "fr",
      "de",
      "es",
      "pt",
      "ja"
    ],
    "tags": [
      "asr",
      "speech-to-text",
      "audio",
      "transcription",
      "gguf",
      "ggml",
      "speech-llm",
      "granite",
      "q-former"
    ],
    "pipeline_tag": "automatic-speech-recognition",
    "library_name": "ggml",
    "frontmatter": {
      "license": "apache-2.0",
      "base_model": "ibm-granite/granite-speech-4.0-1b",
      "language": [
        "en",
        "fr",
        "de",
        "es",
        "pt",
        "ja"
      ],
      "tags": [
        "asr",
        "speech-to-text",
        "audio",
        "transcription",
        "gguf",
        "ggml",
        "speech-llm",
        "granite",
        "q-former"
      ],
      "pipeline_tag": "automatic-speech-recognition",
      "library_name": "ggml"
    },
    "hero_image_url": "",
    "summary": "GGUF conversions of ibm-granite/granite-speech-4.0-1b, IBM's 1B-parameter speech-LLM (Conformer encoder + BLIP-2 Q-Former projector + Granite μP LLM head). Runs through CrispASR's --backend granite path with full feature support including the new prompt-based speech-translation path.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: apache-2.0\nbase_model: ibm-granite/granite-speech-4.0-1b\nlanguage:\n  - en\n  - fr\n  - de\n  - es\n  - pt\n  - ja\ntags:\n  - asr\n  - speech-to-text\n  - audio\n  - transcription\n  - gguf\n  - ggml\n  - speech-llm\n  - granite\n  - q-former\npipeline_tag: automatic-speech-recognition\nlibrary_name: ggml\n---\n\n# Granite-Speech-4.0-1B — GGUF (CrispASR)\n\nGGUF conversions of [`ibm-granite/granite-speech-4.0-1b`](https://huggingface.co/ibm-granite/granite-speech-4.0-1b), IBM's 1B-parameter speech-LLM (Conformer encoder + BLIP-2 Q-Former projector + Granite μP LLM head). Runs through [CrispASR](https://github.com/CrispStrobe/CrispASR)'s `--backend granite` path with full feature support including the new prompt-based speech-translation path.\n\n## What's in the box\n\n| File | Size | Quantization | Notes |\n|---|---|---|---|\n| `granite-speech-4.0-1b-f16.gguf` | 5.58 GB | F16 | Reference precision |\n| `granite-speech-4.0-1b-q8_0.gguf` | 3.86 GB | Q8_0 | Effectively lossless |\n| `granite-speech-4.0-1b-q5_0.gguf` | 3.17 GB | Q5_0 | Slightly slower than Q4_K but a bit more accurate |\n| `granite-speech-4.0-1b-q4_k.gguf` | 2.94 GB | Q4_K | 1.9× compressed; smallest reasonable choice |\n\nAll four contain:\n\n* The Conformer audio encoder (16 layers, d_model 1024, Shaw-style relative-position attention with 200-sample block-local context)\n* The BLIP-2 Q-Former projector (2 layers, 16 heads, 3-query learnable prompt tokens, audio→text dim conversion)\n* The Granite-4.0 1B LLM (40 layers, d_model 2048, 16 heads / 4 KV heads, 8192 ff, 100353 vocab, μP scaling)\n* The full GPT-2-style BPE vocab **and merges table** (the merges are new in this re-conversion — older GGUFs had vocab strings only)\n* Mel filterbank (80 bins, stacked 2× to 160), Hann window\n\n## What's new in this conversion\n\n* **`tokenizer.ggml.merges` field** — the previous granite GGUFs had only `tokenizer.ggml.tokens`. The C++ side could detokenize but couldn't tokenize arbitrary text, which blocked the prompt-based translate path. This re-conversion writes the merges table from `merges.txt` and unblocks `--translate -tl <lang>` end-to-end.\n\nIf you have an older granite GGUF without the merges field, the C++ side still loads it for plain transcribe, but `--translate` is silently a no-op. Re-download these files to get translate.\n\n## Use with CrispASR (no Python at runtime)\n\n```bash\n# Build crispasr (one-time)\ngit clone https://github.com/CrispStrobe/CrispASR\ncd CrispASR\ncmake -B build -DCMAKE_BUILD_TYPE=Release\ncmake --build build -j$(nproc) --target whisper-cli\n\n# Auto-download the recommended quant on first use:\n./build/bin/crispasr --backend granite -m auto -f my_audio.wav\n\n# Or point at a local file:\n./build/bin/crispasr --backend granite \\\n    -m granite-speech-4.0-1b-q4_k.gguf \\\n    -f my_audio.wav\n\n# Speech translation to German via the runtime-tokenized prompt path:\n./build/bin/crispasr --backend granite \\\n    -m granite-speech-4.0-1b-q4_k.gguf \\\n    -f my_audio.wav --translate -tl de\n# → \"und so meine amerikaner, fragen sie nicht, was ihr land für sie\n#    tun kann, fragen sie, was sie für ihr land tun können.\"\n\n# Word-level timestamps via the canary CTC aligner second pass:\n./build/bin/crispasr --backend granite \\\n    -m granite-speech-4.0-1b-q4_k.gguf \\\n    -f my_audio.wav -am canary-ctc-aligner-q5_0.gguf -osrt -ml 1\n```\n\nPure C++ runtime: no Python, no PyTorch, no NeMo. The only build deps beyond C++17 + CMake are `libcurl` or `wget` for auto-download and (optionally) BLAS / OpenMP for the encoder hot-loop speedup.\n\n## Languages\n\nSame as upstream Granite-Speech 4.0: English, French, German, Spanish, Portuguese, Japanese.\n\n## How it was made\n\n```bash\n# 1. Download the base model from HF\nhf download ibm-granite/granite-speech-4.0-1b --local-dir ./granite-speech-4.0-1b\n\n# 2. Convert to F16 GGUF (the new converter writes both vocab and merges)\npython models/convert-granite-speech-to-gguf.py \\\n    --input ./granite-speech-4.0-1b \\\n    --output granite-speech-4.0-1b-f16.gguf\n\n# 3. Quantize\n./build/bin/crispasr-quantize granite-speech-4.0-1b-f16.gguf granite-speech-4.0-1b-q8_0.gguf q8_0\n./build/bin/crispasr-quantize granite-speech-4.0-1b-f16.gguf granite-speech-4.0-1b-q5_0.gguf q5_0\n./build/bin/crispasr-quantize granite-speech-4.0-1b-f16.gguf granite-speech-4.0-1b-q4_k.gguf q4_k\n```\n\nTotal: 940 tensors per file (encoder, projector, LLM, audio frontend, BPE tables).\n\n## Verification\n\nAll four quants tested on samples/jfk.wav under CrispASR's granite backend:\n\n```\ndefault transcribe (any quant):\n  \"and so my fellow americans ask not what your country can do for you\n   ask what you can do for your country\"\n\ntranslate -> German (any quant):\n  \"und so meine amerikaner, fragen sie nicht, was ihr land für sie\n   tun kann, fragen sie, was sie für ihr land tun können.\"\n```\n\nNote: Granite-Speech is a μP-scaled model — the C++ implementation respects the embedding multiplier, attention multiplier, residual multiplier, and logits scale from the upstream config. Bit-identity is preserved across quants on the default transcribe path within q4_k expected float drift.\n\n## License\n\nApache-2.0, same as upstream IBM Granite-Speech-4.0-1B.\n\n## Citation\n\n```bibtex\n@misc{granite-speech,\n    title  = {Granite-Speech-4.0-1B},\n    author = {IBM Research},\n    year   = {2026},\n    url    = {https://huggingface.co/ibm-granite/granite-speech-4.0-1b}\n}\n```\n",
    "related_quantizations": []
  },
  "tags": [
    "ggml",
    "gguf",
    "asr",
    "speech-to-text",
    "audio",
    "transcription",
    "speech-llm",
    "granite",
    "q-former",
    "automatic-speech-recognition",
    "en",
    "fr",
    "de",
    "es",
    "pt",
    "ja",
    "license:apache-2.0",
    "region:us"
  ],
  "likes": 0,
  "downloads": 381,
  "gated": false,
  "private": false,
  "last_modified": "2026-04-11T17:41:27.000Z",
  "created_at": "2026-04-10T20:26:31.000Z",
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "ggml"
}

{
  "_id": "69d95cf7bd7146b181bfb4f7",
  "id": "cstr/granite-speech-4.0-1b-GGUF",
  "modelId": "cstr/granite-speech-4.0-1b-GGUF",
  "sha": "4185313411a24d14fa049a3113067be0174a95b2",
  "createdAt": "2026-04-10T20:26:31.000Z",
  "lastModified": "2026-04-11T17:41:27.000Z",
  "author": "cstr",
  "downloads": 381,
  "likes": 0,
  "gated": false,
  "private": false,
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "ggml",
  "siblings_count": 9
}