oxide-lab/whisper-tiny-gguf overview

{
  "metadata": {},
  "card_data": {
    "license": "mit",
    "language": [
      "multilingual",
      "en",
      "ru"
    ],
    "tags": [
      "whisper",
      "gguf",
      "quantized",
      "speech-recognition",
      "rust",
      "candle"
    ],
    "base_model": [
      "openai/whisper-tiny"
    ],
    "pipeline_tag": "automatic-speech-recognition",
    "frontmatter": {
      "license": "mit",
      "language": [
        "multilingual",
        "en",
        "ru"
      ],
      "tags": [
        "whisper",
        "gguf",
        "quantized",
        "speech-recognition",
        "rust",
        "candle"
      ],
      "base_model": [
        "openai/whisper-tiny"
      ],
      "pipeline_tag": "automatic-speech-recognition"
    },
    "hero_image_url": "",
    "summary": "Quantized versions of openai/whisper-tiny in GGUF format.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlicense: mit\nlanguage:\n- multilingual\n- en\n- ru\ntags:\n- whisper\n- gguf\n- quantized\n- speech-recognition\n- rust\n- candle\nbase_model:\n- openai/whisper-tiny\npipeline_tag: automatic-speech-recognition\n---\n\n# WHISPER-TINY - GGUF Quantized Models\n\nQuantized versions of [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny) in GGUF format.\n\n## Directory Structure\n\n```\ntiny/\n├── whisper-tiny-q*.gguf       # Candle-compatible GGUF models (root)\n├── model-tiny-q80.gguf        # Candle-compatible legacy naming (q8_0 format)\n├── config-tiny.json           # Model configuration for Candle\n├── tokenizer-tiny.json        # Tokenizer for Candle\n└── whisper.cpp/               # whisper.cpp-compatible models\n    └── whisper-tiny-q*.gguf\n\n```\n\n### Format Compatibility\n\n- **Root directory** (`whisper-tiny-*.gguf`): Use with **Candle** (Rust ML framework)\n  - Tensor names include `model.` prefix (e.g., `model.encoder.conv1.weight`)\n  - Requires `config-tiny.json` and `tokenizer-tiny.json`\n  \n- **whisper.cpp/** directory: Use with **whisper.cpp** (C++ implementation)\n  - Tensor names without `model.` prefix (e.g., `encoder.conv1.weight`)\n  - Compatible with whisper.cpp CLI tools\n  - Both directories contain `.gguf` files, not `.bin` files\n\n## Available Formats\n\n| Format |  Quality | Use Case |\n|--------| ---------|----------|\n| q2_k |   Smallest | Extreme compression |\n| q3_k |  Small | Mobile devices |\n| q4_0 |  Good | Legacy compatibility |\n| q4_k |  Good | **Recommended for production** |\n| q4_1 | Good+ | Legacy with bias |\n| q5_0 |  Very Good | Legacy compatibility |\n| q5_k |  Very Good | High quality |\n| q5_1 |  Very Good+ | Legacy with bias |\n| q6_k |   Excellent | Near-lossless |\n| q8_0 |   Excellent | Minimal loss, benchmarking |\n\n## Usage\n\n### With Candle (Rust)\n\n**Command line example:**\n```bash\n# Run Candle Whisper with local quantized model\ncargo run --example whisper --release -- \\\n  --features symphonia \\\n  --quantized \\\n  --model tiny \\\n  --model-id oxide-lab/whisper-tiny-GGUF \n```\n\n### With whisper.cpp (C++)\n\n```bash\n# Use models from whisper.cpp/ subdirectory\n./whisper.cpp/build/bin/whisper-cli \\\n  --model models/openai/tiny/whisper.cpp/whisper-tiny-q4_k.gguf \\\n  --file audio.wav\n```\n\n### Recommended Format\n\nFor most use cases, we recommend **q4_k** format as it provides the best balance of:\n- Size reduction (~65% smaller)\n- Quality (minimal degradation)\n- Speed (faster inference than higher quantizations)\n\n## Quantization Details\n\n- **Source Model**: [openai/whisper-tiny](https://huggingface.co/openai/whisper-tiny)\n- **Quantization Methods**:\n  - **Candle GGUF** (root directory): Python-based quantization. Directly PyTorch → GGUF\n    - Adds `model.` prefix to tensor names for Candle compatibility\n  - **whisper.cpp GGML** (whisper.cpp/ subdirectory): whisper-quantize tool\n    - Uses original tensor names without prefix\n- **Format**: GGUF (GGML Universal Format) for both directories\n- **Total Formats**: 10 quantization levels (q2_k through q8_0)\n\n## License\n\nSame as the original Whisper model (MIT License).\n\n## Citation\n\n```bibtex\n@misc{radford2022whisper,\n  doi = {10.48550/ARXIV.2212.04356},\n  url = {https://arxiv.org/abs/2212.04356},\n  author = {Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},\n  title = {Robust Speech Recognition via Large-Scale Weak Supervision},\n  publisher = {arXiv},\n  year = {2022},\n  copyright = {arXiv.org perpetual, non-exclusive license}\n}\n```",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "whisper",
    "quantized",
    "speech-recognition",
    "rust",
    "candle",
    "automatic-speech-recognition",
    "multilingual",
    "en",
    "ru",
    "arxiv:2212.04356",
    "base_model:openai/whisper-tiny",
    "base_model:quantized:openai/whisper-tiny",
    "license:mit",
    "region:us"
  ],
  "likes": 1,
  "downloads": 704,
  "gated": false,
  "private": false,
  "last_modified": "2026-01-30T09:47:19.000Z",
  "created_at": "2026-01-30T07:38:45.000Z",
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": ""
}

{
  "_id": "697c6005df71de0831a6c624",
  "id": "oxide-lab/whisper-tiny-GGUF",
  "modelId": "oxide-lab/whisper-tiny-GGUF",
  "sha": "94468a6c81edab8c594d9b1d06ea1dfb64292327",
  "createdAt": "2026-01-30T07:38:45.000Z",
  "lastModified": "2026-01-30T09:47:19.000Z",
  "author": "oxide-lab",
  "downloads": 704,
  "likes": 1,
  "gated": false,
  "private": false,
  "pipeline_tag": "automatic-speech-recognition",
  "library_name": "",
  "siblings_count": 18
}