lex-au/orpheus-3b-ft-q4_k_m.gguf overview

{
  "metadata": {},
  "card_data": {
    "language": "en",
    "tags": [
      "text-to-speech",
      "tts",
      "audio",
      "speech-synthesis",
      "orpheus",
      "gguf"
    ],
    "license": "apache-2.0",
    "datasets": [
      "internal"
    ],
    "frontmatter": {
      "language": "en",
      "tags": [
        "text-to-speech",
        "tts",
        "audio",
        "speech-synthesis",
        "orpheus",
        "gguf"
      ],
      "license": "apache-2.0",
      "datasets": [
        "internal"
      ]
    },
    "hero_image_url": "",
    "summary": "This is a quantised version of canopylabs/orpheus-3b-0.1-ft. Orpheus is a high-performance Text-to-Speech model fine-tuned for natural, emotional speech synthesis. This repository hosts the 8-bit quantised version of the 3B parameter model, optimised for efficiency while maintaining high-quality output.",
    "quick_links": [],
    "benchmark_table_html": "",
    "readme_markdown": "---\nlanguage: en\ntags:\n- text-to-speech\n- tts\n- audio\n- speech-synthesis\n- orpheus\n- gguf\nlicense: apache-2.0\ndatasets:\n- internal\n---\n\n# Orpheus-3b-FT-Q4_K_M\n\nThis is a quantised version of [canopylabs/orpheus-3b-0.1-ft](https://huggingface.co/canopylabs/orpheus-3b-0.1-ft).\n\nOrpheus is a high-performance Text-to-Speech model fine-tuned for natural, emotional speech synthesis. This repository hosts the 8-bit quantised version of the 3B parameter model, optimised for efficiency while maintaining high-quality output.\n\n## Model Description\n\n**Orpheus-3b-FT-Q4_K_M** is a 3 billion parameter Text-to-Speech model that converts text inputs into natural-sounding speech with support for multiple voices and emotional expressions. The model has been quantised to 8-bit (Q4_K_M) format for efficient inference, making it accessible on consumer hardware.\n\nKey features:\n- 8 distinct voice options with different characteristics\n- Support for emotion tags like laughter, sighs, etc.\n- Optimised for CUDA acceleration on RTX GPUs\n- Produces high-quality 24kHz mono audio\n- Fine-tuned for conversational naturalness\n\n## How to Use\n\nThis model is designed to be used with an LLM inference server that connects to the [Orpheus-FastAPI](https://github.com/Lex-au/Orpheus-FastAPI) frontend, which provides both a web UI and OpenAI-compatible API endpoints.\n\n### Compatible Inference Servers\n\nThis quantised model can be loaded into any of these LLM inference servers:\n\n- [GPUStack](https://github.com/gpustack/gpustack) - GPU optimised LLM inference server (My pick) - supports LAN/WAN tensor split parallelisation\n- [LM Studio](https://lmstudio.ai/) - Load the GGUF model and start the local server\n- [llama.cpp server](https://github.com/ggerganov/llama.cpp) - Run with the appropriate model parameters\n- Any compatible OpenAI API-compatible server\n\n### Quick Start\n\n1. Download this quantised model from [lex-au's Orpheus-FASTAPI collection](https://huggingface.co/collections/lex-au/orpheus-fastapi-67e125ae03fc96dae0517707)\n\n2. Load the model in your preferred inference server and start the server.\n\n3. Clone the Orpheus-FastAPI repository:\n```bash\ngit clone https://github.com/Lex-au/Orpheus-FastAPI.git\ncd Orpheus-FastAPI\n```\n\n4. Configure the FastAPI server to connect to your inference server by setting the `ORPHEUS_API_URL` environment variable.\n\n5. Follow the complete installation and setup instructions in the [repository README](https://github.com/Lex-au/Orpheus-FastAPI).\n\n### Audio Samples\n\nListen to the model in action with different voices and emotions:\n\n#### Default Voice Sample\n<audio controls>\n  <source src=\"https://lex-au.github.io/Orpheus-FastAPI/DefaultTest.mp3\" type=\"audio/mpeg\">\n  Your browser does not support the audio element.\n</audio>\n\n#### Leah (Happy)\n<audio controls>\n  <source src=\"https://lex-au.github.io/Orpheus-FastAPI/LeahHappy.mp3\" type=\"audio/mpeg\">\n  Your browser does not support the audio element.\n</audio>\n\n#### Tara (Sad)\n<audio controls>\n  <source src=\"https://lex-au.github.io/Orpheus-FastAPI/TaraSad.mp3\" type=\"audio/mpeg\">\n  Your browser does not support the audio element.\n</audio>\n\n#### Zac (Contemplative)\n<audio controls>\n  <source src=\"https://lex-au.github.io/Orpheus-FastAPI/ZacContemplative.mp3\" type=\"audio/mpeg\">\n  Your browser does not support the audio element.\n</audio>\n\n### Available Voices\n\nThe model supports 8 different voices:\n- `tara`: Female, conversational, clear\n- `leah`: Female, warm, gentle\n- `jess`: Female, energetic, youthful\n- `leo`: Male, authoritative, deep\n- `dan`: Male, friendly, casual\n- `mia`: Female, professional, articulate\n- `zac`: Male, enthusiastic, dynamic\n- `zoe`: Female, calm, soothing\n\n### Emotion Tags\n\nYou can add expressiveness to speech by inserting tags:\n- `<laugh>`, `<chuckle>`: For laughter sounds\n- `<sigh>`: For sighing sounds\n- `<cough>`, `<sniffle>`: For subtle interruptions\n- `<groan>`, `<yawn>`, `<gasp>`: For additional emotional expression\n\n## Technical Specifications\n\n- **Architecture**: Specialised token-to-audio sequence model\n- **Parameters**: ~3 billion\n- **Quantisation**: 8-bit (GGUF Q4_K_M format)\n- **Audio Sample Rate**: 24kHz\n- **Input**: Text with optional voice selection and emotion tags\n- **Output**: High-quality WAV audio\n- **Language**: English\n- **Hardware Requirements**: CUDA-compatible GPU (recommended: RTX series)\n- **Integration Method**: External LLM inference server + Orpheus-FastAPI frontend\n\n## Limitations\n\n- Currently supports English text only\n- Best performance achieved on CUDA-compatible GPUs\n- Generation speed depends on GPU capability\n\n## License\n\nThis model is available under the [Apache License 2.0](https://www.apache.org/licenses/LICENSE-2.0).\n\n## Citation & Attribution\n\nThe original Orpheus model was created by Canopy Labs. This repository contains a quantised version optimised for use with the Orpheus-FastAPI server.\n\nIf you use this quantised model in your research or applications, please cite:\n\n```\n@misc{orpheus-tts-2025,\n  author = {Canopy Labs},\n  title = {Orpheus-3b-0.1-ft: Text-to-Speech Model},\n  year = {2025},\n  publisher = {HuggingFace},\n  howpublished = {\\url{https://huggingface.co/canopylabs/orpheus-3b-0.1-ft}}\n}\n\n@misc{orpheus-quantised-2025,\n  author = {Lex-au},\n  title = {Orpheus-3b-FT-Q4_K_M: Quantised TTS Model with FastAPI Server},\n  note = {GGUF quantisation of canopylabs/orpheus-3b-0.1-ft},\n  year = {2025},\n  publisher = {HuggingFace},\n  howpublished = {\\url{https://huggingface.co/lex-au/Orpheus-3b-FT-Q4_K_M.gguf}}\n}\n```",
    "related_quantizations": []
  },
  "tags": [
    "gguf",
    "text-to-speech",
    "tts",
    "audio",
    "speech-synthesis",
    "orpheus",
    "en",
    "dataset:internal",
    "license:apache-2.0",
    "endpoints_compatible",
    "region:us",
    "conversational"
  ],
  "likes": 6,
  "downloads": 584,
  "gated": false,
  "private": false,
  "last_modified": "2025-03-24T10:15:54.000Z",
  "created_at": "2025-03-24T09:25:52.000Z",
  "pipeline_tag": "text-to-speech",
  "library_name": ""
}

{
  "_id": "67e12520253028c81ca7053c",
  "id": "lex-au/Orpheus-3b-FT-Q4_K_M.gguf",
  "modelId": "lex-au/Orpheus-3b-FT-Q4_K_M.gguf",
  "sha": "51079fc5f7de3e137cdb453398baa1d4bc1614dc",
  "createdAt": "2025-03-24T09:25:52.000Z",
  "lastModified": "2025-03-24T10:15:54.000Z",
  "author": "lex-au",
  "downloads": 584,
  "likes": 6,
  "gated": false,
  "private": false,
  "pipeline_tag": "text-to-speech",
  "library_name": "",
  "siblings_count": 3
}