---

license: bsd-2-clause

base_model: pipecat-ai/phonellm-alpha-1

base_model_relation: quantized

pipeline_tag: text-generation

library_name: llama.cpp

language:

• en

tags:

• gguf
• llama.cpp
• quantized
• nemotron_h
• mixture-of-experts
• voice-agent
• phone
• tool-use
• function-calling
• conversational
• pipecat

---

phonellm-alpha-1 — GGUF

Unofficial GGUF quantizations of pipecat-ai/phonellm-alpha-1.

> ⚠️ Disclaimer: This is a community conversion. This repository is not affiliated with, endorsed by, or maintained by Pipecat AI or NVIDIA. All credit for the model goes to the original authors — please see the official model card for full details, intended use, and limitations.

About the model

PhoneLLM Alpha 1 is a voice-agent / phone-conversation model with tool-use and function-calling support, fine-tuned by Pipecat AI from NVIDIA-Nemotron-3-Nano-30B-A3B (a Nemotron-H hybrid Mamba–attention mixture-of-experts model, 30B total / ~3B active parameters).

Conversion details

• Converted with llama.cpp release b10672 (convert_hf_to_gguf.py → F16, then llama-quantize)
• Source weights: BF16 safetensors from the official repository (revision 8e76aaa)

Files

| File | Quant | Size |

|------|-------|------|

| phonellm-alpha-1-F16.gguf | F16 | 63.2 GB |

| phonellm-alpha-1-Q8_0.gguf | Q8_0 | 33.6 GB |

| phonellm-alpha-1-Q4_K_M.gguf | Q4_K_M | 24.5 GB |

| phonellm-alpha-1-Q3_K_M.gguf | Q3_K_M | 19.8 GB |

⚠️ Important: these quants are larger than their names suggest

This architecture's weight tensors have column counts of 1856, 2688 and 3712 — none of which are divisible by 256, the superblock size that llama.cpp's K-quants require. As a result llama-quantize reports 134 of 401 tensor(s) required fallback quantization and substitutes the legacy block-32 types defined in its fallback table (src/llama-quant.cpp): Q3_K → Q4_0, Q4_K → Q5_0, Q6_K → Q8_0.

Only the 29 tensors with a 4096 column count keep their requested K-quant type. The actual composition by size is:

| File | Real tensor mix |

|------|-----------------|

| Q8_0 | 99.9% Q8_0 |

| Q4_K_M | 66.7% Q5_0, 32.5% Q8_0, 0.7% Q4_K |

| Q3_K_M | 49.4% Q5_0, 45.3% Q4_0, 2.5% Q5_1, 1.9% Q8_0, 0.5% Q3_K |

Practical implications:

• The files are larger than you would expect from these quant names on a conventional dense model. Q3_K_M is ~19.8 GB, not the ~13 GB a 30B Q3_K_M would normally be.
• Quality should be better than the names imply, since the fallbacks are higher-precision than the requested types.
• No Q6_K is published. Its fallback type is Q8_0, so a Q6_K build came out 99.1% Q8_0 tensors at 33.5 GB — within 0.1 GB of the Q8_0 file and marginally worse. Use Q8_0 instead.

Usage

Requires a recent llama.cpp build (b10672 or newer) with Nemotron-H support:

llama-cli -m phonellm-alpha-1-Q4_K_M.gguf -cnv

Or serve an OpenAI-compatible API:

llama-server -m phonellm-alpha-1-Q4_K_M.gguf

License

PhoneLLM Alpha 1 is released under the BSD 2-Clause License (see LICENSE).

PhoneLLM is a derivative work of NVIDIA Nemotron 3 Nano 30B-A3B, which is licensed under the NVIDIA Nemotron Open Model License (see LICENSE_NVIDIA.txt). Per Section 3 (Redistribution) of that license, this repository includes a copy of the NVIDIA Nemotron Open Model License and retains the NVIDIA copyright and attribution notices. The BSD 2-Clause terms apply to Pipecat AI's modifications and to the model as a whole, as Section 3 permits; the NVIDIA license continues to apply to the underlying Nemotron work. "Nemotron" and "NVIDIA" are trademarks of NVIDIA Corporation, used here only to describe the origin of the base model.