---

license: openmdw-1.1

base_model: nvidia/Nemotron-3-Embed-1B-BF16

library_name: llama.cpp

tags:

- embedding

- text-embeddings-inference

- llama.cpp

- gguf

---

Nemotron-3-Embed-1B GGUF

GGUF conversion of nvidia/Nemotron-3-Embed-1B-BF16 for use with llama.cpp.

Files

| File | Quant | Size |

|------|-------|------|

| model-BF16.gguf | BF16 (lossless, matches source) | 2.2 GB |

| model-Q8_0.gguf | Q8_0 | 1.2 GB |

Usage

Serve with llama-server for embeddings (mean pooling, L2 normalized output):

llama-server -m model-BF16.gguf \
  --pooling mean \
  --embed-normalize 2 \
  --host 0.0.0.0 --port 8080

Then query:

curl http://localhost:8080/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"input": "hello world"}'

Notes

• Architecture: mistral3 (Ministral3, 16 layers, hidden 2048, 24 Q / 8 KV heads, 262k context with YaRN).
• The model was trained and served as a sentence-transformers model, which does not prepend a BOS token. llama.cpp's tokenizer for this model does add one by default. If you need embeddings that exactly match the Hugging Face reference, run the server with:

--override-kv tokenizer.ggml.add_bos_token=bool:false

Retrieval quality is unaffected either way (the BOS token applies a nearly constant offset to all embeddings), but the raw vectors will differ.

• Converted with convert_hf_to_gguf.py from the BF16 safetensors.