---

license: apache-2.0

base_model: ibm-granite/granite-switch-4.1-3b-preview

tags:

- gguf

- llama.cpp

- ollama

- granite

- granite-switch

- intrinsics

library_name: gguf

---

granite-switch-4.1-3b-preview-GGUF

GGUF (F16) build of ibm-granite/granite-switch-4.1-3b-preview,

for running the model locally with llama.cpp and Ollama.

Files

| File | Quant | Size |

|------|-------|------|

| granite-switch-4.1-3b-preview-f16.gguf | F16 | ~7.8 GB |

About granite-switch

granite-switch packs multiple LoRA/aLoRA intrinsics (answerability, certainty,

guardian, RAG adapters, …) into a single model and selects between them with a

control token placed in the prompt (<|answerability|>, <|guardian-core|>, …).

Running the intrinsics through Ollama requires the patch that recovers per-token

adapter selection in the ggml graph and expects the control token to already be in

the prompt (driven via the raw /api/generate endpoint). See the

granite-switch Ollama patch

and the Mellea + Ollama bridge.

Usage (llama.cpp)

llama-cli -m granite-switch-4.1-3b-preview-f16.gguf -p "Hello"

License

Apache-2.0, inherited from the base model. Refer to the

original model card

for full terms and intended use.