---

license: apache-2.0

base_model: ibm-granite/granite-switch-4.1-3b-preview

tags:

• gguf
• llama.cpp
• granite
• granite-switch

---

granite-switch-4.1-3b-preview — GGUF (PR #25107 build)

GGUF conversion of ibm-granite/granite-switch-4.1-3b-preview,

produced for the in-progress llama.cpp Granite-Switch architecture support in

ggml-org/llama.cpp#25107.

These files are intended to be used with the feature/granite-switch branch of that PR;

they will not load on a stock llama.cpp build until the architecture is merged.

Files

| File | Quant | Size |

|------|-------|------|

| granite-switch-4.1-3b-preview-bf16.gguf | BF16 | 8.80 GB |

The source model weights (including the stacked LoRA adapters) are BF16, so this build keeps

them in BF16 rather than down-casting to F16, avoiding the F16 overflow/underflow issues that

BF16 normally mitigates on Granite models.

Provenance

• Converted with the PR branch's convert_hf_to_gguf.py (arch graniteswitch).
• Source revision: HF snapshot of ibm-granite/granite-switch-4.1-3b-preview.
• GGUF metadata includes the switch-specific keys: num_adapters, max_lora_rank,

adapter_token_ids, adapter_substitute_token_ids.

Usage

# from the PR branch checkout
./build/bin/llama-cli -m granite-switch-4.1-3b-preview-bf16.gguf -p "The capital of France is" -n 24 --temp 0