---

library_name: llama.cpp

license: apache-2.0

base_model: internlm/Intern-Decision-2B

base_model_relation: quantized

quantized_by: bombdefuser-124

pipeline_tag: image-text-to-text

tags:

• gguf
• multimodal
• decision-making
• structured-prediction

---

Intern-Decision-2B GGUF

GGUF conversion of internlm/Intern-Decision-2B, a multimodal structured-decision model fine-tuned from Qwen3.5-2B.

Files

| File | Purpose |

|---|---|

| intern-decision-2b-fp16.gguf | FP16 language model |

| intern-decision-2b-q8_0.gguf | Q8_0 language model |

| mmproj-f16.gguf | FP16 vision projector |

Choose one language model and pair it with mmproj-f16.gguf for multimodal inference.

The language-model GGUFs contain the checkpoint's 24 transformer blocks. The source checkpoint does not include MTP tensors, so the conversion omits the MTP/NextN head declared by the source configuration.

llama.cpp

Use a current llama.cpp build with Qwen3.5 multimodal support:

llama-server \
  --model intern-decision-2b-q8_0.gguf \
  --mmproj mmproj-f16.gguf \
  --host 127.0.0.1 \
  --port 8767 \
  --ctx-size 8192 \
  --parallel 1 \
  --gpu-layers all \
  --flash-attn on

Intern-Decision is a candidate-scoring model rather than a general chat model. Follow the upstream model card's decision-schema prompt format, constrain each field to its allowed single-token answer symbols, and read the candidate probabilities. The upstream calibration temperature is 2.100509348278; it was fitted on BF16 weights, so Q8_0 quantization can change calibration quality.

See the upstream model card for the request schema, inference protocol, benchmarks, and license details.