---

library_name: transformers

tags:

• text-generation-inference
• llama-cpp
• decision-model
• decision-making
• multimodal
• structured-prediction

base_model:

• internlm/Intern-Decision-4B

license: apache-2.0

language:

• en

pipeline_tag: image-text-to-text

---

Intern-Decision-4B-GGUF

> Intern-Decision-4B is a multimodal structured decision model fine-tuned from Qwen3.5-4B by the InternLM team. It takes a shared state, a schema of named questions (choice, score, or yes/no noul, 1-16 questions with up to 62 options each), and up to eight optional images, and returns a calibrated answer distribution for every question in one forward pass. It never calls generate(). Each option is mapped to a single-token symbol, the prompt is rendered with a JSON answer skeleton containing a <decision> placeholder per field, and logits are read just before each placeholder and softmaxed over that field's candidate symbols only. A fitted temperature of about 1.99, trained on 1,728 calibration cases and checked on 1,693 separate validation cases, is then applied. On the card's benchmark table it averages 90.02, edging the Jev baseline at 88.74, and has the best Brier score (0.347 vs Jev's 0.358). Its ECE of 0.065 is better than Jev's 0.095 but not the best, since JevK5 reaches 0.047. The 4B matches Jev's 73.87 on Jevbench-Hard only in the sense that JevK5 also scores that. Jev itself scores 72.07, so the 4B is ahead there. It is weaker on WildJailBreak (89.86 vs 96.29). A separate 96-case known-distribution pilot shows calibration improving Brier/ECE from 0.628/0.213 to 0.550/0.089, against 0.595/0.130 for Jev, so after calibration it has lower ECE but a worse Brier score than Jev. On an RTX 4090 it averages about 44 ms per query versus about 110 ms for Jev. It runs through a local DecisionEngine Hugging Face wrapper whose response follows the Jev envelope, and 0.8B and 2B siblings exist. It is released under Apache 2.0 with the Qwen license preserved.

Model Files

| File Name | Quant Type | File Size | File Link | Description |

|-----------|------------|-----------|-----------|-------------|

| Intern-Decision-4B.BF16.gguf | BF16 | 8.42 GB | Link | Full BF16 weights. Highest quality, largest file size. |

| Intern-Decision-4B.Q3_K_L.gguf | Q3_K_L | 2.42 GB | Link | Lower quality but usable, good for low RAM availability. |

| Intern-Decision-4B.Q3_K_M.gguf | Q3_K_M | 2.26 GB | Link | Low quality. |

| Intern-Decision-4B.Q4_K_M.gguf | Q4_K_M | 2.71 GB | Link | Good quality, default size for most use cases, recommended. |

| Intern-Decision-4B.Q4_K_S.gguf | Q4_K_S | 2.56 GB | Link | Slightly lower quality with more space savings, recommended. |

| Intern-Decision-4B.Q5_K_M.gguf | Q5_K_M | 3.07 GB | Link | High quality, recommended. |

| Intern-Decision-4B.Q5_K_S.gguf | Q5_K_S | 2.99 GB | Link | High quality, recommended. |

| Intern-Decision-4B.Q6_K.gguf | Q6_K | 3.46 GB | Link | Very high quality, near perfect, recommended. |

| Intern-Decision-4B.Q8_0.gguf | Q8_0 | 4.48 GB | Link | Extremely high quality, generally unneeded but max available quant. |

| Intern-Decision-4B.mmproj-bf16.gguf | mmproj-bf16 | 676 MB | Link | Multimodal projection file in BF16 format. Used for vision/language models. |

llama.cpp

LLM inference in C/C++ — https://github.com/ggml-org/llama.cpp