---

base_model: openbmb/MiniCPM-V-4.7-1B

base_model_relation: quantized

quantized_by: AtomicChat

pipeline_tag: image-text-to-text

library_name: gguf

license: apache-2.0

tags:

• gguf
• llama.cpp
• minicpm
• openbmb
• imatrix
• quantized
• conversational
• atomic-chat

---

How to Run MiniCPM V 4.7 1B Locally

<p style="margin-top: 0; margin-bottom: 0;">

<em>Built from openbmb's original weights with our own importance matrix. The <a href="https://huggingface.co/datasets/AtomicChat/calib-corpora">calibration corpora</a> behind our builds are public.</em>

</p>

<div style="display: flex; gap: 8px; align-items: center; margin-top: 10px; margin-bottom: 10px;">

<a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_minicpm_v_4_7_1b&utm_content=btn_atomic"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_atomic.png" width="162" alt="Atomic Chat"></a>

<a href="https://discord.gg/8wGSsvmg4V"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_discord.png" width="119" alt="Discord"></a>

<a href="https://github.com/AtomicBot-ai/Atomic-Chat"><img src="https://huggingface.co/AtomicChat/Qwen3.8-27B-GGUF/resolve/main/btn_github.png" width="115" alt="GitHub"></a>

</div>

<ul style="margin: 0 0 12px 0;">

<li>You can now run MiniCPM in <a href="https://atomic.chat/?utm_source=huggingface&utm_medium=referral&utm_campaign=hf_minicpm_v_4_7_1b&utm_content=bullet_app">Atomic Chat</a>.</li>

<li>See our quantization analysis below for measurements and instructions.</li>

</ul>

<hr style="margin: 0 0 16px 0;">

Pick a file

Every number below is measured, not estimated. How we measured it is at the

bottom, and the raw logs are in

the metrics repo

so you can check any of it yourself.

KL divergence is how far the quantized model's predictions drift from the

original weights. Lower is better, and zero means identical. top-1 is how

often it picks the same next word the original would have picked.

| File | Size | KL divergence | top-1 |

|---|---:|---:|---:|

| Q8_0 | 1,082 MB | 0.00348 | 96.64% |

| AD-Q6_K | 1,003 MB | 0.00630 | 95.51% |

| Q6_K (plain preset) | 838 MB | 0.01686 | 92.66% |

| AD-Q5_K_M | 735 MB | 0.04418 | 89.08% |

| AD-Q5_K_M-Q4_K_M | 693 MB | 0.07105 | 85.97% |

| AD-Q4_K_M | 666 MB | 0.09674 | 83.70% |

| AD-IQ4_XS | 654 MB | 0.10690 | 82.89% |

| AD-IQ4_XS-IQ3_S | 587 MB | 0.19859 | 76.71% |

A 1B model loses more per bit than a large one: even Q8_0 agrees with the original on 96.6% of tokens

here. Below AD-IQ4_XS-IQ3_S the files drop under 75% top-1, so they are not published (their numbers

are in the metrics repo).

AD- marks an Atomic Dynamic layout. The name says what the two largest

tensor groups got: AD-<ffn_down>-<ffn_up>, collapsed to one name when both

match. Nothing is named after a type it does not contain.

Against llama.cpp's plain presets

The same reference, held-out text, machine and llama.cpp build; the plain presets are made with no

importance matrix:

| Plain preset | Size | KL divergence | top-1 | AD file near that size | Size | KL divergence | top-1 |

|---|---:|---:|---:|---|---:|---:|---:|

| Q4_K_M | 672 MB | 0.19053 | 77.42% | AD-Q4_K_M | 666 MB | 0.09674 | 83.70% |

| Q5_K_M | 752 MB | 0.05753 | 87.06% | AD-Q5_K_M | 735 MB | 0.04418 | 89.08% |

| Q6_K | 838 MB | 0.01686 | 92.66% | AD-Q6_K-Q5_K | 875 MB | 0.01887 | 92.52% |

At 4 and 5 bits the AD files are smaller and drift half to three quarters as far. At 6 bits the plain

preset wins. Half of this model is the embedding and the output head (a 248k vocabulary on 1,024

dimensions, a quarter of the weights each), and our 6-bit layouts have not found a better split for

them yet. So Q6_K stays in this repo, and Q4_K_M, Q5_K_M and AD-Q6_K-Q5_K are only in this table.

Which one fits your card

The file has to fit, and so does the context. This model keeps 12 KB of

attention cache per token, which is 0.1 GB at 8k context and 0.4 GB at 32k. Budget

for both.

Every file here fits a 2 GB card with 32k of context.

Running it

llama-server -m MiniCPM-V-4.7-1B-AD-Q6_K.gguf -ngl 99 -c 8192 --jinja

The chat template is inside every file, llama.cpp applies it on its own. With --jinja it starts every answer in thinking mode; turn it off per request with "chat_template_kwargs": {"enable_thinking": false}.

Images and video

This model reads images, not only text. That needs one extra file, the vision

projector, which is separate from the quant and shared by all of them:

• mmproj-MiniCPM-V-4.7-1B-F16.gguf for most people
• mmproj-MiniCPM-V-4.7-1B-BF16.gguf if your setup prefers bfloat16

Download it once alongside whichever quant you picked:

llama-mtmd-cli \
  -m MiniCPM-V-4.7-1B-AD-Q6_K.gguf \
  --mmproj mmproj-MiniCPM-V-4.7-1B-F16.gguf \
  --image your-photo.jpg -ngl 99 -c 8192 \
  -p "What is in this image?"

The calibration data

Our importance matrix comes from a corpus we built for this model and

published: AtomicChat/calib-corpora,

recipe minicpm-v-4.7. It is 4,942,284 tokens across 3,107 documents:

| Part | Share |

|---|---:|

| agentic and tool use | 24.6% |

| code | 17.7% |

| reasoning | 14.7% |

| multilingual | 13.8% |

| long context | 12.2% |

| vocabulary sweep | 10.1% |

| structured data | 3.9% |

| graphics | 3.0% |

Every conversation is rendered through this model's own chat template, so

the chat markup appears as the single tokens the model actually sees.

> [!IMPORTANT]

> If you build your own importance matrix from this corpus, pass

> --parse-special to llama-imatrix. Without it the chat markup is read as

> ordinary punctuation.

How we measured

Reference: the original BF16 weights, converted to GGUF and run unquantized.

Perplexity on the held-out set is 15.9138 plus or minus 0.11294.

Held-out text: eval_neutral from our calibration dataset, never used for

calibration. 87 chunks at 4096 context.

Metric: per-token KL divergence of each file's predictions against the

reference, plus top-1 agreement.

Hardware: 4x H100 80GB HBM3.

The raw logits of the reference are published in the metrics repo, split into

parts because of the file size limit. With them you can measure your own build

against exactly the same point we did:

cat base-neutral.kld.*.part > base-neutral.kld

llama-perplexity -m your-quant.gguf -f eval_neutral.txt \
  --kl-divergence-base base-neutral.kld --kl-divergence -c 4096 -ngl 99

Reproducing a file

The importance matrix was collected on the BF16 weights rather than on a

quantized copy, in one run:

llama-imatrix -m MiniCPM-V-4.7-1B-BF16.gguf -f calib_train.txt -o imatrix.gguf \
  -ngl 99 -c 512 -b 8192 -ub 8192 --parse-special

Then, for AD-Q6_K:

llama-quantize \
  --imatrix imatrix.gguf \
  --tensor-type '^blk\.(0|1|19|20|21|22|23)\.ffn_(down|gate|up)\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.ffn_down\.weight$=q6_k' \
  --tensor-type '^blk\.\d+\.ffn_gate\.weight$=q6_k' \
  --tensor-type '^blk\.\d+\.ffn_up\.weight$=q6_k' \
  --tensor-type '^blk\.\d+\.attn_(q|qkv)\.weight$=q6_k' \
  --tensor-type '^blk\.\d+\.attn_gate\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.ssm_out\.weight$=q8_0' \
  --tensor-type '^output\.weight$=q8_0' \
  --tensor-type '^blk\.\d+\.attn_output\.weight$=q8_0' \
  --tensor-type '^token_embd\.weight$=q8_0' \
  --override-kv general.file_type=int:18 \
  MiniCPM-V-4.7-1B-BF16.gguf MiniCPM-V-4.7-1B-AD-Q6_K.gguf Q8_0

The rules for every other file are in ladder/ in the metrics repo, next to

its quantization log. The first rule that matches a tensor wins.

Model details

24 layers, hidden size 1024, feed-forward size 3584, vocabulary 248,094, context 262,144.

Needs llama.cpp at or after 69f201a (MiniCPM-V 4.7 support, ggml-org/llama.cpp#29416); older builds run text only and get image positions wrong. Built and tested with llama.cpp b11574 (8bb3cf0d2).