---

language:

• en

pipeline_tag: text-generation

library_name: llama.cpp

tags:

• gguf
• mixtral
• moe
• frankenmoe
• mergekit
• quantized

base_model: Petrouil/FrankenCPM-4x1B-A2B

base_model_relation: quantized

---

FrankenCPM-4x1B-A2B-GGUF

GGUF quantized versions of Petrouil/FrankenCPM-4x1B-A2B — a domain-specialized 4x1B MoE model built from MiniCPM5-1B via MergeKit.

4 experts (3 fine-tuned + base), 2 active per token — 2.61B total, ~1.3B active. Specializes in agentic tool use, code generation, and multi-step reasoning.

Available Quantizations

| Quantization | Size | Quality | Recommended For |

|---|---|---|---|

| Q4_K_M | ~2.4 GB | Good | CPU-only, 4GB VRAM |

| Q5_K_M | ~2.8 GB | Very Good | 6GB VRAM |

| Q6_K | ~3.2 GB | Excellent | 8GB VRAM |

| Q8_0 | ~4.2 GB | Near-lossless | 10GB+ VRAM |

Note: Sizes are approximate for the MoE model. Actual sizes depend on the number of experts and quantization method.

Hardware Requirements

• Minimum: 4GB VRAM (Q4_K_M) or 8GB RAM for CPU inference
• Recommended: 8GB+ VRAM for Q6_K/Q8_0, or 16GB+ RAM for CPU
• Context: Supports up to 32K context length

Usage

llama.cpp

# Clone and build llama.cpp
git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make -j

# Run inference
./llama-cli -m FrankenCPM-4x1B-A2B-Q6_K.gguf \
  -p "Write a Python function to find all prime numbers up to n." \
  -n 512 \
  --temp 0.7 \
  -ngl 99

Ollama

# Create a Modelfile
cat > Modelfile << 'EOF'
FROM ./FrankenCPM-4x1B-A2B-Q6_K.gguf

PARAMETER temperature 0.7
PARAMETER num_ctx 8192

SYSTEM "You are a helpful coding assistant."
EOF

# Create the model
ollama create frankencpm-4x1b -f Modelfile

# Run it
ollama run frankencpm-4x1b

LM Studio

1. Download the GGUF file from this repository
2. Open LM Studio and go to the Models tab
3. Click "Import Model" and select the GGUF file
4. Select the model and start chatting

Jan

1. Download the GGUF file
2. Open Jan and go to Settings > Models
3. Click "Add Model" and select the GGUF file
4. Configure context length and GPU layers as needed

Model Details

| Property | Value |

|---|---|

| Architecture | Mixtral-style MoE |

| Base Model | openbmb/MiniCPM5-1B |

| Total Parameters | ~2.61B |

| Active Parameters / Token | ~1.3B |

| Experts | 4 (Agent, Coding, Reasoning, General) |

| Experts per Token | 2 |

| Context Length | 32,768 |

| Quantization | GGUF (llama.cpp) |

| License | Apache 2.0 |

Expert Composition

| Expert | Domain | Specialization |

|---|---|---|

| 0 — Agent | Agentic tool use | Multi-turn tool calls, bash, read, edit, write |

| 1 — Coding | Code generation | Read/write/edit code, refactoring |

| 2 — Reasoning | Multi-step reasoning | Math proofs, logic puzzles, analysis |

| 3 — General | General purpose | Conversational, Q&A, summarization |

Model Family

openbmb/MiniCPM5-1B
  └── Petrouil/FrankenCPM-4x1B-A2B (bf16)
        └── Petrouil/FrankenCPM-4x1B-A2B-GGUF (this repo)

License

Apache 2.0 — same as the base bf16 model.

Citation

@software{frankencpm2026,
  title  = {FrankenCPM-4x1B-A2B: A Domain-Specialized Mixture-of-Experts Model},
  author = {Petrouil},
  year   = {2026},
  url    = {https://huggingface.co/Petrouil/FrankenCPM-4x1B-A2B}
}