---

license: mit

language:

- en

base_model: pinkelephantlimited/pinkelephant-llm-48b-s

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- moe

- mixture-of-experts

- sparse-moe

- llama.cpp

- ollama

- text-generation

- sft

- dpo

- reinforcement-learning-from-human-feedback

- direct-preference-optimization

- pink-elephant

- llm

- 48b

- enterprise

- on-device

- q4_k_m

- blackwell

extra:

lineage: "14B dense -> 48B MoE -> 48B-S -> SFT -> DPO -> merge -> GGUF"

---

<div align="center">

<img src="hero_banner.png" alt="Pink Elephant 48B-S GGUF — hero banner" width="100%">

🐘 Pink Elephant 48B-S — GGUF

Dense-14B brain. Sparse 48B reach. Aligned to be helpful. Run it on your own machine.

**From a single dense core to an SFT + DPO–aligned sparse Mixture-of-Experts flagship —

converted, quantized, and Ollama-ready so you can run the entire success story locally.**

!License: MIT

!Params: 47.7B

!Active: 30.7%-0ea5e9.svg)

!Experts

!Quant: Q4_K_M

!Context: 16k

!Pipeline: SFT+DPO

!Runtime: Ollama

</div>

---

🚀 At a glance

Pink Elephant 48B-S is the culmination of a six-stage lineage: a dense 14B foundation,

upcycled into an 8-expert sparse MoE, refined through a second training epoch (48B-S),

aligned with supervised fine-tuning and direct preference optimization, then merged

bit-exact and converted to GGUF — ready to run in Ollama / llama.cpp on a single

consumer GPU.

| What you get | Why it matters |

|---|---|

| 🧠 47.7B total params — only ~14.66B active / token (30.7%) | Dense-48B quality at a sparse fraction of the compute |

| ⚡ 3.2× fewer FLOPs vs. a dense 48B | Faster inference on the hardware you already own |

| 🎯 8 experts, top-2 routing | Specialized, focused reasoning per token |

| 🛡️ SFT + DPO aligned | Predictable, helpful, preference-tuned behavior |

| 📦 Q4_K_M quantized (8.3 GB) | Fits on an 8–12 GB consumer GPU |

| 🚀 Ollama-ready | One command from zero to running |

| 🆓 MIT license | Free for commercial use & research |

> ## ✅ Tested — corrected build runs clean

> We download the **corrected Q4_K_M GGUF (pe_48b_s_dpo_fix_Q4_K_M.gguf) fresh from this

> repo and ran it end-to-end in llama.cpp / Ollama**: it offloads to GPU, and produces

> coherent, on-topic, non-degenerate output. (The earlier 8-expert build lost the router

> bias and could garble; the shipped files are the corrected 2-expert build — see

> docs/VERIFICATION.md and tools/conversion/pinkelephant.py.) See the

> 📦 GGUF files table for exactly which hardware each file needs.

<p align="center">

<img src="figures/benchmarks.png" alt="Aligned 48B-S benchmark highlights" width="96%">

</p>

**Verified aligned benchmarks (bf16, greedy, single NVIDIA RTX PRO 6000 Blackwell — our own

infrastructure):**

• GSM8K: 68.5%
• MMLU (5-shot): 60.0%




> 🚀 Start here for clean output: docs/HOW_TO_RUN.md — correct ChatML

> template + sampler settings.

> 📘 Read the whole story: docs/ULTIMATE_REPORT.md — full

> traceability for investors, engineers, and users.

---

💡 Why a sparse MoE?

Pretraining a 48B model from scratch is expensive — and wasteful, because most of that compute

goes to reconstructing what a good dense 14B already knows. Pink Elephant took the smarter route:

• Start with a proven dense brain. A 14B foundation, instruction-tuned on code + math,

scoring HumanEval 71.34% and MATH-500 63.40%.

• Upcycle, don't retrain. Split the feed-forward weights into 8 complementary experts,

exactly preserving parent behavior at initialization — so you inherit the dense model's

knowledge instead of relearning it.

• Sparse compute, dense capacity. Only top-2 of 8 experts activate per token (~30.7%),

giving you 47.7B parameters of knowledge with 3.2× fewer FLOPs than a dense 48B.

The result: **more capability per flop, per watt, per dollar — on hardware you can actually

afford to own.**

---

🏆 The Success Journey

<p align="center">

<img src="figures/journey.png" alt="Pink Elephant success journey" width="98%">

</p>

One lineage, six milestones: a proven dense core → upcycled to sparse scale → refined →

supervised fine-tuned → preference-aligned → bit-exact merged and converted to GGUF.

---

🔄 The Lineage

<p align="center">

<img src="figures/lineage.png" alt="Pink Elephant lineage map" width="98%">

</p>

---

🎬 The Story, Act by Act

Act I — The Dense Core: 🐘 Pink Elephant 14B

> pink-elephant-llm-14b

A dense 14.66B decoder-only model, instruction-tuned on 99,661 curated code +

mathematics examples (bigcode/self-oss-instruct + open-r1/OpenR1-Math). It proved the

thesis:

• HumanEval pass@1: 71.34%
• MATH-500: 63.40%

Act II — The Upcycle: Dense 14B → Sparse 48B MoE

> pinkelephant-llm-48b

Rather than train 48B from scratch, Pink Elephant upcycled the 14B foundation into an

8-expert Mixture-of-Experts — feedforward weights split and replicated into eight

complementary experts per layer, *exactly preserving the parent's behavior at

initialization*. A sparse brain with dense-scale capacity at ~30.7% activation per token.

<p align="center">

<img src="figures/architecture.png" alt="48B-S MoE architecture" width="96%">

</p>

Act III — The Refinement: 48B-S

> pinkelephant-llm-48b-s

A second full epoch (3,114 steps) on the same curriculum with a freshly restarted cosine

schedule. Training loss fell 32% (0.0616 → 0.0419) with validation loss steady near 0.50 —

refinement, not overfitting.

<p align="center">

<img src="figures/48bs_training_loss.png" alt="48B-S training loss" width="62%">

</p>

Act IV — Supervised Fine-Tuning (SFT)

> pinkelephant-llm-48b-s-sft

An industrial-standard QLoRA recipe (frozen 4-bit NF4 base purely as a memory-efficient

training vehicle; all math in native bf16; adapters full precision) fine-tuned the whole

47.7B on a single GPU. Training loss fell 0.65 → ~0.44; held-out CE identified

step 90,000 as the best adapter (CE 0.5392).

| | |

|---|---|

| <img src="figures/sft_loss.png" width="340"><br>SFT training loss | <img src="figures/sft_heldout.png" width="340"><br>SFT held-out evaluation → best at step 90k |

Act V — Direct Preference Optimization (DPO)

> pinkelephant-llm-48b-s-dpo

An alignment campaign using a frozen reference + trainable policy (QLoRA-for-DPO). The

defining engineering decision: an evidence-based turning-point early stop — we watched the

loss curve, found its true minimum 0.4763 at step 11,579, then stopped 1,500 steps later.

A clean 2,000-pair held-out preference set shows ~0.65 preference accuracy.

<p align="center">

<img src="figures/dpo_loss.png" alt="DPO training loss with min-loss turning point" width="62%">

</p>

Act VI — The Merge & Conversion (This Repo) 🚀

> pinkelephant-llm-48b-s-gguf — you are here

We fused the best SFT adapter (step 90,000) + best DPO adapter (min-loss step 11,579)

into the 48B-S base — bit-exact — then converted the non-standard fused-layout MoE to

standard GGUF and quantized it.

base → moe → train → sft → dpo → verification → release

<p align="center">

<img src="figures/parameters.png" alt="48B-S parameter summary" width="96%">

</p>

---

🎯 Use cases

Because it is a compact, aligned, MIT-licensed sparse MoE that fits on consumer hardware,

Pink Elephant 48B-S GGUF is ideal for:

| Scenario | Why it fits |

|---|---|

| 🧑‍💻 Local code & math copilot | Trained on code + math; strong HumanEval / GSM8K; runs offline |

| 🔒 Private / on-prem deployment | 8.3 GB model — no data leaves your machine |

| 🏭 Edge & cost-sensitive serving | 3.2× fewer FLOPs than dense 48B; 30.7% activation saves power/cost |

| 📚 Research & experimentation | MIT license, fully reproducible lineage & verification |

| 🎓 Education / demos | One-command Ollama bring-up; the whole story is public & measured |

---

📌 Specifications

| Specification | Value |

|---|---|

| Total parameters | 47,691,290,048 (47.7B) |

| Active params / token | ~14.66B (30.7%) |

| Sparse compute advantage | 3.2× fewer FLOPs vs. dense 48B |

| Architecture | MoE · 8 experts · top-2 routing |

| Transformer layers / hidden / expert | 40 · 5,120 / 8,960 |

| Attention heads | 40 (10 KV heads, head dim 128) |

| Context window | 16,384 tokens |

| Vocabulary | 100,352 |

| Source precision | Native bfloat16 |

| Merge formula | W = W_base + 2·(SFT_B@SFT_A) + 2·(DPO_B@DPO_A) (r=32, α=64) |

| License | MIT — free for commercial use |

---

📦 GGUF files — pick the one that fits your machine

> ✅ Tested (corrected build): the Q4_K_M file below (pe_48b_s_dpo_fix_Q4_K_M.gguf) was

> downloaded fresh from this repo and run end-to-end in llama.cpp / Ollama — created as a

> model, offloaded to GPU, and produced coherent, non-degenerate output. It is the

> corrected 2-expert build (the earlier 8-expert build dropped the router bias and could

> garble; superseded). See docs/VERIFICATION.md and

> tools/conversion/pinkelephant.py for the fix.

| File | Quant | Size | Minimum hardware to run it | Best experience |

|---|---|---|---|---|

| pe_48b_s_dpo_fix_Q4_K_M.gguf | Q4_K_M | 8.3 GB | ≥ 16 GB RAM (CPU-only) or GPU with ≥ 8 GB VRAM for full offload | 8–12 GB GPU (full offload) for interactive speed; the recommended file for Ollama |

| pe_48b_s_dpo_fix.gguf | f16 (BF16) | 29.3 GB | ≥ 32 GB unified memory / VRAM (e.g. 32 GB or 2×16 GB GPU) | Full-precision reference for servers; needs a 32 GB-class host |

How to read this:

• Choose Q4_K_M to run on a typical modern machine. It's the file we verified consistently.

- 8–12 GB GPU → full GPU offload, interactive speed. Best choice for most users.

- No GPU / 16 GB RAM → runs on CPU, slower but works.

• Choose f16 only if you have a 32 GB-class unified host; otherwise use Q4_K_M.

> Note on quality/speed and size: these are the corrected 2-expert (effective) build of

> the SFT+DPO-aligned 48B-S. It ships only the live experts {0,4} (the 8-expert source has 6

> dead experts that were never routed-to), which is why the files are far smaller than the old

> 8-expert GGUF. Q4_K_M trades a small amount of precision for a much smaller file; f16

> preserves full fidelity. Both fix the router-bias bug from the earlier release.

---

🚀 Run it with Ollama

Prerequisite

Install Ollama from <https://ollama.com> and confirm it's running:

ollama --version

Option A — Pull directly from Hugging Face

ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M

Option B — Build from the bundled Modelfile

The repo ships a ready-made Modelfile with the correct ChatML template and safe sampler

settings (this matters — see ⚙️ Sampler & format notes below):

FROM ./pe_48b_s_dpo_fix_Q4_K_M.gguf

# Lower temperature + repeat_penalty 1.0 avoid degenerate/garbled output on this MoE.
PARAMETER temperature 0.3
PARAMETER repeat_penalty 1.0
PARAMETER top_p 0.9
PARAMETER num_ctx 4096

# ChatML (OpenHermes-2.5) template — REQUIRED for correct output.
TEMPLATE """{{- if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{- end }}<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

Download pe_48b_s_dpo_fix_Q4_K_M.gguf from the Files tab into this folder, then:

ollama create pe-48b -f Modelfile
ollama run pe-48b "Write a Python function that checks if a number is prime."

Chat via the API

curl http://localhost:11434/api/chat -d '{
  "model": "pe-48b",
  "messages": [{"role": "user", "content": "What is the capital of France?"}],
  "stream": false
}'

Python:

import requests
r = requests.post("http://localhost:11434/api/chat", json={
    "model": "pe-48b",
    "messages": [{"role": "user", "content": "Explain MoE in one sentence."}],
    "stream": False,
})
print(r.json()["message"]["content"])

⚙️ Sampler & format notes (read this before you run)

This model is ChatML / OpenHermes-2.5 aligned (SFT + DPO). Getting clean output depends on

two things: the chat template, and conservative sampling.

1. It must be prompted in ChatML form. The bundled Modelfile already sets the correct

ChatML TEMPLATE, so plain ollama run pe-48b "..." and /api/chat are handled for you.

If you bypass the template and send a bare/raw prompt (e.g. via /api/generate), you

must wrap it yourself, exactly like this (note the closing assistant tag):

```text

<|im_start|>user\nWhat is the capital of France?<|im_end|>\n<|im_start|>assistant\n

```

A bare prompt such as "The capital of France is" will not auto-complete correctly.

1. Use a low temperature and keep repeat_penalty at 1.0. The default Ollama sampler

(temperature ~0.7, repeat_penalty ~1.1) makes this model garble / repeat words.

The bundled Modelfile pins temperature 0.3, repeat_penalty 1.0, top_p 0.9, which

gives stable, non-degenerate output. For the most deterministic results you may set

temperature 0. Do not raise temperature much above 0.3 without re-testing.

1. Known robustness note (verified). The raw-completion path with an explicit ChatML

prompt (point 1) is the most reliable on this custom MoE — it produced the model's

exact correct answer deterministically. The Ollama /api/chat template path generally

works, but in some Ollama versions it can be less stable for this architecture; if you

ever see odd output via /api/chat, switch to the raw-completion form in point 1 with

"temperature": 0 (deterministic) as a reliable fallback.

> Tip: prefer the chat template (or the explicit ChatML raw-completion form) rather

> than bare completion for natural, instruction-following output from this SFT/DPO-aligned

> model.

🖥️ Hardware fit

• Q4_K_M (8.3 GB): runs fully offloaded on an 8–12 GB GPU (or CPU with ≥ 16 GB RAM). The

corrected 2-expert build is small enough for consumer-range cards.

• f16 (29.3 GB): needs ~32 GB VRAM / unified memory.

---

🔬 Engineering → fully reproducible in this repo

| Path | What it is |

|---|---|

| docs/BUILD_REPORT.md | Full methodology: merge, convert, quantize, verify |

| tools/conversion/pinkelephant.py | Custom converter — splits fused QKV, splits gate/up, stacks 8 experts → LLAMA arch |

| tools/pe_init.patch | One-line registry hook into llama.cpp |

| tools/make_mini_model.py | Cheap mini-model rig that proved the converter |

| scripts/merge_48b.py | Bit-exact SFT+DPO merge (max\|diff\| = 0.0 vs. recomputed) |

| scripts/verify_merge.py | Independent merge sanity check |

| scripts/run_convert.sh / quant_real.sh | GGUF conversion + Q4_K_M quantization |

🧪 Full verification report

See docs/VERIFICATION.md for the complete, measurement-by-measurement evidence:

bit-exact merge, all-403-tensor integrity, router top-2 routing, conversion fidelity, and the HF → Ollama round-trip.

✅ Verification highlights (measured)

• Bit-exact merge: merged == base + 2·SFT + 2·DPO with max|diff| = 0.000e+00.
• Tensor integrity: all 403 tensors (3 global + 40×10 per-layer) with exact shapes —

incl. 3D stacked expert blocks ffn_{gate,up,down}_exps[·,·,8].

• Router check: top-2 selected experts identical to source (overlap 1.0).
• Weights through conversion: o_proj, norms, qkv→q match source within f16 rounding.
• HF → Ollama round-trip: SHA-256 of the downloaded Q4_K_M matches the artifact that

generated valid answers in Ollama.

---

📚 The Pink Elephant Family

| Stage | Repo |

|---|---|

| 🐘 Dense 14B core | pink-elephant-llm-14b |

| 🔀 48B MoE (epoch-1) | pinkelephant-llm-48b |

| 💎 48B-S base | pinkelephant-llm-48b-s |

| 🎓 SFT adapters | pinkelephant-llm-48b-s-sft |

| 🛡️ DPO adapters | pinkelephant-llm-48b-s-dpo |

| 🚀 GGUF (this repo) | pinkelephant-llm-48b-s-gguf |

---

🎉 Get started now

# Fastest path — pull straight from Hugging Face
ollama pull hf.co/pinkelephantlimited/pinkelephant-llm-48b-s-gguf:Q4_K_M
ollama run pinkelephant-llm-48b-s-gguf "Explain mixture of experts like I'm five."

Or watch the whole story come alive in the Ultimate Project Report.

---

License & Commercial Use

MIT License — free for commercial and research use, modification, and redistribution.

Use at your own risk.

<div align="center">

---

*🦾 Pink Elephant Limited · Pink Elephant 48B-S GGUF · 2026 — from a 14B dense core to a

runnable, aligned MoE flagship, in one lineage.*


</div>