---

license: other

license_name: lfm1.0

base_model: LiquidAI/LFM2-8B-A1B

base_model_relation: quantized

tags: [gguf, llama.cpp, rocm, gfx1151, strix-halo, amd, ryzen-ai-max-395, rocmfp4, rocmfpx, lfm2, liquid, moe]

language: [en]

pipeline_tag: text-generation

---

> ### ⚠️ STOCK llama.cpp WILL NOT LOAD THESE

>

> The ROCmFP4 / ROCmFPX quant types exist only in

> charlie12345/ROCmFPX — stock llama.cpp reports

> invalid ggml type. Ignore the auto-generated "Use this model" commands above.

LFM2-8B-A1B — ROCmFP4 / ROCmFPX GGUF

The first ROCmFP4 quantizations of LFM2-8B-A1B. Built for **AMD gfx1151

(Ryzen AI MAX+ 395 / Strix Halo). 24 layers · hidden 2048 · 32 experts**, ~1B active.

Quantized from BF16 GGUF — a lossless source, not a requantization of a lower-bit build.

All four quants — pick by size, not by hoping

| file | ftype | size | bpw | decode (median of 3) | runs | best for |

|---|---|---|---|---|---|---|

| Q4_0_ROCMFP4_COHERENT | 102 | 4.41 GiB | 4.54 | 146.61 | 150.64 / 146.32 / 146.61 | smallest — best fit for a 128 GB Strix Halo |

| Q6_0_ROCMFPX_AGENT | 114 | 7.14 GiB | 7.35 | 104.09 | 104.09 / 104.34 / 103.95 | agent/tool-call routing at 6-bit |

| Q8_0_ROCMFPX | 111 | 8.02 GiB | 8.26 | 102.97 | 103.77 / 102.91 / 102.97 | highest fidelity, plain routing |

| Q8_0_ROCMFPX_AGENT | 115 | 8.13 GiB | 8.37 | 104.31 | 104.31 / 103.87 / 104.78 | highest fidelity + agent routing |

Measured on a Ryzen AI MAX+ 395 (gfx1151, ROCm), -ngl 999 -c 4096 -fa on -fit off,

median of 3 with warm-up discarded on an otherwise-idle box.

Correctness: 3/3 on every quant — 17x23 => 391 - capital of Japan => Tokyo - days in 2024 => 366 (max_tokens 1024)

Speculative decoding: None. No MTP / EAGLE / nextn tensors, so there is no speculative speedup to enable and no acceptance length to quote. Source: unsloth/LFM2-8B-A1B-GGUF BF16.

> ℹ️ Why no "4-bit AGENT"? The AGENT recipes exist only at 3-bit (113), 6-bit (114/117) and

> 8-bit (115) — the entire 4-bit family (100–106) has no AGENT variant. Rather than hand-roll one

> and publish it under a name implying it is standard, the 6-bit Q6_0_ROCMFPX_AGENT fills that

> slot. AGENT routing keeps more tensors at higher precision for tool-call coherence.

> ℹ️ Head protection. These GGUFs contain no separate output.weight — the embedding is

> tied — so --token-embedding-type is what protects the head here and

> --output-tensor-type is a no-op. Audited in the finished files: token_embd is Q6_K on

> the 4-bit build and Q8_0 on the 6/8-bit builds.

---

What was NOT measured

• No perplexity run, and no quality A/B against the BF16 source.
• No long-context testing. · No tool-calling evaluation — note the AGENT builds are

named for tool-call routing but we did not benchmark tool use.

Base model licence inherited; all credit for the weights goes to Liquid AI.