---

license: apache-2.0

base_model: mistralai/Leanstral-1.5-119B-A6B

base_model_relation: quantized

language:

- en

pipeline_tag: text-generation

library_name: gguf

tags:

- gguf

- rocm

- strix

- lean4

- moe

- deepseek2

- quantization

- amd

- rocmfp4

- strix-halo

- gfx1151

- ryzen-ai-max

- llama.cpp

---

Leanstral-1.5-119B-A6B — ROCmFP4 STRIX_LEAN (text-only GGUF) — AMD Ryzen AI Max+ 395 / Strix Halo / gfx1151

ROCmFP4 Strix-Lean quant of mistralai/Leanstral-1.5-119B-A6B for AMD Radeon / Ryzen AI Max+ (gfx1151) inference via the public charlie12345/ROCmFPX llama.cpp fork.

| Field | Value |

|---|---|

| Base model | mistralai/Leanstral-1.5-119B-A6B |

| License | Apache-2.0 (derivative redistributable) |

| Architecture | deepseek2 (DeepSeek-V3 MLA + MoE) |

| MoE | 128 routed experts / 4 active + 1 shared (~6.5B active / 119B total) |

| This file | Text-only LM GGUF (vision / Pixtral projector dropped) |

| Quant | Q4_0_ROCMFP4_STRIX_LEAN |

| Size | ~59.0 GiB (60420 MiB reported by quantizer, 4.26 BPW) |

| Source weights | Community BF16 sahilchachra/Leanstral-1.5-119B-A6B-BF16 (dequantized from Mistral FP8) |

Mistral ships Leanstral FP8-native without HF-format weights or an official GGUF. This quant is converted from the community BF16 transformers dump, then quantized with ROCmFP4 Strix-Lean kernels.

Files

| File | Notes |

|---|---|

| Leanstral-1.5-119B-A6B-Q4_0_ROCMFP4_STRIX_LEAN.gguf | Text-only deepseek2 GGUF |

| SHA256SUMS | Basename-only checksums |

Text-only note

The base model includes a Pixtral vision tower + multimodal projector. This GGUF is language-model only (vision tensors omitted at convert time). It is not a multimodal / vision checkpoint. A text-only deepseek2 GGUF of this model family is a known-good pattern (e.g. public Q4_K_M text-only releases).

Runtime (Ryzen AI Max+ 395 / gfx1151)

export LD_LIBRARY_PATH=/path/to/ROCmFPX/build/bin:/opt/rocm/lib
export HSA_OVERRIDE_GFX_VERSION=11.5.1
export GGML_HIP_ENABLE_UNIFIED_MEMORY=1

./llama-server \
  --host 127.0.0.1 --port 8099 \
  -ngl 999 -fa on -dio --no-warmup --jinja \
  --ctx-size 65536 \
  --cache-type-k q8_0 --cache-type-v q8_0 \
  --parallel 1 \
  --temp 1.0 --top-p 0.95 --min-p 0.05 \
  --model Leanstral-1.5-119B-A6B-Q4_0_ROCMFP4_STRIX_LEAN.gguf

Critical flags

• -dio is mandatory on large unified-memory loads. The mmap path can hang around ~120GB RSS with GTT stuck at 0.
• Use a ROCmFP4-capable build of charlie12345/ROCmFPX (or equivalent) that implements Q4_0_ROCMFP4_STRIX_LEAN.
• Load one ~60GB model at a time on 128GB unified memory.

Convert / quantize recipe

# 1) Convert HF BF16 → F16 GGUF (text-only / deepseek2)
#    Config must resolve as DeepseekV3ForCausalLM text path.
#    MoE experts ship pre-stacked as gate_up_proj / down_proj;
#    split gate_up as (n_exp, 2*n_ff, n_embd) → gate + up.
python convert_hf_to_gguf.py /path/to/leanstral-bf16 \
  --outtype f16 \
  --outfile Leanstral-1.5-119B-A6B-F16.gguf

# 2) Quantize Strix-Lean
./llama-quantize \
  Leanstral-1.5-119B-A6B-F16.gguf \
  Leanstral-1.5-119B-A6B-Q4_0_ROCMFP4_STRIX_LEAN.gguf \
  Q4_0_ROCMFP4_STRIX_LEAN 8

Verify: general.architecture == deepseek2.

A/B — ROCmFP4 STRIX_LEAN vs Q4_K_M (same machine / flags)

Hardware: Ryzen AI Max+ 395 (gfx1151), ROCm, 128GB unified memory.

Server flags identical: -ngl 999 -fa on -dio --no-warmup --jinja --ctx-size 65536 --cache-type-k q8_0 --cache-type-v q8_0 --parallel 1.

Baseline: public-style Q4_K_M text-only GGUF (~68 GiB). Gen cap 256 tokens.

| Prompt size (actual tokens) | Metric | STRIX_LEAN (this) | Q4_K_M | Δ |

|---|---|---:|---:|---:|

| ~7.6K (7616) | Prefill tok/s | 449.9 | 439.2 | +2.4% |

| ~7.6K (7616) | Decode tok/s | 37.41 | 36.85 | +1.5% |

| ~23.1K (23081) | Prefill tok/s | 184.3 | 182.2 | +1.2% |

| ~23.1K (23081) | Decode tok/s | 23.70 | 25.83 | −8.2% |

| — | File size | ~59 GiB | ~68 GiB | ~−13% |

Lean 4 quality spot-check

Both quants produce coherent Lean 4 for:

1. ∀ n : Nat, n + 0 = n — both emit valid Lean (induction / simp / Mathlib-style).
2. Harder: commutativity of Nat addition — both produce structured induction sketches; neither was formally typechecked here.

Did we lose anything?

• Size: clear win for STRIX_LEAN.
• Decode: essentially flat at ~8K context; ~8% slower at ~23K vs Q4_K_M on this run.
• Prefill: slight STRIX_LEAN edge (not the usual −12/−14% tradeoff seen on some other models).
• Quality: no obvious collapse; treat as comparable for Lean-agent use, with Q4_K_M slightly more concise on the easy lemma in this spot-check.

If your workload is decode-heavy at long context, keep Q4_K_M. If you want a smaller footprint on Strix unified memory with comparable short-context decode, this quant is a reasonable option.

SHA256

See SHA256SUMS (basename only).

Links

• Base: https://huggingface.co/mistralai/Leanstral-1.5-119B-A6B
• BF16 source used for convert: https://huggingface.co/sahilchachra/Leanstral-1.5-119B-A6B-BF16
• Runtime fork: https://github.com/charlie12345/ROCmFPX

<!-- CREDITS:START -->

Acknowledgements

This build would not exist without the work below. Please star and follow these

projects — the quantisation format used here is their engineering, not mine.

**ROCmFPX — maintained by

charlie12345 / caf**

The ROCmFP4 / ROCmFPX tensor formats (ggml types 100–106) exist only in this fork.

Every ROCmFP4 file in this repository was produced with its llama-quantize, and

runs on its runtime. The fork also credits collaborators ciru-ai, Tom Turney,

PlunderStruck and Aydan S., and acknowledges AMD for hardware support.

Licensed MIT, based on upstream llama.cpp.

llama.cpp — ggml-org and contributors

The inference engine, GGUF format and conversion tooling everything here is built on.

AMD ROCm

The compute platform these builds target — ROCm 7.2.4 on gfx1151 / Radeon 8060S.

Base model authors — see base_model in the metadata above; all model weights,

licences and capabilities are theirs. This repository contributes quantisation and

measurement only.

If you use these files, please credit ROCmFPX alongside this repository.

<!-- CREDITS:END -->