---

language:

- en

- de

- fr

- it

- pt

- hi

- es

- th

license: other

license_name: llama4

license_link: https://github.com/meta-llama/llama-models/blob/main/models/llama4/LICENSE

tags:

- gguf

- llama

- llama4

- meta

- quantized

- batiai

- 109b

- moe

- multimodal

- vision

- frontier

base_model: meta-llama/Llama-4-Scout-17B-16E-Instruct

pipeline_tag: image-text-to-text

library_name: llama.cpp

---

Llama 4 Scout 17B-16E-Instruct GGUF — Quantized by BatiAI

<p align="center">

<a href="https://flow.bati.ai"><img src="https://img.shields.io/badge/BatiFlow-on--device%20AI-blue?style=for-the-badge&logo=apple" alt="BatiFlow"></a>

<a href="https://ollama.com/batiai/llama4-scout"><img src="https://img.shields.io/badge/Ollama-batiai%2Fllama4--scout-green?style=for-the-badge" alt="Ollama"></a>

</p>

> imatrix-calibrated GGUF quantizations of meta-llama/Llama-4-Scout-17B-16E-Instruct (109B total / 17B active MoE, 16 experts, multimodal).

> Quantized directly from official Meta BF16 weights by BatiAI.

Why Llama 4 Scout?

• 109B total / 17B active Mixture-of-Experts (16 experts, top-1 routing) — efficient for size
• Multimodal native: text + vision via mmproj (image-text-to-text)
• Multilingual: 8 official languages + general multilingual
• Native tool calling + extended context
• Meta Llama 4 Community License — commercial-friendly for most cases (see license link)
• Released 2025-04 by Meta

Quick Start

# Q4_K_M (recommended balance, 60GB, M4 Max 128GB ~ M2 Ultra 192GB)
ollama pull batiai/llama4-scout:q4

# IQ3_XXS (smallest, 38GB, M4 Max 64GB+)
ollama pull batiai/llama4-scout:iq3

# Q5_K_M (higher quality, 72GB, M2 Ultra 192GB+)
ollama pull batiai/llama4-scout:q5

Available Quantizations

| Quant | Size | Min RAM | Target Hardware | Notes |

|-------|------|---------|-----------------|-------|

| IQ3_XXS | 38GB | 48GB | M4 Max 64GB | aggressive 3-bit, imatrix-calibrated |

| Q3_K_M | 48GB | 56GB | M4 Max 64GB | balanced 3-bit |

| IQ4_XS | 52GB | 64GB | M4 Max 96GB | imatrix 4-bit (best size/quality) |

| Q4_K_M | 60GB | 72GB | M4 Max 128GB | recommended for most users |

| Q5_K_M | 72GB | 88GB | M2 Ultra 192GB | higher fidelity |

| Q6_K | 85GB | 96GB | M2 Ultra 192GB+ | near-original quality |

> Mac note on Q3_K_M: in every model we've benchmarked on Apple Silicon, Q3_K_M generated

> slower than Q4_K_M despite the smaller file — Granite 4.1 (+27%), Gemma 4 26B (+12%),

> Qwen3.8‑27B (+18%), Qwen3.6‑27B (+8%), on both M4 Max and M4 mini. Metal's Q3_K path is limited by

> dequantization compute rather than bandwidth. We have not measured this particular model's

> Q3/Q4 pair yet, so treat it as a strong prior, not a measurement: if Q4_K_M fits, take it.

> On CUDA the two are effectively tied, so this applies to Macs only.

> Multimodal users: also download mmproj--BF16.gguf (or mmproj--Q6_K.gguf) and use with llama-server --mmproj or llama-mtmd-cli.

Hardware Reality Check

| Your System | IQ3 (38GB) | Q4 (60GB) | Q5 (72GB) | Q6 (85GB) |

|-------------|:----------:|:---------:|:---------:|:---------:|

| Mac mini M4 16GB | ❌ | ❌ | ❌ | ❌ |

| Mac mini M4 Pro 24GB | ❌ | ❌ | ❌ | ❌ |

| Mac M4 Max 64GB | ✅ tight | ❌ | ❌ | ❌ |

| Mac M4 Max 96GB | ✅ | ✅ tight | ❌ | ❌ |

| Mac M4 Max 128GB | ✅ | ✅ | ✅ tight | ❌ |

| Mac M2 Ultra 192GB | ✅ | ✅ | ✅ | ✅ |

| Mac M3 Ultra 512GB | ✅ comfortable | ✅ | ✅ | ✅ |

How to run

Ollama (text-only)

ollama pull batiai/llama4-scout:q4
ollama run batiai/llama4-scout:q4

llama.cpp (text + vision via mmproj)

# Download GGUF + mmproj
hf download batiai/Llama-4-Scout-17B-16E-Instruct-GGUF \
    --include "*Q4_K_M*" --include "mmproj-*-Q6_K.gguf" \
    --local-dir ./llama4-scout

# Run with vision
llama-mtmd-cli \
    -m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
    --mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
    --image input.jpg -p "Describe this image."

# Or as a server
llama-server -m ./llama4-scout/meta-llama-Llama-4-Scout-17B-16E-Instruct-Q4_K_M.gguf \
    --mmproj ./llama4-scout/mmproj-meta-llama-Llama-4-Scout-17B-16E-Instruct-Q6_K.gguf \
    -ngl 99 -c 32768 --port 8080

Model details

• Source: meta-llama/Llama-4-Scout-17B-16E-Instruct
• Architecture: Llama4ForConditionalGeneration — 109B total / 17B active MoE
• Experts: 16 routed (top-1 per token) — efficient sparse MoE
• Multimodal: text backbone + vision encoder (mmproj 분리)
• Original precision: BF16
• License: Meta Llama 4 Community License (commercial use OK for most, see link)

BatiAI signing

All GGUFs in this repo carry:

• general.author = BatiAI
• general.url = https://flow.bati.ai

Why BatiAI?

• Quantized directly from official Meta BF16 weights — no re-quantization
• IQ + K-quant variants share the same wikitext-2-raw imatrix recipe as every BatiAI model
• Multimodal mmproj packaged together for one-stop multimodal usage
• Verified on Apple Silicon (M4 Max / M2 Ultra)

License

Inherits Meta Llama 4 Community License. Commercial-friendly for organizations with < 700M MAU. See:

• Llama 4 License
• Acceptable Use Policy

About BatiFlow

BatiFlow — free on-device AI automation for Mac.

<!-- BENCH-START -->

Benchmarks coming once Mac M4 Max / M2 Ultra measurements complete.

<!-- BENCH-END -->