---

license: apache-2.0

base_model:

• yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2
• google/gemma-4-12B-it

library_name: llama.cpp

pipeline_tag: image-text-to-text

tags:

• gguf
• gemma4
• multimodal
• vision
• image-to-text
• coding
• agentic
• terminal
• tool-use
• reasoning
• thinking
• speculative-decoding
• llama.cpp
• local-llm
• conversational

---

Swarm Scout 12B GGUF

<p align="center">

<img src="swarm-scout-card.jpg" alt="Swarm Scout mascot" width="520">

</p>

Swarm Scout 12B is a low-refusal, coding/agentic GGUF build derived from

yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2, itself based on google/gemma-4-12B-it.

The upstream model is specialized for coding, terminal workflows, native Gemma 4 tool-use, multi-step agentic tasks, and reasoning. This release keeps that focus, then applies a local Heretic/abliteration pass targeted at reducing opening refusals while preserving behavior as much as possible.

Brand note: this release is now named Swarm Scout. The Hugging Face repo slug and GGUF filenames still contain swarm-sovereign-scout so existing download commands, Modal caches, and copied links keep working.

Build With HivemindOS

Want to build and manage an entire private swarm of agents? HivemindOS gives you shared memory, reusable skills, one-time setup, and a superbrain architecture for coordinating agent work.

• App: HivemindOS
• X: @TheHivemindOS

Artifacts

| File | Size | Purpose |

| --- | ---: | --- |

| swarm-sovereign-scout-Q4_K_M.gguf | ~6.9 GB | Recommended local/runtime artifact |

| MTP/gemma-4-12B-it-MTP-Q8_0.gguf | ~444 MB | Draft model for llama.cpp MTP speculative decoding |

| mmproj-swarm-sovereign-scout-bf16.gguf | ~167 MB | Projector artifact for llama.cpp multimodal image input |

| chat_template.jinja | small | Swarm Scout identity/tool-use template override for llama.cpp |

The local workspace also produced a BF16 GGUF (swarm-sovereign-scout-bf16.gguf, ~22 GB). Q4_K_M is the recommended release/runtime target for speed and practicality.

What It Is Good At

• Coding and debugging assistance.
• Terminal-style agent tasks: read, reason, act, verify.
• Structured tool-use in Gemma 4's native protocol.
• Local agent loops where lower refusal tendency is desired.
• Reasoning-heavy technical work, especially with thinking enabled.

The upstream base reports a large relative gain on local tau2-bench telecom-style agentic tool-use versus google/gemma-4-12B-it; see the upstream model card for the exact methodology and caveats.

Abliteration / Refusal Check

Local verified artifact:

• Winning Heretic trial: 20
• Heretic proxy refusals: 0/6
• KL divergence: 0.008679866790771484
• Merged HF strict refusal check: 0/6 strict full-response refusals
• Served Q4_K_M llama.cpp strict refusal check: 0/6 strict full-response refusals

The strict check measures opening/full-response refusal behavior on a small local prompt set. It is not a broad safety benchmark and should not be treated as a guarantee of behavior in production. Add application-level guardrails where needed.

Speculative Decoding

This release includes the Gemma 4 MTP draft model used for DSpark-style speculative decoding in llama.cpp.

Verified local build:

• llama.cpp b9553, commit 9e3b928fd8c9d14dbf15a8768b9fdd7e5c721d66
• Draft type: draft-mtp
• Recommended Scout settings: --spec-draft-n-min 0 --spec-draft-n-max 2
• Draft file: MTP/gemma-4-12B-it-MTP-Q8_0.gguf

Verified Modal build:

• llama.cpp image: ghcr.io/ggml-org/llama.cpp:server-cuda-b9870
• GPU: Modal A10
• Server command uses -md MTP/gemma-4-12B-it-MTP-Q8_0.gguf --spec-type draft-mtp
• Production smoke returned draft_n / draft_n_accepted, confirming draft tokens were active.
• Hosted endpoint auth: Modal proxy auth is enabled. Server-side callers must send Modal proxy headers; do not expose those credentials in browser/mobile clients.

Older llama.cpp builds may not understand the Gemma 4 assistant draft architecture. For example, server-cuda-b9544 sees CUDA but fails to load this draft with unknown model architecture: 'gemma4-assistant'.

Clean local Apple Silicon measurements with reasoning disabled:

| Prompt set / config | Baseline tok/s | MTP tok/s | Speedup | Acceptance |

| --- | ---: | ---: | ---: | ---: |

| Mixed, n=0..2 | 48.12 | 70.41 | 1.46x | 0.75163 |

| Code, n=0..2 | 33.96 | 47.93 | 1.41x | 0.69159 |

| Mixed, n=1..3, p_min=0.05 | 37.54 | 60.14 | 1.60x | 0.63131 |

Use absolute tokens/sec as the tuning objective. The 1.60x row exists, but n=0..2 produced better real throughput on the local machine.

llama.cpp Usage

Download the model and draft:

hf download LiamVisionary/swarm-sovereign-scout-12b-GGUF \
  swarm-sovereign-scout-Q4_K_M.gguf \
  MTP/gemma-4-12B-it-MTP-Q8_0.gguf \
  mmproj-swarm-sovereign-scout-bf16.gguf \
  chat_template.jinja \
  --local-dir swarm-scout-12b

Run an OpenAI-compatible server:

llama-server \
  -m swarm-scout-12b/swarm-sovereign-scout-Q4_K_M.gguf \
  -md swarm-scout-12b/MTP/gemma-4-12B-it-MTP-Q8_0.gguf \
  --mmproj swarm-scout-12b/mmproj-swarm-sovereign-scout-bf16.gguf \
  --spec-type draft-mtp \
  --spec-draft-n-min 0 \
  --spec-draft-n-max 2 \
  -ngl 99 \
  -ngld 99 \
  -fa on \
  --jinja \
  --chat-template-file swarm-scout-12b/chat_template.jinja \
  --reasoning auto \
  --reasoning-format deepseek \
  --alias swarm-scout-12b \
  --host 127.0.0.1 \
  --port 8080

Then call:

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "swarm-scout-12b",
    "messages": [{"role": "user", "content": "Write a small Python LRU cache."}],
    "max_tokens": 512,
    "temperature": 0
  }'

Image input uses OpenAI-compatible content parts:

{
  "model": "swarm-scout-12b",
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "Read this image and summarize it."},
        {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
      ]
    }
  ],
  "max_tokens": 256,
  "temperature": 0
}

Reasoning

For fast agent loops, run with reasoning off:

--reasoning off --reasoning-format none

For harder debugging/planning, use:

--reasoning auto --reasoning-format deepseek

Compatible clients may also pass thinking_budget_tokens and chat_template_kwargs.enable_thinking per request.

Multimodal Status

The underlying Gemma 4 unified configuration includes image/video/audio processor metadata. The GGUF text model requires the matching mmproj-swarm-sovereign-scout-bf16.gguf projector for multimodal serving in llama.cpp.

Verified on 2026-07-05:

• Local llama.cpp b9553 loaded swarm-sovereign-scout-Q4_K_M.gguf plus mmproj-swarm-sovereign-scout-bf16.gguf.
• Modal llama.cpp server-cuda-b9870 advertised capabilities: ["completion", "multimodal"].
• A receipt image test returned Merchant: SWARM CAFE. Total: $8.37. through the production OpenAI-compatible endpoint.

Image input is verified. The projector metadata also exposes an audio encoder, but audio/video input was not separately quality-tested for this release.

Caveats

• Specialized for coding, terminal, tool-use, and local agentic work.
• General-knowledge answers should be checked.
• Low-refusal behavior is intentional; production applications should add their own policy and abuse-prevention layers.
• English-centric.
• Speculative decoding support is build-sensitive. The local verified path used llama.cpp b9553; the Modal verified path used server-cuda-b9870; newer/older builds may need their own validation.

Lineage

• Base model: google/gemma-4-12B-it
• Agentic/coding base: yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2
• GGUF/MTP reference repo: yuxinlu1/gemma-4-12B-agentic-fable5-composer2.5-v2-3.5x-tau2-GGUF

Released under Apache 2.0, following the upstream/base lineage.