owao/surogate-rune-26b-a4b-GGUF overview

---

library_name: llama.cpp

base_model: surogate/rune-26b-a4b-GGUF

tags:

- gguf

- gemma4

- moe

- decision

- systemone

- openjev

- llama.cpp

pipeline_tag: text-generation

---

> ## ⚠️ Not tested thoroughly! I indeed planned to publish the repo but not that soon, I ran out of private storage!

GGUF builds of the surogate/rune-26b-a4b

decision model (Gemma 4 MoE fine-tune), **patched with System One decision

metadata** so that llama.cpp's llama-server exposes a /v1/systemone

decision endpoint.

You need a llama.cpp build >= b11371.

The weights are unmodified. This is a metadata-only GGUF patch: five

key/value entries were appended to the file header, the tensor data section

was copied byte-for-byte, and the result was verified byte-exact (full tensor

offset chain recomputed and matched; embedded template compared against the

test suite that produced it).

What was patched

| GGUF key | value |

|---|---|

| gemma4.decision.type | openjev |

| gemma4.decision.temperature.choice | 2 |

| gemma4.decision.temperature.noul | 2 |

| gemma4.decision.temperature.score | 2 |

| tokenizer.chat_template.systemone | custom Jinja decision template (6,296 bytes) |

With these keys present, llama-server

auto-detects the model as an openjev decision model at load time:

• gemma4.decision.type — decision type selector (here openjev).
• gemma4.decision.temperature.<question type> — softmax temperatures for the

logits readout per question type (choice, noul, score).

• tokenizer.chat_template.systemone — the Jinja template that renders a

decision request into the model's native two-turn Gemma 4 chat prompt

(system + user turn, thinking disabled) ending exactly at the decision

position. The template implements the surogate decision protocol:

- system prompt instructing the model to treat the state as data, answer with

exactly one option letter, no reasoning;

- user turn carries SHARED STATE (JSON string), QUESTION (instructions),

and OPTIONS as A: <text> / B: <text> / ... lines;

- option text is the option description when given (string raw, anything

else JSON-serialized with Python-compatible float formatting), otherwise

the option key;

- option labels are A..Z then a..z (52 options max, all verified

single-token in this vocab);

- for noul questions, true is listed first, so the reported noul

probability is P(true).

The decision endpoint does no sampling: it renders the prompt, runs one

forward pass, and reads the raw logits at the decision token position over

the label tokens, then softmaxes with the per-type temperature.

Usage

Start a llama.cpp server (because nobody likes ollama):

llama-server \
  --model ggml-model-Q4_K_M.gguf \
  --jinja \
  --ctx-size 32000 \
  --host 0.0.0.0 --port 5810

Then send a decision request:

curl -X POST http://localhost:5810/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "state": {"weather": "rain", "outside_temp_c": 12, "task": "walk the dog"},
    "questions": {
      "take_umbrella": {
        "instructions": "Should I take an umbrella for the walk?",
        "type": "choice",
        "criteria": {"yes": "take the umbrella", "no": "leave it at home"}
      },
      "is_raining": {
        "instructions": "Is it raining right now?",
        "type": "noul",
        "criteria": {"true": "it is raining", "false": "it is not raining"}
      },
      "comfort": {
        "instructions": "How comfortable will the walk be?",
        "type": "score",
        "criteria": ["miserable", "okay", "great"]
      }
    }
  }'

Example response (live output from the Q4_K_M build):

{
  "answers": {
    "take_umbrella": {
      "type": "choice",
      "choice": "yes",
      "probabilities": {"yes": 0.9893468509151246, "no": 0.010653149084875283},
      "confidence": 0.9786937018302493
    },
    "is_raining": {
      "type": "noul",
      "noul": 0.9974877467638251
    },
    "comfort": {
      "type": "score",
      "score": 0.5339092615620311,
      "legend": {"0": "miserable", "1": "okay", "2": "great"},
      "probabilities": {"0": 0.4789796512912832, "1": 0.5081314358554024, "2": 0.012888912853314317},
      "confidence": 0.2621971537831036
    }
  },
  "usage": {"input_tokens": 417, "output_tokens": 0}
}

Notes:

• state is required and passed through verbatim into the prompt (it is

rendered as a JSON string). It may also be a chat-messages array

({"messages": [...]}), in which case image parts are supported.

• questions is an object mapping question ids to question objects

(instructions, type ∈ choice|score|noul, criteria).

• choice: criteria is an object (option key → description), max 52 options.
• score: criteria is an array of 2–10 level labels.
• noul: criteria is an optional object with true/false descriptions.
• All questions in a request share one rendered prompt prefix (prompt caching

across variants), so batch your questions per request.