---

license: apache-2.0

base_model: frontier-infra/jebadiah-9b-v2

base_model_relation: quantized

library_name: gguf

language:

• en

tags:

• gguf
• llama.cpp
• decision-model
• system-one
• typed-decisions
• calibrated-probabilities
• ainode

---

> Just want to run it? Download jebadiah-9b-v2-1-GGUF and open it in llama.cpp, Ollama or LM Studio.

>

> Which one should I use? Start with the 9B. It's smaller and faster, and it's what most people run. Pick the 27B if you have the memory and want the higher score (Decision Index Full 55.94 against the 9B's 44.91, board data of 2026-10-15). On Apple silicon, use the MLX builds of the same models. You can try the 4B in the browser first: jebadiah-playground.

>

> This repo: the 9B v2 GGUF quants (Q4_K_M, Q5_K_M, Q8_0), for llama.cpp, Ollama and LM Studio.

This is an earlier version. For new work, use v2.1 above.

Jebadiah 9B v2 GGUF

Jeb is a decision model, not a chat model. Use jeb serve or the decide scripts (see Run Jeb locally on jebadiah.ai). In an LM Studio or Ollama chat window it will look wrong.

Trying it in a chat window? Use the optional chat profiles: https://github.com/getainode/jebadiah/blob/main/docs/chat-window.md

Official builds are under frontier-infra on Hugging Face (always frontier-infra/jebadiah-..., see jebadiah.ai); the Jebadiah org there is unrelated.

GGUF builds of Jebadiah 9B v2 for llama.cpp, which runs on NVIDIA, AMD and

Apple GPUs and on plain CPUs. Jebadiah answers a typed question (choice, noul or score) with a probability for

every option, read from one forward pass. Nothing is generated. Code, trainer and evals:

getainode/jebadiah.

Results and docs

• Project site, with every result and how to run the models: jebadiah.ai.
• JevBench v1.4.2: on its 231 public items, run through its own harness, Jebadiah 9B v2 scores 0.818 (Jebadiah 27B: 0.866, the same as Jev 1.13.0). This is my own run on the public items, not the official board, which also uses sealed items. Details and caveats.
• JDE blind test (as of 2026-09-26): on 290 real decisions from Titanium Computing's production decision engine, Jebadiah 9B v2 got 277 against Jev's 282, with 0 flips across 5,800 repeat calls. Jev still leads on coverage checks.
• For the family: Decision Index 0.2.1: the 27B scores 54.67, No. 5 of 70 open models (board data generated 2026-09-28). This is the board's own number; the maintainer validated my run and put it on the leaderboard. Run record.
• Decision Index 0.3: we submitted complete public runs, 27B at 55.11 and 9B v2 at 44.09, in apolinario/decision-index PR 96. They are not yet ranked.
• All sizes: the Hugging Face collection, mirrored on ModelScope.

Which one should I use? For local use, start with Jebadiah 9B v2 GGUF. On Apple silicon, use an MLX build: 27B, 9B v2 or 4B v2. For vLLM or fine-tuning, use the full weights: 27B, 9B v2 or 4B v2. Setup for every runtime, and for JDE: Run Jeb locally.

Files

Every file was checked on the 260 held-out questions the merged weights were checked on, and compared with

the merged bf16 weights and with the training run's own eval records.

| File | Size | Same answer as bf16 | Same as the run | choice + noul | score | Prob. diff median / max |

|---|---:|---:|---:|---:|---:|---:|

| jebadiah-9b-v2-Q8_0.gguf | 9.8 GB | 257 / 260 | 256 / 260 | 171 / 173 | 85 / 87 | 0.003 / 0.051 |

| jebadiah-9b-v2-Q5_K_M.gguf | 6.6 GB | 250 / 260 | 251 / 260 | 169 / 173 | 82 / 87 | 0.012 / 0.252 |

| jebadiah-9b-v2-Q4_K_M.gguf | 5.8 GB | 240 / 260 | 239 / 260 | 164 / 173 | 75 / 87 | 0.022 / 0.415 |

| bf16 weights | | | 257 / 260 | 172 / 173 | 85 / 87 | 0.002 / 0.029 |

Which one: Q8_0 if it fits (it changed 3 answers here). On an 8 GB card, Q5_K_M (6.6 GB plus about

1 GB of context fits, Q8_0 does not): none of its 10 changed answers was a confident one, and it got 174 of

260 right against gold, the same as bf16. Q4_K_M only when Q5_K_M does not fit. A file needs about its

own size in GPU or unified memory, plus about 1 GB for a 4k context. Q5_K_M and Q8_0 differ here by one

right answer in 260, too few to rank Q5_K_M above Q8_0.

Q5_K_M changes 10 of 260 answers against bf16 (7 on score questions) and moves probabilities more (median 0.012, max 0.25). None of those changes had a top-2 gap above 0.1 (see the gold scoring below).

Q4_K_M changes 20 of 260 answers against bf16 (10 on score questions) and moves probabilities more (median 0.022, max 0.41). Use it only when a larger build does not fit.

Quantization changes are not all tie-breaks

dipankarsarkar pointed out on the Hugging Face post that the Q4_K_M changes are not near-ties, so I scored all 260

questions against their gold labels (every one has one). Q8_0's 3 changes sit at top-2 gaps of 0.038, 0.0056 and

0.0013 in the file's own probabilities. Q4_K_M's do not: 9 of its 20 changes have a gap above 0.1, and those 9 go

2 toward the gold answer, 6 away from it, and 1 where both answers are wrong. The clearest are PubMedQA 217 and 895:

the gold answer is false, bf16 says false, and Q4_K_M says true at 0.69 and 0.70.

| Build | Changed answers | Confident changes (gap > 0.1) | Toward gold | Away from gold | Accuracy on the 260 | Abstain below 0.05 |

|---|---:|---:|---:|---:|---:|---|

| bf16 | | | | | 174 (66.9%) | answers 224, 71.4% right |

| Q8_0 | 3 | 0 | 0 | 1 | 173 (66.5%) | answers 223, 71.3% right; skips all 3 changes |

| Q5_K_M | 10 | 0 | 3 | 3 | 174 (66.9%) | answers 225, 71.6% right; skips 9 of 10 changes |

| Q4_K_M | 20 | 9 | 5 (2 confident) | 10 (6 confident) | 169 (65.0%) | answers 228, 68.9% right; skips 9 of 20 changes |

The other changes have both answers wrong. Accuracy is exact match: one of 77 intents on Banking77 and the exact 0 to 4

score on HelpSteer2, which is why it sits well below the agreement numbers. Gold is the Jevals suite label for PubMedQA,

Banking77 and HelpSteer2 and Nimble's reference label. "Abstain below 0.05" skips a question when its top two options

are within 0.05. The bf16 row uses the training run's bf16 probabilities, because the merged weights' probabilities are

not in the records (the picks match on 257 of 260 and the probabilities differ by at most 0.029). So a confidence gate

cleans up Q8_0 but not Q4_K_M: if you have the memory, use Q8_0. Per-question results: eval/gold-scoring.json.

Run it

The answer is the log probability of each option label ("A", "B", ...) at the answer position, which

llama-server's /completion returns. The script renders the prompt exactly as AINode does, sends the raw

text (so the server's own chat template is never used), renormalises over the labels and applies

temperatures.json (choice 1.1863, noul 1.0903, score 0.8329). You need a llama.cpp that knows the qwen35 architecture: we checked

v0.5.0 (older builds refuse the file).

hf download frontier-infra/jebadiah-9b-v2-GGUF --include "*Q8_0.gguf" --include "scripts/*" --include "*.json" --include "*.jinja" --include "*.txt" --local-dir jebadiah-9b-v2-GGUF
cd jebadiah-9b-v2-GGUF
llama-server -m jebadiah-9b-v2-Q8_0.gguf -c 4096 -np 1 --port 8080
pip install transformers        # the tokenizer only, no torch
python scripts/decide_gguf.py --server http://127.0.0.1:8080 --request scripts/example-request.json

--no-temperatures returns the raw probabilities. For Ollama and LM Studio, see the next sections.

HelpSteer2-like traffic. A held-out HelpSteer2 check (2026-09-29): on the 418 nvidia/HelpSteer2 validation rows that no reported evaluation set uses (natural label mix, never in the training pool), this model's score answers want a temperature of 1.29, not 0.83, and still 1.11 after reweighting to a flat label mix. If your traffic looks like HelpSteer2, keep the older score temperature, 1.22 (the train fit in temperatures.json; every script here takes --temperatures with a file of your own), or refit on your own labels. v3's calibration split will draw HelpSteer2 from held-out data.

On example-request.json (jebadiah-9b-v2-Q8_0.gguf):

{
 "route": {"type": "choice", "choice": "billing", "confidence": 0.461713, "probabilities": {"billing": 0.641142, "support": 0.036095, "sales": 0.322763}},
 "urgent": {"type": "noul", "noul": 0.167016}
}

Use it in Ollama

pip install jebadiah-decide
jeb doctor         # pulls hf.co/frontier-infra/jebadiah-9b-v2-GGUF:Q8_0 into Ollama the first time, checks it, answers the example
jeb serve          # POST http://localhost:8100/v1/systemone

jeb serve renders the prompt, calls Ollama raw with thinking off, reads the label probabilities and applies

temperatures.json; it serves AINode's /v1/systemone and /v1/decide on localhost for JDE or your code.

Don't use ollama run or the chat window: Jeb isn't a chat model, and what it writes there is nonsense by

design. Tested with Ollama 0.34.4: 257 of 260 answers the same as bf16, and the same

answer as llama-server on all 260. At most 20 options per question. scripts/decide_ollama.py does the same

for one request file. Every runtime, step by step: Run Jeb locally.

Use it in LM Studio

Jeb works in LM Studio through its local server, not the chat window: chat runs with thinking on and shows

text, while a decision needs the probability of every option label. scripts/decide_lmstudio.py takes the same

arguments and prints the same output as decide_gguf.py. It sends AINode's messages to LM Studio's

/v1/chat/completions with thinking off ("reasoning_effort": "none"), where LM Studio renders the same prompt

text the llama-server path sends, and reads the option labels from the top log probabilities that come back. It

stops with an error if LM Studio's prompt length differs from the local tokenizer's.

1. In LM Studio, search for jebadiah-9b-v2 and download jebadiah-9b-v2-Q8_0.gguf from this repository.
2. Open the Developer tab, start the server and load the model. Note the identifier LM Studio shows for it

(for example jebadiah-9b-v2).

1. In a terminal:

hf download frontier-infra/jebadiah-9b-v2-GGUF --include "scripts/*" --include "*.json" --include "*.jinja" --include "*.txt" --local-dir jebadiah-9b-v2-GGUF
pip install transformers        # the tokenizer only, no torch
python jebadiah-9b-v2-GGUF/scripts/decide_lmstudio.py --model jebadiah-9b-v2 --request jebadiah-9b-v2-GGUF/scripts/example-request.json

If Require Authentication is on in LM Studio's server settings, create a token there and

export LM_API_TOKEN=... first.

Tested on the 9B only: LM Studio 0.4.21 with jebadiah-9b-v2-Q8_0, 257 of 260 answers the same as bf16, and the

same answer as llama-server on the same file on 260 of 260. The 27B and 4B builds use the same script but have not been run in LM Studio.

At most 20 options per question. LM Studio returns only the top 20 log probabilities, the same cap AINode's own

route has. On the 77-option Banking77 questions the pick was still right, but the probabilities moved by up to

0.16, so do not rely on them past 20 options.

If LM Studio shows a "Vision" tag, ignore it. Some GGUF repositories, including third-party quants of Jeb, ship a vision file (mmproj) from the Qwen base, and LM Studio labels the model Vision because of it. Jeb was not trained on images, and it does not make decisions in the chat window. It decides only through decide_gguf.py or decide_lmstudio.py, which send the typed question with thinking off and read the probabilities. The mmproj is not needed.

Community quantizations

Thanks to bartowski and mradermacher for building GGUF quantizations of this model:

• bartowski/frontier-infra_jebadiah-9b-v2-GGUF
• mradermacher/jebadiah-9b-v2-GGUF

These are independent builds. The agreement numbers on this card are for the files in this repository, not for these. Whichever GGUF you use, Jeb decides through jeb serve or the scripts above, with thinking off; it is not a chat model, so the chat window will give odd results.

How it was measured

Jevals PubMedQA, Banking77 (77 options) and HelpSteer2, plus Nimble: the merge check's fixed sample (seed

20260925), the run's option order and the run's temperatures, so the probability differences compare like with like (the shipped temperatures.json since 2026-09-29 uses 0.8329 for score; top picks do not depend on it). "Same answer" is the top option; "prob. diff" is the

largest change on any option against the run's CUDA record. llama-server ran on Metal (M3 Ultra) with the same tokens as the Python renderer on every prompt. Records: eval/agreement-*.json.

Images

The vision tower is inherited unchanged from the Qwen3.5 base (mradermacher's GGUF ships an mmproj, so images load), but Jeb was trained and calibrated on text only: decisions on images are untested and the fitted temperatures do not apply to them.

License

Apache-2.0, as the base model. Made in Texas.

PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt and https://jebadiah.ai/support/agent.md.

Support: help@myagents.email (Monday to Friday, 9 AM to 5 PM Central).

Try Jeb locally

Install the client for your existing runtime:

pip install jebadiah-decide

The local setup guide covers Ollama, LM Studio, llama-server, vLLM, MLX and AINode. Pick your runtime there before starting the server.

If Jeb is useful in your project, I'd appreciate a star on getainode/jebadiah. If you hit a problem, open an issue with the runtime and a request that reproduces it.