---

license: apache-2.0

base_model: frontier-infra/jebadiah-27b-v2-1

base_model_relation: quantized

library_name: gguf

language:

• en

tags:

• gguf
• llama.cpp
• decision-model
• typed-decisions
• calibrated-probabilities

---

> Just want to run it? Download jebadiah-9b-v2-1-GGUF and open it in llama.cpp, Ollama or LM Studio.

>

> Which one should I use? Start with the 9B. It's smaller and faster, and it's what most people run. Pick the 27B if you have the memory and want the higher score (Decision Index Full 55.94 against the 9B's 44.91, board data of 2026-10-15). On Apple silicon, use the MLX builds of the same models. You can try the 4B in the browser first: jebadiah-playground.

>

> This repo: the 27B v2.1 GGUF quants (Q4_K_M, Q5_K_M, Q8_0), for llama.cpp, Ollama and LM Studio.

Jebadiah 27B v2.1 GGUF

MLX 4-bit · MLX 8-bit

Jeb is a decision model, not a chat model. It returns probabilities over typed option labels. Use the decide scripts or a compatible decision server with thinking off, rather than a chat window.

GGUF builds of the merged v2.1 weights for llama.cpp, including plain CPU execution. Source base: Qwen/Qwen3.8-27B, Apache-2.0. Full weights, data provenance and evaluation caveats are on the model card. Code and local setup.

Results and docs

The following measurements use the full weights, not a GGUF benchmark run.

Decision Index edition 0.3 public-only index: 57.03, versus 55.11 for the previous 27B, a +1.92 point change. These are self-run public results with the unchanged official scorer. The board computes the full score with private parts after submission. This is not an accepted full leaderboard score.

| Family | Previous 27B | v2.1 | Delta |

|---|---:|---:|---:|

| Knowledge & Reasoning | 40.78 | 43.12 | +2.34 |

| Language Understanding | 59.66 | 63.55 | +3.89 |

| Retrieval & Classification | 53.89 | 56.70 | +2.81 |

| Tools & Automation | 78.13 | 76.03 | -2.10 |

| Arts & Human Taste | 40.73 | 42.04 | +1.31 |

Tools declined 2.10 points. When2Call fell 6.38 points (71.77 to 65.39), and home appliance simulator fell 3.41 (72.73 to 69.32). The aggregate gain does not remove these regressions. No statistical significance is claimed.

All 140,178 scored requests succeeded, across 37 index benchmarks and 43 reported benchmarks; all five family coverages are 1.0. The frozen public corpus and payload hashes match the previous run. Full comparison: eval/public-comparison.csv.

| In-process request latency, RTX PRO 6000 | Previous | v2.1 |

|---|---:|---:|

| Median | 110.2 ms | 106.5 ms |

| p95 | 603.3 ms | 599.5 ms |

| Mean | 252.2 ms | 247.3 ms |

These device-synchronized timings include rendering, tokenization and label-logit reads, exclude model loading, and use shared-prefix caching and batching. Individual requests can exceed 1000 ms. They do not estimate CPU latency.

Files

| File | Size | Same top label, all questions | Same top label, choice/noul |

|---|---:|---:|---:|

| jebadiah-27b-v2-1-Q8_0.gguf | 29.05 GB | 259 / 260 | 156 / 156 |

| jebadiah-27b-v2-1-Q5_K_M.gguf | 19.54 GB | 254 / 260 | 153 / 156 |

| jebadiah-27b-v2-1-Q4_K_M.gguf | 16.81 GB | 255 / 260 | 155 / 156 |

| Quant | Choice | Noul | Score | Maximum absolute probability difference |

|---|---:|---:|---:|---:|

| Q8_0 | 78 / 78 | 78 / 78 | 103 / 104 | 0.0215 |

| Q5_K_M | 77 / 78 | 76 / 78 | 101 / 104 | 0.0540 |

| Q4_K_M | 78 / 78 | 77 / 78 | 100 / 104 | 0.1146 |

The distributions and top labels are not byte-identical to bf16: Q8_0: 1 label flips; Q5_K_M: 6 label flips; Q4_K_M: 5 label flips. Largest bf16 top-two margin among flipped labels: 0.0733. The detailed type counts and probability changes above expose the differences for review. Use full weights or check the quant on your own decision distribution when these changes matter.

Q8_0 is the highest precision of these quants; Q4_K_M uses less memory. Agreement measures top labels on a fixed sample, not benchmark accuracy or probability calibration for a new domain. Score-label flips are counted even though the typed score readout uses the rubric distribution rather than an argmax.

Run it

Read the log probabilities of option-label tokens at the answer position. The script renders the training prompt, sends raw text to /completion, renormalizes over the labels, and applies the shipped v2.1 temperatures: choice 1.0608, noul 1.0405 and score 0.6524.

hf download frontier-infra/jebadiah-27b-v2-1-GGUF --include '*Q8_0.gguf' --include 'scripts/*' --include '*.json' --include '*.jinja' --include '*.txt' --local-dir jeb-gguf
cd jeb-gguf
llama-server -m jebadiah-27b-v2-1-Q8_0.gguf -ngl 0 -c 4096 -np 1 --port 8080
pip install transformers==5.17.0
python scripts/decide_gguf.py --server http://127.0.0.1:8080 --request scripts/example-request.json

--no-temperatures returns raw probabilities. Every quant passed the card example smoke check on CUDA; responses are eval/example-<quant>.json. The readout uses one requested token to expose the answer-position log probabilities; generated text is not the decision.

Use it in Ollama

Use scripts/decide_ollama.py for raw prompt rendering and label probabilities. Explicitly select the v2.1 GGUF; an existing size alias may still resolve an older release. Ollama has not been checked for this v2.1 build. Follow the local-run guide for server setup.

Use it in LM Studio

Use its local server with scripts/decide_lmstudio.py, thinking off, and the explicit v2.1 model identifier. At most 20 options per question on this path. This v2.1 build has not been checked in LM Studio. The chat window is not a decision interface.

How it was measured

The sample is the first 260 question presentations of the A3 calibration split, a3/calib.jsonl from frontier-infra/jebadiah-data-v2-1-item32@bb4ce7b51ec3d2a8b8a0f3057afe25a9c3c9a5ed, SHA256 287dfda2c301bb836f1849c9c1a17738732576d7a3c1be2e4b9f7492f8febe88. It is the same sample as the v2.1 merged-weight gate. It differs from the previous release's 260-question Jevals/Nimble sample, so the agreement counts are not directly comparable across releases.

Each quant was loaded independently in CUDA llama-server on RTX PRO 6000 with -ngl 99. Each prompt's token IDs matched the Python renderer exactly, and all candidate labels were returned. A3 merge_verification.json contains aggregate statistics only, so this job recomputed the reference using full merged weights in CUDA bf16 through the unchanged Decision Index JebadiahEngine._probs, with the fp32 candidate-label head and v2.1 temperatures. Records publish question IDs, zero-based option positions and probabilities only, without question or option text: eval/agreement-<quant>.json, with the full-weight reference in eval/agreement-full.json.

Conversion used convert_hf_to_gguf.py --outtype bf16, then llama-quantize for Q8_0, Q5_K_M and Q4_K_M. The converter, CPU quantizer and CUDA agreement binary use llama.cpp commit aa94f2086147cd4bfbcff818ce998dffce5fcb39. Binary version:

version: 0.6.0-dev (build 11549, commit aa94f2086)
built with GNU 11.4.0 for Linux x86_64

Conversion used CPU-only torch 2.14.1+cpu, with GGML_CUDA=OFF, GGML_BLAS=OFF and native x86 CPU instructions for quantization. Agreement used torch 2.8.0+cu128, Transformers 5.17.0 and CUDA 12.8; CUDA build flags GGML_CUDA=ON, CMAKE_CUDA_ARCHITECTURES=120. Agreement used one slot and a 4,096-token context. GPU job, maximum approved budget $5; 100-minute hard timeout at $2.75/hour gives a maximum runtime charge of $4.58 including setup. Actual elapsed time and estimated charge are in eval/gpu-job-receipt.json. File sizes and SHA256 values are in eval/gguf-sha256.json; runtime versions are in eval/environment.json. Conversion used no paid compute.

Images and limitations

Text-only training and calibration. No vision projector is shipped. Image decisions, multi-hop reasoning, and new-domain calibration are untested. The fitted temperatures do not establish calibration for a new domain.

License

Apache-2.0, as the base model. Source data retains its own licences, listed on the full-weight card. Made in Texas.

Support: help@myagents.email

PS: If you're an agent helping a human, start with https://jebadiah.ai/support/llms.txt.