---

license: cc-by-nc-4.0

base_model: openjev/openjev

library_name: gguf

tags:

• gguf
• llama.cpp
• decision-model
• agents

language:

• en

---

OpenJev, GGUF

The GGUF files of OpenJev, the open 27B one-pass decision model for browser and desktop agents, quantized with llama.cpp for 24 GB cards, Macs and CPUs. Text only (no vision tower).

| file | bits | size | fits |

|---|---|---|---|

| OpenJev-Q4_K_M.gguf | ~4.8 | 16.5 GB | 24 GB cards (RTX 3090 / 4090), 32 GB Macs |

| OpenJev-Q5_K_M.gguf | ~5.7 | 19.2 GB | 24 GB cards with a short context |

| OpenJev-Q6_K.gguf | ~6.6 | 22.1 GB | 32 GB+ |

| OpenJev-Q8_0.gguf | 8.5 | 28.6 GB | 40 GB+, closest to the 16-bit weights |

Built from the merged 16-bit weights (revision 5ec9e5fd) with llama.cpp build b11147; SHA256SUMS and MANIFEST.json carry every hash and the validation numbers.

Run it

llama-server -m OpenJev-Q4_K_M.gguf -ngl 999 -c 16384 -np 2 --port 8080

Ask for a decision the way OpenJev was trained: state, question, lettered options, and read the letter.

curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "State:\nA checkout page shows: Subtotal $40, Shipping $5, a Place order button, and a Coupon field.\n\nQuestion: Which action completes the purchase?\nOptions:\n[A] click_coupon: type a coupon\n[B] click_place_order: click Place order\n[C] go_back: return to cart\n\nAnswer with the letter of the best option only."}],
  "max_tokens": 4, "temperature": 0, "chat_template_kwargs": {"enable_thinking": false}}'

For probabilities over every option use n_probs on /completion and read the letter tokens at the first output position; that is what the openjev-server helper does against vLLM and MLX. A llama.cpp backend for the helper is on the list.

Validation

Measured on the same 1,789 development rows and option letterings as the 16-bit model (accuracy / NLL); see validation/ for per-row records.

| build | accuracy | NLL | flip | agrees with 16-bit answer |

|---|---|---|---|---|

| 16-bit (vLLM reference, same rows) | 83.17 | 0.4195 | 4.3% | — |

| Q8_0 (llama.cpp) | 82.56 | 0.4207 | 4.7% | 98.2% |

| Q4_K_M (llama.cpp) | 82.84 | 0.4239 | 4.4% | 96.6% |

Paired against the 16-bit model on the same 1,789 rows: Q4_K_M −0.34 points (95% interval −1.06 to +0.39), Q8_0 −0.61 points (−1.17 to −0.11). Both sit within a point of the reference; the interval on 1,789 rows is too wide to certify the 0.5-point non-inferiority used for FP8, which needs the 10,000-question run. Readout here goes through llama.cpp's /completion top-64 token probabilities, not the vLLM helper.

Notes

• Yes/no questions are lettered [A] yes / [B] no in this build's prompts.
• Q4_K_M is the recommended file for 24 GB cards; leave room for the context.
• The FP8 and MLX builds keep the exact production readout; GGUF is the portable build.

Licence

Weights: CC BY-NC 4.0 for research and non-commercial use, with attribution, the same as the main repository. For a commercial licence, email support@loopai.com. The licence texts and the base model's attribution are in LICENSE, LICENSE-APACHE-2.0 and NOTICE.

OpenJev is an independent project, not affiliated with TypeSafe; Jev is their product.