openjev/openjev-GGUF

OpenJev, GGUF

The GGUF files of OpenJev, the open 27B one-pass decision model for browser and desktop agents, quantized with llama.cpp for 24 GB cards, Macs and CPUs. Text only (no vision tower).

file	bits	size	fits
OpenJev-Q4_K_M.gguf	~4.8	16.5 GB	24 GB cards (RTX 3090 / 4090), 32 GB Macs
OpenJev-Q5_K_M.gguf	~5.7	19.2 GB	24 GB cards with a short context
OpenJev-Q6_K.gguf	~6.6	22.1 GB	32 GB+
OpenJev-Q8_0.gguf	8.5	28.6 GB	40 GB+, closest to the 16-bit weights

Built from the merged 16-bit weights (revision 5ec9e5fd) with llama.cpp build b11147; SHA256SUMS and MANIFEST.json carry every hash and the validation numbers.

Run it

llama-server -m OpenJev-Q4_K_M.gguf -ngl 999 -c 16384 -np 2 --port 8080

Ask for a decision the way OpenJev was trained: state, question, lettered options, and read the letter.

curl -s localhost:8080/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "messages": [{"role": "user", "content": "State:\nA checkout page shows: Subtotal $40, Shipping $5, a Place order button, and a Coupon field.\n\nQuestion: Which action completes the purchase?\nOptions:\n[A] click_coupon: type a coupon\n[B] click_place_order: click Place order\n[C] go_back: return to cart\n\nAnswer with the letter of the best option only."}],
  "max_tokens": 4, "temperature": 0, "chat_template_kwargs": {"enable_thinking": false}}'

For probabilities over every option use n_probs on /completion and read the letter tokens at the first output position; that is what the openjev-server helper does against vLLM and MLX. A llama.cpp backend for the helper is on the list.

Validation

Measured on the same 1,789 development rows and option letterings as the 16-bit model (accuracy / NLL); see validation/ for per-row records.

build	accuracy	NLL	flip	agrees with 16-bit answer
16-bit (vLLM reference, same rows)	83.17	0.4195	4.3%	—
Q8_0 (llama.cpp)	82.56	0.4207	4.7%	98.2%
Q4_K_M (llama.cpp)	82.84	0.4239	4.4%	96.6%

Paired against the 16-bit model on the same 1,789 rows: Q4_K_M −0.34 points (95% interval −1.06 to +0.39), Q8_0 −0.61 points (−1.17 to −0.11). Both sit within a point of the reference; the interval on 1,789 rows is too wide to certify the 0.5-point non-inferiority used for FP8, which needs the 10,000-question run. Readout here goes through llama.cpp's /completion top-64 token probabilities, not the vLLM helper.

Notes

• Yes/no questions are lettered [A] yes / [B] no in this build's prompts.
• Q4_K_M is the recommended file for 24 GB cards; leave room for the context.
• The FP8 and MLX builds keep the exact production readout; GGUF is the portable build.

OpenJev is an independent project, not affiliated with TypeSafe; Jev is their product.