index.html

<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>OpenJev · Zero-GPU CPU Deploy Kit</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<div class="wrap">

  <div class="hero">
    <h1>OpenJev · Zero-GPU CPU Deploy Kit</h1>
    <span class="tag">⚖️ no GPU · free HF Spaces · exact /v1/systemone contract</span>
    <p>
      Run the <a href="https://huggingface.co/openjev/openjev">OpenJev</a> 27B
      <em>decision API</em> on <strong>pure CPU</strong> — the same
      <code>POST /v1/systemone</code> shape as the hosted
      <a href="https://huggingface.co/spaces/openjev/openjev-server">openjev-server</a>,
      without a GPU. Each answer is read off the model's <em>first output token</em>
      (one forward pass, nothing generated), so a CPU-only 27B is a real, honest API.
    </p>
  </div>

  <div class="card cta">
    <a class="btn" href="https://huggingface.co/spaces/broadfield/openjev-cpu-run?duplicate=true"
       target="_blank" rel="noopener">🚀 Duplicate the zero-GPU runtime Space</a>
    <span class="note" style="margin-left:.5rem">or</span>
    <a class="btn alt" href="https://huggingface.co/datasets/broadfield/openjev-cpu-deploy-kit"
       target="_blank" rel="noopener">📦 Get the deploy kit (Docker, compose, docs)</a>
  </div>

  <div class="card">
    <h2>Why this works — no code hacks</h2>
    <p>
      <code>openjev-server</code>'s <code>vllm</code> backend is really
      <em>"any OpenAI-compatible server that returns logprobs"</em>. When vLLM's
      <code>logprob_token_ids</code> extension is absent, it <strong>falls back to
      matching top-K logprobs by label text</strong> (the documented "old protocol"
      in <code>openjev_server/backends/vllm.py</code>). llama.cpp speaks exactly
      that. So this kit runs the <strong>official, unmodified</strong> server, with
      llama.cpp (CPU, GGUF) in place of vLLM:
    </p>
<pre><code>┌─────────────────────────────┐        ┌──────────────────────────────┐
│  openjev-server (official) │        │  llama.cpp llama-server      │
│  :7860  /v1/systemone      │  HTTP  │  :8080  /v1/chat/completions │
│         /v1/version        │───────▶│  logprobs + top_logprobs    │
│         /healthz /readyz   │  JSON  │  (CPU, OpenJev-Q4_K_M.gguf) │
│         /docs /metrics     │        │  16.2 GB, zero CUDA          │
└─────────────────────────────┘        └──────────────────────────────┘</code></pre>
  </div>

  <div class="card">
    <h2>1) Deploy on Hugging Face (free)</h2>
    <ol>
      <li><strong>Duplicate</strong> the runtime Space
        <a href="https://huggingface.co/spaces/broadfield/openjev-cpu-run?duplicate=true">broadfield/openjev-cpu-run</a>.</li>
      <li>In <em>Settings → Hardware</em>: <strong>CPU basic (free)</strong> &nbsp;·&nbsp;
        in <em>Settings → Persistent storage</em>: <strong>30 GB</strong> (needed for the
        16.2 GB model in <code>/data</code>).</li>
      <li>Wait for the build + first boot. The image compiles llama.cpp CPU, installs
        openjev-server, then downloads the Q4 GGUF. <strong>~20–50 min cold</strong>;
        after that restarts load from persistent storage in ~1–2 min.</li>
      <li>Your Space is now a live decision API. Try it (don't forget your username in
        the subdomain):</li>
    </ol>
<pre><code>BASE=https://YOURUSERNAME-openjev-cpu-run.hf.space

curl -s "$BASE/v1/systemone" -H 'Content-Type: application/json' -d '{
  "state": "Customer message: I was charged twice for my order last week and nobody has replied.",
  "questions": {
    "route":   {"type": "choice", "instructions": "Which team should handle this?",
                "criteria": {"billing": null, "shipping": null, "technical": null}},
    "angry":   {"type": "noul",   "instructions": "Is the customer angry?"},
    "urgency": {"type": "score",  "instructions": "How urgent is this?",
                "criteria": ["can wait", "this week", "today", "right now"]}
  }
}'</code></pre>
<pre><code>{
  "answers": {
    "route":   {"type": "choice", "choice": "billing",
                "probabilities": {"billing": 0.9996, "shipping": 0.0002, "technical": 0.0002},
                "confidence": 0.9995},
    "angry":   {"type": "noul", "noul": 0.56},
    "urgency": {"type": "score", "score": 2.04,
                "legend": {"0": "can wait", "1": "this week", "2": "today", "3": "right now"},
                "probabilities": {"0": 0.02, "1": 0.13, "2": 0.64, "3": 0.21},
                "confidence": 0.71}
  },
  "usage": {"input_tokens": 239, "output_tokens": 0}
}</code></pre>
  </div>

  <div class="card">
    <h2>2) Deploy locally / on any VPS</h2>
<pre><code>git clone https://huggingface.co/datasets/broadfield/openjev-cpu-deploy-kit
cd openjev-cpu-deploy-kit
docker compose up --build -d
curl -s http://localhost:7860/v1/systemone ...   # same payload as above</code></pre>
    <p class="note">plain <code>docker build -t openjev-cpu .</code> +
    <code>docker run -p 7860:7860 -v openjev-data:/data openjev-cpu</code> also works.</p>
  </div>

  <div class="card">
    <h2>API surface (identical to the hosted server)</h2>
    <table>
      <tr><th>endpoint</th><th>method</th><th>what</th></tr>
      <tr><td><code>/v1/systemone</code></td><td>POST</td><td>any mix of <code>choice</code> / <code>noul</code> / <code>score</code> questions</td></tr>
      <tr><td><code>/v1/prewarm</code></td><td>POST</td><td>prefill a long state once, then send its questions</td></tr>
      <tr><td><code>/v1/chat/completions</code></td><td>POST</td><td>text passthrough (thinking forced off)</td></tr>
      <tr><td><code>/v1/version</code></td><td>GET/POST</td><td>model, profile, probe, readout code hash</td></tr>
      <tr><td><code>/healthz</code> <code>/readyz</code> <code>/metrics</code></td><td>GET</td><td>liveness, readiness, Prometheus</td></tr>
      <tr><td><code>/docs</code></td><td>GET</td><td>OpenAPI/Swagger UI</td></tr>
    </table>
    <p>Auth: set <code>OPENJEV_TOKEN</code> → all <code>/v1/*</code> require
    <code>Authorization: Bearer &lt;token&gt;</code>.</p>
  </div>

  <div class="card">
    <h2>3) Ask from code</h2>
    <h2 style="font-size:.95rem;color:var(--muted);margin-top:.4rem">Python</h2>
<pre><code>import httpx

r = httpx.post("http://localhost:7860/v1/systemone", json={
    "state": "Customer message: I was charged twice for my order last week and nobody has replied.",
    "questions": {
        "route": {"type": "choice", "instructions": "Which team should handle this?",
                  "criteria": {"billing": None, "shipping": None, "technical": None}},
        "angry": {"type": "noul", "instructions": "Is the customer angry?"},
    },
}, timeout=60)
answers = r.json()["answers"]
print(answers["route"]["choice"], answers["route"]["probabilities"], answers["angry"]["noul"])</code></pre>
    <h2 style="font-size:.95rem;color:var(--muted)">TypeScript</h2>
<pre><code>const r = await fetch(`${BASE}/v1/systemone`, {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({
    state: "Customer message: I was charged twice for my order last week and nobody has replied.",
    questions: {
      route: { type: "choice", instructions: "Which team should handle this?",
               criteria: { billing: null, shipping: null, technical: null } },
      angry: { type: "noul", instructions: "Is the customer angry?" },
    },
  }),
});
const { answers } = await r.json();</code></pre>
  </div>

  <div class="card">
    <h2>Config & tuning</h2>
    <table>
      <tr><th>var</th><th>default</th><th>what it does</th></tr>
      <tr><td><code>MODEL_FILE</code></td><td><code>OpenJev-Q4_K_M.gguf</code></td><td>Q4 (16 GB) · Q5 (19 GB) · Q6 (22 GB) · Q8 (28 GB)</td></tr>
      <tr><td><code>MODEL_REPO</code></td><td><code>openjev/openjev-GGUF</code></td><td>where the GGUF lives</td></tr>
      <tr><td><code>LLAMA_CTX</code></td><td><code>8192</code></td><td>context window</td></tr>
      <tr><td><code>LLAMA_THREADS</code></td><td>auto</td><td>pin cores, e.g. <code>8</code> / <code>8</code></td></tr>
      <tr><td><code>LLAMA_MAX_LOGPROBS</code></td><td><code>64</code></td><td>top-K; keep ≥ #options</td></tr>
      <tr><td><code>LLAMA_PARALLEL</code></td><td><code>1</code></td><td>concurrent sequences</td></tr>
      <tr><td><code>OPENJEV_TOKEN</code></td><td>(empty)</td><td>bearer auth on <code>/v1/*</code></td></tr>
      <tr><td><code>OPENJEV_PROFILE</code></td><td><code>openjev</code></td><td>published calibration constants</td></tr>
    </table>
  </div>

  <div class="card">
    <h2>Latency expectations (be honest with yourself)</h2>
    <p>
      This is a 27B model on a shared free CPU — <strong>not</strong> 125 ms.
      Expect ~2–6 s per single question on a 16-core Space (Q4). But because a
      decision only ever generates <strong>1 token</strong>, batching questions that
      share one state into a single call makes each extra answer nearly free:
    </p>
    <table>
      <tr><th>questions / call</th><th>est. latency @ ~3 SPE</th></tr>
      <tr><td>1</td><td>~4 s</td></tr>
      <tr><td>4</td><td>~4.5 s</td></tr>
      <tr><td>10</td><td>~5.5 s (10 answers, still one pass)</td></tr>
    </table>
    <p class="note">Bench the live Space: <code>node bench.mjs http://…:7860</code>.</p>
  </div>

  <div class="card">
    <h2>Files in this kit</h2>
    <table>
      <tr><th>path</th><th>role</th><th>raw</th></tr>
      <tr><td><code>Dockerfile</code></td><td>multi-stage: llama.cpp CPU + official openjev-server</td>
        <td><a href="https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/Dockerfile">raw</a></td></tr>
      <tr><td><code>entrypoint.sh</code></td><td>GGUF → /data, llama-server, then <code>openjev serve</code></td>
        <td><a href="https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/entrypoint.sh">raw</a></td></tr>
      <tr><td><code>docker-compose.yml</code></td><td>local/VPS one-liner</td>
        <td><a href="https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/docker-compose.yml">raw</a></td></tr>
      <tr><td><code>README.md</code></td><td>full deploy + benchmarks</td>
        <td><a href="https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/README.md">raw</a></td></tr>
      <tr><td><code>bench.mjs</code></td><td>throughput/latency smoke bench</td>
        <td><a href="https://huggingface.co/spaces/broadfield/openjev-cpu-deploy/raw/main/bench.mjs">raw</a></td></tr>
    </table>
    <p class="note">Everything is also mirrored to the dataset repo
      <a href="https://huggingface.co/datasets/broadfield/openjev-cpu-deploy-kit">broadfield/openjev-cpu-deploy-kit</a>.</p>
  </div>

  <footer>
    <p class="badge">
      Model: <a href="https://huggingface.co/openjev/openjev">openjev/openjev</a>
      (CC BY-NC 4.0) · Server:
      <a href="https://huggingface.co/spaces/openjev/openjev-server">openjev-server</a>
      (Apache 2.0) · llama.cpp (MIT) · This kit: Apache 2.0.<br>
      OpenJev is an independent project, not affiliated with TypeSafe (Jev is their product).
    </p>
  </footer>

</div>
</body>
</html>