Jev AI Context Compression: Preserve Tool Calls, Cut Agent Noise

Long-running agents do not run out of context only because users write long messages. They accumulate plans, file excerpts, tool inputs, tool results, test logs, retries, and stale conclusions. Jev AI can help an application decide which bounded pieces of evidence deserve to stay in the next context window—but the application must still pair tool calls with results, enforce the token budget, and keep a recoverable source history.

This guide adapts the engineering problem explored in a Chinese Jev AI field report into a portable design: treat context compression as a selection problem over evidence units, not as permission to delete arbitrary history. The report’s local demo is a useful example of a workflow; its compression ratio and test results are not a universal Jev benchmark. Original article

What Jev should decide in a compaction pipeline

Jev is TypeSafe AI’s System One model: it accepts state and typed questions, and returns typed decisions rather than generated prose. That makes it a candidate for repeated semantic judgments such as “Does this tool result contain evidence needed for the current task?” It does not automatically summarize a transcript, manage a context window, or guarantee that a particular detail is safe to drop. Those responsibilities remain in application code. Official introduction · System One input and question types

A robust pipeline separates the work into three layers:

1. Deterministic preparation: parse messages, pair tool calls with results, pin system constraints, and identify exact facts such as exit codes.

2. Semantic selection: use Jev to judge whether candidate evidence is relevant, risky to omit, or needed for the next action.

3. Deterministic fitting: apply a token budget, preserve required pairs, store the original trace, and construct the next state.

   

Start with evidence units, not raw messages

Do not ask Jev to select from an unstructured transcript containing every token and event. First convert the history into evidence units. A unit might be:

• the user’s current goal and explicit constraints;
• one file excerpt plus the path and why it was read;
• one tool call and its corresponding result;
• a test failure with command, failing case, exit code, and relevant excerpt;
• a verified decision already made by the agent;
• an unresolved question or next action. Tool calls and results frequently appear in different messages. Before compression, match them by the framework’s call ID and store them as a single unit. Preserve failed tool results when their cause matters; a failure mentioning a permission check, missing test, or changed file is often more useful than a long successful directory listing.

Never let semantic selection remove system/developer instructions, access-control policy, a user’s active constraints, or application-owned state. Pin those outside the model’s candidate set. Also keep the full trace in your own storage if audit or replay matters: the compressed context is a working view, not the canonical record.

Ask a small, explicit decision

For each candidate unit, present a compact state that includes the current task, the next planned action, and a bounded description of the evidence. Then ask one atomic question, such as:

Given the current task and the next action, should this evidence unit be retained in the next model context because omitting it could change the decision or break a tool exchange?

A Noul question can return a yes-probability signal for that proposition. A separate Score can estimate task relevance against a clearly defined rubric. A Choice can classify a unit as must_keep, useful, or discardable if the application needs those categories. Avoid combining relevance, privacy, authorization, and output formatting into one vague question.

Questions in a System One request are evaluated independently against the same state. If one decision depends on the result of another, sequence that dependency in code. For example, first determine whether the task involves a destructive operation using a policy rule; then apply a stricter retention or review policy to the relevant evidence. Do not expect the second Jev question to read the first answer as hidden context.

Build the candidate state safely

A candidate state might be a JSON object like this (the values below are fictional):

{
  "goal": "Fix the order export authorization regression",
  "next_action": "Inspect the failed test and patch the export route",
  "evidence_unit": {
    "kind": "tool_exchange",
    "tool": "test_runner",
    "summary": "order-export test failed after 500 ms; expected authorization header to be preserved",
    "linked_call_result": true,
    "contains_secret": false,
    "source_ref": "trace://run-42/tool-3"
  }
}

Do not ship secrets or unnecessary personal data to an external API. Redact credentials before any model call; a contains_secret decision should not be your only secret-protection mechanism. Prefer local deterministic redaction and policy checks before building the state.

A request to the Jev AI Model API has a state, model, and named questions according to that site’s documentation. Keep the concrete endpoint and API key tied to the service that issued them. Best Jev AI documents a separate independent gateway and explicitly distinguishes it from TypeSafe’s official API. Do not copy an endpoint or key between the two services without checking their current docs and terms.

The following pseudocode is deliberately provider-neutral after the decision call:

type EvidenceUnit = {
  id: string;
  kind: "user_constraint" | "file" | "tool_exchange" | "test_failure";
  summary: string;
  sourceRef: string;
  linkedCallResult?: boolean;
  pinned: boolean;
};

function mayDrop(unit: EvidenceUnit, decision: number, threshold: number) {
  if (unit.pinned) return false;
  if (unit.kind === "tool_exchange" && !unit.linkedCallResult) return false;
  return decision < threshold;
}

The exact response field depends on the API adapter you choose. Read the selected service’s API reference and validate its schema rather than assuming a response shape from another Jev endpoint.

Fit the selected evidence into a real budget

Selection alone does not guarantee the remaining context fits. Use deterministic code to measure tokens with the tokenizer for the downstream model, then apply a fitting policy. A practical policy can:

1. Keep pinned instructions and current user constraints.
2. Keep the most recent turns needed to preserve conversational continuity.
3. Keep evidence selected for the next action, including both sides of relevant tool exchanges.
4. Add short summaries for older verified decisions, each with a trace reference.
5. Remove lower-value units only after all must-keep rules are satisfied.

If the context still exceeds budget, do not keep lowering one global confidence threshold until the result looks small. Try a second pass over larger groups, shorten deterministic content such as repeated logs, or switch to a safer fallback such as a longer-context model. Record which policy caused each omission so a bad continuation can be debugged.

Preserve reversibility

Compaction should be reversible whenever possible. Keep an immutable source trace, record the IDs of retained and omitted evidence units, and make every retained summary point back to its source. If the next model asks for a dropped detail, retrieve the original unit instead of hallucinating or silently treating the summary as complete.

For an early implementation, maintain both active_context and source_trace. The first is optimized for the next call; the second is your recoverable history. This separation lets you change compaction rules without losing the evidence needed to inspect a decision.

Use uncertainty to choose a fallback

A probability is not a universal delete threshold. Build a labeled evaluation set from your own agent traces and define errors by consequence. Omitting a repeated search result is different from omitting the only failing test, a user constraint, or the result of a security-sensitive tool call.

A conservative policy might have three paths:

Choose thresholds by task and risk, not by habit. TypeSafe’s docs describe probabilities and confidence signals, but those signals still need calibration checks against your own outcomes. Begin in shadow mode: let Jev make retention judgments without changing the live context, then compare them with human labels and downstream task success.

Common failure modes

Dropping one side of a tool exchange

A later model may see that a tool was called but not what it returned—or see an error without knowing which command produced it. Pair by call ID and preserve both sides when the exchange is relevant.

Treating a summary as the source of truth

Summaries can omit a qualifier. Keep source references and allow retrieval of the original file excerpt, tool result, or user instruction.

Asking the model to enforce policy

Jev can assess a narrowly framed proposition, but access control, secret handling, and permissions belong in deterministic code. Do not ask a model to decide whether a user is authorized to perform a privileged action.

Optimizing only for token reduction

A shorter context is not automatically better. Track downstream task completion, regression rate, tool-call validity, recovery frequency, latency, and cost alongside tokens saved.

A safe rollout checklist

• Start with one task type and a small set of representative traces.
• Pin instructions, constraints, permissions, and application-owned facts outside the candidate set.
• Pair tool calls and results before scoring.
• Redact secrets deterministically and minimize state sent to the selected provider.
• Keep a source trace and references from every summary.
• Measure both token reduction and task quality in shadow mode.
• Set stricter thresholds for security-related or hard-to-reverse evidence.
• Add a retrieval or human-review fallback before enabling automatic omission.

You can experiment with question design in the Jev AI Model playground, then read its API documentation. If you choose the separate Best Jev AI playground, use that site’s own documentation and credentials.

Further reading

• TypeSafe AI: Introduction
• TypeSafe AI: System One
• Best Jev AI API documentation
• Original Chinese Jev AI field report