Carnot VJEPA v2 — Three-Tier LLM Verification
OOD AUC: 0.9211 | Milestone 2026.04.68 | Apache 2.0
Overview
VariationalJEPAPredictor is the Tier 2 energy-based verifier in the Carnot three-tier pipeline. It uses a variational (KL-regularised) JEPA encoder to detect hallucinations and reasoning errors in LLM outputs across multiple domains without re-running the upstream LLM.
The key innovation over deterministic JEPA (Exp 834, which collapsed to AUC=0.0 on out-of-distribution domains) is the variational encoder that produces (mu, log_var) posteriors rather than a single point. The KL term forces the model to maintain probability mass across the latent space even on unfamiliar inputs, preventing the trivial constant-predictor collapse.
Architecture
| Component | Detail |
|---|---|
| Encoder q(z|x) | 2-layer MLP: in_dim → 128 → 64 → (mu:32, logvar:32) |
| Prior p(z|c) | GRU cell: context_dim → 64 → (mu:32, logvar:32) |
| Classifier | Linear: 32 → 1 (sigmoid) |
| Loss | BCE + 0.1 × KL[q || p] (beta-VAE convention) |
| Framework | JAX / Optax |
| Serialisation | safetensors |
Training
- Corpus: 207 pairs: 57 FoVer + 100 synthetic GSM8K + 30 ARC + 20 SVAMP (plus 146 pairs used for final v2 deployment run)
- Epochs: 200
- Final KL magnitude: 0.624
- OOD held-out set: 10 ARC + 10 SVAMP (seed 999)
Key Metrics (Milestone .68)
| Metric | Value |
|---|---|
| OOD AUC | 0.9211 |
| Cascade deployment | confirmed (Exp 884) |
| Prior JEPA v24 OOD AUC | 0.0 (collapsed) |
| Improvement | +0.92 AUC points |
Tier Context
Carnot uses three tiers of energy-based verification:
| Tier | Model | Role |
|---|---|---|
| 0h | SpectralAttentionProbe | Lightweight syntactic probe |
| 2 | VJEPA v2 (this model) | Variational semantic verifier |
| 3 | Self-Learning Relay | FR-11 closed, adaptive relay |
Usage
from carnot.models.vjepa_predictor import VariationalJEPAPredictor
from pathlib import Path
# Load from safetensors
model = VariationalJEPAPredictor.load(Path("model.safetensors"))
# Score a candidate: returns float in [0, 1] (higher = more likely correct)
score = model.score(prompt_embedding, candidate_embedding)
print(f"Verification score: {score:.4f}")
Citation
@misc{carnot2026vjepa,
title = {Carnot VJEPA v2: Variational Energy-Based LLM Verification},
author = {Carnot Project},
year = {2026},
note = {Milestone 2026.04.68, ood\_auc=0.9211},
url = {https://huggingface.co/Carnot-EBM/carnot-vjepa-v2}
}
License
Apache 2.0. See LICENSE.