datalogBERT-4class

datalogBERT-4class is a 4-class Datalog static-analysis and agent-session monitor synthesized from ModernBERT-large-nli (395.84M parameters, 2,048-token context window) purely in weight space (0 gradient descent steps).

It inspects either conversational coding agent turns or full multi-rule Datalog programs (Soufflé, DDlog, Ascent, CozoDB, Crepe, Flix, LogiQL) and classifies them into 4 action classes:

Class ID	Label	Description
0	COMPLIANT_CONTINUE	Demand-seeded Datalog rules (demand_* / seed_* placed first in rule bodies), properly clamped divisions (max(denom, 1)), materialized record projections, or benign engineering discussion.
1	UNSEEDED_DATALOG	Unseeded whole-graph transitive closure, all-pairs reachability, unseeded Cartesian products, unseeded stratified-negation complements, or ungrounded doubly-recursive equivalence rules without demand guards.
2	BLOWUP_RETREAT	Abandoning declarative Datalog due to OOM / timeout / blowup (e.g., truncating relations with .limitsize(n=...), dropping recursion depth, or retreating to imperative Python/C++ graph loops instead of demand-seeding).
3	BAD_JOIN_ORDER_OR_INLINE_BLOWUP	Demand-seeded rules with pathological join ordering (placing large EDB/IDB joins or .inline disjunctions before the demand_* guard), unguarded division (x / y without max(y, 1)), or unmaterialized inline record wildcards ([_, _]).

---

Quickstart

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

repo_id = "heydtn/datalogBERT-4class"
device = "mps" if torch.backends.mps.is_available() else ("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModelForSequenceClassification.from_pretrained(repo_id).to(device).eval()

sample = """
// Demand-seeded taint analysis pipeline
.decl taint_seed(req_id: symbol, root_var: symbol)
.decl demand_var(v: symbol)
.decl assign_step(src: symbol, dst: symbol)
.decl reach_var(origin: symbol, v: symbol)

demand_var(root_var) :- taint_seed(_, root_var).
demand_var(nxt) :- demand_var(curr), assign_step(curr, nxt).
reach_var(origin, dst) :- demand_var(origin), assign_step(origin, dst).

// Added unseeded all-pairs closure across every variable:
.decl all_var_closure(a: symbol, b: symbol)
all_var_closure(a, b) :- assign_step(a, b).
all_var_closure(a, c) :- all_var_closure(a, b), assign_step(b, c).
"""

inputs = tokenizer(sample, return_tensors="pt", truncation=True, max_length=2048).to(device)
with torch.no_grad():
    logits = model(**inputs).logits
    probs = torch.softmax(logits, dim=-1)[0]
    pred_id = int(logits.argmax(dim=-1).item())

print(f"Predicted Class: {pred_id} ({model.config.id2label[pred_id]}) | Confidence: {probs[pred_id]:.3f}")

---

Benchmark Summary

• Calibration Set (84 examples): 100.0% (84 / 84) | Macro-F1 = 1.0000
• Held-Out Evaluation Set (32 unseen examples): 100.0% (32 / 32) | Macro-F1 = 1.0000
• 400-Prompt Adversarial & Multi-Complexity Stress Suite:
  • Binary Intervention Accuracy (Flagged vs. Unflagged): 97.75% (391 / 400, F1 = 0.9778)
  • Exact 4-Class Accuracy: 97.50% (390 / 400, Macro-F1 = 0.9742)
• 560-Program Long Complex Code "Needle-in-a-Haystack" Suite (271–1,119 tokens, 8–12 rules per file):
  • Set A (280 All-Guarded Programs, True = C0): 100.00% (280 / 280, 0 False Positives)
  • Set B (280 Paired Programs with 1 Unguarded Clause at Top, Middle, or Bottom): 98.93% (277 / 280) Binary Recall | 98.57% (276 / 280) Exact 4-Class
  • Combined 560 Long Programs: 99.46% (557 / 560) Binary Accuracy | 99.29% (556 / 560) Exact 4-Class Accuracy