Learned code-watermark policy heads (OpenCoder-1.5B, Qwen2.5-Coder-1.5B)
12 small watermark-policy networks ("heads") trained for the course project "Who wrote this code?". Code, data, every result and every failure: https://github.com/sushant-k-ray/LLM-Watermarking, sections 5.4, 6.6, 6.7 and 7 of its README.
A head reads the previous 2 tokens and outputs (a) a green list (the top half of the vocabulary by its logits) and
(b) a switch saying whether this position is watermarked. During generation, Ξ΄ is added to the green-list logits where the
switch is on. Detection is blind: it needs only the head and the tokenizer (not the LLM and not the prompt) and runs a
one-proportion z-test over the switched positions. The architecture follows CodeTracer's CodeModel (6 transformer layers,
d = 512, 8 attention heads, FF 2048, context width 2, dropout 0; ~118 M parameters). The training code is ours, sized for
2Γ T4: a GRPO-style policy-gradient trainer plus a differentiable soft-z detection surrogate with a null contrast.
These are not CodeTracer's own checkpoints. CodeTracer's training recipe needs an A100; that run is still pending.
Files
| folder | base model (generator + tokenizer) | vocab | heads |
|---|---|---|---|
opencoder/ |
infly/OpenCoder-1.5B-Instruct |
96,640 | tests, tests-rob, tests-s1, tests-rob-s1, static, static-rob, tests-nosur, tests-nonull |
qwen/ |
Qwen/Qwen2.5-Coder-1.5B-Instruct |
151,936 | tests, tests-rob, static, static-rob |
Head names: tests = functionality reward from unit tests; static = functionality reward from static analysis instead;
-rob = plus a refactoring-robustness reward; -s1 = seed 1 (the others are seed 0); tests-nosur = ablation without the
detection surrogate (RL reward only); tests-nonull = ablation without the null contrast. All were trained on MBPP-train
rollouts at temperature 0.8 for 760β1,000 steps. SHA256SUMS lists the checksum of every file.
Each .pt is a torch.save dict: state_dict (fp16), vocab, context_width, gamma (0.5), delta (2.0), model,
variant, step. It is a pickle, so load it only with torch.load from a source you trust (this one).
Usage
import sys, torch
from huggingface_hub import hf_hub_download
sys.path.insert(0, "LLM-Watermarking/analysis/common") # git clone https://github.com/sushant-k-ray/LLM-Watermarking
from wm_policy import PolicyHead, PolicyProcessor, detect
ck = torch.load(hf_hub_download("Blech24/llm-watermarking-heads", "opencoder/head_tests.pt"), map_location="cpu")
head = PolicyHead(ck["vocab"], ck["context_width"]); head.load_state_dict({k: v.float() for k, v in ck["state_dict"].items()})
head.eval()
# generation: logits_processor=[PolicyProcessor(head, gamma=0.5, delta=2.0)] in model.generate(...)
# detection: z, n_scored, n_green = detect(head, token_ids, prompt_len) # token_ids from the base model's tokenizer
Results (held-out; details and caveats in section 6.7 of the GitHub README)
One harness for every method: HumanEval, MBPP-test, HumanEvalPack Python / JavaScript / Java; 5 samples per problem at T = 0.2, Ξ΄ = 2. Means over the 5 tasks. Negatives = human reference solutions + unwatermarked generations.
| model | method | pass@1 Ξ (pts) | AUROC | TPR at 5 % FPR | FPR at z > 4 | AUROC after refactor* |
|---|---|---|---|---|---|---|
| OpenCoder | SWEET (baseline) | β0.8 | 84.2 | 34.9 | 0.0 % | 52.9 |
| OpenCoder | WLLM (baseline) | β4.5 | 68.8 | 16.2 | 1.8 % | 56.5 |
| OpenCoder | heads (6, excl. ablations) | β1.2 β¦ β5.4 | 72.6 β¦ 79.4 | 14.8 β¦ 26.4 | 5.9 β¦ 20.3 % | 63.8 β¦ 72.1 |
| OpenCoder | tests-nosur |
β2.8 | 80.8 | 37.3 | 0.8 % | 65.7 |
| Qwen | SWEET (baseline) | β3.6 | 87.7 | 49.6 | 0.0 % | 54.8 |
| Qwen | WLLM (baseline) | β6.5 | 73.7 | 25.8 | 0.1 % | 57.4 |
| Qwen | heads (4) | β3.3 β¦ β6.3 | 71.9 β¦ 75.4 | 20.2 β¦ 23.3 | 23.2 β¦ 33.0 % | 69.7 β¦ 74.1 |
* Python tasks, after the combined refactor (ruff + identifier renaming + comment stripping), applied to negatives too.
Known problems. Read these before using the heads:
- Without edits, SWEET detects better than every head, for both models.
- After refactoring, the heads are more robust: 64β74 AUROC against 53β57 for SWEET and WLLM.
- The fixed z > 4 threshold is miscalibrated for the heads. On average 6β33 % of unwatermarked code scores above it (up to 48 % on one task). Use an empirical-null threshold (as in the TPR column), not z > 4.
- They work best on their training distribution. For example, Qwen heads reach 88β90 AUROC on MBPP-test, but transfer to JavaScript and Java is weak.
- The detection surrogate did not help on held-out data:
tests-nosuris the best OpenCoder head. The refactoring reward (-rob) does not help reliably, and seed variance is large. tests-nonullis a negative ablation (AUROC 58, FPR 94 %), kept for completeness.- LLM paraphrasing (Llama-3.1-8B, Qwen2.5-Coder-7B) is still being evaluated. A small smoke test suggests it removes WLLM/SWEET entirely and that the heads' high z on paraphrased code is the miscalibration above, not a surviving watermark.
Model tree for Blech24/llm-watermarking-heads
Base model
Qwen/Qwen2.5-1.5B