Learned code-watermark policy heads (OpenCoder-1.5B, Qwen2.5-Coder-1.5B)

12 small watermark-policy networks ("heads") trained for the course project "Who wrote this code?". Code, data, every result and every failure: https://github.com/sushant-k-ray/LLM-Watermarking, sections 5.4, 6.6, 6.7 and 7 of its README.

A head reads the previous 2 tokens and outputs (a) a green list (the top half of the vocabulary by its logits) and (b) a switch saying whether this position is watermarked. During generation, Ξ΄ is added to the green-list logits where the switch is on. Detection is blind: it needs only the head and the tokenizer (not the LLM and not the prompt) and runs a one-proportion z-test over the switched positions. The architecture follows CodeTracer's CodeModel (6 transformer layers, d = 512, 8 attention heads, FF 2048, context width 2, dropout 0; ~118 M parameters). The training code is ours, sized for 2Γ— T4: a GRPO-style policy-gradient trainer plus a differentiable soft-z detection surrogate with a null contrast.

These are not CodeTracer's own checkpoints. CodeTracer's training recipe needs an A100; that run is still pending.

Files

folder base model (generator + tokenizer) vocab heads
opencoder/ infly/OpenCoder-1.5B-Instruct 96,640 tests, tests-rob, tests-s1, tests-rob-s1, static, static-rob, tests-nosur, tests-nonull
qwen/ Qwen/Qwen2.5-Coder-1.5B-Instruct 151,936 tests, tests-rob, static, static-rob

Head names: tests = functionality reward from unit tests; static = functionality reward from static analysis instead; -rob = plus a refactoring-robustness reward; -s1 = seed 1 (the others are seed 0); tests-nosur = ablation without the detection surrogate (RL reward only); tests-nonull = ablation without the null contrast. All were trained on MBPP-train rollouts at temperature 0.8 for 760–1,000 steps. SHA256SUMS lists the checksum of every file.

Each .pt is a torch.save dict: state_dict (fp16), vocab, context_width, gamma (0.5), delta (2.0), model, variant, step. It is a pickle, so load it only with torch.load from a source you trust (this one).

Usage

import sys, torch
from huggingface_hub import hf_hub_download
sys.path.insert(0, "LLM-Watermarking/analysis/common")       # git clone https://github.com/sushant-k-ray/LLM-Watermarking
from wm_policy import PolicyHead, PolicyProcessor, detect

ck = torch.load(hf_hub_download("Blech24/llm-watermarking-heads", "opencoder/head_tests.pt"), map_location="cpu")
head = PolicyHead(ck["vocab"], ck["context_width"]); head.load_state_dict({k: v.float() for k, v in ck["state_dict"].items()})
head.eval()
# generation: logits_processor=[PolicyProcessor(head, gamma=0.5, delta=2.0)] in model.generate(...)
# detection:  z, n_scored, n_green = detect(head, token_ids, prompt_len)   # token_ids from the base model's tokenizer

Results (held-out; details and caveats in section 6.7 of the GitHub README)

One harness for every method: HumanEval, MBPP-test, HumanEvalPack Python / JavaScript / Java; 5 samples per problem at T = 0.2, Ξ΄ = 2. Means over the 5 tasks. Negatives = human reference solutions + unwatermarked generations.

model method pass@1 Ξ” (pts) AUROC TPR at 5 % FPR FPR at z > 4 AUROC after refactor*
OpenCoder SWEET (baseline) βˆ’0.8 84.2 34.9 0.0 % 52.9
OpenCoder WLLM (baseline) βˆ’4.5 68.8 16.2 1.8 % 56.5
OpenCoder heads (6, excl. ablations) βˆ’1.2 … βˆ’5.4 72.6 … 79.4 14.8 … 26.4 5.9 … 20.3 % 63.8 … 72.1
OpenCoder tests-nosur βˆ’2.8 80.8 37.3 0.8 % 65.7
Qwen SWEET (baseline) βˆ’3.6 87.7 49.6 0.0 % 54.8
Qwen WLLM (baseline) βˆ’6.5 73.7 25.8 0.1 % 57.4
Qwen heads (4) βˆ’3.3 … βˆ’6.3 71.9 … 75.4 20.2 … 23.3 23.2 … 33.0 % 69.7 … 74.1

* Python tasks, after the combined refactor (ruff + identifier renaming + comment stripping), applied to negatives too.

Known problems. Read these before using the heads:

  • Without edits, SWEET detects better than every head, for both models.
  • After refactoring, the heads are more robust: 64–74 AUROC against 53–57 for SWEET and WLLM.
  • The fixed z > 4 threshold is miscalibrated for the heads. On average 6–33 % of unwatermarked code scores above it (up to 48 % on one task). Use an empirical-null threshold (as in the TPR column), not z > 4.
  • They work best on their training distribution. For example, Qwen heads reach 88–90 AUROC on MBPP-test, but transfer to JavaScript and Java is weak.
  • The detection surrogate did not help on held-out data: tests-nosur is the best OpenCoder head. The refactoring reward (-rob) does not help reliably, and seed variance is large.
  • tests-nonull is a negative ablation (AUROC 58, FPR 94 %), kept for completeness.
  • LLM paraphrasing (Llama-3.1-8B, Qwen2.5-Coder-7B) is still being evaluated. A small smoke test suggests it removes WLLM/SWEET entirely and that the heads' high z on paraphrased code is the miscalibration above, not a surviving watermark.
Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Blech24/llm-watermarking-heads

Finetuned
(245)
this model