README.md

---
title: LLM Text Watermark Microscope
emoji: "\U0001F52C"
colorFrom: blue
colorTo: indigo
sdk: static
app_file: index.html
pinned: false
license: apache-2.0
short_description: Inspect three generations of LLM text watermarks in-browser
---

# LLM Text Watermark Microscope

An interactive research/education tool that shows **how LLM text watermarking evolved** —
not just *that* watermarks work, but *what happens inside the decoding loop* when they do.
You supply the secrets and walk the decoding forward one stage at a time.

The screen is split: an unwatermarked **baseline** on the left, held fixed for the prompt, and the
watermark you are studying on the right. The baseline mirrors whatever stage you are on, so every
claim has a control beside it. Each method is a four-stage wizard that unlocks as you go:

| stage | Kirchenbauer | k-SemStamp | TextSeal |
|---|---|---|---|
| 1 | Seed & Split — type a seed, watch the vocabulary partition, click a token for the arithmetic | Seed & Clusters — which semantic clusters your secret allows next | Seed & R-values — the PRF numbers behind each candidate |
| 2 | Bias & Sampling — move δ and see any real decoding step re-weighted | Rejection Sampling — candidate sentences accepted/rejected live | Gumbel Sampling — dual-key routing vs. the baseline draw |
| 3 | Result — every generated token coloured by the list it came from | Result — every sentence by its cluster verdict | Evidence & Regions — per-token evidence heatmap, localized spans |
| 4 | Edit & Detect — all three detectors re-score as you type | | |

The four runs share sampling parameters:

| run | year | carrier | detection |
|---|---|---|---|
| Baseline | — | none | (reference) |
| [Kirchenbauer et al.](https://arxiv.org/abs/2301.10226) | 2023 | token identities (green/red vocabulary partition, +δ logit bias) | one-proportion z-test on green counts |
| [k-SemStamp](https://arxiv.org/abs/2402.11399) (Hou et al.) | 2024 | sentence semantics (k-means clusters over embeddings, rejection sampling) | z-test on valid-region sentence counts |
| [TextSeal](https://arxiv.org/abs/2605.12456) (Sander et al., Meta FAIR) | 2026 | distortion-free Gumbel-max sampling (dual-key PRF) | entropy-weighted Gamma p-value + **localized multi-region detection** |

Every stage exposes its internals rather than summarizing them: PRF seeds and the U-vs-γ
comparison for a token you click, per-step logits before and after biasing, candidate sentences
with their cluster assignments and accept/reject reasons, per-token Gumbel evidence, and the
localized region scan. Stage 4 runs **all three detectors on whatever you type**, which is where
the generations visibly separate — token-level edits destroy Kirchenbauer evidence, meaning-preserving
rewrites leave k-SemStamp intact, and excerpting shrinks a global score while the localized scan
still finds the span. Each method exports its seed, parameters, trace and scores as JSON.

Measured on a run of 96 tokens with Qwen3-0.6B (your numbers will differ):
Kirchenbauer reached z = 5.48 against a baseline of −0.61; k-SemStamp reached z = 3.00 against
2.31; TextSeal's unweighted score sat just under threshold at −log₁₀p = 3.39 and crossed it at
4.51 once entropy weighting was enabled — a direct demonstration of what that part of the paper
buys you.

## Everything runs in your browser

- **No server GPU, no API keys.** Model inference runs on your GPU via **WebGPU**
  ([Transformers.js](https://huggingface.co/docs/transformers.js) v3 / ONNX Runtime Web).
  Your prompt and generated text never leave the browser.
- Default model: [Qwen3-0.6B](https://huggingface.co/onnx-community/Qwen3-0.6B-ONNX) q4f16
  (~570 MB download, cached by the browser; thinking mode disabled). Swappable to
  Qwen2.5-0.5B-Instruct (~483 MB) or SmolLM2-360M-Instruct (~273 MB).
- Sentence embeddings for k-SemStamp:
  [EmbeddingGemma-300m](https://huggingface.co/onnx-community/embeddinggemma-300m-ONNX)
  (~167 MB, downloaded only when you open that method). It is multilingual, so the semantic
  clusters mean something in every language the demo model can write.
- **Fallbacks:** GPUs without `shader-f16` get `q4`; browsers without WebGPU fall back to WASM
  (CPU — slow; keep max tokens low). Chrome/Edge ≥ 113 recommended.

All four runs share one custom autoregressive loop (manual `forward()` + KV cache — the
framework's `generate()` is bypassed so each algorithm can hook logits or replace sampling).
Baseline and Kirchenbauer consume the **same base RNG stream** with the same seed, isolating the
watermark's effect from sampling randomness. TextSeal replaces sampling with a deterministic PRF
rule and k-SemStamp's rejection loop changes randomness consumption, so stream-sharing is
impossible there — the UI says so rather than hiding it.

## What this tool is not

This is a **mechanism verifier, not an AI-text detector**. It checks for a *specific* watermark
inserted with a *known secret key*. No detection ≠ human-written; detection ≠ proof of
authorship. The editor also demonstrates honestly that iterative editing against a visible score
degrades detection — detectability and unremovability are different properties.

## Faithfulness notes (educational deviations from the papers)

| component | reference | this implementation | consequence |
|---|---|---|---|
| Kirchenbauer PRNG | `torch.Generator` (MT19937) + `randperm`, seed = `15485863 × prev_token mod 2⁶⁴−1` | same seed formula, but splitmix64-based per-token judgment `U(seed, key, id) < γ`, with the user's seed mixed in at the judgment rather than the seed so the displayed derivation stays literal | green-list size is Binomial(\|V\|, γ) instead of exactly γ\|V\|; z-test unaffected. **Not interoperable** with the reference detector |
| Kirchenbauer scheme | `lefthash` and `selfhash` | `lefthash` only | selfhash (context 4 + self-salt) not implemented |
| k-SemStamp encoder | SBERT fine-tuned with paraphrase-contrastive loss per domain | general-purpose EmbeddingGemma-300m, called through `AutoModel` for its `sentence_embedding` output (the feature-extraction pipeline would mean-pool hidden states and skip the model's dense projection) | paraphrase robustness weaker than published results, but multilingual |
| k-SemStamp centroids | k-means on 8k in-domain paragraphs | k-means (K=8) on ~440 wikitext-2, narrative and Korean sentences, shipped as JSON (`scripts/compute-centroids.mjs`). The script uses the same encoder and the same `task: clustering \| query: ` prefix as the browser — a mismatch there would put centroids and runtime assignments in different spaces | clusters are generic; margin default relaxed (0.01 vs paper 0.035) |
| k-SemStamp trials | `MAX_TRIALS = 100` | default 16 (browser latency), adjustable; each retry raises the temperature slightly, capped at 1.5x, because an unbounded climb buys candidate diversity by turning the text into gibberish | occasional forced accepts, shown in the trace |
| k-SemStamp counting | every sentence scored | a sentence in the same cluster as the one before it is **not counted again**, and fewer than three independent sentences reports **inconclusive** | consecutive same-cluster sentences re-ask one question ("is q in valid(q)?"), so counting them separately turns one coin flip into a confident-looking z. This is the same reasoning as `ignore_repeated_ngrams` on the Kirchenbauer side |
| TextSeal PRF | cryptographic hash (paper appendix A.1 / [official repo](https://github.com/facebookresearch/textseal)) | splitmix64 mixing over (key, k=3 context tokens, candidate) | statistically equivalent construction, **not interoperable** with official TextSeal |
| TextSeal localization penalties | exact combinatorial multi-region penalty | per-region `log₁₀M + log₁₀Y_max` simplification, ensemble `+log₁₀3` | slightly conservative p-values |
| TextSeal entropy weights | teacher-forced model entropies | optional (one extra forward pass ≤512 tokens); default unweighted | unweighted variant is also calibrated, per the paper |

KV-cache rollback for k-SemStamp re-computes the prefix on rejection instead of snapshotting
GPU tensors — simpler and leak-free at demo scale.

## Reproducibility

`seed`, `secret key`, sampling parameters, model repo/dtype and per-algorithm parameters are all
visible in each result's metadata and adjustable under Advanced settings. Identical
(model, prompt, parameters, key, seed) reproduce identical Baseline/Kirchenbauer outputs;
TextSeal is deterministic given the key and routing seed. WebGPU floating-point differences
across GPU vendors may change low-probability tie-breaks; bit-level determinism across devices
is not guaranteed.

## Development

```bash
npm install
npm run dev        # local dev server
npm test           # 74 unit + integration tests (node; mock model, no GPU needed)
npm run build      # production build (dist/)
node scripts/compute-centroids.mjs   # regenerate k-SemStamp centroids (downloads the encoder)
```

Architecture: `src/lib/model` (adapter + custom generation loop) · `src/lib/watermark`
(algorithms) · `src/lib/detectors` · `src/lib/workers/inference-worker.ts` (dedicated worker
owning the model) · `src/lib/stores/app-state.ts` (per-method wizard state) ·
`src/components/stages/<method>` (the stage screens) · `src/components/BaselinePanel.svelte`
(the mirror view).

## Citations

```bibtex
@inproceedings{kirchenbauer2023watermark,
  title={A Watermark for Large Language Models},
  author={Kirchenbauer, John and Geiping, Jonas and Wen, Yuxin and Katz, Jonathan and Miers, Ian and Goldstein, Tom},
  booktitle={ICML}, year={2023}
}
@inproceedings{hou2024ksemstamp,
  title={k-SemStamp: A Clustering-Based Semantic Watermark for Detection of Machine-Generated Text},
  author={Hou, Abe Bohan and Zhang, Jingyu and Wang, Yichen and Khashabi, Daniel and He, Tianxing},
  booktitle={Findings of ACL}, year={2024}
}
@article{sander2026textseal,
  title={TextSeal: A Localized LLM Watermark for Provenance and Distillation Protection},
  author={Sander, Tom and Chang, Hongyan and Sou{\v{c}}ek, Tom{\'a}{\v{s}} and others},
  journal={arXiv preprint arXiv:2605.12456}, year={2026}
}
```