| --- |
| title: AlienLM |
| emoji: 🛸 |
| colorFrom: purple |
| colorTo: green |
| sdk: gradio |
| sdk_version: 6.19.0 |
| app_file: app.py |
| short_description: AlienLM privacy layer for black-box LLMs |
| python_version: "3.12" |
| startup_duration_timeout: 30m |
| --- |
| |
| # AlienLM Space |
|
|
| Interactive research demo for [AlienLM](https://github.com/KimJaehee0725/AlienLM), |
| using the public |
| [`dsba-lab/gemma2-9b-it-alienlm-full`](https://huggingface.co/dsba-lab/gemma2-9b-it-alienlm-full) |
| checkpoint. |
|
|
| The Space uses the official repository's token-ID translation rule and startup |
| compatibility checks. Its inference path matches the |
| [`SingleAlienTokenizer`](https://github.com/KimJaehee0725/AlienLM/blob/icml/training/axolotl/tokenizers/single_alien_tokenizer.py) |
| used for training: |
|
|
| 1. Render the natural-language chat with the original Gemma 2 tokenizer, then |
| translate the rendered text into alien text. |
| 2. Encode the alien text with the original tokenizer, matching the |
| `SingleAlienTokenizer` path used during training. |
| 3. Decode generated token IDs with the unchanged original tokenizer, matching a |
| black-box server that returns text and no token metadata. |
| 4. On the client side, invert the original tokenizer's decoded-piece lattice, |
| keep the minimum-token segmentations, and rank their alien-decoded natural |
| text locally. The server tokenizer is not modified. |
|
|
| The Space includes the alien tokenizer files required by the model. Access to |
| the gated `google/gemma-2-9b-it` tokenizer is supplied through the Space's |
| `HF_TOKEN` secret. |
|
|
| ## Privacy scope |
|
|
| This hosted Space is an interactive pipeline demo: natural-language input first |
| reaches the Space backend, where translation and model inference run in the same |
| process. In the deployment described by the paper, the translator and bijection |
| remain on a trusted client so that only alienized text crosses the black-box LLM |
| API boundary. AlienLM is a research method, not a formal cryptographic privacy |
| guarantee. |
|
|
| The inverse segmentation step is necessarily heuristic when the server returns |
| only a fused string: multiple token-ID sequences can decode to the same text. |
| The demo uses minimum token count as a strong prior and resolves collisions |
| using only tokenizer-internal signals: alien-side canonicality, structural |
| validity, and the original tokenizer's BPE vocabulary rank. It does not use a |
| second language model. Special tokens intentionally removed by the server |
| cannot be reconstructed, but are not part of the visible response content. |
| The search budget is intentionally tuned for the Space's short demo responses; |
| long-form reconstruction is outside this demo's scope. |
|
|
| ## Links |
|
|
| - [Paper](https://arxiv.org/abs/2601.22710) |
| - [Project page](https://kimjaehee0725.github.io/AlienLM/) |
| - [Official code](https://github.com/KimJaehee0725/AlienLM) |
| - [Model collection](https://huggingface.co/collections/dsba-lab/alienlm) |
|
|
