README.md

---
title: Compression-Lens
emoji: 🔬
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: mit
---

# UncheatableEval: Qwen3 vs RWKV7 Byte-Level Comparison

Compare the byte-level prediction performance between models.

## Features

- **Byte-level analysis**: See exactly where each model performs better or worse
- **Interactive visualization**: Hover over tokens to see detailed predictions
- **Color-coded comparison**:
  - 🟢 Green = Qwen3 predicts better (lower loss)
  - 🔴 Red = RWKV7 predicts better (lower loss)
- **Top-10 predictions**: View each model's top predictions for every token
- **Word occurrence linking**: See how repeated words are predicted differently

## How to Use

1. Enter or paste your text (max 4000 characters)
2. Click "Run Comparison"
3. Explore the interactive visualization
4. Download the HTML file for offline viewing

## Models

| Model | Type | Parameters | Architecture |
|-------|------|------------|--------------|
| Qwen3-1.7B-Base | Transformer | 1.7B | Dense attention |
| RWKV7-G1C-1.5B | RWKV | 1.5B | Linear attention |

## Technical Details

This tool uses the [UncheatableEval](https://github.com/Jellyfish042/UncheatableEval) framework to:

1. Tokenize input text with each model's tokenizer
2. Calculate per-token cross-entropy loss
3. Map token losses to byte-level losses
4. Generate interactive HTML visualization

## Local Development

```bash
# Clone the repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/UncheatableEval-Visualization

# Install dependencies
pip install -r requirements.txt

# Run locally
python app.py
```

## Requirements

- CUDA-capable GPU (16GB+ VRAM recommended)
- Python 3.10+
- See `requirements.txt` for package dependencies

## License

MIT License

## Acknowledgments

- [UncheatableEval](https://github.com/Jellyfish042/UncheatableEval) - Original evaluation framework
- [Qwen](https://github.com/QwenLM/Qwen) - Qwen model family
- [RWKV](https://github.com/BlinkDL/RWKV-LM) - RWKV model family