Spaces:
Running
Running
File size: 2,813 Bytes
9afeeeb ddd27a5 9afeeeb ddd27a5 9afeeeb 350392a 9afeeeb |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
---
title: Compression-Lens
emoji: 🔬
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: mit
---
# UncheatableEval: Qwen3 vs RWKV7 Byte-Level Comparison
Compare the byte-level prediction performance between models.
## Features
- **Byte-level analysis**: See exactly where each model performs better or worse
- **Interactive visualization**: Hover over tokens to see detailed predictions
- **Color-coded comparison**:
- 🟢 Green = Qwen3 predicts better (lower loss)
- 🔴 Red = RWKV7 predicts better (lower loss)
- **Top-10 predictions**: View each model's top predictions for every token
- **Word occurrence linking**: See how repeated words are predicted differently
## How to Use
1. Enter or paste your text (max 4000 characters)
2. Click "Run Comparison"
3. Explore the interactive visualization
4. Download the HTML file for offline viewing
## Models
| Model | Type | Parameters | Architecture |
|-------|------|------------|--------------|
| Qwen3-1.7B-Base | Transformer | 1.7B | Dense attention |
| RWKV7-G1C-1.5B | RWKV | 1.5B | Linear attention |
## Technical Details
This tool uses the [UncheatableEval](https://github.com/Jellyfish042/UncheatableEval) framework to:
1. Tokenize input text with each model's tokenizer
2. Calculate per-token cross-entropy loss
3. Map token losses to byte-level losses
4. Generate interactive HTML visualization
## Local Development
```bash
# Clone the repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/UncheatableEval-Visualization
# Install dependencies
pip install -r requirements.txt
# Run locally
python app.py
```
## Regression Checks (Recommended)
Run these after UI or rendering changes:
```bash
# Generate baseline snapshots
conda run -n torch2 python tests/generate_snapshots.py --out tests/golden
# Generate candidate snapshots
conda run -n torch2 python tests/generate_snapshots.py --out tests/_out
# Compare render-model JSON
conda run -n torch2 python tests/compare_snapshots.py --baseline tests/golden/stress.render_model.json --candidate tests/_out/stress.render_model.json
# Compare HTML output
conda run -n torch2 python tests/compare_html.py --baseline tests/golden/stress.output.html --candidate tests/_out/stress.output.html
# Optional: visual smoke placeholder
conda run -n torch2 python tests/visual_smoke.py --html tests/_out/stress.output.html
```
## Requirements
- CUDA-capable GPU (16GB+ VRAM recommended)
- Python 3.10+
- See `requirements.txt` for package dependencies
## License
MIT License
## Acknowledgments
- [UncheatableEval](https://github.com/Jellyfish042/UncheatableEval) - Original evaluation framework
- [Qwen](https://github.com/QwenLM/Qwen) - Qwen model family
- [RWKV](https://github.com/BlinkDL/RWKV-LM) - RWKV model family
|