README.md

metadata

title: Compression-Lens
emoji: 🔬
colorFrom: blue
colorTo: green
sdk: gradio
sdk_version: 4.44.0
app_file: app.py
pinned: false
license: mit

UncheatableEval: Qwen3 vs RWKV7 Byte-Level Comparison

Compare the byte-level prediction performance between models.

Features

• Byte-level analysis: See exactly where each model performs better or worse
• Interactive visualization: Hover over tokens to see detailed predictions
• Color-coded comparison:
  • 🟢 Green = Qwen3 predicts better (lower loss)
  • 🔴 Red = RWKV7 predicts better (lower loss)
• Top-10 predictions: View each model's top predictions for every token
• Word occurrence linking: See how repeated words are predicted differently

How to Use

1. Enter or paste your text (max 4000 characters)
2. Click "Run Comparison"
3. Explore the interactive visualization
4. Download the HTML file for offline viewing

Models

Model	Type	Parameters	Architecture
Qwen3-1.7B-Base	Transformer	1.7B	Dense attention
RWKV7-G1C-1.5B	RWKV	1.5B	Linear attention

Technical Details

This tool uses the UncheatableEval framework to:

1. Tokenize input text with each model's tokenizer
2. Calculate per-token cross-entropy loss
3. Map token losses to byte-level losses
4. Generate interactive HTML visualization

Local Development

# Clone the repository
git clone https://huggingface.co/spaces/YOUR_USERNAME/UncheatableEval-Visualization

# Install dependencies
pip install -r requirements.txt

# Run locally
python app.py

Requirements

• CUDA-capable GPU (16GB+ VRAM recommended)
• Python 3.10+
• See requirements.txt for package dependencies

License

MIT License

Acknowledgments

• UncheatableEval - Original evaluation framework
• Qwen - Qwen model family
• RWKV - RWKV model family