Spaces:

---

infosave

/

bounce-archiver

like 8

Sleeping

🚀 Problem Statement

Downloading large AI models (10-100 GB) from Hugging Face Hub is:

• Time-consuming for users on slower connections
• Expensive in terms of bandwidth costs for both HF and users
• Storage-intensive for Hugging Face infrastructure

Current compression options (gzip, zstd) are not optimized for neural network weights (IEEE-754 float tensors).

💡 Proposed Solution

Integrate bounce compression natively into Hugging Face Hub:

Key Benefits:

• ⚡ 25% average compression on model weights (.safetensors, .pt, .gguf)
• 🚀 1069 MB/s decompression speed — faster than most network connections
• 🧠 Specialized for ML: byte-shuffle transform optimized for IEEE-754 tensors
• 🔒 CRC-32 integrity verification built-in
• 🧑‍💻 Zero dependencies: pure Rust, Apache-2.0 license

📊 Benchmark: Safetensors Model Weights (255.5 MB)

bounce saves 17.2 MB (7% better) than the next best tool (brotli) while maintaining 5x faster decompression.

🛠️ Proposed Integration

1. CLI Integration (huggingface-cli)

# Download with automatic decompression
huggingface-cli download model/name --compress bounce

# Upload with compression
huggingface-cli upload model/name --compress bounce

2. Python SDK (huggingface_hub)

from huggingface_hub import hf_hub_download

# Automatic transparent decompression
path = hf_hub_download(
    repo_id="model/name",
    filename="model.safetensors",
    compression="bounce"  # auto-decompress .bnc files
)

3. Server-side optimization

• Store popular models in .bnc format
• On-the-fly compression for new uploads
• Backward compatibility: serve both .safetensors and .safetensors.bnc

💰 ROI for Hugging Face

Storage Savings:

• 25% reduction across millions of models
• Example: 1 PB of models → 250 TB saved

Bandwidth Savings:

• 25% less egress traffic
• Significant cost reduction for CDN/bandwidth

User Experience:

• Faster downloads for users worldwide
• Lower data costs for mobile/metered connections
• Improved accessibility in regions with slower internet

🔗 Resources

• Demo Space: bounce-archiver
• GitHub: https://github.com/infosave2007/bounce
• Install: cargo install nvg-bounce
• License: Apache-2.0

❓ Open Questions

1. Should this be opt-in or automatic for large files?
2. Backward compatibility strategy for existing downloads?
3. Integration timeline with huggingface_hub Python package?
4. Server-side infrastructure requirements?

I'm happy to collaborate on implementation — the bounce compressor is production-ready, well-tested, and designed specifically for this use case.

What do you think? 🤔