HUGGINGFACE_SETUP.md

Hugging Face Hub Storage Setup Guide

⚠️ Important Limitations

Hugging Face Hub will NOT solve Vercel's 2GB upload limit because:

• Files still go through Vercel serverless functions
• Vercel has a 4.5MB (free) / 50MB (Pro) request body limit
• Vercel has a 10-second (Hobby) / 60-second (Pro) timeout

For true 2GB uploads, you need:

• Direct client uploads (bypassing Vercel)
• Services like AWS S3, Cloudflare R2, or Backblaze B2 with pre-signed URLs

Setup Instructions

1. Create Hugging Face Account

1. Go to https://huggingface.co/join
2. Create a free account

2. Create a Repository

1. Go to https://huggingface.co/new-dataset (or https://huggingface.co/new-model)
2. Choose:
   • Name: voltedge-storage (or your preferred name)
   • Type: Dataset (recommended) or Model
   • Visibility: Public (free unlimited) or Private (limited storage)
3. Click "Create repository"

3. Generate Access Token

1. Go to https://huggingface.co/settings/tokens
2. Click "New token"
3. Set:
   • Name: VoltEdge Storage
   • Type: Write (needed for uploads)
4. Copy the token (you'll only see it once!)

4. Configure Environment Variables

Add to your .env.local or Vercel environment variables:

# Hugging Face Hub Configuration
HF_TOKEN=your_huggingface_token_here
HF_REPO_ID=your-username/voltedge-storage
HF_REPO_TYPE=dataset
USE_HF_STORAGE=true

Variables:

• HF_TOKEN: Your Hugging Face access token (from step 3)
• HF_REPO_ID: Your repository ID (format: username/repo-name)
• HF_REPO_TYPE: Either dataset or model (default: dataset)
• USE_HF_STORAGE: Set to true to enable HF Hub, false to use Telegram

5. Deploy

After setting environment variables, redeploy your Vercel app. The system will:

• Use HF Hub if USE_HF_STORAGE=true and credentials are set
• Fall back to Telegram if HF Hub fails or is not configured

How It Works

1. Upload Flow:

   Client → Vercel API → HF Hub API → File stored in HF repository
   

2. File Access:

   Client → Vercel API → HF CDN URL → File served from HF CDN
   

3. Storage Location:

   • Files are stored in: files/{fileId}.{ext}
   • Accessible via: https://cdn.huggingface.co/{repo-id}/main/files/{fileId}.{ext}

Testing

1. Set USE_HF_STORAGE=true in your environment
2. Upload a small file (< 50MB) via the web interface
3. Check your HF repository to see the uploaded file
4. Access the file via the generated URL

Troubleshooting

Error: "Hugging Face credentials not configured"

• Check that HF_TOKEN and HF_REPO_ID are set
• Verify the token has "Write" permissions

Error: "Repository not found"

• Check HF_REPO_ID format: username/repo-name
• Ensure the repository exists and is accessible

Upload fails for large files

• This is expected! Vercel limits apply
• Files > 50MB will fail on Vercel
• Consider using direct client uploads (S3, R2, etc.)

Files not appearing in repository

• Check HF Hub API rate limits
• Verify token permissions
• Check repository visibility settings

Rate Limits

• Free Tier: ~1000 API calls/hour
• Bandwidth: ~10GB/month download
• Upload: No hard limit, but rate-limited

Alternative: Direct Client Uploads

For true 2GB support, consider:

• Cloudflare R2: Free tier, S3-compatible, pre-signed URLs
• AWS S3: Industry standard, pre-signed URLs
• Backblaze B2: Cheap, S3-compatible

These allow direct client uploads, bypassing Vercel completely.