README.md

---
title: RustFS S3-Compatible Storage
emoji: 🗄️
colorFrom: yellow
colorTo: red
sdk: docker
pinned: false
license: apache-2.0
tags:
  - storage
  - s3
  - rustfs
  - object-storage
  - learning
---

# 🗄️ RustFS S3-Compatible Storage

**High-performance Rust-based object storage for learning and experimentation**

[![Status](https://img.shields.io/badge/status-experimental-orange.svg)](https://huggingface.co/spaces/YOUR_USERNAME/rustfs-storage)
[![License](https://img.shields.io/badge/license-Apache%202.0-blue.svg)](LICENSE)
[![RustFS](https://img.shields.io/badge/RustFS-1.0.0--alpha.72-red.svg)](https://github.com/rustfs/rustfs)

## 📋 Overview

This is a **learning project** demonstrating RustFS deployment on HuggingFace Spaces with:

- ✅ **S3-Compatible API** - Works with AWS SDKs and tools
- ✅ **Public Read Access** - Anyone can view/download files
- ✅ **Authenticated Write** - Only authenticated users can upload/delete
- ✅ **Web Console** - Built-in management UI
- ✅ **Presigned URLs** - Generate temporary upload/download links
- ⚠️ **Alpha Software** - Use for learning, not production!

## 🚀 Quick Start

### Access Points

| Service | URL | Authentication |
|---------|-----|----------------|
| **S3 API** | `https://YOUR_USERNAME-rustfs-storage.hf.space` | Required for writes |
| **Console** | `https://YOUR_USERNAME-rustfs-storage.hf.space/console/` | Required |
| **Health** | `https://YOUR_USERNAME-rustfs-storage.hf.space/health` | Public |

### Default Bucket

- **Name**: `storage`
- **Public Read**: ✅ Yes (anyone can download)
- **Public Write**: ❌ No (authentication required)

## 🔐 Authentication

Set these as **HuggingFace Secrets** (not in code!):

```bash
RUSTFS_ACCESS_KEY=your-access-key
RUSTFS_SECRET_KEY=your-secret-key
```

## 💻 Usage Examples

### 1. Upload File (Authenticated)

```bash
# Using AWS CLI
aws configure set aws_access_key_id YOUR_ACCESS_KEY
aws configure set aws_secret_access_key YOUR_SECRET_KEY

aws s3 cp myfile.jpg \
  s3://storage/images/myfile.jpg \
  --endpoint-url https://YOUR_USERNAME-rustfs-storage.hf.space
```

### 2. Download File (Public - No Auth)

```bash
# Direct download
curl "https://YOUR_USERNAME-rustfs-storage.hf.space/storage/images/myfile.jpg" \
  -o downloaded.jpg
```

### 3. Generate Presigned Upload URL

```python
import boto3
from botocore.config import Config

# Configure S3 client for RustFS
s3_client = boto3.client(
    's3',
    endpoint_url='https://YOUR_USERNAME-rustfs-storage.hf.space',
    aws_access_key_id='YOUR_ACCESS_KEY',
    aws_secret_access_key='YOUR_SECRET_KEY',
    config=Config(signature_version='s3v4'),
    region_name='us-east-1'
)

# Generate presigned upload URL (expires in 1 hour)
presigned_url = s3_client.generate_presigned_url(
    'put_object',
    Params={
        'Bucket': 'storage',
        'Key': 'uploads/file.jpg',
        'ContentType': 'image/jpeg'
    },
    ExpiresIn=3600
)

print(f"Upload URL: {presigned_url}")

# Use the URL to upload
import requests
with open('myfile.jpg', 'rb') as f:
    requests.put(presigned_url, data=f, headers={'Content-Type': 'image/jpeg'})
```

### 4. Generate Presigned Download URL

```python
# Generate presigned download URL (expires in 1 hour)
presigned_url = s3_client.generate_presigned_url(
    'get_object',
    Params={
        'Bucket': 'storage',
        'Key': 'images/file.jpg'
    },
    ExpiresIn=3600
)

print(f"Download URL: {presigned_url}")
```

### 5. List Files

```bash
# Using AWS CLI
aws s3 ls s3://storage/ \
  --endpoint-url https://YOUR_USERNAME-rustfs-storage.hf.space
```

### 6. Delete File (Authenticated)

```bash
# Using AWS CLI
aws s3 rm s3://storage/images/myfile.jpg \
  --endpoint-url https://YOUR_USERNAME-rustfs-storage.hf.space
```

## 🎯 Use Cases

### ✅ Perfect For:
- **Learning S3 API** - Experiment with object storage
- **Testing Applications** - Develop S3-compatible apps
- **Prototyping** - Quick storage solution for demos
- **Understanding Object Storage** - Hands-on experience

### ❌ Not Recommended For:
- **Production Data** - This is alpha software
- **Critical Files** - No backup guarantees on free tier
- **Large Scale** - HuggingFace Spaces has storage limits
- **Sensitive Data** - Public space, use with caution

## 🏗️ Architecture

```
┌─────────────┐
│   Client    │
└──────┬──────┘
       │
       ▼
┌─────────────┐     ┌──────────────┐
│    Nginx    │────▶│   RustFS     │
│   (Port     │     │   S3 API     │
│    7860)    │     │  (Port 9000) │
└─────────────┘     └──────────────┘
       │                    │
       │                    ▼
       │            ┌──────────────┐
       └───────────▶│   Console    │
                    │  (Port 9001) │
                    └──────────────┘
```

## 🔧 Configuration

### Environment Variables

| Variable | Default | Description |
|----------|---------|-------------|
| `RUSTFS_ACCESS_KEY` | `admin` | S3 access key (⚠️ Set in Secrets!) |
| `RUSTFS_SECRET_KEY` | `verysecret` | S3 secret key (⚠️ Set in Secrets!) |
| `DEFAULT_BUCKET` | `storage` | Default bucket name |
| `CLIENT_MAX_BODY_SIZE` | `0` | Max upload size (0 = unlimited) |
| `ALLOWED_ORIGINS` | `*` | CORS allowed origins |
| `WORKER_CONNECTIONS` | `1024` | Nginx worker connections |
| `NGINX_LOG_LEVEL` | `error` | Nginx log level |

### Storage Limits

- **Max File Size**: Unlimited (but HF Spaces has overall storage limits)
- **Total Storage**: Depends on HuggingFace Space tier
- **Concurrent Connections**: 1024

## 📊 Performance

RustFS claims **2.3x faster than MinIO** for small (4KB) objects, but:
- ⚠️ This is alpha software with known issues
- Performance varies based on workload
- Not yet optimized for all scenarios

## ⚠️ Important Notes

### Alpha Software Warning

- **RustFS is in alpha** (v1.0.0-alpha.72 - Latest: Dec 5, 2024)
- May have bugs and breaking changes
- Not production-ready
- Use for **learning purposes only**

### Known Limitations

- Some S3 features may be incomplete
- Console UI has minor bugs
- Performance not fully optimized
- Limited community support compared to MinIO

### Security Considerations

- **Public Read Access**: All files in bucket are publicly readable
- **No Directory Listing**: Bucket contents not publicly listed
- **Authentication Required**: Upload/delete needs valid credentials
- **HTTPS**: All traffic encrypted via HuggingFace proxy

## 📚 Documentation

- [RustFS GitHub](https://github.com/rustfs/rustfs)
- [RustFS Documentation](https://docs.rustfs.com)
- [S3 API Reference](https://docs.aws.amazon.com/AmazonS3/latest/API/)
- [MinIO Client (mc) Guide](https://min.io/docs/minio/linux/reference/minio-mc.html)

## 🛠️ Troubleshooting

### "403 Forbidden" on Upload

**Problem**: Can't upload files
**Solution**: Make sure you're using correct credentials and authenticating properly

```bash
# Check your credentials
aws configure list

# Test with explicit credentials
aws s3 cp test.txt s3://storage/test.txt \
  --endpoint-url YOUR_ENDPOINT \
  --profile YOUR_PROFILE
```

### CORS Errors in Browser

**Problem**: Browser blocks API requests
**Solution**: Make sure `ALLOWED_ORIGINS` includes your domain

```bash
# In HuggingFace Secrets
ALLOWED_ORIGINS=https://yourdomain.com
```

### Presigned URL Not Working

**Problem**: Presigned URLs return errors
**Solution**: 
- Make sure signature version is `s3v4`
- Check that endpoint URL matches exactly
- Verify region is set (use `us-east-1`)

## 🤝 Contributing

This is a learning project! Feel free to:
- Fork and modify
- Report issues (but remember it's alpha software)
- Share your experiments
- Compare with MinIO/SeaweedFS

## 📄 License

- **This Space**: Apache 2.0 (Free to use!)
- **RustFS**: Apache 2.0
- **Learning Project**: Use freely for education

## 🙏 Acknowledgments

- **RustFS Team**: For building this awesome Rust-based storage
- **HuggingFace**: For free Spaces hosting
- **Community**: For testing and feedback

---

## 🎓 Learning Resources

### Compare with Other Storage Solutions

I've also deployed these for learning:
- **MinIO**: `YOUR_USERNAME/minio-storage` (Most mature)
- **SeaweedFS**: `YOUR_USERNAME/seaweedfs-storage` (Go-based)
- **Garage**: `YOUR_USERNAME/garage-storage` (Rust-based)
- **RustFS**: This space (Newest, experimental)

### Why Learn Object Storage?

- **Cloud Skills**: S3 is the standard for cloud storage
- **Scalability**: Learn distributed systems concepts
- **Modern Apps**: Most apps use object storage
- **Career Growth**: High demand skill

---

**Made with ❤️ for learning about distributed storage systems**

*⚠️ This is experimental software - use MinIO for production workloads!*