--- title: fe_03f1 emoji: 😂 colorFrom: blue colorTo: gray sdk: docker app_port: 3000 pinned: false --- # Vico - Hugging Face File Upload API API Python Flask untuk upload file ke Hugging Face Dataset dengan response format JSON. Menggunakan library resmi `huggingface_hub`. ## 🚀 Fitur - ✅ Upload file ke Hugging Face Dataset via REST API - ✅ Menggunakan library resmi `huggingface_hub` - ✅ Validasi file dan error handling - ✅ Response JSON terstruktur - ✅ Support subfolder dalam repository - ✅ File size limit 100MB - ✅ Environment variables support (HF_TOKEN, HF_REPO_ID) - ✅ Automatic temporary file cleanup ## 📋 Prerequisites - Python 3.8 atau lebih tinggi - pip (Python package manager) - Hugging Face account dengan access token ## 🔧 Instalasi ### 1. Clone Repository ```bash git clone https://github.com/4cko/vico.git cd vico ``` ### 2. Install Dependencies ```bash pip install -r requirements.txt ``` ### 3. Setup Environment Variables ```bash cp .env.example .env # Edit .env sesuai kebutuhan (opsional) ``` ### 4. Dapatkan Hugging Face Token 1. Buka https://huggingface.co/settings/tokens 2. Buat token baru dengan akses 'write' 3. Simpan token tersebut ## 🎯 Cara Menggunakan ### 1. Jalankan Server ```bash # Production mode python3 server.py # Development mode (dengan auto-reload) python3 -m flask run --reload ``` Server akan berjalan di `http://localhost:3000` ### 2. Upload File **Endpoint:** `POST /api/upload` **Required Fields:** - `file` - File yang akan di-upload (form-data) - `repo_id` - Hugging Face repo ID (format: `username/dataset-name`) *atau set di .env* - `token` - Hugging Face API token *atau set di .env* **Optional Fields:** - `subfolder` - Subfolder dalam repository (default: root) **Catatan:** Jika `repo_id` dan `token` sudah di-set di `.env` (HF_REPO_ID dan HF_TOKEN), Anda tidak perlu mengirimnya di request. Nilai dari request akan override nilai di .env. #### Contoh menggunakan cURL: ```bash curl -X POST http://localhost:3000/api/upload \ -F "file=@/path/to/file.csv" \ -F "repo_id=username/my-dataset" \ -F "token=hf_xxxxxxxxxxxxxxxxxxxxxxx" \ -F "subfolder=data" ``` #### Contoh menggunakan Python: ```python import requests files = { 'file': open('/path/to/file.csv', 'rb') } data = { 'repo_id': 'username/my-dataset', 'token': 'hf_xxxxxxxxxxxxxxxxxxxxxxx', 'subfolder': 'data' } response = requests.post( 'http://localhost:3000/api/upload', files=files, data=data ) print(response.json()) ``` #### Direct Upload dengan huggingface_hub Anda juga bisa upload langsung tanpa API server: ```python from huggingface_hub import login, upload_file, upload_folder # Authenticate login(token='hf_your_token_here') # Upload single file upload_file( path_or_fileobj='/path/to/file.csv', path_in_repo='u/file.csv', repo_id='username/my-dataset', repo_type='dataset', commit_message='Upload data' ) # Atau upload entire folder upload_folder( folder_path='/path/to/folder', repo_id='username/my-dataset', repo_type='dataset', path_in_repo='u/folder', commit_message='Upload folder' ) ``` #### Dengan Environment Variables (.env) Jika sudah set `.env` dengan `HF_TOKEN` dan `HF_REPO_ID`: **cURL:** ```bash curl -X POST http://localhost:3000/api/upload \ -F "file=@/path/to/file.csv" ``` **Python:** ```python import requests files = {'file': open('file.csv', 'rb')} # repo_id dan token akan diambil dari .env r = requests.post('http://localhost:3000/api/upload', files=files) print(r.json()) ``` #### Contoh menggunakan huggingface_hub (Direct Upload) ```python from huggingface_hub import login, upload_file # Authenticate login(token='hf_your_token_here') # Upload file upload_file( path_or_fileobj='/path/to/file.csv', path_in_repo='u/file.csv', repo_id='username/my-dataset', repo_type='dataset', commit_message='Upload data' ) print('✅ File uploaded!') ``` ### 3. Response Format #### ✅ Sukses Upload (Status 200): ```json { "success": true, "message": "File berhasil di-upload ke Hugging Face", "data": { "file_name": "data.csv", "file_size": 12345, "repo_id": "username/my-dataset", "path_prefix": "/u/", "subfolder": "data", "url": "https://huggingface.co/datasets/username/my-dataset/blob/main/u/data/data.csv", "file_path": "u/data/data.csv", "upload_timestamp": "2024-06-24T10:30:00Z" } } ``` #### ❌ Error Response (Status 400/500): ```json { "success": false, "error": "repo_id dan token diperlukan (kirim via request atau set di .env)", "required_fields": ["repo_id", "token"], "hint": "Set HF_REPO_ID dan HF_TOKEN di .env atau kirim via form data" } ``` ```json { "success": false, "error": "Token tidak valid atau expired" } ``` ```json { "success": false, "error": "Repository tidak ditemukan: invalid/repo" } ``` ## 📝 API Documentation ### GET / Health check endpoint **Response:** ```json { "status": "ok", "message": "Hugging Face File Upload API", "version": "1.0.0", "language": "Python", "framework": "Flask", "endpoints": { "upload": { "method": "POST", "path": "/api/upload", "description": "Upload file ke Hugging Face dataset", "required": ["file", "repo_id", "token"], "optional": ["subfolder"] } } } ``` ## 🔐 Security Notes 1. **Token Protection:** - Jangan commit `.env` file ke repository - Gunakan environment variables untuk production - Rotate token secara berkala 2. **File Validation:** - Max file size: 100MB - File validation dilakukan pada server - Automatic cleanup file lokal setelah upload 3. **Error Handling:** - Token tidak valid - Repository tidak ditemukan - Access denied - Network errors ## 📂 Project Structure ``` vico/ ├── server.py # Main Flask server ├── hf_uploader.py # Hugging Face upload utility ├── requirements.txt # Python dependencies ├── .env.example # Environment variables template ├── README.md # Dokumentasi ├── examples/ │ ├── upload-file.py # Contoh upload via API │ ├── direct-hf-upload.py # Contoh direct upload ke HF │ ├── curl-upload.sh # Contoh cURL │ ├── env-upload.sh # Contoh menggunakan .env │ └── sample-data.csv # Sample data file └── uploads/ # Temporary folder untuk uploaded files ``` ## 🐛 Troubleshooting ### Token tidak valid ``` Error: Token tidak valid atau expired ``` **Solusi:** Check token di https://huggingface.co/settings/tokens ### Repository tidak ditemukan ``` Error: Repository tidak ditemukan: username/dataset ``` **Solusi:** - Pastikan dataset sudah di-create di Hugging Face - Format: `username/dataset-name` (lowercase) ### File terlalu besar ``` Error: File too large ``` **Solusi:** Max file size adalah 100MB. Ubah di `server.py` jika perlu ### Connection timeout ``` Error: Connection timeout ``` **Solusi:** Untuk file besar, process mungkin memerlukan waktu lebih lama. Tunggu lebih lama atau tingkatkan timeout. ### Module not found ``` ModuleNotFoundError: No module named 'flask' ``` **Solusi:** Install dependencies dengan `pip install -r requirements.txt` ## 🚀 Deployment ### Menggunakan Gunicorn (Production) ```bash pip install gunicorn gunicorn -w 4 -b 0.0.0.0:3000 server:app ``` ### Menggunakan Docker ```bash docker build -t vico-api . docker run -p 3000:3000 \ -e HF_TOKEN=your_token \ -e HF_REPO_ID=username/dataset \ vico-api ``` ## 📄 License MIT ## 👤 Author 4cko ## 💬 Support Untuk masalah atau pertanyaan, buka issue di repository ini. --- **Catatan:** API ini menggunakan library resmi Hugging Face Hub. Pastikan token memiliki akses 'write' untuk repository yang dituju. Dokumentasi library: https://huggingface.co/docs/hub/security-tokens **Catatan:** API ini menggunakan Hugging Face Hub API. Pastikan token memiliki akses 'write' untuk repository yang dituju.