PYTHON_MIGRATION.md

🐍 Python Conversion Summary

Vico telah dikonversi dari Node.js menjadi Python dengan menggunakan library resmi huggingface_hub.

✨ Perubahan Utama

File Python Baru

• server.py - Flask API server (menggantikan server.js)
• hf_uploader.py - Utility upload dengan huggingface_hub library
• requirements.txt - Dependencies Python
• examples/upload-file.py - Contoh upload via API (Python)
• examples/direct-hf-upload.py - Contoh direct upload ke HF

Fitur Baru

✅ Menggunakan library resmi huggingface_hub
✅ Support direct upload dengan upload_file() dan upload_folder()
✅ Better error handling
✅ Support upload folder (tidak hanya file)
✅ Path structure: /u/ untuk semua file

🚀 Quick Start

1. Install Dependencies

pip install -r requirements.txt

2. Setup Environment

cp .env.example .env
# Edit .env dan set HF_TOKEN dan HF_REPO_ID

3. Run Server

python3 server.py

4. Upload File

Via API:

curl -X POST http://localhost:3000/api/upload \
  -F "file=@data.csv"

Via Python (Direct):

from huggingface_hub import login, upload_file

login(token='hf_...')
upload_file(
    path_or_fileobj='data.csv',
    path_in_repo='u/data.csv',
    repo_id='username/dataset',
    repo_type='dataset'
)

📝 API Endpoints

POST /api/upload

Upload single file ke Hugging Face dataset

Params:

• file - File untuk di-upload
• repo_id - Target dataset (atau dari .env)
• token - HF token (atau dari .env)
• subfolder - Optional subfolder

Response:

{
  "success": true,
  "data": {
    "url": "https://huggingface.co/datasets/.../blob/main/u/...",
    "file_path": "u/...",
    "file_size": 12345,
    "upload_timestamp": "2024-06-24T..."
  }
}

🔄 Migration Guide

Dari Node.js (Old)

// Old: Node.js
npm start

Ke Python (New)

# New: Python
python3 server.py

📚 Library: huggingface_hub

Dokumentasi: https://huggingface.co/docs/hub/security-tokens

Contoh Penggunaan

from huggingface_hub import login, upload_file, upload_folder

# Authenticate
login(token='hf_...')

# Upload single file
upload_file(
    path_or_fileobj='path/to/file.csv',
    path_in_repo='u/file.csv',
    repo_id='username/dataset-name',
    repo_type='dataset',
    commit_message='Upload data'
)

# Upload entire folder
upload_folder(
    folder_path='path/to/folder',
    repo_id='username/dataset-name',
    repo_type='dataset',
    path_in_repo='u/folder',
    commit_message='Upload folder'
)

🔑 Environment Variables

Setup di .env:

HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxxxxx
HF_REPO_ID=username/dataset-name
PORT=3000
NODE_ENV=development

📦 Dependencies

Flask==2.3.3          # Web framework
python-dotenv==1.0.0  # Environment variables
huggingface-hub==0.16.4  # Official HF library
requests==2.31.0      # HTTP client

🧪 Testing

Test server health

curl http://localhost:3000

Test upload (dengan .env set)

python3 examples/upload-file.py

Test direct upload

python3 examples/direct-hf-upload.py

✅ Checklist

• Server berjalan di port 3000
• API endpoint /api/upload working
• Environment variables support
• Response format JSON
• Error handling
• Direct upload support
• Folder upload support
• Docker support

📖 File Reference

File	Purpose
server.py	Flask web server
hf_uploader.py	Upload utility dengan HF library
requirements.txt	Python dependencies
examples/upload-file.py	Upload via API
examples/direct-hf-upload.py	Direct upload
Dockerfile	Docker image

🚀 Production Deployment

Dengan Gunicorn

pip install gunicorn
gunicorn -w 4 -b 0.0.0.0:3000 server:app

Dengan Docker

docker build -t vico-api .
docker run -p 3000:3000 \
  -e HF_TOKEN=your_token \
  -e HF_REPO_ID=username/dataset \
  vico-api

---

Note: Old Node.js files (server.js, package.json) masih ada untuk referensi tapi tidak lagi digunakan. Gunakan Python files untuk development baru.