| # π Setup Complete - Next Steps |
|
|
| ## β
What Was Created |
|
|
| ### 1. **Model Loading System** |
| - `scripts/model_loader.py` - Downloads & caches models from HuggingFace Hub |
| - `scripts/upload_models_to_hub.py` - Upload script for your 3 models |
| - `model_config.yaml` - Model metadata configuration |
|
|
| ### 2. **Configuration** |
| - `.env.example` - Template for environment variables |
| - `.gitignore` - Prevents committing large files and secrets |
|
|
| ### 3. **Documentation** |
| - `MODEL_SETUP.md` - Complete setup guide |
| - `README.md` - Updated with new architecture |
|
|
| ### 4. **App Updates** |
| - `app.py` - Now loads models from HuggingFace Hub automatically |
| - `requirements.txt` - Added `huggingface_hub` and `pyyaml` |
|
|
| --- |
|
|
| ## π Your Next Steps (In Order) |
|
|
| ### Step 1: Create `.env` File |
| ```bash |
| cd d:\DamageLens |
| copy .env.example .env |
| ``` |
|
|
| Edit `.env` in VS Code: |
| ```env |
| HF_MODEL_REPO=junaid17/damagelens-models |
| HF_TOKEN=hf_xxxxxxxxxxxxxxxxxxxx # β Replace with your actual token |
| MODEL_CACHE_DIR=./model_cache |
| ``` |
|
|
| **Get HF_TOKEN:** |
| 1. Go to https://huggingface.co/settings/tokens |
| 2. Click "New token" |
| 3. Choose "Write" permission |
| 4. Copy and paste in `.env` |
| |
| ### Step 2: Install New Dependencies |
| ```bash |
| pip install -r requirements.txt |
| ``` |
| |
| This adds: |
| - `huggingface_hub` - for model downloads |
| - `pyyaml` - for config reading |
| |
| ### Step 3: Upload Models to HuggingFace |
| ```bash |
| python scripts/upload_models_to_hub.py |
| ``` |
| |
| This will: |
| - Create repo: `https://huggingface.co/junaid17/damagelens-models` |
| - Upload all 3 models from `checkpoints/` folder |
| - Show progress for each upload |
| |
| **This is one-time setup!** After this, you don't need the local `checkpoints/` folder. |
| |
| ### Step 4: Test the App |
| ```bash |
| python app.py |
| ``` |
| |
| On first run, it will: |
| - Download models from HuggingFace (~1.4 GB) |
| - Cache them locally in `model_cache/` |
| - Start the server |
| |
| On subsequent runs, it will: |
| - Use cached models |
| - Start in ~5 seconds |
| |
| --- |
| |
| ## π How It Works Now |
| |
| ``` |
| Your Computer |
| β |
| [app.py starts] |
| β |
| [calls model_loader.py] |
| β |
| ββ Check if model cached locally? |
| β |
| ββ Yes β Use cached model β
(fast) |
| β |
| ββ No β Download from HuggingFace Hub β Cache it β Use it β
|
| ``` |
| |
| **Benefits:** |
| - β
No large files in git repo |
| - β
Easy deployment to Docker/HF Spaces |
| - β
Models shared across multiple servers |
| - β
Version control for models |
| - β
Offline usage after caching |
| |
| --- |
| |
| ## π File Structure After Setup |
| |
| ``` |
| DamageLens/ |
| βββ .env β Your config (do NOT commit) |
| βββ .env.example β Template (commit this) |
| βββ .gitignore β Prevents committing large files |
| βββ app.py β Now uses model_loader |
| βββ requirements.txt β Updated |
| βββ MODEL_SETUP.md β Detailed guide |
| βββ README.md β Updated |
| βββ model_config.yaml β Model metadata |
| βββ scripts/ |
| β βββ model_loader.py β NEW: Downloads models |
| β βββ upload_models_to_hub.py β NEW: Upload script |
| β βββ prediction_helper.py β Unchanged |
| β βββ gradcam.py β Unchanged |
| β βββ yolo.py β Unchanged |
| βββ checkpoints/ β (Optional after upload) |
| β βββ best_resnet_model.pt β Will be on HuggingFace |
| β βββ best_fusion_model.pt β Will be on HuggingFace |
| β βββ damage_detector.pt β Will be on HuggingFace |
| βββ model_cache/ β AUTO CREATED: Downloaded models |
| βββ best_resnet_model.pt β Cached after first download |
| βββ best_fusion_model.pt β Cached after first download |
| βββ damage_detector.pt β Cached after first download |
| ``` |
| |
| --- |
| |
| ## π Security Notes |
| |
| **DO NOT commit `.env` file!** |
| - `.gitignore` already prevents this |
| - Contains sensitive HuggingFace token |
|
|
| **For Deployment:** |
| - Add `HF_TOKEN` to platform secrets |
| - Example (HuggingFace Spaces): |
| - Settings β Secrets β Add `HF_TOKEN` |
|
|
| --- |
|
|
| ## β¨ What Changed in App |
|
|
| **Before:** |
| ```python |
| resnet_checkpoint = "checkpoints/best_resnet_model.pt" |
| Resnet_Model = ResnetCarDamagePredictor(resnet_checkpoint, class_map) |
| ``` |
|
|
| **After:** |
| ```python |
| from scripts.model_loader import initialize_models |
| Resnet_Model, Fusion_Model, loader = initialize_models(class_map) |
| ``` |
|
|
| The app now: |
| - β
Automatically downloads models from HuggingFace |
| - β
Caches them locally |
| - β
Requires no changes to prediction endpoints |
| - β
Works offline after first run |
|
|
| --- |
|
|
| ## π Troubleshooting |
|
|
| | Issue | Solution | |
| |-------|----------| |
| | `ModuleNotFoundError: huggingface_hub` | Run `pip install -r requirements.txt` | |
| | `HF_TOKEN not found` | Create `.env` file with token | |
| | `Repository not found` | Check token has write permission | |
| | `Slow startup` | Normal on first run (downloading ~1.4GB) | |
| | `Out of disk space` | Delete `model_cache/` and re-download | |
|
|
| --- |
|
|
| ## π Further Reading |
|
|
| See [MODEL_SETUP.md](MODEL_SETUP.md) for: |
| - Detailed troubleshooting |
| - Docker deployment |
| - HuggingFace Spaces setup |
| - Model verification |
|
|
| --- |
|
|
| ## β
Checklist |
|
|
| - [ ] Create `.env` file with HF_TOKEN |
| - [ ] Run `pip install -r requirements.txt` |
| - [ ] Run `python scripts/upload_models_to_hub.py` |
| - [ ] Run `python app.py` and verify it downloads models |
| - [ ] Test the web UI at http://localhost:8000 |
| - [ ] Commit code (without `.env` and `checkpoints/`) |
|
|
| --- |
|
|
| **You're all set!** π The app is now ready to load models from HuggingFace Hub. |
|
|