aiBatteryLifeCycle / STRUCTURE.md
NeerajCodz's picture
feat: full project β€” ML simulation, dashboard UI, models on HF Hub
f381be8
# Project Structure (v2.0)
## Root Level Organization
```
aiBatteryLifecycle/
β”œβ”€β”€ πŸ“‚ api/ FastAPI backend with model registry
β”œβ”€β”€ πŸ“‚ artifacts/ Versioned model artifacts & results
β”‚ β”œβ”€β”€ v1/ Legacy models (cross-battery train/test)
β”‚ └── v2/ Production models (intra-battery split) βœ“ ACTIVE
β”‚ β”œβ”€β”€ models/ Trained models: {classical, deep, ensemble}
β”‚ β”œβ”€β”€ scalers/ Feature scalers (StandardScaler for linear models)
β”‚ β”œβ”€β”€ results/ CSV results, feature matrices, metrics
β”‚ β”œβ”€β”€ figures/ Visualizations: PNG charts, HTML reports
β”‚ β”œβ”€β”€ logs/ Training/inference logs
β”‚ └── features/ Feature engineering artifacts
β”œβ”€β”€ πŸ“‚ cleaned_dataset/ Raw battery test data
β”‚ β”œβ”€β”€ data/ CSV files per battery (00001.csv - 00137.csv)
β”‚ β”œβ”€β”€ extra_infos/ Supplementary metadata
β”‚ └── metadata.csv Battery inventory
β”œβ”€β”€ πŸ“‚ docs/ Documentation markdown files
β”œβ”€β”€ πŸ“‚ frontend/ React SPA (TypeScript, Vite)
β”œβ”€β”€ πŸ“‚ notebooks/ Jupyter analysis & training (01-09)
β”œβ”€β”€ πŸ“‚ reference/ External papers & reference notebooks
β”œβ”€β”€ πŸ“‚ scripts/ Organized utility scripts
β”‚ β”œβ”€β”€ data/ Data processing (write_nb03_v2, patch_dl_notebooks_v2)
β”‚ β”œβ”€β”€ models/ Model training (retrain_classical)
β”‚ β”œβ”€β”€ __init__.py Package marker
β”‚ └── README (in data/models/)
β”œβ”€β”€ πŸ“‚ src/ Core Python library
β”‚ β”œβ”€β”€ data/ Data loading & preprocessing
β”‚ β”œβ”€β”€ evaluation/ Metrics & validation
β”‚ β”œβ”€β”€ models/ Model architectures
β”‚ β”œβ”€β”€ utils/ Config, logging, helpers
β”‚ └── __init__.py
β”œβ”€β”€ πŸ“‚ tests/ βœ“ NEW: Test & validation scripts
β”‚ β”œβ”€β”€ test_v2_models.py Comprehensive v2 validation
β”‚ β”œβ”€β”€ test_predictions.py Quick endpoint test
β”‚ β”œβ”€β”€ __init__.py
β”‚ └── README.md
β”œβ”€β”€ πŸ“„ CHANGELOG.md Version history & updates
β”œβ”€β”€ πŸ“„ VERSION.md βœ“ NEW: Versioning & versioning guide
β”œβ”€β”€ πŸ“„ README.md Project overview
β”œβ”€β”€ πŸ“„ requirements.txt Python dependencies
β”œβ”€β”€ πŸ“„ package.json Node.js dependencies (frontend)
β”œβ”€β”€ πŸ“„ Dockerfile Docker configuration
β”œβ”€β”€ πŸ“„ docker-compose.yml Multi-container orchestration
β”œβ”€β”€ πŸ“„ tsconfig.json TypeScript config (frontend)
└── πŸ“„ vite.config.ts Vite bundler config (frontend)
```
## Key Changes in V2 Reorganization
### βœ“ Completed
1. **Versioned Artifacts**
- Moved `artifacts/models/` β†’ `artifacts/v2/models/`
- Moved `artifacts/scalers/` β†’ `artifacts/v2/scalers/`
- Moved `artifacts/figures/` β†’ `artifacts/v2/figures/`
- All result CSVs β†’ `artifacts/v2/results/`
- Clean `artifacts/` root (only v1 and v2 subdirs)
2. **Organized Scripts**
- Created `scripts/data/` for data processing utilities
- Created `scripts/models/` for model training scripts
- All scripts now using `get_version_paths('v2')`
- Path: `scripts/retrain_classical.py` β†’ `scripts/models/retrain_classical.py`
3. **Centralized Tests**
- Created `tests/` folder at project root
- Moved `test_v2_models.py` β†’ `tests/`
- Moved `test_predictions.py` β†’ `tests/`
- Added `tests/README.md` with usage guide
- All tests now using `artifacts/v2/` paths
4. **Updated Imports & Paths**
- `test_v2_models.py`: Uses `v2['results']` for data loading
- `retrain_classical.py`: Uses `get_version_paths('v2')` for artifact saving
- API: Already defaults to `registry_v2`
- Notebooks NB03-09: Already use `get_version_paths()`
### Code Changes Summary
| File | Change | Result |
|------|--------|--------|
| `tests/test_v2_models.py` | Updated artifact paths to use v2 | Output β†’ `artifacts/v2/{results,figures}` |
| `scripts/models/retrain_classical.py` | Uses `get_version_paths('v2')` | Models saved to `artifacts/v2/models/classical/` |
| `api/model_registry.py` | Already has versioning support | No changes needed |
| `src/utils/config.py` | Already supports versioning | No changes needed |
| Notebooks NB03-09 | Already use `get_version_paths()` | No changes needed |
## Running Tests After Reorganization
```bash
# From project root
python tests/test_v2_models.py # Full v2 validation
python tests/test_predictions.py # Quick endpoint test
python scripts/models/retrain_classical.py # Retrain models
```
## Artifact Access in Code
### Before (V1 - hardcoded paths)
```python
model_path = "artifacts/models/classical/rf.joblib"
results_csv = "artifacts/results.csv"
```
### After (V2 - versioned paths via config)
```python
from src.utils.config import get_version_paths
v2 = get_version_paths('v2')
model_path = v2['models_classical'] / 'rf.joblib'
results_csv = v2['results'] / 'results.csv'
```
## Production Readiness
| Aspect | Status | Notes |
|--------|--------|-------|
| Versioning | βœ“ Complete | All artifacts under `v2/` |
| Structure | βœ“ Organized | Scripts, tests, notebooks organized |
| Configuration | βœ“ Active | `ACTIVE_VERSION = 'v2'` in config |
| API | βœ“ Ready | Defaults to `registry_v2` |
| Tests | βœ“ Available | `tests/test_v2_models.py` for validation |
| Documentation | βœ“ Added | VERSION.md and README files created |
## Forward Compatibility
### For Future Versions (v3, v4, etc.)
Simply copy the v2 folder structure and update:
```python
# In src/utils/config.py
ACTIVE_VERSION: str = "v3"
# In scripts
v3 = get_version_paths('v3')
ensure_version_dirs('v3')
```
The system will automatically create versioned paths and maintain backward compatibility.