aiBatteryLifeCycle / STRUCTURE.md
NeerajCodz's picture
feat: full project β€” ML simulation, dashboard UI, models on HF Hub
f381be8

Project Structure (v2.0)

Root Level Organization

aiBatteryLifecycle/
β”œβ”€β”€ πŸ“‚ api/                    FastAPI backend with model registry
β”œβ”€β”€ πŸ“‚ artifacts/              Versioned model artifacts & results
β”‚   β”œβ”€β”€ v1/                    Legacy models (cross-battery train/test)
β”‚   └── v2/                    Production models (intra-battery split) βœ“ ACTIVE
β”‚       β”œβ”€β”€ models/            Trained models: {classical, deep, ensemble}
β”‚       β”œβ”€β”€ scalers/           Feature scalers (StandardScaler for linear models)
β”‚       β”œβ”€β”€ results/           CSV results, feature matrices, metrics
β”‚       β”œβ”€β”€ figures/           Visualizations: PNG charts, HTML reports
β”‚       β”œβ”€β”€ logs/              Training/inference logs
β”‚       └── features/          Feature engineering artifacts
β”œβ”€β”€ πŸ“‚ cleaned_dataset/        Raw battery test data
β”‚   β”œβ”€β”€ data/                  CSV files per battery (00001.csv - 00137.csv)
β”‚   β”œβ”€β”€ extra_infos/           Supplementary metadata
β”‚   └── metadata.csv           Battery inventory
β”œβ”€β”€ πŸ“‚ docs/                   Documentation markdown files
β”œβ”€β”€ πŸ“‚ frontend/               React SPA (TypeScript, Vite)
β”œβ”€β”€ πŸ“‚ notebooks/              Jupyter analysis & training (01-09)
β”œβ”€β”€ πŸ“‚ reference/              External papers & reference notebooks
β”œβ”€β”€ πŸ“‚ scripts/                Organized utility scripts
β”‚   β”œβ”€β”€ data/                  Data processing (write_nb03_v2, patch_dl_notebooks_v2)
β”‚   β”œβ”€β”€ models/                Model training (retrain_classical)
β”‚   β”œβ”€β”€ __init__.py            Package marker
β”‚   └── README (in data/models/)
β”œβ”€β”€ πŸ“‚ src/                    Core Python library
β”‚   β”œβ”€β”€ data/                  Data loading & preprocessing
β”‚   β”œβ”€β”€ evaluation/            Metrics & validation
β”‚   β”œβ”€β”€ models/                Model architectures
β”‚   β”œβ”€β”€ utils/                 Config, logging, helpers
β”‚   └── __init__.py
β”œβ”€β”€ πŸ“‚ tests/                  βœ“ NEW: Test & validation scripts
β”‚   β”œβ”€β”€ test_v2_models.py      Comprehensive v2 validation
β”‚   β”œβ”€β”€ test_predictions.py    Quick endpoint test
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── README.md
β”œβ”€β”€ πŸ“„ CHANGELOG.md            Version history & updates  
β”œβ”€β”€ πŸ“„ VERSION.md              βœ“ NEW: Versioning & versioning guide
β”œβ”€β”€ πŸ“„ README.md               Project overview
β”œβ”€β”€ πŸ“„ requirements.txt        Python dependencies
β”œβ”€β”€ πŸ“„ package.json            Node.js dependencies (frontend)
β”œβ”€β”€ πŸ“„ Dockerfile              Docker configuration
β”œβ”€β”€ πŸ“„ docker-compose.yml      Multi-container orchestration
β”œβ”€β”€ πŸ“„ tsconfig.json           TypeScript config (frontend)
└── πŸ“„ vite.config.ts          Vite bundler config (frontend)

Key Changes in V2 Reorganization

βœ“ Completed

  1. Versioned Artifacts

    • Moved artifacts/models/ β†’ artifacts/v2/models/
    • Moved artifacts/scalers/ β†’ artifacts/v2/scalers/
    • Moved artifacts/figures/ β†’ artifacts/v2/figures/
    • All result CSVs β†’ artifacts/v2/results/
    • Clean artifacts/ root (only v1 and v2 subdirs)
  2. Organized Scripts

    • Created scripts/data/ for data processing utilities
    • Created scripts/models/ for model training scripts
    • All scripts now using get_version_paths('v2')
    • Path: scripts/retrain_classical.py β†’ scripts/models/retrain_classical.py
  3. Centralized Tests

    • Created tests/ folder at project root
    • Moved test_v2_models.py β†’ tests/
    • Moved test_predictions.py β†’ tests/
    • Added tests/README.md with usage guide
    • All tests now using artifacts/v2/ paths
  4. Updated Imports & Paths

    • test_v2_models.py: Uses v2['results'] for data loading
    • retrain_classical.py: Uses get_version_paths('v2') for artifact saving
    • API: Already defaults to registry_v2
    • Notebooks NB03-09: Already use get_version_paths()

Code Changes Summary

File Change Result
tests/test_v2_models.py Updated artifact paths to use v2 Output β†’ artifacts/v2/{results,figures}
scripts/models/retrain_classical.py Uses get_version_paths('v2') Models saved to artifacts/v2/models/classical/
api/model_registry.py Already has versioning support No changes needed
src/utils/config.py Already supports versioning No changes needed
Notebooks NB03-09 Already use get_version_paths() No changes needed

Running Tests After Reorganization

# From project root
python tests/test_v2_models.py      # Full v2 validation
python tests/test_predictions.py    # Quick endpoint test
python scripts/models/retrain_classical.py  # Retrain models

Artifact Access in Code

Before (V1 - hardcoded paths)

model_path = "artifacts/models/classical/rf.joblib"
results_csv = "artifacts/results.csv"

After (V2 - versioned paths via config)

from src.utils.config import get_version_paths
v2 = get_version_paths('v2')

model_path = v2['models_classical'] / 'rf.joblib'
results_csv = v2['results'] / 'results.csv'

Production Readiness

Aspect Status Notes
Versioning βœ“ Complete All artifacts under v2/
Structure βœ“ Organized Scripts, tests, notebooks organized
Configuration βœ“ Active ACTIVE_VERSION = 'v2' in config
API βœ“ Ready Defaults to registry_v2
Tests βœ“ Available tests/test_v2_models.py for validation
Documentation βœ“ Added VERSION.md and README files created

Forward Compatibility

For Future Versions (v3, v4, etc.)

Simply copy the v2 folder structure and update:

# In src/utils/config.py
ACTIVE_VERSION: str = "v3"

# In scripts
v3 = get_version_paths('v3')
ensure_version_dirs('v3')

The system will automatically create versioned paths and maintain backward compatibility.