docs/FILE_ORGANIZATION.md

# File Organization

## Directory Structure

The YLFF package follows a clear organizational structure:

```
ylff/
├── __init__.py          # Package initialization with backward compatibility
├── __main__.py          # Entry point for `python -m ylff`
├── app.py               # Unified CLI/API entry point
├── cli.py               # CLI command definitions (Typer)
├── api.py               # Backward compatibility wrapper
├── config.py            # Configuration management (Pydantic Settings)
│
├── models/              # Pydantic API models
│   ├── __init__.py
│   └── api_models.py
│
├── routers/             # FastAPI route handlers
│   ├── __init__.py
│   ├── health.py
│   ├── jobs.py
│   ├── models.py
│   ├── profiling.py
│   ├── training.py
│   ├── validation.py
│   └── visualization.py
│
├── services/            # Business logic
│   ├── __init__.py
│   ├── arkit_processor.py
│   ├── ba_validator.py
│   ├── data_pipeline.py
│   ├── evaluate.py
│   ├── fine_tune.py
│   └── pretrain.py
│
└── utils/               # Utilities and helpers
    ├── __init__.py
    ├── api_middleware.py
    ├── coordinate_utils.py
    ├── exceptions.py     # Custom exception classes
    ├── job_manager.py
    ├── losses.py         # Loss functions for training
    ├── model_loader.py   # ML model loading utilities
    ├── profiler.py
    ├── visualization_gui.py
    └── wandb_utils.py
```

## Organization Principles

### Root Level (`ylff/`)
Only core application files:
- **`app.py`**: Unified entry point (CLI/API)
- **`cli.py`**: CLI command definitions
- **`config.py`**: Configuration management
- **`api.py`**: Backward compatibility wrapper
- **`__init__.py`**: Package initialization

### `models/` - Pydantic API Models
- Request/response models for the API
- Data validation and serialization
- Example: `JobResponse`, `ValidateSequenceRequest`

### `routers/` - API Route Handlers
- FastAPI route definitions
- One router per domain (health, jobs, validation, etc.)
- Thin layer that delegates to services

### `services/` - Business Logic
- Core application logic
- Domain-specific functionality
- Examples: `BAValidator`, `ARKitProcessor`, `fine_tune_da3`

### `utils/` - Utilities and Helpers
- Reusable utilities across the codebase
- **`exceptions.py`**: Custom exception classes
- **`losses.py`**: Loss functions for training
- **`model_loader.py`**: ML model loading utilities
- **`profiler.py`**: Performance profiling
- **`coordinate_utils.py`**: Coordinate system conversions
- **`job_manager.py`**: Background job management
- **`visualization_gui.py`**: GUI utilities
- **`wandb_utils.py`**: Weights & Biases integration

## Import Patterns

### From Utils
```python
# Exceptions
from ylff.utils.exceptions import DataError, ModelLoadError

# Loss functions
from ylff.utils.losses import pose_loss, geodesic_rotation_loss

# Model loading
from ylff.utils.model_loader import load_da3_model, get_recommended_model

# Other utilities
from ylff.utils.profiler import Profiler
from ylff.utils.coordinate_utils import convert_arkit_to_opencv
```

### From Services
```python
from ylff.services.ba_validator import BAValidator
from ylff.services.arkit_processor import ARKitProcessor
from ylff.services.fine_tune import fine_tune_da3
```

### From Models
```python
from ylff.models import JobResponse, ValidateSequenceRequest
```

### From Routers
```python
from ylff.routers import health_router, validation_router
```

## Migration History

### Moved to `utils/`
- **`exceptions.py`** → `utils/exceptions.py` (was at root)
- **`losses.py`** → `utils/losses.py` (was at root)
- **`model_loader.py`** → `utils/model_loader.py` (was at root, renamed from `models.py`)

### Rationale
- All three files are utilities used across multiple modules
- Keeps the root `ylff/` directory clean and focused
- Makes it clear these are reusable utilities, not core application logic
- Follows the principle: "If it's used by multiple services, it's a utility"

## Backward Compatibility

The `ylff/__init__.py` provides backward compatibility imports:
```python
# Still works (via __getattr__)
from ylff import BAValidator, Profiler, load_da3_model
```

But preferred imports are:
```python
from ylff.utils.model_loader import load_da3_model
from ylff.utils.losses import pose_loss
from ylff.utils.exceptions import DataError
```