docs/implementation_plan.md

# KRONECTOR — Implementation Plan (v2 — Corrected)

> **Tagline:** Every sector. Every timeline. Predicted.
> **Domain:** F1 Race Intelligence — self-improving multi-agent AI system

---

## Month 1, Week 1 — Data Pipelines (CURRENT FOCUS)

### Deliverables
1. `data/build_driver_map.py` — Generate `drivers_map.json` (run once at init)
2. `data/fastf1_pipeline.py` — Fetch telemetry + session data for 2018–2024 (includes `fetch_lap_data`)
3. `data/jolpica_pipeline.py` — Backfill race results, grid, pit stops, standings for 2014–2017
4. `data/__init__.py` — Merge logic combining both sources on `(season, round, driver_id)`
5. Project scaffolding — all `__init__.py` files, `.env`, `requirements.txt`, `.gitignore`

### Execution Order
1. Run `build_driver_map.py` → generates `drivers_map.json`
2. Run `jolpica_pipeline.py` for 2014–2017 → Jolpica backfill
3. Run `fastf1_pipeline.py` for **2023 only** → verify schema
4. Run merge logic → verify unified 2023 dataset
5. Scale `fastf1_pipeline.py` to 2018–2024
6. Run full merge → final unified 2014–2024 dataset
7. Run `pytest tests/test_data_pipelines.py -v`

---

### [NEW] Project Scaffolding

```
kronector/
├── agents/__init__.py
├── ml/__init__.py
├── data/
│   ├── __init__.py          (merge logic)
│   ├── fastf1_pipeline.py
│   ├── jolpica_pipeline.py  (renamed from ergast)
│   └── build_driver_map.py
├── api/__init__.py
├── ui/
├── mlflow_config/
├── tests/
├── cache/fastf1/            (gitignored)
├── drivers_map.json         (generated by build_driver_map.py)
├── requirements.txt
├── .env
├── .gitignore
└── README.md
```

---

### [NEW] [build_driver_map.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/build_driver_map.py)

**Purpose:** One-time init script — maps FastF1 3-letter abbreviations to Jolpica slugs.

- Pull FastF1 driver list for 2014–2024 via `fastf1.get_event_schedule()` + session drivers
- Pull Jolpica driver list via `/api/f1/drivers.json`
- Match on `full_name` → derive Jolpica slug
- Output: `drivers_map.json` at project root

**Rule:** FastF1 abbreviation (`VER`) = master `driver_id` throughout entire system. Jolpica slug only for Jolpica API calls.

---

### [NEW] [fastf1_pipeline.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/fastf1_pipeline.py)

**Purpose:** Fetch telemetry + session data from FastF1 for 2018–2024.

| Function | Description |
|---|---|
| `enable_cache(cache_dir)` | Configure FastF1 cache directory |
| `fetch_race_results(season, round_num)` | Race finishing order, grid positions |
| `fetch_qualifying(season, round_num)` | Qualifying sector times + **missing data guard** |
| `fetch_practice(season, round_num)` | FP2/FP3 average lap times |
| `fetch_tire_data(season, round_num)` | Tire compounds, stint lengths, fresh/used |
| `fetch_pit_stops(season, round_num)` | Pit stop count + **team_pit_speed computed inline** |
| `fetch_weather(season, round_num)` | Track temp, rainfall from session weather |
| `fetch_lap_data(season, round_num)` | **NEW** — Lap-by-lap data with `track_status` for safety car |
| `build_season_dataframe(season)` | Orchestrate all fetchers for a full season |
| `build_full_dataset(start, end)` | Build complete FastF1 dataset |

**Correction 2 — `fetch_lap_data`:** Returns `(season, round, driver_id, lap_number, track_status)`. `track_status == '4'` = safety car, `'6'` = VSC. Used in merge to compute `safety_car_probability`.

**Correction 3 — `team_pit_speed`:** Computed inside `fetch_pit_stops()` as mean pit duration per team per race. Returned as column, no separate function.

**Correction 8 — Sector time guard:**
```python
if session.laps['Sector1Time'].isna().mean() > 0.5:
    logger.warning(f"Season {season} R{round_num}: >50% sector times missing.")
```
No dropping, no imputing. Imputation deferred to `feature_engineering.py` (Week 2).

**No `championship_standing` from FastF1** — comes from Jolpica only (Correction 1).

---

### [NEW] [jolpica_pipeline.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/jolpica_pipeline.py)

**Purpose:** Backfill 2014–2017 data + championship standings for ALL years.

**Base URL:** `https://api.jolpi.ca/ergast/f1`

| Function | Description |
|---|---|
| `jolpica_get(url, retries, base_delay)` | **Request wrapper with exponential backoff** |
| `fetch_race_results(season, round_num)` | Results + grid from Jolpica JSON API |
| `fetch_pit_stops(season, round_num)` | Pit stop count per driver |
| `fetch_driver_standings(season, round_num)` | **Championship standings — sole source for all years** |
| `fetch_circuit_info(season, round_num)` | Circuit metadata (circuitId, locality) |
| `build_season_dataframe(season)` | Orchestrate fetchers for full season |
| `build_jolpica_dataset(start, end)` | Build complete backfill dataset |

**Correction 5 — Rate limiting:**
```python
def jolpica_get(url, retries=3, base_delay=0.2):
    for attempt in range(retries):
        try:
            response = requests.get(url, timeout=10)
            response.raise_for_status()
            time.sleep(base_delay)
            return response.json()
        except requests.exceptions.RequestException as e:
            wait = base_delay * (2 ** attempt)
            time.sleep(wait)
    return None
```

**Correction 1:** `championship_standing` fetched here only, joined onto FastF1 rows in merge step.

---

### [NEW] Merge Logic — [data/__init__.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/__init__.py)

1. Align column schemas (union of all columns)
2. `pd.concat([jolpica_df, fastf1_df])`
3. Sort by `(season, round, grid_position)`
4. Add `regulation_era`: 2014–2021 → `hybrid_era`, 2022–2024 → `ground_effect_era`
5. Add `track_type` from circuit mapping
6. Compute `driver_form_last3` (rolling avg finish, last 3 races)
7. **Compute `safety_car_probability`** from `fetch_lap_data` output:
   - Group by `circuit_id`, count laps where `track_status == '4'` / total laps
8. **Join `championship_standing`** from Jolpica onto all rows
9. Add `win_probability` target (1 if `finish_position == 1`, else 0)
10. Validate `telemetry_available` flag integrity

**Driver ID mapping:** All Jolpica slugs converted to FastF1 abbreviations via `DRIVER_MAP`.

---

### Verification Target (2023 First — Correction 7)
- 22 races × ~20 drivers = ~440 rows
- Spot-check sector times vs official F1 results
- `telemetry_available = True` for all FastF1 rows
- `telemetry_available = False` for all Jolpica rows
- Merged dataset sorted by `(season, round, grid_position)`

---

## Month 1, Week 2–4 (Unchanged)

| Week | Deliverable |
|---|---|
| W2 | `feature_engineering.py` — era normalization, encoding, TimeSeriesSplit, imputation of missing sector times |
| W3 | `train.py` + `predict.py` — LightGBM + SHAP TreeExplainer + MLflow logging |
| W4 | `main.py` — FastAPI `/predict/f1` endpoint, all agents as plain Python functions |

## Month 2 — Agent Refactor + Drift + Auto-Retrain (Unchanged)
## Month 3 — UI + Deploy + Polish (Unchanged)

---

## .env Template (Updated)
```
GROQ_API_KEY=
MLFLOW_TRACKING_URI=
CHROMA_PERSIST_DIR=./chroma
JOLPICA_BASE_URL=https://api.jolpi.ca/ergast/f1
FASTF1_CACHE_DIR=./cache/fastf1
```