File size: 7,379 Bytes
2532605 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | # KRONECTOR — Implementation Plan (v2 — Corrected)
> **Tagline:** Every sector. Every timeline. Predicted.
> **Domain:** F1 Race Intelligence — self-improving multi-agent AI system
---
## Month 1, Week 1 — Data Pipelines (CURRENT FOCUS)
### Deliverables
1. `data/build_driver_map.py` — Generate `drivers_map.json` (run once at init)
2. `data/fastf1_pipeline.py` — Fetch telemetry + session data for 2018–2024 (includes `fetch_lap_data`)
3. `data/jolpica_pipeline.py` — Backfill race results, grid, pit stops, standings for 2014–2017
4. `data/__init__.py` — Merge logic combining both sources on `(season, round, driver_id)`
5. Project scaffolding — all `__init__.py` files, `.env`, `requirements.txt`, `.gitignore`
### Execution Order
1. Run `build_driver_map.py` → generates `drivers_map.json`
2. Run `jolpica_pipeline.py` for 2014–2017 → Jolpica backfill
3. Run `fastf1_pipeline.py` for **2023 only** → verify schema
4. Run merge logic → verify unified 2023 dataset
5. Scale `fastf1_pipeline.py` to 2018–2024
6. Run full merge → final unified 2014–2024 dataset
7. Run `pytest tests/test_data_pipelines.py -v`
---
### [NEW] Project Scaffolding
```
kronector/
├── agents/__init__.py
├── ml/__init__.py
├── data/
│ ├── __init__.py (merge logic)
│ ├── fastf1_pipeline.py
│ ├── jolpica_pipeline.py (renamed from ergast)
│ └── build_driver_map.py
├── api/__init__.py
├── ui/
├── mlflow_config/
├── tests/
├── cache/fastf1/ (gitignored)
├── drivers_map.json (generated by build_driver_map.py)
├── requirements.txt
├── .env
├── .gitignore
└── README.md
```
---
### [NEW] [build_driver_map.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/build_driver_map.py)
**Purpose:** One-time init script — maps FastF1 3-letter abbreviations to Jolpica slugs.
- Pull FastF1 driver list for 2014–2024 via `fastf1.get_event_schedule()` + session drivers
- Pull Jolpica driver list via `/api/f1/drivers.json`
- Match on `full_name` → derive Jolpica slug
- Output: `drivers_map.json` at project root
**Rule:** FastF1 abbreviation (`VER`) = master `driver_id` throughout entire system. Jolpica slug only for Jolpica API calls.
---
### [NEW] [fastf1_pipeline.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/fastf1_pipeline.py)
**Purpose:** Fetch telemetry + session data from FastF1 for 2018–2024.
| Function | Description |
|---|---|
| `enable_cache(cache_dir)` | Configure FastF1 cache directory |
| `fetch_race_results(season, round_num)` | Race finishing order, grid positions |
| `fetch_qualifying(season, round_num)` | Qualifying sector times + **missing data guard** |
| `fetch_practice(season, round_num)` | FP2/FP3 average lap times |
| `fetch_tire_data(season, round_num)` | Tire compounds, stint lengths, fresh/used |
| `fetch_pit_stops(season, round_num)` | Pit stop count + **team_pit_speed computed inline** |
| `fetch_weather(season, round_num)` | Track temp, rainfall from session weather |
| `fetch_lap_data(season, round_num)` | **NEW** — Lap-by-lap data with `track_status` for safety car |
| `build_season_dataframe(season)` | Orchestrate all fetchers for a full season |
| `build_full_dataset(start, end)` | Build complete FastF1 dataset |
**Correction 2 — `fetch_lap_data`:** Returns `(season, round, driver_id, lap_number, track_status)`. `track_status == '4'` = safety car, `'6'` = VSC. Used in merge to compute `safety_car_probability`.
**Correction 3 — `team_pit_speed`:** Computed inside `fetch_pit_stops()` as mean pit duration per team per race. Returned as column, no separate function.
**Correction 8 — Sector time guard:**
```python
if session.laps['Sector1Time'].isna().mean() > 0.5:
logger.warning(f"Season {season} R{round_num}: >50% sector times missing.")
```
No dropping, no imputing. Imputation deferred to `feature_engineering.py` (Week 2).
**No `championship_standing` from FastF1** — comes from Jolpica only (Correction 1).
---
### [NEW] [jolpica_pipeline.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/jolpica_pipeline.py)
**Purpose:** Backfill 2014–2017 data + championship standings for ALL years.
**Base URL:** `https://api.jolpi.ca/ergast/f1`
| Function | Description |
|---|---|
| `jolpica_get(url, retries, base_delay)` | **Request wrapper with exponential backoff** |
| `fetch_race_results(season, round_num)` | Results + grid from Jolpica JSON API |
| `fetch_pit_stops(season, round_num)` | Pit stop count per driver |
| `fetch_driver_standings(season, round_num)` | **Championship standings — sole source for all years** |
| `fetch_circuit_info(season, round_num)` | Circuit metadata (circuitId, locality) |
| `build_season_dataframe(season)` | Orchestrate fetchers for full season |
| `build_jolpica_dataset(start, end)` | Build complete backfill dataset |
**Correction 5 — Rate limiting:**
```python
def jolpica_get(url, retries=3, base_delay=0.2):
for attempt in range(retries):
try:
response = requests.get(url, timeout=10)
response.raise_for_status()
time.sleep(base_delay)
return response.json()
except requests.exceptions.RequestException as e:
wait = base_delay * (2 ** attempt)
time.sleep(wait)
return None
```
**Correction 1:** `championship_standing` fetched here only, joined onto FastF1 rows in merge step.
---
### [NEW] Merge Logic — [data/__init__.py](file:///c:/Users/Lenovo/OneDrive/Desktop/kronector/data/__init__.py)
1. Align column schemas (union of all columns)
2. `pd.concat([jolpica_df, fastf1_df])`
3. Sort by `(season, round, grid_position)`
4. Add `regulation_era`: 2014–2021 → `hybrid_era`, 2022–2024 → `ground_effect_era`
5. Add `track_type` from circuit mapping
6. Compute `driver_form_last3` (rolling avg finish, last 3 races)
7. **Compute `safety_car_probability`** from `fetch_lap_data` output:
- Group by `circuit_id`, count laps where `track_status == '4'` / total laps
8. **Join `championship_standing`** from Jolpica onto all rows
9. Add `win_probability` target (1 if `finish_position == 1`, else 0)
10. Validate `telemetry_available` flag integrity
**Driver ID mapping:** All Jolpica slugs converted to FastF1 abbreviations via `DRIVER_MAP`.
---
### Verification Target (2023 First — Correction 7)
- 22 races × ~20 drivers = ~440 rows
- Spot-check sector times vs official F1 results
- `telemetry_available = True` for all FastF1 rows
- `telemetry_available = False` for all Jolpica rows
- Merged dataset sorted by `(season, round, grid_position)`
---
## Month 1, Week 2–4 (Unchanged)
| Week | Deliverable |
|---|---|
| W2 | `feature_engineering.py` — era normalization, encoding, TimeSeriesSplit, imputation of missing sector times |
| W3 | `train.py` + `predict.py` — LightGBM + SHAP TreeExplainer + MLflow logging |
| W4 | `main.py` — FastAPI `/predict/f1` endpoint, all agents as plain Python functions |
## Month 2 — Agent Refactor + Drift + Auto-Retrain (Unchanged)
## Month 3 — UI + Deploy + Polish (Unchanged)
---
## .env Template (Updated)
```
GROQ_API_KEY=
MLFLOW_TRACKING_URI=
CHROMA_PERSIST_DIR=./chroma
JOLPICA_BASE_URL=https://api.jolpi.ca/ergast/f1
FASTF1_CACHE_DIR=./cache/fastf1
```
|