README.md

---
title: Forge
emoji: 🔥
colorFrom: yellow
colorTo: red
sdk: docker
app_port: 7860
pinned: false
license: mit
---

# Forge — ML Training Compatibility Graph

Pick your training ingredients (optimizer, scheduler, quantization, technique, architecture). Incompatible or not-yet-satisfied components grey out **live**, each with a cited reason and the fix. Output: a validated training recipe assembled from real practitioner evidence across the web.

Built for the Bright Data **Web Data UNLOCKED** hackathon.

## How it works

- **Data (Bright Data):** `discover` (AI-intent ranking) + Web Unlocker + SERP (EN + CN geo) + Scraper API + Scraping Browser mine compatibility signal from docs, papers, GitHub repos/issues, and forums.
- **Extraction (local, free):** a local Qwen2.5-3B model (llama.cpp) pulls `(A, relation, B, conditions)` triples — no paid LLM API. A keyless heuristic is the fallback.
- **Trust:** tier-1 = verified practitioner research + a real LR-scheduler benchmark; tier-2 = docs / 2+ independent sources; tier-3 = single scrape (held in a review queue until corroborated). GitHub is used as a secondary source to verify candidates.
- **Graph:** SQLite (`forge.db`). The grey-out resolves client-side for instant feedback.

## Architecture

The heavy work (scrape + extraction) runs at **build time** on the maintainer's machine. This Space just serves the prebuilt `forge.db` via FastAPI + a static D3 frontend — one origin, CPU-only.

```
brightdata CLI + local model  ──build──>  forge.db  ──serve──>  FastAPI (/graph,/resolve,/recipe,/feed) + static UI
```

## Run locally

```bash
pip install -r requirements.txt
python -m forge.seed                 # build forge.db
uvicorn forge.api:app --port 7860    # serves API + UI at http://localhost:7860
```

To grow the graph from live web data (needs the Bright Data CLI authenticated + a local `llama-server`):

```bash
python -m forge.campaign             # discover EN+CN across components
python -m forge.github_collect corroborate   # verify candidates against GitHub
```