README.md

metadata

title: Forge
emoji: 🔥
colorFrom: yellow
colorTo: red
sdk: docker
app_port: 7860
pinned: false
license: mit

Forge — ML Training Compatibility Graph

Pick your training ingredients (optimizer, scheduler, quantization, technique, architecture). Incompatible or not-yet-satisfied components grey out live, each with a cited reason and the fix. Output: a validated training recipe assembled from real practitioner evidence across the web.

Built for the Bright Data Web Data UNLOCKED hackathon.

How it works

• Data (Bright Data): discover (AI-intent ranking) + Web Unlocker + SERP (EN + CN geo) + Scraper API + Scraping Browser mine compatibility signal from docs, papers, GitHub repos/issues, and forums.
• Extraction (local, free): a local Qwen2.5-3B model (llama.cpp) pulls (A, relation, B, conditions) triples — no paid LLM API. A keyless heuristic is the fallback.
• Trust: tier-1 = verified practitioner research + a real LR-scheduler benchmark; tier-2 = docs / 2+ independent sources; tier-3 = single scrape (held in a review queue until corroborated). GitHub is used as a secondary source to verify candidates.
• Graph: SQLite (forge.db). The grey-out resolves client-side for instant feedback.

Architecture

The heavy work (scrape + extraction) runs at build time on the maintainer's machine. This Space just serves the prebuilt forge.db via FastAPI + a static D3 frontend — one origin, CPU-only.

brightdata CLI + local model  ──build──>  forge.db  ──serve──>  FastAPI (/graph,/resolve,/recipe,/feed) + static UI

Run locally

pip install -r requirements.txt
python -m forge.seed                 # build forge.db
uvicorn forge.api:app --port 7860    # serves API + UI at http://localhost:7860

To grow the graph from live web data (needs the Bright Data CLI authenticated + a local llama-server):

python -m forge.campaign             # discover EN+CN across components
python -m forge.github_collect corroborate   # verify candidates against GitHub