fix: proper rewards for all steps and clean CSV output

fix: intelligent navigation scraper with GitHub trending support

feat: implement intelligent agentic web scraper

fix: resolve scraper functionality and plugin issues

test: add end-to-end scraper test script with successful validation

feat: add WebSocket support for real-time scraper progress updates

feat: add providers API endpoints with model and cost tracking

feat: add embeddings service and update plugins with NVIDIA support

fix: support settings endpoint without trailing slash

fix: support plugins endpoint without trailing slash

test: add comprehensive API and core module tests

fix: add list_providers method and agents list endpoint

feat: add modular plugin system for extensions

feat: add client API key configuration and model selection

feat: add API routes and utility modules