Spaces:
Sleeping
Sleeping
Commit History
-
feat: Major scraper enhancements - consistent format, better extraction, per-month costs
Riley +1: ClaudeThis commit significantly improves the grant scraper to extract more complete and consistent data: ## Scraper Improvements (snapshot.py) ### 1. Consistent JSON Structure - Added `id` and `competition_id` fields to all snapshots - Ensures compatibility between manual (snapshot.py) and auto (discover_grants.py) scraping - All existing code handles both old and new formats gracefully ### 2. Enhanced Funding Rates Extraction - Pattern 1: Standard format "up to 60% if you are micro/small" - Pattern 2: Combined format "up to 60% if you are a micro, small or medium" - Pattern 3: KTP format "large companies...50%...SMEs...67%" - Now extracts rates for diverse grant types ### 3. Enhanced Duration Extraction - Range: "last/must be between 12 and 36 months" - Single max: "last up to nine months" (supports word numbers like "nine") - Maximum: "maximum 24 months" - Added word-to-number conversion for written numbers ### 4. Per-Month Cost Calculation (NEW) - Detects patterns like "£8,500 per mo…2ae7490 -
feat: Major system enhancements - GPT-5 support, monitoring, translation, and optimizations
Riley +1: ClaudeThis commit includes 15 major improvements to the grant-analyst system: ## New Features ### GPT-5 Model Support (Step 1) - Implemented three-tier GPT-5 model system: - gpt-5-nano for routing and simple decisions - gpt-5-mini for translations and summaries - gpt-5 for complex analysis - Updated token encoding to use o200k_base (GPT-5) with cl100k_base fallback - Updated all configuration files and documentation ### Grant Translation Endpoint (Step 14) - New `/translate` API endpoint for converting grant jargon to simple language - Structured output with 4 sections: eligibility, process, timeline, requirements - Uses gpt-5-mini model for cost-effective translations - Permanent caching in MongoDB (translations don't change) - Batch translation support via `/translate/batch` ### Monitoring & Metrics System (Step 15) - New metrics collection system tracking: - P50/P95/P99 latency per endpoint - Token usage per request (prompt + completion) - Cache hit/miss rates (Redis + Mon…057c21e