# Analiza struktury GrantForge — Hybrid Grant Intelligence System

Data: 2026-06-23 | Ścieżka wdrożenia: `grantforge-patch/backend/`

## 1. Modele grantów i naborów

| Plik | Rola |
|------|------|
| `core/grants/models.py` | `Grant` ORM (pełne pola HGIS), `GrantVersion`, `ResearchJob`, `HumanVerificationItem` |
| `core/grants/wyszukiwarka_mapper.py` | Mapowanie rekordów JSON WYSZUKIWARKA → payload API (PKD, regiony, linki, program_year) |
| `core/grants/wyszukiwarka_import.py` | Upsert DB, wersjonowanie przy imporcie, enrichment; **nie** seeduje snapshotów research |
| `core/grants/eligibility_fields.py` | Parser PKD, beneficjentów, regionów, rozmiarów firm |
| `core/grants/catalog_visibility.py` | Jedno źródło prawdy: `resolve_catalog_hidden`, `is_catalog_visible`, wiek >3 lat |
| `core/grants/catalog_filters.py` | Re-eksport widoczności + `build_catalog_description` |
| `core/grants/completeness.py` | `compute_completeness`, `grant_dict_from_row` |
| `core/grants/versioning.py` | `detect_field_changes`, `record_changes`, `get_recent_versions` |
| `core/grants/enrichment.py` | Uzupełnianie pól z surowego JSON (regulamin, linki, PKD) |
| `core/grants/research_snapshot.py` | Pure helpers: hash/diff snapshotów dla Live Research |

## 2. Endpointy wyszukiwania i rekomendacji

| Plik | Endpointy / funkcje |
|------|---------------------|
| `endpoints/grants_nabory_enhanced.py` | `GET /api/grants/nabory` — filtry + `hybrid=true` |
| `endpoints/grants_catalog.py` | `POST /catalog/search`, stats — `use_hybrid` |
| `endpoints/grants_intelligence.py` | `/intelligence/dashboard`, `/recommendations`, `/hybrid-search`, `/research/*`, `/verification/*` |
| `endpoints/grants_loader.py` | Rejestracja routerów grants |
| `core/grants/catalog_service.py` | `nabory_search`, `search_catalog`, `get_catalog_stats`, filtry PKD |
| `core/grants/hybrid_search.py` | FTS + Pinecone (gdy klucz) lub `fts_only` bez klucza |
| `core/grants/regulation_backfill.py` | Backfill `regulation_url` z raw_data + raport kompletności + próbka HTTP |
| `core/grants/catalog_vector_index.py` | Indeksowanie grantów do Pinecone (`grants_catalog` namespace) |
| `core/grants/recommendations.py` | `recommend_grants_for_company`, `score_grant_for_company` + `why_it_fits` |

## 3. Agenci LangGraph

| Plik | Rola |
|------|------|
| `agents/generator_agent.py` | Wzorzec StateGraph (generator wniosków) |
| `agents/researcher.py` | `researcher_node` — katalog PostgreSQL |
| `agents/grant_research_agent.py` | HGIS: `analyze_changes` → `enrich_metadata` → `summarize` → `recommend` |
| `agents/matcher.py` | Dopasowanie grantów do profilu firmy (istniejący) |

## 4. Scraping / sync / research

| Plik | Rola |
|------|------|
| `rag_pipeline/scraper.py` | Scraping treści programów |
| `rag_pipeline/refresh_job.py` | `ContentRefreshJob` + hash diff URL |
| `rag_pipeline/change_detector.py` | Hash diff treści (plik JSON) |
| `rag_pipeline/vector_store.py` | Pinecone store |
| `rag_pipeline/hybrid_retriever.py` | RRF FTS + vector |
| `core/grants/live_research.py` | `detect_grant_changes`, `establish_research_baselines`, kolejka `research_jobs` |
| `core/grants/source_research.py` | Organic refresh: dwa snapshoty agregatora + `detect_url_content_changes` |
| `core/grants/hybrid_search_status.py` | Jawny status FTS vs Pinecone (`get_hybrid_search_status`) |
| `core/grants/credibility.py` | `score_source_credibility` |
| `core/grants/sync_service.py` | `sync_wyszukiwarka_catalog`, live scrape |
| `core/grants/catalog_freshness.py` | Sprawdzanie URL |
| `core/grants/human_verification.py` | Human Verification Queue |
| `scripts/full_catalog_cron.py` | Cron katalogu |
| `scripts/sync_grants_cron.py` | Sync grantów |
| `scripts/import_wyszukiwarka.py` | CLI import |
| `bootstrap_wyszukiwarka.py` | Hook bootstrap przy starcie |

## 5. Admin / diagnostyka

| Plik | Rola |
|------|------|
| `endpoints/admin_diagnostics.py` | Diagnostyka admin (istniejący) |
| `endpoints/grants_intelligence.py` | Dashboard HGIS (4 warstwy) |
| `grantforge_bootstrap.py` | Migracja HGIS przy starcie |
| `scripts/migrate_hgis_schema.py` | Idempotentna migracja kolumn/tabel |
| `scripts/verify_hgis.py` | Gating verification (6 kroków) |
| `scripts/verify_catalog.py` | Weryfikacja katalogu |
| `scripts/test_intelligence_local.py` | Lokalne endpointy bez serwera HTTP |

## 6. Testy (shipped)

| Plik | Zakres |
|------|--------|
| `tests/test_hgis.py` | completeness, versioning, recs, agent, filtry (fixture PARP + verify DB) |
| `tests/fixtures/real_grant_sample.json` | Rzeczywisty rekord PARP z `dotacje-latest.json` |
| `tests/test_research_snapshot.py` | Unit snapshot helpers |
| `tests/test_wyszukiwarka_pipeline.py` | Import JSON → baseline → file mutation → detect |
| `tests/test_catalog_filters.py` | Widoczność, catalog_hidden |
| `tests/test_catalog_pipeline.py` | Pipeline katalogu |

## 7. Przepływ danych (skrót)

```
dotacje-latest.json
  → wyszukiwarka_mapper (map)
  → enrichment + catalog_visibility
  → wyszukiwarka_import (upsert, NO snapshot seed)
  → establish_research_baselines (detect_grant_changes)
  → run_organic_aggregator_refresh (older JSON → newer JSON, real field diffs)
  → [cron] detect_grant_changes + detect_url_content_changes
  → versioning + research_jobs + HVQ
  → grant_research_agent (LangGraph)
  → nabory_search / hybrid_search / recommendations
```

## 8. Ograniczenia środowiska (ujawnione w verify)

| Obszar | Stan w sandbox |
|--------|----------------|
| `regulation_url` w DB | ~26% widocznych rekordów (limit źródła JSON ~23% surowych) |
| `PINECONE_API_KEY` | Brak → `search_mode=fts_only`; semantic wymaga klucza (`hybrid_search_status`) |
| Organic change detection | `dotacje-2026-06-20.json` → `dotacje-latest.json`: 149 zmian w verify |
| Mutacje importu | Tylko przez edycję pliku JSON (brak `mutations=` w API) |
| Neo4j / pełny server HTTP | Opcjonalne; verify używa direct function calls |

## 9. Zmienne środowiskowe (HGIS)

| Zmienna | Domyślnie | Opis |
|---------|-----------|------|
| `ENABLE_LIVE_RESEARCH` | `true` | Warstwa 2 |
| `ENABLE_GRANT_RESEARCH_AGENT` | `true` | LangGraph agent |
| `ENABLE_HYBRID_SEARCH` | `true` | Hybrydowe wyszukiwanie |
| `ENABLE_PINECONE_SEMANTIC` | `true` | Wymaga `PINECONE_API_KEY` |
| `HUMAN_VERIF_QUEUE` | `true` | Kolejka weryfikacji |
| `ENABLE_IMPORT_MUTATIONS` | `false` | Dev-only patch importu |