Add autonomous dataset intelligence engine

README.md

@@ -211,6 +211,59 @@ The objective is not to predict stock prices. The objective is to learn how Blum
 
 The autonomous Blum Financial Model cycle is server-side and evidence-bound. When `BLUM_ENABLE_LEARNING_LOOP=true`, it runs on startup, during market refresh and on its own interval controlled by `BLUM_MODEL_CYCLE_MINUTES` and `BLUM_MODEL_CYCLE_LIMIT`. Each cycle captures recent signal reasoning, evaluates matured thesis outcomes, refreshes training examples and logs a `blum_model_autonomous_cycle` learning event. It updates database memory only; it does not self-modify source code and it does not execute trades.
 
+## Autonomous Research Engine
+
+Blum now runs a server-side autonomous research cycle by default. Manual refresh buttons are diagnostics; normal operation does not require user input. The default cycle is controlled by:
+
+```bash
+export BLUM_ENABLE_AUTONOMOUS_ENGINE=true
+export BLUM_AUTONOMOUS_CYCLE_MINUTES=20
+```
+
+The autonomous cycle executes in this strict order:
+
+1. Refresh Hugging Face financial dataset catalog.
+2. Update macro context.
+3. Update SEC companyfacts fundamentals.
+4. Hydrate historical and recent OHLCV prices from public providers.
+5. Ingest public news and sentiment.
+6. Generate signal snapshots.
+7. Update ETF intelligence.
+8. Update IPO radar.
+9. Run accuracy audit.
+10. Run Blum Financial Model reasoning and outcome learning.
+11. Persist embedded PostgreSQL backup when configured.
+
+Every cycle creates an `autonomous_engine_runs` row and a `learning_events` audit entry. If a provider fails, the run is marked `degraded` with the failing stage and traceback, rather than hiding the issue.
+
+New APIs:
+
+- `GET /autonomous/status`
+- `POST /autonomous/run`
+- `GET /datasets/sources`
+- `POST /datasets/refresh`
+
+## Hugging Face Dataset Intelligence
+
+Blum catalogs real Hugging Face datasets for historical prices, SEC filings, earnings transcripts, finance reasoning and benchmark evidence. The catalog is metadata-first and incremental by design: large corpora are validated through Dataset Viewer/parquet metadata before any targeted ingestion is attempted.
+
+Initial curated sources include:
+
+- `defeatbeta/yahoo-finance-data`
+- `paperswithbacktest/Stocks-Daily-Price`
+- `TeraflopAI/SEC-EDGAR`
+- `kurry/sp500_earnings_transcripts`
+- `glopardo/sp500-earnings-transcripts`
+- `paperswithbacktest/Stocks-Quarterly-Earnings`
+- `c3po-ai/edgar-corpus`
+- `PatronusAI/financebench`
+- `BUPT-Reasoning-Lab/FinanceReasoning`
+- `jlh-ibm/earnings_call`
+- `younginpiniti/us-stocks-daily-all`
+- `sfd-anonymous/edgar-forecast-benchmark`
+
+The catalog is stored in `external_dataset_sources` with dataset id, source domain, license, priority, ingestion mode, Dataset Viewer status, parquet metadata and usage policy. Blum does not copy massive datasets blindly and does not fabricate missing evidence.
+
 ## Chart Vision Technical Analyst
 
 Blum includes a dedicated technical chart intelligence module. It is designed to read financial chart images when a vision model is configured, but it never relies only on visual interpretation.
@@ -444,6 +497,8 @@ export BLUM_ENABLE_LEARNING_LOOP=true
 export BLUM_LEARNING_LOOP_MINUTES=360
 export BLUM_MODEL_CYCLE_MINUTES=5
 export BLUM_MODEL_CYCLE_LIMIT=120
+export BLUM_ENABLE_HF_DATASET_CATALOG=true
+export BLUM_HF_DATASET_REFRESH_HOURS=24
 ```
 
 The learning loops only update database memory, confidence adjustments, proprietary reasoning examples and reversible scoring-weight versions.
@@ -485,6 +540,7 @@ The UI exposes `/system/status` in the sidebar and dashboard. If the GUI looks u
 - `app_version` must show the latest deployed version.
 - `feature_set` must show the expected feature bundle.
 - `persistence.mode` must be `external_postgres` for strict no-reset durability, or `embedded_postgres` with a populated backup file plus persistent `/data` storage for demo durability.
+- `GET /autonomous/status` shows the latest autonomous run, stage diagnostics, readiness score and dataset catalog status.
 - `POST /system/persistence/backup` can force an immediate embedded PostgreSQL backup after a learning cycle or manual operations.
 - `Financial Brain` shows `fallback mode` unless `BLUM_ENABLE_FINANCIAL_BRAIN_MODEL=true`.
 - Hugging Face serves the previous Docker image until the new build finishes successfully.

backend/alembic/versions/0009_autonomous_dataset_intelligence.py

@@ -0,0 +1,79 @@
+from __future__ import annotations
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+
+revision = "0009_autonomous_dataset_intelligence"
+down_revision = "0008_blum_financial_model"
+branch_labels = None
+depends_on = None
+
+
+json_type = postgresql.JSONB(astext_type=sa.Text())
+
+
+def upgrade() -> None:
+    op.create_table(
+        "external_dataset_sources",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column("dataset_id", sa.String(length=220), nullable=False),
+        sa.Column("provider", sa.String(length=80), nullable=False),
+        sa.Column("title", sa.String(length=260), nullable=False),
+        sa.Column("primary_domain", sa.String(length=80), nullable=False),
+        sa.Column("data_domains", json_type, nullable=False),
+        sa.Column("license", sa.String(length=80), nullable=False),
+        sa.Column("priority", sa.Integer(), nullable=False),
+        sa.Column("ingestion_mode", sa.String(length=80), nullable=False),
+        sa.Column("status", sa.String(length=80), nullable=False),
+        sa.Column("dataset_url", sa.Text(), nullable=False),
+        sa.Column("viewer_status", json_type, nullable=False),
+        sa.Column("parquet_files", json_type, nullable=False),
+        sa.Column("size_summary", json_type, nullable=False),
+        sa.Column("usage_policy", json_type, nullable=False),
+        sa.Column("last_checked_at", sa.DateTime(), nullable=True),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), nullable=False),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("dataset_id", name="uq_external_dataset_source_dataset_id"),
+    )
+    for column in ["dataset_id", "provider", "primary_domain", "license", "priority", "ingestion_mode", "status", "last_checked_at", "created_at", "updated_at"]:
+        op.create_index(f"ix_external_dataset_sources_{column}", "external_dataset_sources", [column])
+    op.create_index("ix_external_dataset_sources_status_priority", "external_dataset_sources", ["status", "priority"])
+    op.create_index("ix_external_dataset_sources_domain_updated", "external_dataset_sources", ["primary_domain", "updated_at"])
+
+    op.create_table(
+        "autonomous_engine_runs",
+        sa.Column("id", sa.Integer(), nullable=False),
+        sa.Column("run_id", sa.String(length=100), nullable=False),
+        sa.Column("trigger", sa.String(length=80), nullable=False),
+        sa.Column("status", sa.String(length=80), nullable=False),
+        sa.Column("started_at", sa.DateTime(), nullable=False),
+        sa.Column("completed_at", sa.DateTime(), nullable=True),
+        sa.Column("stage_results", json_type, nullable=False),
+        sa.Column("readiness_score", sa.Float(), nullable=False),
+        sa.Column("data_coverage_score", sa.Float(), nullable=False),
+        sa.Column("reasoning_memory_created", sa.Integer(), nullable=False),
+        sa.Column("warning_count", sa.Integer(), nullable=False),
+        sa.Column("error_payload", json_type, nullable=False),
+        sa.Column("created_at", sa.DateTime(), nullable=False),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("run_id", name="uq_autonomous_engine_run_id"),
+    )
+    for column in ["run_id", "trigger", "status", "started_at", "completed_at", "readiness_score", "data_coverage_score", "created_at"]:
+        op.create_index(f"ix_autonomous_engine_runs_{column}", "autonomous_engine_runs", [column])
+    op.create_index("ix_autonomous_engine_runs_status_started", "autonomous_engine_runs", ["status", "started_at"])
+
+
+def downgrade() -> None:
+    op.drop_index("ix_autonomous_engine_runs_status_started", table_name="autonomous_engine_runs")
+    for column in ["created_at", "data_coverage_score", "readiness_score", "completed_at", "started_at", "status", "trigger", "run_id"]:
+        op.drop_index(f"ix_autonomous_engine_runs_{column}", table_name="autonomous_engine_runs")
+    op.drop_table("autonomous_engine_runs")
+
+    op.drop_index("ix_external_dataset_sources_domain_updated", table_name="external_dataset_sources")
+    op.drop_index("ix_external_dataset_sources_status_priority", table_name="external_dataset_sources")
+    for column in ["updated_at", "created_at", "last_checked_at", "status", "ingestion_mode", "priority", "license", "primary_domain", "provider", "dataset_id"]:
+        op.drop_index(f"ix_external_dataset_sources_{column}", table_name="external_dataset_sources")
+    op.drop_table("external_dataset_sources")

backend/app/api/routes.py

@@ -17,6 +17,7 @@ from app.models import (
     AIInsight,
     AccuracySnapshot,
     Asset,
+    AutonomousEngineRun,
     BlumDatasetExport,
     BlumKnowledgeGraphEdge,
     BlumKnowledgeGraphNode,
@@ -33,6 +34,7 @@ from app.models import (
     ChartPatternMemory,
     ConfidenceAdjustment,
     EmbeddingVector,
+    ExternalDatasetSource,
     FundamentalSnapshot,
     HistoricalSimilarityCase,
     IntelligenceReport,
@@ -57,6 +59,7 @@ from app.models import (
 )
 from app.schemas import AssetOut, MarketUpdateRequest, NewsOut, NewsUpdateRequest, SemanticSearchRequest, SignalRunRequest
 from app.services.accuracy import asset_accuracy_profile, latest_accuracy_snapshot, market_accuracy_overview, run_accuracy_audit, signal_validation_report
+from app.services.autonomous_engine import AutonomousResearchEngine, latest_autonomous_status
 from app.services.blum_financial_model import (
     build_training_dataset,
     capture_ai_insight_reasoning,
@@ -93,6 +96,7 @@ from app.services.financial_brain_learning import (
     run_learning_cycle,
 )
 from app.services.hybrid_chart_intelligence import HybridChartIntelligence
+from app.services.huggingface_datasets import dataset_catalog_status, refresh_huggingface_dataset_catalog
 from app.services.ipo import ipo_radar, sec_company_submissions, update_ipo_radar
 from app.services.live import live_news, market_sentiment
 from app.services.macro import macro_overview, update_macro_snapshots
@@ -134,7 +138,7 @@ def system_status(db: Session = Depends(get_db)) -> dict:
     return {
         "service": "blum-ai-financial-intelligence",
         "app_version": settings.app_version,
-        "feature_set": "persistent-autonomous-blum-financial-model-v0.7.1",
+        "feature_set": "autonomous-dataset-intelligence-engine-v0.8.0",
         "environment": settings.environment,
         "generated_at": datetime.utcnow().isoformat(),
         "hugging_face": {
@@ -147,6 +151,8 @@ def system_status(db: Session = Depends(get_db)) -> dict:
             "model_loading_enabled": settings.enable_model_loading,
             "financial_brain_model_enabled": settings.enable_financial_brain_model,
             "live_startup_enabled": settings.enable_live_startup,
+            "autonomous_engine_enabled": settings.enable_autonomous_engine,
+            "autonomous_cycle_minutes": settings.autonomous_cycle_minutes,
             "yfinance_fallback_enabled": settings.enable_yfinance_fallback,
             "historical_price_seed_enabled": settings.seed_historical_prices_on_startup,
             "startup_signal_seed_enabled": settings.seed_signals_on_startup,
@@ -161,6 +167,8 @@ def system_status(db: Session = Depends(get_db)) -> dict:
             "chart_vision_min_confidence": settings.chart_vision_min_confidence,
             "fundamentals_refresh_minutes": settings.fundamentals_refresh_minutes,
             "macro_refresh_minutes": settings.macro_refresh_minutes,
+            "hf_dataset_catalog_enabled": settings.enable_hf_dataset_catalog,
+            "hf_dataset_refresh_hours": settings.hf_dataset_refresh_hours,
         },
         "persistence": database_persistence_status(),
         "active_models": {
@@ -189,6 +197,8 @@ def system_status(db: Session = Depends(get_db)) -> dict:
             "financial_knowledge_graph": True,
             "portfolio_scenario": True,
             "watchlist": True,
+            "autonomous_research_engine": True,
+            "huggingface_dataset_catalog": True,
         },
         "database_counts": {
             "assets": int(db.scalar(select(func.count(Asset.id))) or 0),
@@ -227,13 +237,15 @@ def system_status(db: Session = Depends(get_db)) -> dict:
             "blum_graph_edges": int(db.scalar(select(func.count(BlumKnowledgeGraphEdge.id))) or 0),
             "blum_dataset_exports": int(db.scalar(select(func.count(BlumDatasetExport.id))) or 0),
             "blum_training_jobs": int(db.scalar(select(func.count(BlumModelTrainingJob.id))) or 0),
+            "external_dataset_sources": int(db.scalar(select(func.count(ExternalDatasetSource.id))) or 0),
+            "autonomous_engine_runs": int(db.scalar(select(func.count(AutonomousEngineRun.id))) or 0),
         },
         "latest_news_created_at": latest_brain,
         "why_gui_can_look_unchanged": [
             "Hugging Face serves the previous image until the Docker build finishes successfully.",
             "The finance-domain 7B model is disabled by default unless BLUM_ENABLE_FINANCIAL_BRAIN_MODEL=true.",
             "Existing snapshots must be regenerated with Run brain or full pipeline after a new deployment.",
-            "Browser cache can keep old static Next.js chunks; hard refresh if app_version is not 0.7.1.",
+            "Browser cache can keep old static Next.js chunks; hard refresh if app_version is not 0.8.0.",
         ],
     }
 
@@ -243,6 +255,26 @@ def trigger_database_backup() -> dict:
     return backup_embedded_postgres_if_configured(reason="manual_api_trigger")
 
 
+@router.get("/autonomous/status")
+def autonomous_status(db: Session = Depends(get_db)) -> dict:
+    return latest_autonomous_status(db)
+
+
+@router.post("/autonomous/run")
+def autonomous_run(db: Session = Depends(get_db)) -> dict:
+    return AutonomousResearchEngine().run_cycle(db, trigger="manual_diagnostic")
+
+
+@router.get("/datasets/sources")
+def datasets_sources(limit: int = Query(default=80, ge=1, le=200), db: Session = Depends(get_db)) -> dict:
+    return dataset_catalog_status(db, limit=limit)
+
+
+@router.post("/datasets/refresh")
+def datasets_refresh(db: Session = Depends(get_db)) -> dict:
+    return refresh_huggingface_dataset_catalog(db, validate=True)
+
+
 @router.get("/brain/status")
 def financial_brain_status(db: Session = Depends(get_db)) -> dict:
     return brain_status(db)

backend/app/core/config.py

@@ -5,7 +5,7 @@ from pydantic_settings import BaseSettings
 
 class Settings(BaseSettings):
     app_name: str = "Blum AI Financial Intelligence"
-    app_version: str = "0.7.1"
+    app_version: str = "0.8.0"
     environment: str = Field(default="demo", alias="ENVIRONMENT")
     database_url: str = Field(
         default="postgresql+psycopg2://postgres:postgres@127.0.0.1:5432/blum",
@@ -28,12 +28,14 @@ class Settings(BaseSettings):
     chart_vision_min_confidence: float = Field(default=0.70, alias="CHART_VISION_MIN_CONFIDENCE")
     default_benchmark: str = Field(default="SPY", alias="BLUM_DEFAULT_BENCHMARK")
     enable_yfinance_fallback: bool = Field(default=False, alias="BLUM_ENABLE_YFINANCE_FALLBACK")
-    max_update_assets: int = Field(default=36, alias="BLUM_MAX_UPDATE_ASSETS")
+    max_update_assets: int = Field(default=80, alias="BLUM_MAX_UPDATE_ASSETS")
     enable_live_startup: bool = Field(default=True, alias="BLUM_ENABLE_LIVE_STARTUP")
+    enable_autonomous_engine: bool = Field(default=True, alias="BLUM_ENABLE_AUTONOMOUS_ENGINE")
+    autonomous_cycle_minutes: int = Field(default=20, alias="BLUM_AUTONOMOUS_CYCLE_MINUTES")
     seed_historical_prices_on_startup: bool = Field(default=True, alias="BLUM_SEED_HISTORICAL_PRICES_ON_STARTUP")
     seed_signals_on_startup: bool = Field(default=True, alias="BLUM_SEED_SIGNALS_ON_STARTUP")
     seed_accuracy_on_startup: bool = Field(default=True, alias="BLUM_SEED_ACCURACY_ON_STARTUP")
-    startup_pipeline_limit: int = Field(default=36, alias="BLUM_STARTUP_PIPELINE_LIMIT")
+    startup_pipeline_limit: int = Field(default=80, alias="BLUM_STARTUP_PIPELINE_LIMIT")
     news_refresh_minutes: int = Field(default=10, alias="BLUM_NEWS_REFRESH_MINUTES")
     market_refresh_minutes: int = Field(default=45, alias="BLUM_MARKET_REFRESH_MINUTES")
     data_gap_repair_minutes: int = Field(default=180, alias="BLUM_DATA_GAP_REPAIR_MINUTES")
@@ -48,12 +50,15 @@ class Settings(BaseSettings):
     minimum_history_rows: int = Field(default=220, alias="BLUM_MINIMUM_HISTORY_ROWS")
     ipo_refresh_minutes: int = Field(default=120, alias="BLUM_IPO_REFRESH_MINUTES")
     news_fetch_workers: int = Field(default=10, alias="BLUM_NEWS_FETCH_WORKERS")
-    max_dynamic_asset_news_feeds: int = Field(default=36, alias="BLUM_MAX_DYNAMIC_ASSET_NEWS_FEEDS")
+    max_dynamic_asset_news_feeds: int = Field(default=60, alias="BLUM_MAX_DYNAMIC_ASSET_NEWS_FEEDS")
     historical_price_period: str = Field(default="max", alias="BLUM_HISTORICAL_PRICE_PERIOD")
     refresh_price_period: str = Field(default="6mo", alias="BLUM_REFRESH_PRICE_PERIOD")
     sec_user_agent: str = Field(default="Blum-AI-Financial-Intelligence research demo", alias="BLUM_SEC_USER_AGENT")
     blum_model_repository: str = Field(default="Italianhype/Blum", alias="BLUM_MODEL_REPOSITORY")
     training_export_dir: str = Field(default="/tmp/blum_training_exports", alias="BLUM_TRAINING_EXPORT_DIR")
+    enable_hf_dataset_catalog: bool = Field(default=True, alias="BLUM_ENABLE_HF_DATASET_CATALOG")
+    hf_dataset_refresh_hours: int = Field(default=24, alias="BLUM_HF_DATASET_REFRESH_HOURS")
+    hf_dataset_max_sources: int = Field(default=40, alias="BLUM_HF_DATASET_MAX_SOURCES")
 
     class Config:
         env_file = ".env"

backend/app/data/seed_assets.py

@@ -69,3 +69,47 @@ SEED_ASSETS = [
     {"ticker": "ENR.DE", "name": "Siemens Energy AG", "category": "Clean Energy", "sector": "Industrials", "industry": "Power Infrastructure", "country": "Germany", "asset_type": "Stock", "currency": "EUR", "exchange": "XETRA", "description": "Grid, turbine and power infrastructure transition exposure."},
 ]
 
+SEED_ASSETS.extend([
+    {"ticker": "VOO", "name": "Vanguard S&P 500 ETF", "category": "Broad Market", "sector": "US Equity", "industry": "Large Cap Blend", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Low-cost S&P 500 benchmark exposure."},
+    {"ticker": "VTI", "name": "Vanguard Total Stock Market ETF", "category": "Broad Market", "sector": "US Equity", "industry": "Total Market", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Total US equity market breadth proxy."},
+    {"ticker": "XLI", "name": "Industrial Select Sector SPDR Fund", "category": "Sector", "sector": "Industrials", "industry": "Industrials", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Industrial cyclicality and capex proxy."},
+    {"ticker": "XLY", "name": "Consumer Discretionary Select Sector SPDR Fund", "category": "Sector", "sector": "Consumer Discretionary", "industry": "Consumer Discretionary", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Consumer discretionary and risk appetite proxy."},
+    {"ticker": "XLP", "name": "Consumer Staples Select Sector SPDR Fund", "category": "Sector", "sector": "Consumer Staples", "industry": "Consumer Staples", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Defensive consumer staples proxy."},
+    {"ticker": "XLU", "name": "Utilities Select Sector SPDR Fund", "category": "Sector", "sector": "Utilities", "industry": "Utilities", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Utilities, rates sensitivity and power demand proxy."},
+    {"ticker": "XLC", "name": "Communication Services Select Sector SPDR Fund", "category": "Sector", "sector": "Communication Services", "industry": "Media and Platforms", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Communication services sector rotation proxy."},
+    {"ticker": "SOXX", "name": "iShares Semiconductor ETF", "category": "Semiconductors", "sector": "Technology", "industry": "Semiconductors", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NASDAQ", "description": "Semiconductor ETF confirmation source."},
+    {"ticker": "IGV", "name": "iShares Expanded Tech-Software Sector ETF", "category": "Software", "sector": "Technology", "industry": "Software", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "Cboe", "description": "Software sector confirmation source."},
+    {"ticker": "CIBR", "name": "First Trust Nasdaq Cybersecurity ETF", "category": "Cyber Security", "sector": "Technology", "industry": "Cyber Security", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NASDAQ", "description": "Cybersecurity thematic confirmation ETF."},
+    {"ticker": "URA", "name": "Global X Uranium ETF", "category": "Nuclear Energy", "sector": "Energy", "industry": "Uranium and Nuclear", "country": "United States", "asset_type": "ETF", "currency": "USD", "exchange": "NYSE Arca", "description": "Uranium and nuclear power thematic proxy."},
+    {"ticker": "ORCL", "name": "Oracle Corporation", "category": "AI Infrastructure", "sector": "Technology", "industry": "Cloud Software", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Cloud infrastructure, database and enterprise AI workload exposure."},
+    {"ticker": "CRM", "name": "Salesforce Inc.", "category": "Software", "sector": "Technology", "industry": "Enterprise Software", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Enterprise software and AI-enabled workflow platform."},
+    {"ticker": "NOW", "name": "ServiceNow Inc.", "category": "Software", "sector": "Technology", "industry": "Workflow Software", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Enterprise workflow automation and AI software exposure."},
+    {"ticker": "PLTR", "name": "Palantir Technologies Inc.", "category": "AI", "sector": "Technology", "industry": "Data Analytics Software", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "AI data platform, defense and commercial analytics exposure."},
+    {"ticker": "NFLX", "name": "Netflix Inc.", "category": "Growth", "sector": "Communication Services", "industry": "Streaming Media", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Streaming media, advertising tier and global content platform."},
+    {"ticker": "SMCI", "name": "Super Micro Computer Inc.", "category": "AI Infrastructure", "sector": "Technology", "industry": "AI Servers", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "AI server infrastructure and data center buildout exposure."},
+    {"ticker": "MU", "name": "Micron Technology Inc.", "category": "Semiconductors", "sector": "Technology", "industry": "Memory Semiconductors", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Memory cycle and high-bandwidth memory AI exposure."},
+    {"ticker": "QCOM", "name": "Qualcomm Inc.", "category": "Semiconductors", "sector": "Technology", "industry": "Wireless Semiconductors", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Mobile, edge AI and wireless semiconductor exposure."},
+    {"ticker": "AMAT", "name": "Applied Materials Inc.", "category": "Semiconductors", "sector": "Technology", "industry": "Semiconductor Equipment", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Wafer fabrication equipment and semiconductor capex proxy."},
+    {"ticker": "LRCX", "name": "Lam Research Corporation", "category": "Semiconductors", "sector": "Technology", "industry": "Semiconductor Equipment", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Etch and deposition equipment exposure."},
+    {"ticker": "KLAC", "name": "KLA Corporation", "category": "Semiconductors", "sector": "Technology", "industry": "Semiconductor Equipment", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Process control and inspection equipment exposure."},
+    {"ticker": "INTC", "name": "Intel Corporation", "category": "Semiconductors", "sector": "Technology", "industry": "Integrated Semiconductors", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Foundry turnaround, CPUs and domestic semiconductor policy exposure."},
+    {"ticker": "IBM", "name": "International Business Machines Corporation", "category": "AI Infrastructure", "sector": "Technology", "industry": "Hybrid Cloud and Consulting", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Hybrid cloud, enterprise AI and consulting exposure."},
+    {"ticker": "PANW", "name": "Palo Alto Networks Inc.", "category": "Cyber Security", "sector": "Technology", "industry": "Cyber Security", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Enterprise cybersecurity platform leader."},
+    {"ticker": "CRWD", "name": "CrowdStrike Holdings Inc.", "category": "Cyber Security", "sector": "Technology", "industry": "Endpoint Security", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Cloud-native endpoint and security operations platform."},
+    {"ticker": "UBER", "name": "Uber Technologies Inc.", "category": "Growth", "sector": "Industrials", "industry": "Mobility and Delivery", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Mobility, delivery and platform operating leverage exposure."},
+    {"ticker": "COST", "name": "Costco Wholesale Corporation", "category": "Quality", "sector": "Consumer Staples", "industry": "Warehouse Retail", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "High-quality consumer staples and membership retail proxy."},
+    {"ticker": "WMT", "name": "Walmart Inc.", "category": "Quality", "sector": "Consumer Staples", "industry": "Retail", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Defensive retail, consumer health and scale logistics proxy."},
+    {"ticker": "HD", "name": "Home Depot Inc.", "category": "Housing", "sector": "Consumer Discretionary", "industry": "Home Improvement Retail", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Housing, repair/remodel and consumer discretionary proxy."},
+    {"ticker": "BA", "name": "Boeing Company", "category": "Aerospace", "sector": "Industrials", "industry": "Aerospace", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Commercial aerospace and defense manufacturing exposure."},
+    {"ticker": "CAT", "name": "Caterpillar Inc.", "category": "Industrials", "sector": "Industrials", "industry": "Machinery", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Global machinery, construction, mining and infrastructure proxy."},
+    {"ticker": "RTX", "name": "RTX Corporation", "category": "Defense", "sector": "Industrials", "industry": "Aerospace and Defense", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Defense systems and commercial aerospace supplier."},
+    {"ticker": "LMT", "name": "Lockheed Martin Corporation", "category": "Defense", "sector": "Industrials", "industry": "Defense", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Defense prime contractor and aerospace systems leader."},
+    {"ticker": "NOC", "name": "Northrop Grumman Corporation", "category": "Defense", "sector": "Industrials", "industry": "Defense", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Defense, space and strategic systems exposure."},
+    {"ticker": "UNH", "name": "UnitedHealth Group Incorporated", "category": "Healthcare", "sector": "Healthcare", "industry": "Managed Care", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Managed care and health services bellwether."},
+    {"ticker": "MRK", "name": "Merck & Co. Inc.", "category": "Healthcare", "sector": "Healthcare", "industry": "Pharmaceuticals", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Pharmaceutical innovation and oncology exposure."},
+    {"ticker": "ISRG", "name": "Intuitive Surgical Inc.", "category": "Healthcare Innovation", "sector": "Healthcare", "industry": "Medical Devices", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NASDAQ", "description": "Robotic surgery and medical device innovation exposure."},
+    {"ticker": "V", "name": "Visa Inc.", "category": "Quality", "sector": "Financials", "industry": "Payments", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Global card network and payments volume proxy."},
+    {"ticker": "MA", "name": "Mastercard Incorporated", "category": "Quality", "sector": "Financials", "industry": "Payments", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Global payments network and consumer spending proxy."},
+    {"ticker": "BAC", "name": "Bank of America Corporation", "category": "Value", "sector": "Financials", "industry": "Banks", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Rates, credit and US banking cycle exposure."},
+    {"ticker": "GS", "name": "Goldman Sachs Group Inc.", "category": "Capital Markets", "sector": "Financials", "industry": "Investment Banking", "country": "United States", "asset_type": "Stock", "currency": "USD", "exchange": "NYSE", "description": "Capital markets, deal activity and institutional risk appetite proxy."},
+])

backend/app/models.py

@@ -955,3 +955,53 @@ class BlumModelTrainingJob(Base):
     updated_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow, index=True)
 
     dataset_export = relationship("BlumDatasetExport")
+
+
+class ExternalDatasetSource(Base):
+    __tablename__ = "external_dataset_sources"
+    __table_args__ = (
+        UniqueConstraint("dataset_id", name="uq_external_dataset_source_dataset_id"),
+        Index("ix_external_dataset_sources_status_priority", "status", "priority"),
+        Index("ix_external_dataset_sources_domain_updated", "primary_domain", "updated_at"),
+    )
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    dataset_id: Mapped[str] = mapped_column(String(220), unique=True, index=True)
+    provider: Mapped[str] = mapped_column(String(80), default="hugging_face", index=True)
+    title: Mapped[str] = mapped_column(String(260), default="")
+    primary_domain: Mapped[str] = mapped_column(String(80), default="market_data", index=True)
+    data_domains: Mapped[dict] = mapped_column(JsonType, default=dict)
+    license: Mapped[str] = mapped_column(String(80), default="unknown", index=True)
+    priority: Mapped[int] = mapped_column(Integer, default=50, index=True)
+    ingestion_mode: Mapped[str] = mapped_column(String(80), default="catalog_only", index=True)
+    status: Mapped[str] = mapped_column(String(80), default="discovered", index=True)
+    dataset_url: Mapped[str] = mapped_column(Text, default="")
+    viewer_status: Mapped[dict] = mapped_column(JsonType, default=dict)
+    parquet_files: Mapped[dict] = mapped_column(JsonType, default=dict)
+    size_summary: Mapped[dict] = mapped_column(JsonType, default=dict)
+    usage_policy: Mapped[dict] = mapped_column(JsonType, default=dict)
+    last_checked_at: Mapped[datetime | None] = mapped_column(DateTime, index=True)
+    created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, index=True)
+    updated_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, onupdate=datetime.utcnow, index=True)
+
+
+class AutonomousEngineRun(Base):
+    __tablename__ = "autonomous_engine_runs"
+    __table_args__ = (
+        UniqueConstraint("run_id", name="uq_autonomous_engine_run_id"),
+        Index("ix_autonomous_engine_runs_status_started", "status", "started_at"),
+    )
+
+    id: Mapped[int] = mapped_column(Integer, primary_key=True)
+    run_id: Mapped[str] = mapped_column(String(100), unique=True, index=True)
+    trigger: Mapped[str] = mapped_column(String(80), default="scheduled", index=True)
+    status: Mapped[str] = mapped_column(String(80), default="running", index=True)
+    started_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, index=True)
+    completed_at: Mapped[datetime | None] = mapped_column(DateTime, index=True)
+    stage_results: Mapped[dict] = mapped_column(JsonType, default=dict)
+    readiness_score: Mapped[float] = mapped_column(Float, default=0.0, index=True)
+    data_coverage_score: Mapped[float] = mapped_column(Float, default=0.0, index=True)
+    reasoning_memory_created: Mapped[int] = mapped_column(Integer, default=0)
+    warning_count: Mapped[int] = mapped_column(Integer, default=0)
+    error_payload: Mapped[dict] = mapped_column(JsonType, default=dict)
+    created_at: Mapped[datetime] = mapped_column(DateTime, default=datetime.utcnow, index=True)

backend/app/services/autonomous_engine.py

@@ -0,0 +1,182 @@
+from __future__ import annotations
+
+from datetime import datetime, timedelta
+import traceback
+from uuid import uuid4
+
+from sqlalchemy import func, select
+from sqlalchemy.orm import Session
+
+from app.core.config import get_settings
+from app.ingestion.news_ingestor import NewsIngestor
+from app.models import (
+    Asset,
+    AutonomousEngineRun,
+    BlumKnowledgeRecord,
+    ExternalDatasetSource,
+    LearningEvent,
+    NewsArticle,
+    PriceHistory,
+    SignalSnapshot,
+)
+from app.services.accuracy import run_accuracy_audit
+from app.services.blum_financial_model import run_model_learning_cycle
+from app.services.etf import update_etf_trends
+from app.services.fundamentals import update_fundamentals
+from app.services.huggingface_datasets import dataset_catalog_status, refresh_huggingface_dataset_catalog
+from app.services.ipo import update_ipo_radar
+from app.services.macro import update_macro_snapshots
+from app.services.market_data import MarketDataService
+from app.services.persistence import backup_embedded_postgres_if_configured
+from app.signals.engine import SignalEngine
+
+
+settings = get_settings()
+
+
+class AutonomousResearchEngine:
+    """Runs Blum's server-side intelligence cycle in a strict, auditable sequence."""
+
+    def run_cycle(self, db: Session, trigger: str = "scheduled") -> dict:
+        run_id = f"auto-{datetime.utcnow().strftime('%Y%m%d%H%M%S')}-{uuid4().hex[:8]}"
+        stage_results: dict[str, dict] = {}
+        started_at = datetime.utcnow()
+
+        def stage(name: str, work) -> dict:
+            stage_started = datetime.utcnow()
+            try:
+                result = work()
+                payload = {
+                    "status": "ok",
+                    "started_at": stage_started.isoformat(),
+                    "completed_at": datetime.utcnow().isoformat(),
+                    "result": result or {},
+                }
+            except Exception as exc:
+                payload = {
+                    "status": "error",
+                    "started_at": stage_started.isoformat(),
+                    "completed_at": datetime.utcnow().isoformat(),
+                    "error": f"{type(exc).__name__}: {exc}",
+                    "traceback": traceback.format_exc(limit=4),
+                }
+            stage_results[name] = payload
+            return payload
+
+        stage("hf_dataset_catalog", lambda: self.refresh_dataset_catalog_if_needed(db))
+        stage("macro_context", lambda: update_macro_snapshots(db))
+        stage("fundamentals", lambda: update_fundamentals(db, limit=min(settings.max_update_assets, 32)))
+        stage("historical_prices", lambda: MarketDataService().update_prices(db, period=self.price_period(trigger), limit=settings.max_update_assets))
+        stage("news_sentiment", lambda: NewsIngestor().update_news(db, lookback_hours=96, limit_per_feed=45))
+        stage("signals", lambda: SignalEngine().run(db, limit=settings.max_update_assets))
+        stage("etf_intelligence", lambda: update_etf_trends(db))
+        stage("ipo_radar", lambda: update_ipo_radar(db, limit_per_form=55))
+        stage("accuracy_audit", lambda: run_accuracy_audit(db, limit=settings.max_update_assets))
+        if settings.enable_learning_loop:
+            stage("blum_financial_model", lambda: run_model_learning_cycle(db, limit=settings.blum_model_cycle_limit))
+        stage("persistence_backup", lambda: backup_embedded_postgres_if_configured(reason="autonomous_engine_cycle"))
+
+        readiness = self.readiness(db, stage_results)
+        status = "ok" if readiness["warning_count"] == 0 else "degraded"
+        if any(item.get("status") == "error" for item in stage_results.values()):
+            status = "degraded"
+        run = AutonomousEngineRun(
+            run_id=run_id,
+            trigger=trigger,
+            status=status,
+            started_at=started_at,
+            completed_at=datetime.utcnow(),
+            stage_results=stage_results,
+            readiness_score=readiness["readiness_score"],
+            data_coverage_score=readiness["data_coverage_score"],
+            reasoning_memory_created=readiness["reasoning_memory_created"],
+            warning_count=readiness["warning_count"],
+            error_payload=readiness["errors"],
+        )
+        db.add(run)
+        db.add(
+            LearningEvent(
+                event_type="autonomous_research_engine_cycle",
+                severity="Info" if status == "ok" else "Warning",
+                title="Autonomous Blum research cycle completed",
+                description="Blum executed the full research pipeline without manual input.",
+                payload={"run_id": run_id, "trigger": trigger, "status": status, "readiness": readiness, "stage_results": stage_results},
+            )
+        )
+        db.commit()
+        return {"run_id": run_id, "trigger": trigger, "status": status, "readiness": readiness, "stage_results": stage_results}
+
+    def refresh_dataset_catalog_if_needed(self, db: Session) -> dict:
+        if not settings.enable_hf_dataset_catalog:
+            return {"status": "disabled"}
+        latest = db.scalar(select(func.max(ExternalDatasetSource.updated_at)))
+        if latest and latest >= datetime.utcnow() - timedelta(hours=settings.hf_dataset_refresh_hours):
+            return {"status": "fresh", "catalog": dataset_catalog_status(db, limit=settings.hf_dataset_max_sources)}
+        return refresh_huggingface_dataset_catalog(db, validate=True)
+
+    def price_period(self, trigger: str) -> str:
+        if trigger in {"startup", "manual"}:
+            return settings.historical_price_period
+        return settings.refresh_price_period
+
+    def readiness(self, db: Session, stage_results: dict) -> dict:
+        active_assets = int(db.scalar(select(func.count(Asset.id)).where(Asset.is_active.is_(True))) or 0)
+        priced_assets = int(db.scalar(select(func.count(func.distinct(PriceHistory.asset_id)))) or 0)
+        signal_count = int(db.scalar(select(func.count(SignalSnapshot.id))) or 0)
+        news_count = int(db.scalar(select(func.count(NewsArticle.id))) or 0)
+        reasoning_records = int(db.scalar(select(func.count(BlumKnowledgeRecord.id))) or 0)
+        dataset_sources = int(db.scalar(select(func.count(ExternalDatasetSource.id))) or 0)
+        coverage = (priced_assets / active_assets * 100) if active_assets else 0.0
+        evidence_components = [
+            min(100.0, coverage),
+            100.0 if signal_count > 0 else 0.0,
+            min(100.0, news_count / 100 * 100),
+            min(100.0, reasoning_records / 100 * 100),
+            min(100.0, dataset_sources / max(1, settings.hf_dataset_max_sources) * 100),
+        ]
+        errors = {name: result for name, result in stage_results.items() if result.get("status") == "error"}
+        warnings = sum(1 for result in stage_results.values() if result.get("status") != "ok")
+        model_stage = stage_results.get("blum_financial_model", {}).get("result", {})
+        return {
+            "active_assets": active_assets,
+            "priced_assets": priced_assets,
+            "signal_count": signal_count,
+            "news_count": news_count,
+            "reasoning_records": reasoning_records,
+            "dataset_sources": dataset_sources,
+            "data_coverage_score": round(coverage, 2),
+            "readiness_score": round(sum(evidence_components) / len(evidence_components), 2),
+            "reasoning_memory_created": int(model_stage.get("knowledge_records_created", 0) or 0),
+            "warning_count": warnings,
+            "errors": errors,
+            "policy": "Autonomous research improves evidence quality and calibration; it does not guarantee market outperformance or execute trades.",
+        }
+
+
+def latest_autonomous_status(db: Session) -> dict:
+    latest = db.scalar(select(AutonomousEngineRun).order_by(AutonomousEngineRun.started_at.desc()).limit(1))
+    runs = db.scalars(select(AutonomousEngineRun).order_by(AutonomousEngineRun.started_at.desc()).limit(20)).all()
+    return {
+        "enabled": settings.enable_autonomous_engine,
+        "cycle_minutes": settings.autonomous_cycle_minutes,
+        "latest_run": serialize_run(latest) if latest else None,
+        "recent_runs": [serialize_run(run) for run in runs],
+        "dataset_catalog": dataset_catalog_status(db, limit=settings.hf_dataset_max_sources),
+        "policy": "Blum runs sequential server-side research cycles. Manual buttons are optional diagnostics, not required for normal operation.",
+    }
+
+
+def serialize_run(run: AutonomousEngineRun) -> dict:
+    return {
+        "run_id": run.run_id,
+        "trigger": run.trigger,
+        "status": run.status,
+        "started_at": run.started_at.isoformat() if run.started_at else None,
+        "completed_at": run.completed_at.isoformat() if run.completed_at else None,
+        "readiness_score": run.readiness_score,
+        "data_coverage_score": run.data_coverage_score,
+        "reasoning_memory_created": run.reasoning_memory_created,
+        "warning_count": run.warning_count,
+        "stage_results": run.stage_results,
+        "error_payload": run.error_payload,
+    }

backend/app/services/fundamentals.py

@@ -28,6 +28,37 @@ CIK_MAP = {
     "NVO": "0000353278",
     "ASML": "0000937966",
     "SAP": "0001000184",
+    "ORCL": "0001341439",
+    "CRM": "0001108524",
+    "NOW": "0001373715",
+    "PLTR": "0001321655",
+    "NFLX": "0001065280",
+    "SMCI": "0001375365",
+    "MU": "0000723125",
+    "QCOM": "0000804328",
+    "AMAT": "0000006951",
+    "LRCX": "0000707549",
+    "KLAC": "0000319201",
+    "INTC": "0000050863",
+    "IBM": "0000051143",
+    "PANW": "0001327567",
+    "CRWD": "0001535527",
+    "UBER": "0001543151",
+    "COST": "0000909832",
+    "WMT": "0000104169",
+    "HD": "0000354950",
+    "BA": "0000012927",
+    "CAT": "0000018230",
+    "RTX": "0000101829",
+    "LMT": "0000936468",
+    "NOC": "0001133421",
+    "UNH": "0000731766",
+    "MRK": "0000310158",
+    "ISRG": "0001035267",
+    "V": "0001403161",
+    "MA": "0001141391",
+    "BAC": "0000070858",
+    "GS": "0000886982",
 }
 
 SEC_METRICS = {

backend/app/services/huggingface_datasets.py

@@ -0,0 +1,263 @@
+from __future__ import annotations
+
+from datetime import datetime
+from urllib.parse import quote
+
+import requests
+from sqlalchemy import desc, select
+from sqlalchemy.orm import Session
+
+from app.models import ExternalDatasetSource, LearningEvent
+
+
+DATASETS_SERVER = "https://datasets-server.huggingface.co"
+HF_DATASET_URL = "https://huggingface.co/datasets"
+
+
+CURATED_HF_DATASETS = [
+    {
+        "dataset_id": "defeatbeta/yahoo-finance-data",
+        "title": "Yahoo Finance data, stock news and earnings call transcripts",
+        "primary_domain": "multi_source_finance",
+        "data_domains": ["market_data", "stock_news", "earnings_transcripts", "treasury_data"],
+        "license": "odc-by",
+        "priority": 98,
+        "ingestion_mode": "metadata_and_incremental_evidence",
+        "usage_policy": "Public dataset metadata is cataloged automatically. Large table ingestion must remain incremental and evidence-tracked.",
+    },
+    {
+        "dataset_id": "paperswithbacktest/Stocks-Daily-Price",
+        "title": "Daily price data for 7000+ US stocks",
+        "primary_domain": "historical_prices",
+        "data_domains": ["daily_ohlcv", "us_equities", "backtest_history"],
+        "license": "other",
+        "priority": 94,
+        "ingestion_mode": "metadata_first_price_backfill_candidate",
+        "usage_policy": "Use as a candidate for historical OHLCV backfill after schema validation. No synthetic prices are created.",
+    },
+    {
+        "dataset_id": "TeraflopAI/SEC-EDGAR",
+        "title": "Large-scale SEC EDGAR filings corpus",
+        "primary_domain": "filings",
+        "data_domains": ["sec_filings", "10k", "10q", "risk_factors", "management_discussion"],
+        "license": "apache-2.0",
+        "priority": 92,
+        "ingestion_mode": "catalog_and_targeted_retrieval",
+        "usage_policy": "Very large corpus. The Space should retrieve targeted company evidence rather than materializing the full corpus.",
+    },
+    {
+        "dataset_id": "kurry/sp500_earnings_transcripts",
+        "title": "S&P 500 and US large-cap earnings transcripts from 2005 to 2025",
+        "primary_domain": "earnings_transcripts",
+        "data_domains": ["earnings_calls", "management_language", "guidance", "sentiment"],
+        "license": "mit",
+        "priority": 90,
+        "ingestion_mode": "metadata_first_transcript_retrieval",
+        "usage_policy": "Use for company narrative history and management tone. Respect dataset license and avoid fabricating missing transcripts.",
+    },
+    {
+        "dataset_id": "glopardo/sp500-earnings-transcripts",
+        "title": "S&P 500 earnings call transcripts and quarterly context",
+        "primary_domain": "earnings_transcripts",
+        "data_domains": ["earnings_calls", "question_answering", "summarization", "retrieval"],
+        "license": "unknown",
+        "priority": 82,
+        "ingestion_mode": "metadata_first_transcript_retrieval",
+        "usage_policy": "Secondary transcript source for cross-source validation.",
+    },
+    {
+        "dataset_id": "paperswithbacktest/Stocks-Quarterly-Earnings",
+        "title": "Quarterly earnings reports for 7000+ US stocks",
+        "primary_domain": "fundamentals",
+        "data_domains": ["quarterly_earnings", "fundamentals", "us_equities"],
+        "license": "other",
+        "priority": 78,
+        "ingestion_mode": "gated_metadata_only",
+        "usage_policy": "Gated source. Catalog only unless access is explicitly available.",
+    },
+    {
+        "dataset_id": "c3po-ai/edgar-corpus",
+        "title": "Annual SEC 10-K filing corpus from 1993 to 2020",
+        "primary_domain": "filings",
+        "data_domains": ["10k", "sec_filings", "long_document_understanding"],
+        "license": "apache-2.0",
+        "priority": 75,
+        "ingestion_mode": "catalog_and_targeted_retrieval",
+        "usage_policy": "Historical filing text source for long-term company memory.",
+    },
+    {
+        "dataset_id": "PatronusAI/financebench",
+        "title": "FinanceBench open-book financial QA benchmark",
+        "primary_domain": "reasoning_benchmark",
+        "data_domains": ["financial_qa", "reasoning_evaluation", "open_book_benchmark"],
+        "license": "cc-by-nc-4.0",
+        "priority": 68,
+        "ingestion_mode": "evaluation_only",
+        "usage_policy": "Use for evaluation ideas and benchmark structure, not for commercial training unless license allows it.",
+    },
+    {
+        "dataset_id": "BUPT-Reasoning-Lab/FinanceReasoning",
+        "title": "Finance reasoning dataset",
+        "primary_domain": "reasoning_benchmark",
+        "data_domains": ["financial_reasoning", "thesis_quality", "reasoning_evaluation"],
+        "license": "cc-by-4.0",
+        "priority": 66,
+        "ingestion_mode": "evaluation_only",
+        "usage_policy": "Use to benchmark reasoning style; Blum proprietary reasoning remains separate.",
+    },
+    {
+        "dataset_id": "jlh-ibm/earnings_call",
+        "title": "Earnings call transcripts with stock and sector price context",
+        "primary_domain": "earnings_transcripts",
+        "data_domains": ["earnings_calls", "stock_prices", "sector_index"],
+        "license": "cc0-1.0",
+        "priority": 64,
+        "ingestion_mode": "metadata_first_small_reference",
+        "usage_policy": "Useful as a small validation corpus for transcript and price-link logic.",
+    },
+    {
+        "dataset_id": "younginpiniti/us-stocks-daily-all",
+        "title": "US stocks daily historical dataset",
+        "primary_domain": "historical_prices",
+        "data_domains": ["daily_ohlcv", "us_equities"],
+        "license": "unknown",
+        "priority": 63,
+        "ingestion_mode": "metadata_first_price_backfill_candidate",
+        "usage_policy": "Candidate source for redundant daily OHLCV validation after schema inspection.",
+    },
+    {
+        "dataset_id": "sfd-anonymous/edgar-forecast-benchmark",
+        "title": "EDGAR grounded numerical forecasting benchmark",
+        "primary_domain": "reasoning_benchmark",
+        "data_domains": ["sec_filings", "forecasting_benchmark", "numerical_reasoning"],
+        "license": "cc-by-4.0",
+        "priority": 60,
+        "ingestion_mode": "evaluation_only",
+        "usage_policy": "Use to evaluate grounded reasoning, not as direct trading signal evidence.",
+    },
+]
+
+
+def refresh_huggingface_dataset_catalog(db: Session, validate: bool = True) -> dict:
+    upserted = 0
+    validation = []
+    for item in CURATED_HF_DATASETS:
+        dataset_id = item["dataset_id"]
+        status_payload = validate_dataset(dataset_id) if validate else {"status": "not_checked"}
+        source = db.scalar(select(ExternalDatasetSource).where(ExternalDatasetSource.dataset_id == dataset_id))
+        payload = {
+            "provider": "hugging_face",
+            "title": item["title"],
+            "primary_domain": item["primary_domain"],
+            "data_domains": {"items": item["data_domains"]},
+            "license": item["license"],
+            "priority": item["priority"],
+            "ingestion_mode": item["ingestion_mode"],
+            "status": source_status(status_payload),
+            "dataset_url": f"{HF_DATASET_URL}/{dataset_id}",
+            "viewer_status": status_payload,
+            "parquet_files": status_payload.get("parquet_files", {}),
+            "size_summary": status_payload.get("size", {}),
+            "usage_policy": {"policy": item["usage_policy"], "no_synthetic_data": True},
+            "last_checked_at": datetime.utcnow(),
+        }
+        if source is None:
+            source = ExternalDatasetSource(dataset_id=dataset_id, **payload)
+            db.add(source)
+        else:
+            for key, value in payload.items():
+                setattr(source, key, value)
+        upserted += 1
+        validation.append({"dataset_id": dataset_id, "status": payload["status"], "priority": item["priority"]})
+    db.add(
+        LearningEvent(
+            event_type="huggingface_dataset_catalog_refresh",
+            severity="Info",
+            title="Hugging Face financial dataset catalog refreshed",
+            description="Blum refreshed its catalog of real public datasets for market data, filings, earnings and reasoning benchmarks.",
+            payload={"sources_seen": len(CURATED_HF_DATASETS), "validation": validation},
+        )
+    )
+    db.commit()
+    return {"status": "ok", "sources_upserted": upserted, "validation": validation}
+
+
+def dataset_catalog_status(db: Session, limit: int = 80) -> dict:
+    rows = db.scalars(select(ExternalDatasetSource).order_by(ExternalDatasetSource.priority.desc(), desc(ExternalDatasetSource.updated_at)).limit(limit)).all()
+    ready = [row for row in rows if row.status in {"viewer_ready", "metadata_ready"}]
+    by_domain: dict[str, int] = {}
+    for row in rows:
+        by_domain[row.primary_domain] = by_domain.get(row.primary_domain, 0) + 1
+    return {
+        "status": "ready" if rows else "not_initialized",
+        "source_count": len(rows),
+        "ready_count": len(ready),
+        "domains": by_domain,
+        "sources": [serialize_source(row) for row in rows],
+        "policy": "Cataloged Hugging Face datasets are real public sources. Large datasets are validated and ingested incrementally, not copied blindly.",
+    }
+
+
+def validate_dataset(dataset_id: str) -> dict:
+    encoded = quote(dataset_id, safe="")
+    payload = {"dataset_id": dataset_id}
+    payload["is_valid"] = get_json(f"{DATASETS_SERVER}/is-valid?dataset={encoded}")
+    payload["splits"] = get_json(f"{DATASETS_SERVER}/splits?dataset={encoded}")
+    payload["parquet_files"] = slim_parquet(get_json(f"{DATASETS_SERVER}/parquet?dataset={encoded}"))
+    payload["size"] = get_json(f"{DATASETS_SERVER}/size?dataset={encoded}")
+    return payload
+
+
+def get_json(url: str) -> dict:
+    try:
+        response = requests.get(url, timeout=4)
+        if response.status_code >= 400:
+            return {"status": "error", "status_code": response.status_code, "detail": response.text[:240]}
+        return response.json()
+    except Exception as exc:
+        return {"status": "error", "error": f"{type(exc).__name__}: {exc}"}
+
+
+def slim_parquet(payload: dict) -> dict:
+    files = payload.get("parquet_files") if isinstance(payload, dict) else None
+    if not isinstance(files, list):
+        return payload
+    return {
+        "status": "ok",
+        "file_count": len(files),
+        "sample_files": files[:8],
+    }
+
+
+def source_status(payload: dict) -> str:
+    valid = payload.get("is_valid", {})
+    splits = payload.get("splits", {})
+    parquet = payload.get("parquet_files", {})
+    if isinstance(valid, dict) and valid.get("valid") is False:
+        return "unavailable"
+    if isinstance(parquet, dict) and parquet.get("file_count", 0) > 0:
+        return "viewer_ready"
+    if isinstance(splits, dict) and splits.get("splits"):
+        return "metadata_ready"
+    if any(isinstance(payload.get(key), dict) and payload[key].get("status") == "error" for key in ["is_valid", "splits", "parquet_files", "size"]):
+        return "metadata_partial"
+    return "discovered"
+
+
+def serialize_source(row: ExternalDatasetSource) -> dict:
+    return {
+        "dataset_id": row.dataset_id,
+        "title": row.title,
+        "primary_domain": row.primary_domain,
+        "data_domains": row.data_domains,
+        "license": row.license,
+        "priority": row.priority,
+        "ingestion_mode": row.ingestion_mode,
+        "status": row.status,
+        "dataset_url": row.dataset_url,
+        "viewer_status": row.viewer_status,
+        "parquet_files": row.parquet_files,
+        "size_summary": row.size_summary,
+        "usage_policy": row.usage_policy,
+        "last_checked_at": row.last_checked_at.isoformat() if row.last_checked_at else None,
+    }

backend/app/services/realtime.py

@@ -10,6 +10,7 @@ from app.core.config import get_settings
 from app.core.database import SessionLocal
 from app.ingestion.news_ingestor import NewsIngestor
 from app.services.accuracy import run_accuracy_audit
+from app.services.autonomous_engine import AutonomousResearchEngine
 from app.services.blum_financial_model import run_model_learning_cycle
 from app.services.data_continuity import repair_data_gaps
 from app.services.etf import update_etf_trends
@@ -44,6 +45,12 @@ def start_realtime_services() -> None:
     _scheduler = BackgroundScheduler(timezone="UTC")
     if settings.enable_live_startup:
         threading.Thread(target=run_startup_pipeline, daemon=True).start()
+    if settings.enable_autonomous_engine:
+        _scheduler.add_job(run_autonomous_engine_job, "interval", minutes=settings.autonomous_cycle_minutes, id="autonomous_research_engine", replace_existing=True, max_instances=1)
+        _scheduler.start()
+        with _state_lock:
+            _state["started"] = True
+        return
     _scheduler.add_job(run_news_refresh, "interval", minutes=settings.news_refresh_minutes, id="news_refresh", replace_existing=True, max_instances=1)
     _scheduler.add_job(run_market_refresh, "interval", minutes=settings.market_refresh_minutes, id="market_refresh", replace_existing=True, max_instances=1)
     _scheduler.add_job(run_data_gap_repair, "interval", minutes=settings.data_gap_repair_minutes, id="data_gap_repair", replace_existing=True, max_instances=1)
@@ -72,6 +79,10 @@ def realtime_status() -> dict:
 
 
 def run_startup_pipeline() -> None:
+    if settings.enable_autonomous_engine:
+        _run_job("autonomous_startup", lambda db: AutonomousResearchEngine().run_cycle(db, trigger="startup"))
+        return
+
     def work(db):
         pipeline = PipelineService().run(db, limit=settings.startup_pipeline_limit, period=settings.historical_price_period)
         learning = run_learning_cycle(db, limit=settings.max_update_assets * 6) if settings.enable_learning_loop else {}
@@ -128,6 +139,10 @@ def run_blum_model_cycle_job() -> None:
     _run_job("blum_financial_model_cycle", lambda db: run_model_learning_cycle(db, limit=settings.blum_model_cycle_limit))
 
 
+def run_autonomous_engine_job() -> None:
+    _run_job("autonomous_research_engine", lambda db: AutonomousResearchEngine().run_cycle(db, trigger="scheduled"))
+
+
 def _run_job(job_name: str, work):
     with _state_lock:
         if _state["running"]:

backend/app/services/thesis_engine.py

@@ -134,6 +134,7 @@ def build_signal_thesis_payload(asset: Asset, score: dict, technical: dict, narr
         narrative=narrative,
         related_news=[],
         market_context=market_context,
+        accuracy=narrative.get("accuracy_profile", {}),
     )
     return {
         "executive_thesis": thesis["executive_thesis"],
@@ -206,6 +207,7 @@ def enrich_theme_lifecycle(theme: dict, linked_assets: list[str] | None = None,
 
 
 def observed_facts(ticker: str, signal: dict, technical: dict, narrative: dict, related_news: list[dict], historical: dict) -> list[str]:
+    fundamentals = narrative.get("fundamentals") or {}
     facts = [
         f"{ticker} classification is {signal.get('classification', 'Insufficient Evidence')}.",
         f"Blum score is {display(signal.get('blum_score'))} and confidence is {display(signal.get('confidence_score'))}.",
@@ -213,6 +215,8 @@ def observed_facts(ticker: str, signal: dict, technical: dict, narrative: dict,
         f"7D sentiment is {display(narrative.get('sentiment_7d'))} across {int(number(narrative.get('news_count_7d')))} linked recent news records.",
         f"Stored related news items supplied to the thesis: {len(related_news)}.",
     ]
+    if fundamentals.get("status") == "ready":
+        facts.append(f"SEC fundamental score is {display(fundamentals.get('fundamental_score'))} from provider {fundamentals.get('provider', 'unknown')}.")
     if historical:
         facts.append(f"Historical similarity data mode is {historical.get('data_mode', historical.get('statistical_reliability', 'available'))}.")
     return facts
@@ -275,6 +279,7 @@ def causal_reasoning(ticker: str, technical: dict, narrative: dict, related_news
 def supporting_evidence(signal: dict, technical: dict, narrative: dict, related_news: list[dict], historical: dict) -> list[str]:
     rows = []
     breakdown = signal.get("score_breakdown") or {}
+    fundamentals = narrative.get("fundamentals") or {}
     if number(breakdown.get("momentum_score")) >= 60:
         rows.append(f"Momentum score is {number(breakdown.get('momentum_score')):.1f}.")
     if number(breakdown.get("trend_score")) >= 60:
@@ -285,6 +290,8 @@ def supporting_evidence(signal: dict, technical: dict, narrative: dict, related_
         rows.append("Price is above both SMA20 and SMA50.")
     if number(narrative.get("news_count_7d")) >= 2:
         rows.append(f"News flow is active with {int(number(narrative.get('news_count_7d')))} linked 7D records.")
+    if number(breakdown.get("fundamental_score"), number(fundamentals.get("fundamental_score"))) >= 62:
+        rows.append(f"Fundamental evidence is supportive with score {number(breakdown.get('fundamental_score'), number(fundamentals.get('fundamental_score'))):.1f}.")
     if related_news:
         best = max(related_news, key=lambda item: number(item.get("quality_score")))
         rows.append(f"Highest-quality linked news source is {best.get('source', 'unknown')} with quality {display(best.get('quality_score'))}.")
@@ -295,6 +302,7 @@ def supporting_evidence(signal: dict, technical: dict, narrative: dict, related_
 
 def contradicting_evidence(signal: dict, technical: dict, narrative: dict, related_news: list[dict], accuracy: dict, historical: dict) -> list[str]:
     rows = []
+    fundamentals = narrative.get("fundamentals") or {}
     perf_5d = number(technical.get("perf_5d"))
     sentiment_7d = number(narrative.get("sentiment_7d"))
     if perf_5d > 4 and sentiment_7d < -0.12:
@@ -305,6 +313,10 @@ def contradicting_evidence(signal: dict, technical: dict, narrative: dict, relat
         rows.append("RSI is elevated, increasing the risk of crowded short-term momentum.")
     if signal.get("risk_level") == "High":
         rows.append("The latest signal is explicitly marked High risk.")
+    if fundamentals.get("status") == "missing":
+        rows.append("Fundamental evidence is missing, so the thesis is less complete.")
+    elif number(fundamentals.get("fundamental_score")) < 45:
+        rows.append("Fundamental score is weak relative to the technical or narrative setup.")
     if number(accuracy.get("blum_confidence_score"), 100) < 55:
         rows.append("Evidence confidence is limited by data quality checks.")
     if historical.get("data_mode") == "demonstration_mode":
@@ -331,7 +343,8 @@ def uncertainty_points(signal: dict, technical: dict, narrative: dict, related_n
 
 def missing_information(signal: dict, technical: dict, narrative: dict, related_news: list[dict], historical: dict, accuracy: dict) -> list[str]:
     rows = []
-    if technical.get("fundamental_confirmation") is None:
+    fundamentals = narrative.get("fundamentals") or {}
+    if fundamentals.get("status") != "ready":
         rows.append("Fundamental confirmation is not fully connected to the signal.")
     if number(narrative.get("news_count_7d")) == 0:
         rows.append("Recent source-backed catalyst is missing.")
@@ -370,6 +383,7 @@ def invalidation_conditions(technical: dict, narrative: dict, risk_level: str) -
 
 def thesis_risks(signal: dict, technical: dict, narrative: dict, accuracy: dict, regime: str) -> list[str]:
     rows = []
+    fundamentals = narrative.get("fundamentals") or {}
     if signal.get("risk_level") == "High":
         rows.append("High signal risk can make the thesis unstable even when momentum is strong.")
     if regime in {"Risk-Off", "Panic", "Bull Exhaustion"}:
@@ -380,6 +394,8 @@ def thesis_risks(signal: dict, technical: dict, narrative: dict, accuracy: dict,
         rows.append("Data quality limits conviction.")
     if number(narrative.get("sentiment_polarization")) > 0.8:
         rows.append("Sentiment polarization is high; narrative interpretation may be unstable.")
+    if fundamentals.get("status") == "missing":
+        rows.append("Fundamental coverage is missing or incomplete.")
     return rows or ["No dominant single risk, but thesis quality depends on continued evidence alignment."]
 
 
@@ -422,10 +438,11 @@ def conviction_score(**kwargs) -> dict:
             number(breakdown.get("trend_score")) >= 60,
             number(breakdown.get("sentiment_score")) >= 58 or number(narrative.get("sentiment_7d")) > 0.12,
             number(breakdown.get("etf_confirmation_score")) >= 55,
+            number(breakdown.get("fundamental_score")) >= 62,
             number(historical.get("case_count")) >= 12,
         ]
     )
-    confirmation_score = independent_confirmations / 5 * 100
+    confirmation_score = independent_confirmations / 6 * 100
     contradiction_score = clamp(100 - max(0, len(contradicting) - 1) * 18)
     narrative_score = clamp(number(breakdown.get("semantic_trend_score"), number(narrative.get("semantic_trend_score"))))
     technical_score = mean_safe([number(breakdown.get("momentum_score")), number(breakdown.get("trend_score"))])
@@ -455,6 +472,7 @@ def conviction_score(**kwargs) -> dict:
             "technical_coherence": round(technical_score, 1),
             "historical_support": round(historical_score, 1),
             "market_context": round(context_score, 1),
+            "fundamental_support": round(number(breakdown.get("fundamental_score")), 1),
         },
         "reducers": reducers or ["No major conviction reducer was detected, but the thesis remains conditional."],
     }

backend/app/signals/engine.py

@@ -6,13 +6,13 @@ from sqlalchemy import delete, desc, select
 from sqlalchemy.orm import Session
 
 from app.ai.orchestrator import AIOrchestrator
-from app.models import Asset, NewsAssetLink, PriceHistory, SentimentAnalysis, SignalSnapshot, TechnicalIndicator
+from app.models import AccuracySnapshot, Asset, FundamentalSnapshot, NewsAssetLink, PriceHistory, SentimentAnalysis, SignalSnapshot, TechnicalIndicator
 from app.services.blum_financial_model import capture_signal_reasoning
 from app.services.thesis_engine import build_signal_thesis_payload
 from app.signals.indicators import compute_indicators
 
 
-SCORE_VERSION = "blum-thesis-score-v0.6"
+SCORE_VERSION = "blum-thesis-score-v0.8"
 
 
 class SignalEngine:
@@ -34,9 +34,12 @@ class SignalEngine:
             indicators = compute_indicators(frame, benchmark_frame)
             ts = self.ai.time_series.analyze(frame)
             narrative = self.narrative_features(db, asset)
-            score = build_score(indicators, narrative, ts, asset)
+            fundamentals = fundamental_features(db, asset)
+            accuracy = accuracy_features(db, asset)
+            narrative = {**narrative, "fundamentals": fundamentals, "accuracy_profile": accuracy}
+            score = build_score(indicators, narrative, ts, asset, fundamentals, accuracy)
             previous = latest_signal_for_asset(db, asset.id)
-            confidence = confidence_score(frame, indicators, narrative, ts)
+            confidence = confidence_score(frame, indicators, narrative, ts, fundamentals, accuracy)
             score["confidence_score"] = confidence
             lifecycle = lifecycle_state(previous, score, confidence)
             thesis = build_signal_thesis_payload(asset, score, indicators, narrative, ts)
@@ -127,7 +130,9 @@ def load_prices(db: Session, asset_id: int) -> pd.DataFrame:
     return pd.DataFrame(rows, columns=["date", "open", "high", "low", "close", "volume"])
 
 
-def build_score(indicators: dict, narrative: dict, ts: dict, asset: Asset) -> dict:
+def build_score(indicators: dict, narrative: dict, ts: dict, asset: Asset, fundamentals: dict | None = None, accuracy: dict | None = None) -> dict:
+    fundamentals = fundamentals or {}
+    accuracy = accuracy or {}
     momentum_score = avg(
         scale(indicators.get("perf_5d"), -8, 8),
         scale(indicators.get("perf_1m"), -14, 16),
@@ -159,16 +164,20 @@ def build_score(indicators: dict, narrative: dict, ts: dict, asset: Asset) -> di
     )
     semantic_trend_score = narrative.get("semantic_trend_score", 0)
     etf_confirmation_score = etf_confirmation_proxy(asset, indicators)
+    fundamental_score = float(fundamentals.get("fundamental_score", 45.0) or 45.0)
+    historical_accuracy_score = float(accuracy.get("accuracy_score", 50.0) or 50.0)
     risk_adjustment = avg(volatility_score, scale(indicators.get("beta_vs_benchmark"), 2.1, 0.55), scale(indicators.get("recent_drawdown"), -30, 0))
     blum_score = (
-        momentum_score * 0.18
-        + trend_score * 0.18
-        + sentiment_score * 0.16
-        + volatility_score * 0.12
-        + anomaly_score * 0.10
-        + semantic_trend_score * 0.12
-        + etf_confirmation_score * 0.08
-        + risk_adjustment * 0.06
+        momentum_score * 0.16
+        + trend_score * 0.16
+        + sentiment_score * 0.13
+        + volatility_score * 0.10
+        + anomaly_score * 0.09
+        + semantic_trend_score * 0.10
+        + etf_confirmation_score * 0.07
+        + fundamental_score * 0.10
+        + historical_accuracy_score * 0.06
+        + risk_adjustment * 0.03
     )
     classification = classify_signal(blum_score, indicators, narrative, anomaly_score)
     return {
@@ -184,6 +193,8 @@ def build_score(indicators: dict, narrative: dict, ts: dict, asset: Asset) -> di
             "anomaly_score": round(anomaly_score, 1),
             "semantic_trend_score": round(semantic_trend_score, 1),
             "etf_confirmation_score": round(etf_confirmation_score, 1),
+            "fundamental_score": round(fundamental_score, 1),
+            "historical_accuracy_score": round(historical_accuracy_score, 1),
             "risk_adjustment": round(risk_adjustment, 1),
         },
     }
@@ -250,7 +261,7 @@ def build_rule_explanation(asset: Asset, score: dict, indicators: dict, narrativ
     return (
         f"{asset.ticker} is classified as {score['classification']} with a Blum Intelligence Score of "
         f"{score['blum_score']}. The engine combines momentum, trend quality, sentiment, volatility, "
-        f"semantic intensity, ETF confirmation and anomaly pressure. Current 5D performance is "
+        f"semantic intensity, ETF confirmation, fundamentals, historical accuracy and anomaly pressure. Current 5D performance is "
         f"{indicators.get('perf_5d', 0):.2f}%, 1M performance is {indicators.get('perf_1m', 0):.2f}%, "
         f"7D sentiment is {narrative.get('sentiment_7d', 0):.2f}, and the time-series regime is "
         f"{ts.get('regime', 'unknown')}."
@@ -266,13 +277,17 @@ def latest_signal_for_asset(db: Session, asset_id: int) -> SignalSnapshot | None
     )
 
 
-def confidence_score(frame: pd.DataFrame, indicators: dict, narrative: dict, ts: dict) -> float:
+def confidence_score(frame: pd.DataFrame, indicators: dict, narrative: dict, ts: dict, fundamentals: dict | None = None, accuracy: dict | None = None) -> float:
+    fundamentals = fundamentals or {}
+    accuracy = accuracy or {}
     history_depth = scale(len(frame), 60, 900)
     indicator_completeness = sum(1 for key in ["sma20", "sma50", "sma200", "rsi", "macd_hist", "atr_percent", "support", "resistance"] if indicators.get(key) is not None) / 8 * 100
     news_support = scale(narrative.get("news_count_30d", 0), 0, 12)
     sentiment_quality = scale(abs(narrative.get("sentiment_30d", 0)), 0, 0.55)
     time_series_depth = 80 if ts.get("regime") not in {"unknown", "insufficient_history"} else 35
-    return round(avg(history_depth, indicator_completeness, news_support, sentiment_quality, time_series_depth), 1)
+    fundamental_quality = float(fundamentals.get("quality_score", 0.0) or 0.0)
+    historical_accuracy = float(accuracy.get("accuracy_score", 50.0) or 50.0)
+    return round(avg(history_depth, indicator_completeness, news_support, sentiment_quality, time_series_depth, fundamental_quality, historical_accuracy), 1)
 
 
 def lifecycle_state(previous: SignalSnapshot | None, score: dict, confidence: float) -> str:
@@ -298,6 +313,72 @@ def etf_confirmation_proxy(asset: Asset, indicators: dict) -> float:
     return base
 
 
+def fundamental_features(db: Session, asset: Asset) -> dict:
+    snapshot = db.scalar(
+        select(FundamentalSnapshot)
+        .where(FundamentalSnapshot.asset_id == asset.id)
+        .order_by(desc(FundamentalSnapshot.period_end), desc(FundamentalSnapshot.created_at))
+        .limit(1)
+    )
+    if snapshot is None:
+        return {"status": "missing", "quality_score": 0.0, "fundamental_score": 42.0, "issues": ["No SEC fundamental snapshot is stored."]}
+    metrics = snapshot.metrics or {}
+    revenue = metric_value(metrics, "revenue")
+    net_income = metric_value(metrics, "net_income")
+    assets = metric_value(metrics, "assets")
+    liabilities = metric_value(metrics, "liabilities")
+    operating_cash_flow = metric_value(metrics, "operating_cash_flow")
+    profit_margin = net_income / revenue if revenue else None
+    leverage = liabilities / assets if assets else None
+    cash_conversion = operating_cash_flow / net_income if operating_cash_flow is not None and net_income and net_income > 0 else None
+    score = avg(
+        float(snapshot.quality_score or 0.0),
+        scale(profit_margin, -0.18, 0.30) if profit_margin is not None else 45,
+        scale(1 - leverage, 0.05, 0.72) if leverage is not None else 45,
+        scale(cash_conversion, 0.25, 1.45) if cash_conversion is not None else 45,
+    )
+    return {
+        "status": "ready",
+        "provider": snapshot.provider,
+        "period_end": snapshot.period_end.isoformat() if snapshot.period_end else None,
+        "quality_score": float(snapshot.quality_score or 0.0),
+        "fundamental_score": round(score, 1),
+        "profit_margin": round(profit_margin, 4) if profit_margin is not None else None,
+        "liabilities_to_assets": round(leverage, 4) if leverage is not None else None,
+        "cash_conversion": round(cash_conversion, 4) if cash_conversion is not None else None,
+        "data_policy": "SEC companyfacts metrics only. Missing values are not estimated.",
+    }
+
+
+def accuracy_features(db: Session, asset: Asset) -> dict:
+    snapshot = db.scalar(
+        select(AccuracySnapshot)
+        .where(AccuracySnapshot.asset_id == asset.id, AccuracySnapshot.scope == "asset")
+        .order_by(desc(AccuracySnapshot.created_at))
+        .limit(1)
+    )
+    if snapshot is None:
+        return {"status": "missing", "accuracy_score": 50.0, "confidence_label": "Unknown"}
+    return {
+        "status": "ready",
+        "accuracy_score": float(snapshot.score or 50.0),
+        "confidence_label": snapshot.confidence_label,
+        "components": snapshot.components,
+        "issues": snapshot.issues,
+        "created_at": snapshot.created_at.isoformat() if snapshot.created_at else None,
+    }
+
+
+def metric_value(metrics: dict, key: str) -> float | None:
+    payload = metrics.get(key)
+    if not isinstance(payload, dict):
+        return None
+    try:
+        return float(payload.get("value"))
+    except Exception:
+        return None
+
+
 def scale(value, low: float, high: float) -> float:
     try:
         number = float(value)

frontend/lib/types.ts

@@ -138,6 +138,8 @@ export type SystemStatus = {
     model_loading_enabled: boolean;
     financial_brain_model_enabled: boolean;
     live_startup_enabled: boolean;
+    autonomous_engine_enabled?: boolean;
+    autonomous_cycle_minutes?: number;
     yfinance_fallback_enabled?: boolean;
     historical_price_seed_enabled?: boolean;
     startup_signal_seed_enabled?: boolean;
@@ -150,6 +152,8 @@ export type SystemStatus = {
     blum_model_cycle_limit?: number;
     fundamentals_refresh_minutes?: number;
     macro_refresh_minutes?: number;
+    hf_dataset_catalog_enabled?: boolean;
+    hf_dataset_refresh_hours?: number;
   };
   persistence?: {
     mode: string;

frontend/package.json

@@ -1,6 +1,6 @@
 {
   "name": "blum-ai-financial-intelligence-frontend",
-  "version": "0.7.1",
+  "version": "0.8.0",
   "private": true,
   "scripts": {
     "dev": "next dev -p 3000",

package.json

@@ -1,6 +1,6 @@
 {
   "name": "blum-ai-financial-intelligence",
-  "version": "0.7.1",
+  "version": "0.8.0",
   "private": true,
   "scripts": {
     "frontend:dev": "npm --prefix frontend run dev",