Spaces:

---

Duplicated from  DataEyond/Agentic-Service-Data-Eyond

DataEyond

/

Agentic-Service-Data-Eyond-Catalog

Like 0

Running

pr/23: what changed

Documents

• PDF/DOCX/TXT ingestion moved to Python: /api/v2/documents/doctypes, /api/v2/documents/{user_id}, /api/v2/document/upload, /api/v2/document/process. Same shapes as Go's /api/v1 routes; only the prefix changes. Tabular files and delete (all types) stay on Go.
• Retrieval is limited to the v2 documents bound to each analysis. Documents processed by Go before this change aren't searched and need re-uploading.
• Answers can show a document's own figures. They arrive as asset://<id> markers; the FE swaps them for URLs from POST /api/v2/document/image-token on every render (URLs expire after 1 hour, so never store them).

Chat and planning

• Documents and data in one answer: the planner can combine a document lookup with data queries.
• Retrieve-then-plan: up to 3 relevant excerpts reach the planner and the answer writer, and are used for definitions, formulas and thresholds, never as data.
• The router separates a figure in a document from a chart made from data. "What do I have?" lists both data and documents.
• Replies follow the question's language, not the documents'.

Correctness

• Queries capped at 10,000 rows now return the same rows every time.
• A mistyped column in a step-to-step handoff is caught at plan time.
• Saved analysis records are trimmed; charts are kept.

Prompts and evals

• Planner prompt −17% per call; router prompt −9%, examples 32 → 21.
• New eval cases for scoped turns, DB joins and documents; the router's tool-group choice is scored; new faithfulness check; paid runs keep full plans and logs.
• Results: planner 39/40 (all 22 regression cases green), router 57/57 (0 wrong tool groups), tools 35/35.

Notes for reviewers

• The v2 document routes don't check auth yet (#43), which is why delete stays on Go.
• Python now writes documents, langchain_pg_embedding (collection documents, parser_version: v2) and bucket objects, for pdf/docx/txt only (signed off).
• The planner's Langfuse trace is masked when document excerpts are in its prompt. Traceability gains an additive retrieve_knowledge entry.
• No new dependencies, env vars or DDL.