Spaces:

DataEyond
/

Agentic-Service-Data-Eyond

Sleeping

App Files Files Community

Merge dev_new to main (improve retriever and add querying)

#13

by rhbt6767 - opened 10 days ago

base: refs/heads/main

←

from: refs/pr/13

Discussion Files changed

[noticket] add gitignorec87f27f4

[NOTICKET]: add document pipeline, simplify document APIfb871f3d

[NOTICKET]: update folder document_pipelines after pipelinesa4cf97ab

[NOTICKET][DB] refactor code to new repo7f3bb978

[KM-441] add mean and median9b593342

[NOTICKET] new metadata format for cleaner code6b590d94

update document5a69e0ee

delete duplicate file3848d7b2

edit document for new pipeline425e0210

[NOTICKET]: add CSV and XLSX file type31920c3b

[DB] fix/rename db_pipeline.pyd913315c

[NOTICKET][DB] menyesuaikan format struktur db_pipeline sesuai dengan file laine13a9017

[NOTICKET][DB] pisahin db credential ke folder model. add ingestion endpoint at db_client to use db pipeline. add router db_client di main.347a73aa

[NOTICKET]: use tesseract for extract PDF6b9a13d4

[NOTICKET]: add Tesseract and Poppler binaries via Git LFS0a9101a1

[NOTICKET]: update uv.lockbb79f64b

[NOTICKET][DB] update credential & databaseclient. update settings0e079550

[NOTICKET] update settings65a5c6b1

[KM-437][DB] add mysql, sqlserver, bigquery, snowflake connections43539293

[NOTICKET]: adjusted pyproject.toml for OCR PDFa00e2ad5

[NOTICKET]: fix merge conflict6c873460

[NOTICKET][DB] fix mysql pipeline060c8cc8

[NOTICKET] edit importsb145c06e

[NOTICKET] minor code refactor52415b6a

[NOTICKET] add duplicate check for storing databased310770f

[NOTICKET][DB] add supported dbtype for frontenda531fcc7

[NOTICKET]: add doctypes endpoint & 10MB file size limit9debae56

[NOTICKET]: add comments to flag that file type lists must stay in sync023b7cfe

[NOTICKET]: add to gitignorebbc8c584

Merge branch 'main' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond into dev_new7757da18

Merge branch 'main' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond into dev_new9c090a04

Merge branch 'main' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond into dev_new5398fec4

Merge branch 'dev_new' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond into dev_new20bf3f8f

[NOTICKET] add total token loggingb9703fc5

[NOTICKET] add updated_at field for metadata & delete old embedding before appendingcb5ab327

[NO TICKET][document]: add updated_at on metadatad2f7a483

[NO TICKET][document]: delete vector embedding on table langchain_pg_embedding if user delete document on knowledgeac3d8c19

[NOTICKET][document]: make a clean output to status error unsupported file type2814813f

[KM-438][KM-439] framework for knowledge retrieverd1e12641

Merge branch 'main' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond into dev_newa701ac37

[NOTICKET] fix single source to multiple sources589ca324

[NOTICKET] fixed multiple sourcese9f2a263

[KM-507] add multiple retrieval method to compare (dense, mmr, bm25, hybrid)ac6b78d1

[KM-507] add changes to methods82186504

[NOTICKET] add db_client for queryinge49db601

add to gitignore83ed7447

[KM-507] add different methods, now using dense cosine145bca39

[KM-512] create folder for querying from bd/tabular docs2c8a3e89

[NOTICKET] minor fix in chat.py, add package for query, change schema used to hybrid (cosine+bm25)15cd3a7f

[KM-512] add Pydantic model the LLM fills via function calling in sql_query, and add same signature for db and tabular220f59eb

[NOTICKET] rename file name, updated after uv sync948d6dda

[NOTICKET] update .gitignore240251c4

Merge branch 'dev_new' of https://huggingface.co/spaces/DataEyond/Agentic-Service-Data-Eyond into dev_new29efec67

[KM-513][document] add convert to parquet if type file is XLSX and CSV770f26b1

add to gitignore1fef470b

[KM-512] connect query executor to user question. add logging for db_executorabc494f9

[NOTICKET] fix delete, now can filter by userf273db05

[NOTICKET] db_executor: CTE DML check now walks entire AST root, schema: cast instead of string interpolationbd2b1d9d

[NOTICKET] fix-revert string change110ee343

[KM-520] Integrate db query executor pipeline with existing rag retrieve pipelinea25febe2

[KM-516][KM-517] add new feature; ai can now see table & column names that have fk relationship with retrieved resultf86da27b

[NOTICKET] fix query now use orchestrator msg, rework db pipeline replace ingestion logicbe9bbd9d

DataEyond org 10 days ago

No description provided.

[NOTICKET] edit settings9e16c227

[KM 436-439] adjust endpoint for new features0707f2ba

[KM-436-437] edit knowledge handler pipeline767625e5

[KM-438-439] add retriever featureba550a5a

DataEyond org 10 days ago

Merge from dev_new to main (fix and improve knowledge handler pipeline, add new retriever for docs and tabular (and query logic for tabular)).
new commit starting from: [KM-507] add multiple retrieval method to compare (dense, mmr, bm25, hybrid)ac6b78d1

rhbt6767 changed pull request status to open 10 days ago

DataEyond org 10 days ago

Merge from dev_new to main (fix and improve knowledge handler pipeline, add new retriever for docs and tabular (and query logic for tabular)).
new commit starting from: [KM-507] add multiple retrieval method to compare (dense, mmr, bm25, hybrid)ac6b78d1

DataEyond org 10 days ago

nice

ishaq101 changed pull request status to merged 10 days ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment