Prism / docs /structure.md
benroshan's picture
feat: Stage 8 β€” eval dashboard, mandatory web search, docs update
65cee4e
|
Raw
History Blame Contribute Delete
3.71 kB

Project Structure β€” Prism

prism/
β”œβ”€β”€ CLAUDE.md
β”œβ”€β”€ README.md
β”œβ”€β”€ requirements.txt
β”œβ”€β”€ .env.example
β”œβ”€β”€ config.yaml                      # All tunable params (chunk sizes, weights, eval flags)
β”œβ”€β”€ Dockerfile
β”œβ”€β”€ render.yaml
β”œβ”€β”€ .gitignore
β”‚
β”œβ”€β”€ .claude/
β”‚   β”œβ”€β”€ settings.json
β”‚   β”œβ”€β”€ hooks/
β”‚   β”‚   └── session-reflect.sh
β”‚   └── rules/
β”‚       β”œβ”€β”€ memory-profile.md        # Facts about Ben
β”‚       β”œβ”€β”€ memory-preferences.md    # How Ben likes things done
β”‚       β”œβ”€β”€ memory-decisions.md      # Technical decisions log
β”‚       β”œβ”€β”€ memory-sessions.md       # Session log
β”‚       └── coding-standards.md     # Code style rules
β”‚
β”œβ”€β”€ docs/
β”‚   β”œβ”€β”€ architecture.md
β”‚   β”œβ”€β”€ decisions.md
β”‚   β”œβ”€β”€ api-spec.md
β”‚   └── structure.md                 # This file
β”‚
β”œβ”€β”€ server/
β”‚   β”œβ”€β”€ __init__.py
β”‚   β”œβ”€β”€ main.py                      # FastAPI app, lifespan startup
β”‚   β”œβ”€β”€ ingest.py                    # Load β†’ ParentDocumentRetriever β†’ embed β†’ store
β”‚   β”œβ”€β”€ retriever.py                 # Hybrid: dense + BM25 + RRF + reranker
β”‚   β”œβ”€β”€ bm25_index.py               # BM25 index singleton
β”‚   β”œβ”€β”€ reranker.py                  # Cross-encoder reranker singleton
β”‚   β”œβ”€β”€ memory.py                    # ConversationBufferWindowMemory
β”‚   β”œβ”€β”€ chain.py                     # ConversationalRetrievalChain
β”‚   β”œβ”€β”€ utils.py                     # Config loader, logger, token counter
β”‚   β”œβ”€β”€ routes/
β”‚   β”‚   β”œβ”€β”€ __init__.py
β”‚   β”‚   β”œβ”€β”€ chat.py                  # POST /api/chat, DELETE /api/chat/memory
β”‚   β”‚   β”œβ”€β”€ upload.py               # POST /api/upload
β”‚   β”‚   └── eval.py                  # GET /api/eval/session, POST /api/eval/precision|ragas
β”‚   └── eval/
β”‚       β”œβ”€β”€ __init__.py
β”‚       β”œβ”€β”€ precision.py             # Precision@K
β”‚       β”œβ”€β”€ faithfulness.py          # LLM-as-Judge (1–5 score)
β”‚       └── ragas_eval.py           # RAGAS (4 metrics)
β”‚
β”œβ”€β”€ frontend/
β”‚   β”œβ”€β”€ package.json
β”‚   β”œβ”€β”€ vite.config.js
β”‚   β”œβ”€β”€ tailwind.config.js
β”‚   β”œβ”€β”€ index.html
β”‚   └── src/
β”‚       β”œβ”€β”€ main.jsx
β”‚       β”œβ”€β”€ App.jsx                  # Tab nav: Chat | Eval | Upload
β”‚       β”œβ”€β”€ api.js                   # Axios client
β”‚       └── components/
β”‚           β”œβ”€β”€ ChatTab.jsx
β”‚           β”œβ”€β”€ EvalDashboard.jsx    # RAGAS scorecard + Precision@K + LangSmith link
β”‚           β”œβ”€β”€ UploadTab.jsx        # Drag-and-drop upload
β”‚           β”œβ”€β”€ MessageBubble.jsx
β”‚           └── SourceExpander.jsx
β”‚
β”œβ”€β”€ scripts/
β”‚   β”œβ”€β”€ run_ingest.py               # CLI ingestion
β”‚   β”œβ”€β”€ run_eval.py                  # CLI Precision@K eval
β”‚   β”œβ”€β”€ run_ragas_eval.py           # CLI RAGAS eval
β”‚   └── benchmark_chunks.py         # Sweep chunk sizes, plot Precision@K
β”‚
β”œβ”€β”€ tests/
β”‚   β”œβ”€β”€ test_ingest.py
β”‚   β”œβ”€β”€ test_retriever.py
β”‚   β”œβ”€β”€ test_reranker.py
β”‚   β”œβ”€β”€ test_chain.py
β”‚   β”œβ”€β”€ test_eval.py
β”‚   └── test_ragas.py
β”‚
β”œβ”€β”€ data/
β”‚   β”œβ”€β”€ raw/                         # Drop PDFs here (gitignored)
β”‚   └── ground_truth/
β”‚       └── eval_pairs.json          # 20 query/chunk pairs + ground_truth field
β”‚
β”œβ”€β”€ sample_data/                     # Seeded sample PDFs for demo
└── chroma_db/                       # Auto-created, gitignored