dhammawatthumpra commited on
Commit
5fd7c4e
Β·
1 Parent(s): 0ea2446

docs: update README.md to reflect V2 architecture (Qdrant, Hybrid Search, new features)

Browse files
Files changed (1) hide show
  1. README.md +202 -9
README.md CHANGED
@@ -9,15 +9,208 @@ pinned: false
9
 
10
  # πŸ“– ΰΈžΰΈ£ΰΈ°ΰΉ„ΰΈ•ΰΈ£ΰΈ›ΰΈ΄ΰΈŽΰΈ ΰΈ‰ΰΈšΰΈ±ΰΈšΰΈ‘ΰΈ«ΰΈ²ΰΈˆΰΈΈΰΈ¬ΰΈ²ΰΈ₯ΰΈ‡ΰΈΰΈ£ΰΈ“ΰΈ£ΰΈ²ΰΈŠΰΈ§ΰΈ΄ΰΈ—ΰΈ’ΰΈ²ΰΈ₯ΰΈ±ΰΈ’
11
 
12
- Tipitaka web application β€” React + FastAPI
13
 
14
- - Frontend: React + TypeScript + Tailwind CSS (Vite)
15
- - Backend: FastAPI (Python)
16
- - Database: SQLite (loaded in-memory)
17
- - Search: ChromaDB with Qwen3-Embedding-0.6B
18
- - AI: DeepSeek API (deepseek-chat / deepseek-reasoner)
19
 
20
- ## Deployment
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
21
 
22
- - Docker multi-stage build
23
- - DB + vector index downloaded from HF bucket at runtime
 
9
 
10
  # πŸ“– ΰΈžΰΈ£ΰΈ°ΰΉ„ΰΈ•ΰΈ£ΰΈ›ΰΈ΄ΰΈŽΰΈ ΰΈ‰ΰΈšΰΈ±ΰΈšΰΈ‘ΰΈ«ΰΈ²ΰΈˆΰΈΈΰΈ¬ΰΈ²ΰΈ₯ΰΈ‡ΰΈΰΈ£ΰΈ“ΰΈ£ΰΈ²ΰΈŠΰΈ§ΰΈ΄ΰΈ—ΰΈ’ΰΈ²ΰΈ₯ΰΈ±ΰΈ’
11
 
12
+ **Tipitaka Web App** β€” ΰΉ€ΰΈ§ΰΉ‡ΰΈšΰΈ­ΰΉˆΰΈ²ΰΈ™ΰΈžΰΈ£ΰΈ°ΰΉ„ΰΈ•ΰΈ£ΰΈ›ΰΈ΄ΰΈŽΰΈ ΰΈ‰ΰΈšΰΈ±ΰΈš ฑจร. ΰΉΰΈšΰΈšΰΈˆΰΈ΄ΰΈ•ΰΈ§ΰΈ΄ΰΉ€ΰΈ§ΰΈ ΰΈžΰΈ£ΰΉ‰ΰΈ­ΰΈ‘ AI ΰΈœΰΈΉΰΉ‰ΰΈŠΰΉˆΰΈ§ΰΈ’ΰΈ­ΰΈ±ΰΈˆΰΈ‰ΰΈ£ΰΈ΄ΰΈ’ΰΈ°
13
 
14
+ > **Stack:** React + Vite | FastAPI + Python | SQLite FTS5 | Qdrant Vector | ONNX Reranker | DeepSeek API
 
 
 
 
15
 
16
+ ---
17
+
18
+ ## ✨ Features
19
+
20
+ | Feature | Description |
21
+ |---------|-------------|
22
+ | **πŸ“– Reader** | 45+ ΰΉ€ΰΈ₯ΰΉˆΰΈ‘ΰΈžΰΈ£ΰΈ°ΰΉ„ΰΈ•ΰΈ£ΰΈ›ΰΈ΄ΰΈŽΰΈ β€” ΰΈ«ΰΈ™ΰΉ‰ΰΈ²-ΰΈ•ΰΉˆΰΈ­-ΰΈ«ΰΈ™ΰΉ‰ΰΈ², ΰΈ›ΰΈ£ΰΈ±ΰΈšΰΈ‚ΰΈ™ΰΈ²ΰΈ”ΰΈ•ΰΈ±ΰΈ§ΰΈ­ΰΈ±ΰΈΰΈ©ΰΈ£, 3 themes (dark/light/classic) |
23
+ | **πŸ” Hybrid Search** | FTS5 + Vector (Qdrant) + ONNX Reranker β€” ถูกต้องทั้ง keyword แΰΈ₯ΰΈ°ΰΈ„ΰΈ§ΰΈ²ΰΈ‘ΰΈ«ΰΈ‘ΰΈ²ΰΈ’ |
24
+ | **πŸ€– AI Assistant** | Floating chat ΰΈžΰΈ£ΰΉ‰ΰΈ­ΰΈ‘ RAG context, ΰΈ£ΰΈ­ΰΈ‡ΰΈ£ΰΈ±ΰΈš DeepSeek-chat / DeepSeek-reasoner |
25
+ | **πŸ”— Cross-Volume RAG** | ΰΈ„ΰΉ‰ΰΈ™ΰΈ‚ΰΉ‰ΰΈ²ΰΈ‘ΰΉ€ΰΈ₯่ฑจาก Qdrant vector store + jina-embeddings-v3, rerank ΰΈ”ΰΉ‰ΰΈ§ΰΈ’ jina-reranker-v2 |
26
+ | **πŸ“± Responsive** | Desktop + Mobile, swipe gesture, keyboard nav |
27
+ | **🎨 Markdown Rendering** | AI ΰΈ•ΰΈ­ΰΈšΰΉ€ΰΈ›ΰΉ‡ΰΈ™ markdown ΰΈžΰΈ£ΰΉ‰ΰΈ­ΰΈ‘ prose-invert ΰΈͺำหรับ dark mode |
28
+ | **↔️ Resizable Panel** | ΰΈ₯ΰΈ²ΰΈΰΈ‚ΰΈ­ΰΈšΰΈ‚ΰΈ§ΰΈ²ΰΉ€ΰΈžΰΈ·ΰΉˆΰΈ­ΰΈ›ΰΈ£ΰΈ±ΰΈšΰΈ‚ΰΈ™ΰΈ²ΰΈ” AI chat β€” persist ΰΈ‚ΰΈ™ΰΈ²ΰΈ”ΰΉ„ΰΈ§ΰΉ‰ |
29
+ | **🏷️ Pali Autocorrect** | ΰΉΰΈΰΉ‰ΰΈ„ΰΈ³ΰΈšΰΈ²ΰΈ₯ΰΈ΅ΰΈœΰΈ΄ΰΈ”ΰΉ€ΰΈ­ΰΈ‡ ΰΈΰΉˆΰΈ­ΰΈ™ΰΈͺΰΉˆΰΈ‡ΰΈ„ΰΉ‰ΰΈ™ΰΈ«ΰΈ² β€” PyThaiNLP + custom dict |
30
+
31
+ ---
32
+
33
+ ## πŸ—οΈ Architecture (V2)
34
+
35
+ ```
36
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
37
+ β”‚ Frontend (Vite + React) β”‚
38
+ β”‚ AppShell β†’ NavDrawer | ReaderPanel | RightToolbar | AIPopup β”‚
39
+ β”‚ Zustand (5 stores) Β· Framer Motion Β· Tailwind CSS v4 β”‚
40
+ β”‚ react-markdown + remark-gfm Β· Lucide React β”‚
41
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
42
+ β”‚ HTTP / SSE
43
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
44
+ β”‚ Backend (FastAPI Python 3.11) β”‚
45
+ β”‚ β”‚
46
+ β”‚ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β” β”‚
47
+ β”‚ β”‚ Page β”‚ β”‚ Search β”‚ β”‚ RAG β”‚ β”‚ LLM β”‚ β”‚
48
+ β”‚ β”‚ Service β”‚ β”‚ Service β”‚ β”‚ Service β”‚ β”‚ Service β”‚ β”‚
49
+ β”‚ β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”˜ β”‚
50
+ β”‚ β”‚ β”‚ β”‚ β”‚ β”‚
51
+ β”‚ β”Œβ”€β”€β”€β”€β–Όβ”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β–Όβ”€β”€β”€β”€β” β”Œβ”€β”€β”€β”€β–Όβ”€β”€β”€β”€β”€β” β”‚
52
+ β”‚ β”‚ SQLite β”‚ β”‚ SQLite β”‚ β”‚ Qdrant β”‚ β”‚ DeepSeek β”‚ β”‚
53
+ β”‚ β”‚ FTS5 β”‚ β”‚ LIKE β”‚ β”‚ Vector β”‚ β”‚ API β”‚ β”‚
54
+ β”‚ β”‚ (memory)β”‚ β”‚ fallback β”‚ β”‚ + ONNX β”‚ β”‚ Streamingβ”‚ β”‚
55
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚ Rerankerβ”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
56
+ β”‚ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜ β”‚
57
+ β”‚ Services: page_service Β· search_service Β· pali_utils β”‚
58
+ β”‚ rag_service Β· onnx_reranker Β· llm_service β”‚
59
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
60
+ ```
61
+
62
+ ### πŸ”„ Hybrid Search Pipeline
63
+
64
+ ```
65
+ ΰΈ„ΰΈ³ΰΈ–ΰΈ²ΰΈ‘ β†’ Pali Autocorrect β†’ FTS5 (SQLite, 30 results)
66
+ ─→ Qdrant Vector (jina-embeddings-v3, 30 results)
67
+ β†’ Merge & Deduplicate
68
+ β†’ ONNX Rerank (jina-reranker-v2)
69
+ β†’ Top N β†’ AI context
70
+ ```
71
+
72
+ ---
73
+
74
+ ## πŸ—‚οΈ Project Structure
75
+
76
+ ```
77
+ RAG/
78
+ β”œβ”€β”€ webapp/
79
+ β”‚ β”œβ”€β”€ tipitaka-api/ # FastAPI backend
80
+ β”‚ β”‚ β”œβ”€β”€ app/
81
+ β”‚ β”‚ β”‚ β”œβ”€β”€ main.py # FastAPI entry + lifespan
82
+ β”‚ β”‚ β”‚ β”œβ”€β”€ config.py # pydantic-settings
83
+ β”‚ β”‚ β”‚ β”œβ”€β”€ schemas.py # Pydantic models
84
+ β”‚ β”‚ β”‚ β”œβ”€β”€ routers/ # pages, search, ai, health
85
+ β”‚ β”‚ β”‚ β”œβ”€β”€ services/ # 6 service modules
86
+ β”‚ β”‚ β”‚ └── database/ # sqlite_db.py
87
+ β”‚ β”‚ β”œβ”€β”€ tests/ # 35 tests
88
+ β”‚ β”‚ β”œβ”€β”€ models/ # ONNX reranker
89
+ β”‚ β”‚ └── startup.sh # HF Spaces entry
90
+ β”‚ β”‚
91
+ β”‚ └── tipitaka-web/ # React frontend
92
+ β”‚ └── src/
93
+ β”‚ β”œβ”€β”€ components/ # layout, reader, toolbar, ai
94
+ β”‚ β”œβ”€β”€ stores/ # Zustand (5 stores)
95
+ β”‚ β”œβ”€β”€ hooks/ # swipe, keyboard
96
+ β”‚ └── lib/ # api.ts
97
+ β”‚
98
+ β”œβ”€β”€ qdrant_storage/ # Local Qdrant data
99
+ β”œβ”€β”€ snapshots/ # Qdrant snapshot files
100
+ β”œβ”€β”€ Dockerfile # Multi-stage build
101
+ └── TIPITAKA_WEB_ARCHITECTURE_V2.md # Full architecture doc
102
+ ```
103
+
104
+ ---
105
+
106
+ ## πŸš€ Quick Start
107
+
108
+ ### Prerequisites
109
+ - Python 3.11+
110
+ - Node.js 20+
111
+ - Ollama (for embedding β€” **jina-embeddings-v3**)
112
+ - Qdrant (optional β€” auto-detect localhost:6333)
113
+ - DeepSeek API key
114
+
115
+ ### 1. Setup Backend
116
+ ```bash
117
+ cd webapp/tipitaka-api
118
+ python -m venv .venv
119
+ .venv\Scripts\activate # Windows
120
+ pip install -r requirements.txt
121
+
122
+ # Copy .env.example β†’ .env and fill DEEPSEEK_API_KEY
123
+ python -m app.main # starts on :8000
124
+ ```
125
+
126
+ ### 2. Setup Frontend
127
+ ```bash
128
+ cd webapp/tipitaka-web
129
+ npm install
130
+ npm run dev # starts on :5173
131
+ ```
132
+
133
+ ### 3. Setup Embedding
134
+ ```bash
135
+ # Download jina-embeddings-v3 GGUF
136
+ ollama pull hf.co/second-state/jina-embeddings-v3-GGUF:jina-embeddings-v3-Q4_K_M.gguf
137
+ ```
138
+
139
+ ### 4. Run Tests
140
+ ```bash
141
+ cd webapp/tipitaka-api
142
+ python -m pytest tests/ -v # 35 tests, all passing
143
+ ```
144
+
145
+ ---
146
+
147
+ ## πŸ“‘ API Endpoints
148
+
149
+ | Method | Path | Description |
150
+ |--------|------|-------------|
151
+ | `GET` | `/api/pages/volumes` | ราฒการเΰΈ₯ΰΉˆΰΈ‘ΰΈžΰΈ£ΰΈ°ΰΉ„ΰΈ•ΰΈ£ΰΈ›ΰΈ΄ΰΈŽΰΈ |
152
+ | `GET` | `/api/pages/volumes/{id}/toc` | ΰΈͺารบัญ (dedup + noise filter) |
153
+ | `GET` | `/api/pages/{vol}/{page}` | ΰΉ€ΰΈ™ΰΈ·ΰΉ‰ΰΈ­ΰΈ«ΰΈ²ΰΈ«ΰΈ™ΰΉ‰ΰΈ² |
154
+ | `GET` | `/api/search?q=&limit=&offset=` | ΰΈ„ΰΉ‰ΰΈ™ΰΈ«ΰΈ² FTS5 + LIKE + highlight |
155
+ | `GET` | `/api/search/suggestions?q=` | Autocomplete |
156
+ | `POST` | `/api/ask/stream` | AI ΰΈ–ΰΈ²ΰΈ‘-ΰΈ•ΰΈ­ΰΈš (SSE streaming) |
157
+ | `POST` | `/api/ask` | AI ΰΈ–ΰΈ²ΰΈ‘-ΰΈ•ΰΈ­ΰΈš (non-stream β€” Discord) |
158
+ | `GET` | `/api/ask/rag-status` | ΰΈͺΰΈ–ΰΈ²ΰΈ™ΰΈ° Qdrant |
159
+ | `GET` | `/health` | Health check |
160
+
161
+ ---
162
+
163
+ ## πŸ§ͺ Test Suite
164
+
165
+ ```
166
+ tests/
167
+ β”œβ”€β”€ test_pali_utils.py # 90 assertions β€” autocorrect, digits, similarity
168
+ β”œβ”€β”€ test_search_service.py # FTS5 + LIKE merge, suggestions
169
+ └── test_rag_service.py # Embedding LRU cache
170
+
171
+ Total: 35 tests βœ…
172
+ ```
173
+
174
+ ---
175
+
176
+ ## 🧰 Tech Stack Detail
177
+
178
+ | Layer | Technology |
179
+ |-------|-----------|
180
+ | **Frontend** | React 18, TypeScript 5, Vite 6, Tailwind CSS v4 |
181
+ | **State** | Zustand (5 stores) |
182
+ | **Animation** | Framer Motion |
183
+ | **Icons** | Lucide React |
184
+ | **Markdown** | react-markdown + remark-gfm |
185
+ | **Backend** | FastAPI, Python 3.11 |
186
+ | **Database** | SQLite (FTS5, loaded to `:memory:` at startup) |
187
+ | **Vector Store** | Qdrant (local/server dual-mode, snapshot restore) |
188
+ | **Embedding** | jina-embeddings-v3 (Q4_K_M GGUF via Ollama, 1024 dims) |
189
+ | **Reranker** | jina-reranker-v2-base-multilingual (ONNX, CPU) |
190
+ | **AI** | DeepSeek API (OpenAI-compatible, SSE streaming) |
191
+ | **CI/CD** | HF Spaces (Docker multi-stage, auto-build) |
192
+
193
+ ---
194
+
195
+ ## πŸ“š Documents
196
+
197
+ | File | Description |
198
+ |------|-------------|
199
+ | `DESIGN.md` | Design system β€” colors, typography, components, themes |
200
+ | `PRODUCT.md` | Product vision, users, principles |
201
+ | `TIPITAKA_WEB_ARCHITECTURE_V2.md` | Full architecture V2 document |
202
+ | `TIPITAKA_WEB_ARCHITECTURE.md` | Original V1 architecture (legacy reference) |
203
+
204
+ ---
205
+
206
+ ## πŸ™ Acknowledgements
207
+
208
+ - **ฑหาจุฬาΰΈ₯ΰΈ‡ΰΈΰΈ£ΰΈ“ΰΈ£ΰΈ²ΰΈŠΰΈ§ΰΈ΄ΰΈ—ΰΈ’ΰΈ²ΰΈ₯ΰΈ±ΰΈ’ (MCU)** β€” ΰΉ€ΰΈ™ΰΈ·ΰΉ‰ΰΈ­ΰΈ«ΰΈ²ΰΈžΰΈ£ΰΈ°ΰΉ„ΰΈ•ΰΈ£ΰΈ›ΰΈ΄ΰΈŽΰΈ
209
+ - **Qdrant** β€” Vector database
210
+ - **jina.ai** β€” Embedding + Reranker models
211
+ - **Ollama** β€” Local model serving
212
+ - **Hugging Face Spaces** β€” Deployment platform
213
+
214
+ ---
215
 
216
+ *"อุปฑาเหฑือนแΰΈͺΰΈ‡ΰΉ€ΰΈ—ΰΈ΅ΰΈ’ΰΈ™ΰΉƒΰΈ™ΰΈ«ΰΉ‰ΰΈ­ΰΈ‡ΰΈ‘ΰΈ·ΰΈ” คือปัญญาทำΰΈ₯าฒอวิชชา"*