Spaces:
Sleeping
Sleeping
Introduce Project ID for context management, finalizing the stateful API pipeline from file submission to graph construction.
Browse files- .env.example +8 -0
- .gitignore +9 -1
- backend/README.md +243 -0
- backend/app/__init__.py +52 -0
- backend/app/api/__init__.py +10 -0
- backend/app/api/graph.py +657 -0
- backend/app/config.py +53 -0
- backend/app/models/__init__.py +9 -0
- backend/app/models/project.py +305 -0
- backend/app/models/task.py +178 -0
- backend/app/services/__init__.py +10 -0
- backend/app/services/graph_builder.py +457 -0
- backend/app/services/ontology_generator.py +361 -0
- backend/app/services/text_processor.py +71 -0
- backend/app/utils/__init__.py +9 -0
- backend/app/utils/file_parser.py +141 -0
- backend/app/utils/llm_client.py +91 -0
- backend/app/utils/logger.py +107 -0
- backend/requirements.txt +22 -0
- backend/run.py +57 -0
.env.example
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Zep Cloud API Key
|
| 2 |
+
# ไป https://app.getzep.com ่ทๅ
|
| 3 |
+
ZEP_API_KEY=
|
| 4 |
+
|
| 5 |
+
# LLM API Key
|
| 6 |
+
LLM_API_KEY=
|
| 7 |
+
LLM_BASE_URL=
|
| 8 |
+
LLM_MODEL_NAME=
|
.gitignore
CHANGED
|
@@ -11,6 +11,7 @@
|
|
| 11 |
.env.development.local
|
| 12 |
.env.test.local
|
| 13 |
.env.production.local
|
|
|
|
| 14 |
|
| 15 |
.vscode
|
| 16 |
.idea
|
|
@@ -19,4 +20,11 @@
|
|
| 19 |
.cursor/
|
| 20 |
|
| 21 |
mydoc/
|
| 22 |
-
mytest/
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 11 |
.env.development.local
|
| 12 |
.env.test.local
|
| 13 |
.env.production.local
|
| 14 |
+
__pycache__/
|
| 15 |
|
| 16 |
.vscode
|
| 17 |
.idea
|
|
|
|
| 20 |
.cursor/
|
| 21 |
|
| 22 |
mydoc/
|
| 23 |
+
mytest/
|
| 24 |
+
|
| 25 |
+
# ๆฅๅฟๆไปถ
|
| 26 |
+
backend/logs/
|
| 27 |
+
*.log
|
| 28 |
+
|
| 29 |
+
# ไธไผ ๆไปถ
|
| 30 |
+
backend/uploads/
|
backend/README.md
ADDED
|
@@ -0,0 +1,243 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# MiroFish Backend
|
| 2 |
+
|
| 3 |
+
็คพไผ่่ฎบๆจกๆ็ณป็ปๅ็ซฏๆๅก๏ผๅบไบFlaskๆกๆถใ
|
| 4 |
+
|
| 5 |
+
## ้กน็ฎ็ปๆ
|
| 6 |
+
|
| 7 |
+
```
|
| 8 |
+
backend/
|
| 9 |
+
โโโ app/
|
| 10 |
+
โ โโโ __init__.py # Flaskๅบ็จๅทฅๅ
|
| 11 |
+
โ โโโ config.py # ้
็ฝฎ็ฎก็
|
| 12 |
+
โ โโโ api/ # API่ทฏ็ฑ
|
| 13 |
+
โ โ โโโ __init__.py
|
| 14 |
+
โ โ โโโ graph.py # ๅพ่ฐฑ็ธๅ
ณๆฅๅฃ
|
| 15 |
+
โ โโโ services/ # ไธๅก้ป่พๅฑ
|
| 16 |
+
โ โ โโโ ontology_generator.py # ๆฌไฝ็ๆๆๅก
|
| 17 |
+
โ โ โโโ graph_builder.py # ๅพ่ฐฑๆๅปบๆๅก
|
| 18 |
+
โ โ โโโ text_processor.py # ๆๆฌๅค็ๆๅก
|
| 19 |
+
โ โโโ models/ # ๆฐๆฎๆจกๅ
|
| 20 |
+
โ โ โโโ task.py # ไปปๅก็ถๆ็ฎก็
|
| 21 |
+
โ โ โโโ project.py # ้กน็ฎไธไธๆ็ฎก็
|
| 22 |
+
โ โโโ utils/ # ๅทฅๅ
ทๆจกๅ
|
| 23 |
+
โ โโโ file_parser.py # ๆไปถ่งฃๆ
|
| 24 |
+
โ โโโ llm_client.py # LLMๅฎขๆท็ซฏ
|
| 25 |
+
โโโ requirements.txt
|
| 26 |
+
โโโ run.py # ๅฏๅจๅ
ฅๅฃ
|
| 27 |
+
```
|
| 28 |
+
|
| 29 |
+
## ๅฎ่ฃ
|
| 30 |
+
|
| 31 |
+
```bash
|
| 32 |
+
conda activate MiroFish
|
| 33 |
+
cd backend
|
| 34 |
+
pip install -r requirements.txt
|
| 35 |
+
```
|
| 36 |
+
|
| 37 |
+
## ้
็ฝฎ
|
| 38 |
+
|
| 39 |
+
ๅจ้กน็ฎๆ น็ฎๅฝ `MiroFish/.env` ไธญ้
็ฝฎ๏ผ
|
| 40 |
+
|
| 41 |
+
```bash
|
| 42 |
+
# LLM้
็ฝฎ
|
| 43 |
+
LLM_API_KEY=your-llm-api-key
|
| 44 |
+
LLM_BASE_URL=https://openrouter.ai/api/v1
|
| 45 |
+
LLM_MODEL_NAME=gpt-4o-mini
|
| 46 |
+
|
| 47 |
+
# Zep้
็ฝฎ
|
| 48 |
+
ZEP_API_KEY=your-zep-api-key
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
## ๅฏๅจๆๅก
|
| 52 |
+
|
| 53 |
+
```bash
|
| 54 |
+
python run.py
|
| 55 |
+
```
|
| 56 |
+
|
| 57 |
+
ๆๅก้ป่ฎค่ฟ่กๅจ http://localhost:5000
|
| 58 |
+
|
| 59 |
+
---
|
| 60 |
+
|
| 61 |
+
## APIๆฅๅฃ
|
| 62 |
+
|
| 63 |
+
### ๆ ธๅฟๅทฅไฝๆต็จ
|
| 64 |
+
|
| 65 |
+
```
|
| 66 |
+
1. ๅๅปบ้กน็ฎ๏ผๅฏ้๏ผ
|
| 67 |
+
POST /api/graph/project/create
|
| 68 |
+
|
| 69 |
+
2. ไธไผ ๆไปถ + ็ๆๆฌไฝ๏ผๆฅๅฃ1๏ผ
|
| 70 |
+
POST /api/graph/ontology/generate
|
| 71 |
+
โ ่ฟๅ project_id
|
| 72 |
+
|
| 73 |
+
3. ๆๅปบๅพ่ฐฑ๏ผๆฅๅฃ2๏ผ
|
| 74 |
+
POST /api/graph/build
|
| 75 |
+
โ ไผ ๅ
ฅ project_id
|
| 76 |
+
โ ่ฟๅ task_id
|
| 77 |
+
|
| 78 |
+
4. ๆฅ่ฏขไปปๅก่ฟๅบฆ
|
| 79 |
+
GET /api/graph/task/{task_id}
|
| 80 |
+
|
| 81 |
+
5. ่ทๅๅพ่ฐฑๆฐๆฎ
|
| 82 |
+
GET /api/graph/data/{graph_id}
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
---
|
| 86 |
+
|
| 87 |
+
### ๆฅๅฃ1๏ผ็ๆๆฌไฝๅฎไน
|
| 88 |
+
|
| 89 |
+
**POST** `/api/graph/ontology/generate`
|
| 90 |
+
|
| 91 |
+
ไธไผ ๆๆกฃ๏ผๅๆ็ๆ้ๅ็คพไผๆจกๆ็ๅฎไฝๅๅ
ณ็ณป็ฑปๅๅฎไนใ
|
| 92 |
+
|
| 93 |
+
**่ฏทๆฑ๏ผform-data๏ผ๏ผ**
|
| 94 |
+
|
| 95 |
+
| ๅญๆฎต | ็ฑปๅ | ๅฟ
ๅกซ | ่ฏดๆ |
|
| 96 |
+
|------|------|------|------|
|
| 97 |
+
| `files` | File | โ
| PDF/MD/TXTๆไปถ๏ผๅฏๅคไธช |
|
| 98 |
+
| `simulation_requirement` | Text | โ
| ๆจกๆ้ๆฑๆ่ฟฐ |
|
| 99 |
+
| `project_name` | Text | โ | ้กน็ฎๅ็งฐ |
|
| 100 |
+
| `additional_context` | Text | โ | ้ขๅค่ฏดๆ |
|
| 101 |
+
|
| 102 |
+
**ๅๅบ็คบไพ๏ผ**
|
| 103 |
+
```json
|
| 104 |
+
{
|
| 105 |
+
"success": true,
|
| 106 |
+
"data": {
|
| 107 |
+
"project_id": "proj_abc123def456",
|
| 108 |
+
"project_name": "ๆญฆๆฑๅคงๅญฆ่ๆ
ๅๆ",
|
| 109 |
+
"ontology": {
|
| 110 |
+
"entity_types": [
|
| 111 |
+
{
|
| 112 |
+
"name": "Person",
|
| 113 |
+
"description": "Individuals who can express opinions",
|
| 114 |
+
"attributes": [...]
|
| 115 |
+
}
|
| 116 |
+
],
|
| 117 |
+
"edge_types": [
|
| 118 |
+
{
|
| 119 |
+
"name": "AFFILIATED_WITH",
|
| 120 |
+
"description": "Indicates affiliation",
|
| 121 |
+
"source_targets": [...]
|
| 122 |
+
}
|
| 123 |
+
]
|
| 124 |
+
},
|
| 125 |
+
"analysis_summary": "ๅๆ่ฏดๆ...",
|
| 126 |
+
"files": [
|
| 127 |
+
{"filename": "ๆฅๅ.pdf", "size": 123456}
|
| 128 |
+
],
|
| 129 |
+
"total_text_length": 20833
|
| 130 |
+
}
|
| 131 |
+
}
|
| 132 |
+
```
|
| 133 |
+
|
| 134 |
+
---
|
| 135 |
+
|
| 136 |
+
### ๆฅๅฃ2๏ผๆๅปบๅพ่ฐฑ
|
| 137 |
+
|
| 138 |
+
**POST** `/api/graph/build`
|
| 139 |
+
|
| 140 |
+
ๆ นๆฎ `project_id` ๆๅปบZep็ฅ่ฏๅพ่ฐฑ๏ผๅผๆญฅไปปๅก๏ผใ
|
| 141 |
+
|
| 142 |
+
**่ฏทๆฑ๏ผJSON๏ผ๏ผ**
|
| 143 |
+
```json
|
| 144 |
+
{
|
| 145 |
+
"project_id": "proj_abc123def456",
|
| 146 |
+
"graph_name": "ๅพ่ฐฑๅ็งฐ",
|
| 147 |
+
"chunk_size": 500,
|
| 148 |
+
"chunk_overlap": 50
|
| 149 |
+
}
|
| 150 |
+
```
|
| 151 |
+
|
| 152 |
+
| ๅญๆฎต | ็ฑปๅ | ๅฟ
ๅกซ | ่ฏดๆ |
|
| 153 |
+
|------|------|------|------|
|
| 154 |
+
| `project_id` | string | โ
| ๆฅ่ชๆฅๅฃ1็่ฟๅ |
|
| 155 |
+
| `graph_name` | string | โ | ๅพ่ฐฑๅ็งฐ |
|
| 156 |
+
| `chunk_size` | int | โ | ๆๆฌๅๅคงๅฐ๏ผ้ป่ฎค500 |
|
| 157 |
+
| `chunk_overlap` | int | โ | ๅ้ๅ ๅญ็ฌฆ๏ผ้ป่ฎค50 |
|
| 158 |
+
|
| 159 |
+
**ๅๅบ๏ผ**
|
| 160 |
+
```json
|
| 161 |
+
{
|
| 162 |
+
"success": true,
|
| 163 |
+
"data": {
|
| 164 |
+
"project_id": "proj_abc123def456",
|
| 165 |
+
"task_id": "task_xyz789",
|
| 166 |
+
"message": "ๅพ่ฐฑๆๅปบไปปๅกๅทฒๅฏๅจ"
|
| 167 |
+
}
|
| 168 |
+
}
|
| 169 |
+
```
|
| 170 |
+
|
| 171 |
+
---
|
| 172 |
+
|
| 173 |
+
### ไปปๅก็ถๆๆฅ่ฏข
|
| 174 |
+
|
| 175 |
+
**GET** `/api/graph/task/{task_id}`
|
| 176 |
+
|
| 177 |
+
```json
|
| 178 |
+
{
|
| 179 |
+
"success": true,
|
| 180 |
+
"data": {
|
| 181 |
+
"task_id": "task_xyz789",
|
| 182 |
+
"status": "processing",
|
| 183 |
+
"progress": 45,
|
| 184 |
+
"message": "ๆทปๅ ๆๆฌๅ (15/30)...",
|
| 185 |
+
"result": null
|
| 186 |
+
}
|
| 187 |
+
}
|
| 188 |
+
```
|
| 189 |
+
|
| 190 |
+
**็ถๆๅผ๏ผ**
|
| 191 |
+
- `pending` - ็ญๅพ
ไธญ
|
| 192 |
+
- `processing` - ๅค็ไธญ
|
| 193 |
+
- `completed` - ๅทฒๅฎๆ
|
| 194 |
+
- `failed` - ๅคฑ่ดฅ
|
| 195 |
+
|
| 196 |
+
---
|
| 197 |
+
|
| 198 |
+
### ้กน็ฎ็ฎก็ๆฅๅฃ
|
| 199 |
+
|
| 200 |
+
| ๆนๆณ | ่ทฏๅพ | ่ฏดๆ |
|
| 201 |
+
|------|------|------|
|
| 202 |
+
| POST | `/api/graph/project/create` | ๅๅปบ้กน็ฎ |
|
| 203 |
+
| GET | `/api/graph/project/{project_id}` | ่ทๅ้กน็ฎ่ฏฆๆ
|
|
| 204 |
+
| GET | `/api/graph/project/list` | ๅๅบๆๆ้กน็ฎ |
|
| 205 |
+
| DELETE | `/api/graph/project/{project_id}` | ๅ ้ค้กน็ฎ |
|
| 206 |
+
|
| 207 |
+
---
|
| 208 |
+
|
| 209 |
+
### ๅพ่ฐฑๆฐๆฎๆฅๅฃ
|
| 210 |
+
|
| 211 |
+
| ๆนๆณ | ่ทฏๅพ | ่ฏดๆ |
|
| 212 |
+
|------|------|------|
|
| 213 |
+
| GET | `/api/graph/data/{graph_id}` | ่ทๅๅพ่ฐฑ่็นๅ่พน |
|
| 214 |
+
| DELETE | `/api/graph/delete/{graph_id}` | ๅ ้คZepๅพ่ฐฑ |
|
| 215 |
+
|
| 216 |
+
---
|
| 217 |
+
|
| 218 |
+
## ๅฎไฝ่ฎพ่ฎกๅๅ
|
| 219 |
+
|
| 220 |
+
ๆฌ็ณป็ปไธไธบ็คพไผ่่ฎบๆจกๆ่ฎพ่ฎก๏ผๅฎไฝๅฟ
้กปๆฏ๏ผ
|
| 221 |
+
|
| 222 |
+
**โ
ๅฏไปฅๆฏ๏ผ**
|
| 223 |
+
- ๅ
ทไฝ็ไธชไบบ๏ผๆๅๆๅง๏ผ
|
| 224 |
+
- ๆณจๅ็ๅ
ฌๅธใ็ป็ปใๆบๆ
|
| 225 |
+
- ๅชไฝๆบๆ
|
| 226 |
+
- ๆฟๅบ้จ้จ
|
| 227 |
+
|
| 228 |
+
**โ ไธๅฏไปฅๆฏ๏ผ**
|
| 229 |
+
- ๆฝ่ฑกๆฆๅฟต๏ผๅฆ"ๆๆฏ"ใ"ๅๆฐ"๏ผ
|
| 230 |
+
- ๆ
็ปชใ่ง็นใ่ถๅฟ
|
| 231 |
+
- ๆณๆ็็พคไฝ๏ผๅฆ"็จๆท"ใ"ๆถ่ดน่
"๏ผ
|
| 232 |
+
|
| 233 |
+
่ฟๆฏๅ ไธบๅ็ปญ้่ฆๆจกๆๅๅฎไฝๅฏน่่ฎบ็ๅๅบๅไผ ๆญ๏ผๆฝ่ฑกๆฆๅฟตๆ ๆณๅไธ่ฟ็งๆจกๆใ
|
| 234 |
+
|
| 235 |
+
---
|
| 236 |
+
|
| 237 |
+
## ้กน็ฎ็ถๆๆต่ฝฌ
|
| 238 |
+
|
| 239 |
+
```
|
| 240 |
+
created โ ontology_generated โ graph_building โ graph_completed
|
| 241 |
+
โ
|
| 242 |
+
failed
|
| 243 |
+
```
|
backend/app/__init__.py
ADDED
|
@@ -0,0 +1,52 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
MiroFish Backend - Flaskๅบ็จๅทฅๅ
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
from flask import Flask, request
|
| 6 |
+
from flask_cors import CORS
|
| 7 |
+
|
| 8 |
+
from .config import Config
|
| 9 |
+
from .utils.logger import setup_logger, get_logger
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
def create_app(config_class=Config):
|
| 13 |
+
"""Flaskๅบ็จๅทฅๅๅฝๆฐ"""
|
| 14 |
+
app = Flask(__name__)
|
| 15 |
+
app.config.from_object(config_class)
|
| 16 |
+
|
| 17 |
+
# ่ฎพ็ฝฎๆฅๅฟ
|
| 18 |
+
logger = setup_logger('mirofish')
|
| 19 |
+
logger.info("=" * 50)
|
| 20 |
+
logger.info("MiroFish Backend ๅฏๅจไธญ...")
|
| 21 |
+
logger.info("=" * 50)
|
| 22 |
+
|
| 23 |
+
# ๅฏ็จCORS
|
| 24 |
+
CORS(app, resources={r"/api/*": {"origins": "*"}})
|
| 25 |
+
|
| 26 |
+
# ่ฏทๆฑๆฅๅฟไธญ้ดไปถ
|
| 27 |
+
@app.before_request
|
| 28 |
+
def log_request():
|
| 29 |
+
logger = get_logger('mirofish.request')
|
| 30 |
+
logger.debug(f"่ฏทๆฑ: {request.method} {request.path}")
|
| 31 |
+
if request.content_type and 'json' in request.content_type:
|
| 32 |
+
logger.debug(f"่ฏทๆฑไฝ: {request.get_json(silent=True)}")
|
| 33 |
+
|
| 34 |
+
@app.after_request
|
| 35 |
+
def log_response(response):
|
| 36 |
+
logger = get_logger('mirofish.request')
|
| 37 |
+
logger.debug(f"ๅๅบ: {response.status_code}")
|
| 38 |
+
return response
|
| 39 |
+
|
| 40 |
+
# ๆณจๅ่ๅพ
|
| 41 |
+
from .api import graph_bp
|
| 42 |
+
app.register_blueprint(graph_bp, url_prefix='/api/graph')
|
| 43 |
+
|
| 44 |
+
# ๅฅๅบทๆฃๆฅ
|
| 45 |
+
@app.route('/health')
|
| 46 |
+
def health():
|
| 47 |
+
return {'status': 'ok', 'service': 'MiroFish Backend'}
|
| 48 |
+
|
| 49 |
+
logger.info("MiroFish Backend ๅฏๅจๅฎๆ")
|
| 50 |
+
|
| 51 |
+
return app
|
| 52 |
+
|
backend/app/api/__init__.py
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
API่ทฏ็ฑๆจกๅ
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
from flask import Blueprint
|
| 6 |
+
|
| 7 |
+
graph_bp = Blueprint('graph', __name__)
|
| 8 |
+
|
| 9 |
+
from . import graph # noqa: E402, F401
|
| 10 |
+
|
backend/app/api/graph.py
ADDED
|
@@ -0,0 +1,657 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๅพ่ฐฑ็ธๅ
ณAPI่ทฏ็ฑ
|
| 3 |
+
้็จ้กน็ฎไธไธๆๆบๅถ๏ผๆๅก็ซฏๆไน
ๅ็ถๆ
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import os
|
| 7 |
+
import traceback
|
| 8 |
+
import threading
|
| 9 |
+
from flask import request, jsonify
|
| 10 |
+
|
| 11 |
+
from . import graph_bp
|
| 12 |
+
from ..config import Config
|
| 13 |
+
from ..services.ontology_generator import OntologyGenerator
|
| 14 |
+
from ..services.graph_builder import GraphBuilderService
|
| 15 |
+
from ..services.text_processor import TextProcessor
|
| 16 |
+
from ..utils.file_parser import FileParser
|
| 17 |
+
from ..utils.logger import get_logger
|
| 18 |
+
from ..models.task import TaskManager, TaskStatus
|
| 19 |
+
from ..models.project import ProjectManager, ProjectStatus
|
| 20 |
+
|
| 21 |
+
# ่ทๅๆฅๅฟๅจ
|
| 22 |
+
logger = get_logger('mirofish.api')
|
| 23 |
+
|
| 24 |
+
|
| 25 |
+
def allowed_file(filename: str) -> bool:
|
| 26 |
+
"""ๆฃๆฅๆไปถๆฉๅฑๅๆฏๅฆๅ
่ฎธ"""
|
| 27 |
+
if not filename or '.' not in filename:
|
| 28 |
+
return False
|
| 29 |
+
ext = os.path.splitext(filename)[1].lower().lstrip('.')
|
| 30 |
+
return ext in Config.ALLOWED_EXTENSIONS
|
| 31 |
+
|
| 32 |
+
|
| 33 |
+
# ============== ้กน็ฎ็ฎก็ๆฅๅฃ ==============
|
| 34 |
+
|
| 35 |
+
@graph_bp.route('/project/create', methods=['POST'])
|
| 36 |
+
def create_project():
|
| 37 |
+
"""
|
| 38 |
+
ๅๅปบๆฐ้กน็ฎ
|
| 39 |
+
|
| 40 |
+
่ฏทๆฑ๏ผJSON๏ผ๏ผ
|
| 41 |
+
{
|
| 42 |
+
"name": "้กน็ฎๅ็งฐ" // ๅฏ้
|
| 43 |
+
}
|
| 44 |
+
|
| 45 |
+
่ฟๅ๏ผ
|
| 46 |
+
{
|
| 47 |
+
"success": true,
|
| 48 |
+
"data": {
|
| 49 |
+
"project_id": "proj_xxxx",
|
| 50 |
+
"name": "...",
|
| 51 |
+
"status": "created",
|
| 52 |
+
...
|
| 53 |
+
}
|
| 54 |
+
}
|
| 55 |
+
"""
|
| 56 |
+
try:
|
| 57 |
+
data = request.get_json() or {}
|
| 58 |
+
name = data.get('name', 'Unnamed Project')
|
| 59 |
+
|
| 60 |
+
project = ProjectManager.create_project(name=name)
|
| 61 |
+
|
| 62 |
+
return jsonify({
|
| 63 |
+
"success": True,
|
| 64 |
+
"data": project.to_dict()
|
| 65 |
+
})
|
| 66 |
+
|
| 67 |
+
except Exception as e:
|
| 68 |
+
return jsonify({
|
| 69 |
+
"success": False,
|
| 70 |
+
"error": str(e),
|
| 71 |
+
"traceback": traceback.format_exc()
|
| 72 |
+
}), 500
|
| 73 |
+
|
| 74 |
+
|
| 75 |
+
@graph_bp.route('/project/<project_id>', methods=['GET'])
|
| 76 |
+
def get_project(project_id: str):
|
| 77 |
+
"""
|
| 78 |
+
่ทๅ้กน็ฎ่ฏฆๆ
|
| 79 |
+
"""
|
| 80 |
+
project = ProjectManager.get_project(project_id)
|
| 81 |
+
|
| 82 |
+
if not project:
|
| 83 |
+
return jsonify({
|
| 84 |
+
"success": False,
|
| 85 |
+
"error": f"้กน็ฎไธๅญๅจ: {project_id}"
|
| 86 |
+
}), 404
|
| 87 |
+
|
| 88 |
+
return jsonify({
|
| 89 |
+
"success": True,
|
| 90 |
+
"data": project.to_dict()
|
| 91 |
+
})
|
| 92 |
+
|
| 93 |
+
|
| 94 |
+
@graph_bp.route('/project/list', methods=['GET'])
|
| 95 |
+
def list_projects():
|
| 96 |
+
"""
|
| 97 |
+
ๅๅบๆๆ้กน็ฎ
|
| 98 |
+
"""
|
| 99 |
+
limit = request.args.get('limit', 50, type=int)
|
| 100 |
+
projects = ProjectManager.list_projects(limit=limit)
|
| 101 |
+
|
| 102 |
+
return jsonify({
|
| 103 |
+
"success": True,
|
| 104 |
+
"data": [p.to_dict() for p in projects],
|
| 105 |
+
"count": len(projects)
|
| 106 |
+
})
|
| 107 |
+
|
| 108 |
+
|
| 109 |
+
@graph_bp.route('/project/<project_id>', methods=['DELETE'])
|
| 110 |
+
def delete_project(project_id: str):
|
| 111 |
+
"""
|
| 112 |
+
ๅ ้ค้กน็ฎ
|
| 113 |
+
"""
|
| 114 |
+
success = ProjectManager.delete_project(project_id)
|
| 115 |
+
|
| 116 |
+
if not success:
|
| 117 |
+
return jsonify({
|
| 118 |
+
"success": False,
|
| 119 |
+
"error": f"้กน็ฎไธๅญๅจๆๅ ้คๅคฑ่ดฅ: {project_id}"
|
| 120 |
+
}), 404
|
| 121 |
+
|
| 122 |
+
return jsonify({
|
| 123 |
+
"success": True,
|
| 124 |
+
"message": f"้กน็ฎๅทฒๅ ้ค: {project_id}"
|
| 125 |
+
})
|
| 126 |
+
|
| 127 |
+
|
| 128 |
+
@graph_bp.route('/project/<project_id>/reset', methods=['POST'])
|
| 129 |
+
def reset_project(project_id: str):
|
| 130 |
+
"""
|
| 131 |
+
้็ฝฎ้กน็ฎ็ถๆ๏ผ็จไบ้ๆฐๆๅปบๅพ่ฐฑ๏ผ
|
| 132 |
+
"""
|
| 133 |
+
project = ProjectManager.get_project(project_id)
|
| 134 |
+
|
| 135 |
+
if not project:
|
| 136 |
+
return jsonify({
|
| 137 |
+
"success": False,
|
| 138 |
+
"error": f"้กน็ฎไธๅญๅจ: {project_id}"
|
| 139 |
+
}), 404
|
| 140 |
+
|
| 141 |
+
# ้็ฝฎๅฐๆฌไฝๅทฒ็ๆ็ถๆ
|
| 142 |
+
if project.ontology:
|
| 143 |
+
project.status = ProjectStatus.ONTOLOGY_GENERATED
|
| 144 |
+
else:
|
| 145 |
+
project.status = ProjectStatus.CREATED
|
| 146 |
+
|
| 147 |
+
project.graph_id = None
|
| 148 |
+
project.graph_build_task_id = None
|
| 149 |
+
project.error = None
|
| 150 |
+
ProjectManager.save_project(project)
|
| 151 |
+
|
| 152 |
+
return jsonify({
|
| 153 |
+
"success": True,
|
| 154 |
+
"message": f"้กน็ฎๅทฒ้็ฝฎ: {project_id}",
|
| 155 |
+
"data": project.to_dict()
|
| 156 |
+
})
|
| 157 |
+
|
| 158 |
+
|
| 159 |
+
# ============== ๆฅๅฃ1๏ผไธไผ ๆไปถๅนถ็ๆๆฌไฝ ==============
|
| 160 |
+
|
| 161 |
+
@graph_bp.route('/ontology/generate', methods=['POST'])
|
| 162 |
+
def generate_ontology():
|
| 163 |
+
"""
|
| 164 |
+
ๆฅๅฃ1๏ผไธไผ ๆไปถ๏ผๅๆ็ๆๆฌไฝๅฎไน
|
| 165 |
+
|
| 166 |
+
่ฏทๆฑๆนๅผ๏ผmultipart/form-data
|
| 167 |
+
|
| 168 |
+
ๅๆฐ๏ผ
|
| 169 |
+
files: ไธไผ ็ๆไปถ๏ผPDF/MD/TXT๏ผ๏ผๅฏๅคไธช
|
| 170 |
+
simulation_requirement: ๆจกๆ้ๆฑๆ่ฟฐ๏ผๅฟ
ๅกซ๏ผ
|
| 171 |
+
project_name: ้กน็ฎๅ็งฐ๏ผๅฏ้๏ผ
|
| 172 |
+
additional_context: ้ขๅค่ฏดๆ๏ผๅฏ้๏ผ
|
| 173 |
+
|
| 174 |
+
่ฟๅ๏ผ
|
| 175 |
+
{
|
| 176 |
+
"success": true,
|
| 177 |
+
"data": {
|
| 178 |
+
"project_id": "proj_xxxx",
|
| 179 |
+
"ontology": {
|
| 180 |
+
"entity_types": [...],
|
| 181 |
+
"edge_types": [...],
|
| 182 |
+
"analysis_summary": "..."
|
| 183 |
+
},
|
| 184 |
+
"files": [...],
|
| 185 |
+
"total_text_length": 12345
|
| 186 |
+
}
|
| 187 |
+
}
|
| 188 |
+
"""
|
| 189 |
+
try:
|
| 190 |
+
logger.info("=== ๅผๅง็ๆๆฌไฝๅฎ๏ฟฝ๏ฟฝ ===")
|
| 191 |
+
|
| 192 |
+
# ่ทๅๅๆฐ
|
| 193 |
+
simulation_requirement = request.form.get('simulation_requirement', '')
|
| 194 |
+
project_name = request.form.get('project_name', 'Unnamed Project')
|
| 195 |
+
additional_context = request.form.get('additional_context', '')
|
| 196 |
+
|
| 197 |
+
logger.debug(f"้กน็ฎๅ็งฐ: {project_name}")
|
| 198 |
+
logger.debug(f"ๆจกๆ้ๆฑ: {simulation_requirement[:100]}...")
|
| 199 |
+
|
| 200 |
+
if not simulation_requirement:
|
| 201 |
+
return jsonify({
|
| 202 |
+
"success": False,
|
| 203 |
+
"error": "่ฏทๆไพๆจกๆ้ๆฑๆ่ฟฐ (simulation_requirement)"
|
| 204 |
+
}), 400
|
| 205 |
+
|
| 206 |
+
# ่ทๅไธไผ ็ๆไปถ
|
| 207 |
+
uploaded_files = request.files.getlist('files')
|
| 208 |
+
if not uploaded_files or all(not f.filename for f in uploaded_files):
|
| 209 |
+
return jsonify({
|
| 210 |
+
"success": False,
|
| 211 |
+
"error": "่ฏท่ณๅฐไธไผ ไธไธชๆๆกฃๆไปถ"
|
| 212 |
+
}), 400
|
| 213 |
+
|
| 214 |
+
# ๅๅปบ้กน็ฎ
|
| 215 |
+
project = ProjectManager.create_project(name=project_name)
|
| 216 |
+
project.simulation_requirement = simulation_requirement
|
| 217 |
+
logger.info(f"ๅๅปบ้กน็ฎ: {project.project_id}")
|
| 218 |
+
|
| 219 |
+
# ไฟๅญๆไปถๅนถๆๅๆๆฌ
|
| 220 |
+
document_texts = []
|
| 221 |
+
all_text = ""
|
| 222 |
+
|
| 223 |
+
for file in uploaded_files:
|
| 224 |
+
if file and file.filename and allowed_file(file.filename):
|
| 225 |
+
# ไฟๅญๆไปถๅฐ้กน็ฎ็ฎๅฝ
|
| 226 |
+
file_info = ProjectManager.save_file_to_project(
|
| 227 |
+
project.project_id,
|
| 228 |
+
file,
|
| 229 |
+
file.filename
|
| 230 |
+
)
|
| 231 |
+
project.files.append({
|
| 232 |
+
"filename": file_info["original_filename"],
|
| 233 |
+
"size": file_info["size"]
|
| 234 |
+
})
|
| 235 |
+
|
| 236 |
+
# ๆๅๆๆฌ
|
| 237 |
+
text = FileParser.extract_text(file_info["path"])
|
| 238 |
+
text = TextProcessor.preprocess_text(text)
|
| 239 |
+
document_texts.append(text)
|
| 240 |
+
all_text += f"\n\n=== {file_info['original_filename']} ===\n{text}"
|
| 241 |
+
|
| 242 |
+
if not document_texts:
|
| 243 |
+
ProjectManager.delete_project(project.project_id)
|
| 244 |
+
return jsonify({
|
| 245 |
+
"success": False,
|
| 246 |
+
"error": "ๆฒกๆๆๅๅค็ไปปไฝๆๆกฃ๏ผ่ฏทๆฃๆฅๆไปถๆ ผๅผ"
|
| 247 |
+
}), 400
|
| 248 |
+
|
| 249 |
+
# ไฟๅญๆๅ็ๆๆฌ
|
| 250 |
+
project.total_text_length = len(all_text)
|
| 251 |
+
ProjectManager.save_extracted_text(project.project_id, all_text)
|
| 252 |
+
logger.info(f"ๆๆฌๆๅๅฎๆ๏ผๅ
ฑ {len(all_text)} ๅญ็ฌฆ")
|
| 253 |
+
|
| 254 |
+
# ็ๆๆฌไฝ
|
| 255 |
+
logger.info("่ฐ็จ LLM ็ๆๆฌไฝๅฎไน...")
|
| 256 |
+
generator = OntologyGenerator()
|
| 257 |
+
ontology = generator.generate(
|
| 258 |
+
document_texts=document_texts,
|
| 259 |
+
simulation_requirement=simulation_requirement,
|
| 260 |
+
additional_context=additional_context if additional_context else None
|
| 261 |
+
)
|
| 262 |
+
|
| 263 |
+
# ไฟๅญๆฌไฝๅฐ้กน็ฎ
|
| 264 |
+
entity_count = len(ontology.get("entity_types", []))
|
| 265 |
+
edge_count = len(ontology.get("edge_types", []))
|
| 266 |
+
logger.info(f"ๆฌไฝ็ๆๅฎๆ: {entity_count} ไธชๅฎไฝ็ฑปๅ, {edge_count} ไธชๅ
ณ็ณป็ฑปๅ")
|
| 267 |
+
|
| 268 |
+
project.ontology = {
|
| 269 |
+
"entity_types": ontology.get("entity_types", []),
|
| 270 |
+
"edge_types": ontology.get("edge_types", [])
|
| 271 |
+
}
|
| 272 |
+
project.analysis_summary = ontology.get("analysis_summary", "")
|
| 273 |
+
project.status = ProjectStatus.ONTOLOGY_GENERATED
|
| 274 |
+
ProjectManager.save_project(project)
|
| 275 |
+
logger.info(f"=== ๆฌไฝ็ๆๅฎๆ === ้กน็ฎID: {project.project_id}")
|
| 276 |
+
|
| 277 |
+
return jsonify({
|
| 278 |
+
"success": True,
|
| 279 |
+
"data": {
|
| 280 |
+
"project_id": project.project_id,
|
| 281 |
+
"project_name": project.name,
|
| 282 |
+
"ontology": project.ontology,
|
| 283 |
+
"analysis_summary": project.analysis_summary,
|
| 284 |
+
"files": project.files,
|
| 285 |
+
"total_text_length": project.total_text_length
|
| 286 |
+
}
|
| 287 |
+
})
|
| 288 |
+
|
| 289 |
+
except Exception as e:
|
| 290 |
+
return jsonify({
|
| 291 |
+
"success": False,
|
| 292 |
+
"error": str(e),
|
| 293 |
+
"traceback": traceback.format_exc()
|
| 294 |
+
}), 500
|
| 295 |
+
|
| 296 |
+
|
| 297 |
+
# ============== ๆฅๅฃ2๏ผๆๅปบๅพ่ฐฑ ==============
|
| 298 |
+
|
| 299 |
+
@graph_bp.route('/build', methods=['POST'])
|
| 300 |
+
def build_graph():
|
| 301 |
+
"""
|
| 302 |
+
ๆฅๅฃ2๏ผๆ นๆฎproject_idๆๅปบๅพ่ฐฑ
|
| 303 |
+
|
| 304 |
+
่ฏทๆฑ๏ผJSON๏ผ๏ผ
|
| 305 |
+
{
|
| 306 |
+
"project_id": "proj_xxxx", // ๅฟ
ๅกซ๏ผๆฅ่ชๆฅๅฃ1
|
| 307 |
+
"graph_name": "ๅพ่ฐฑๅ็งฐ", // ๅฏ้
|
| 308 |
+
"chunk_size": 500, // ๅฏ้๏ผ้ป่ฎค500
|
| 309 |
+
"chunk_overlap": 50 // ๅฏ้๏ผ้ป่ฎค50
|
| 310 |
+
}
|
| 311 |
+
|
| 312 |
+
่ฟๅ๏ผ
|
| 313 |
+
{
|
| 314 |
+
"success": true,
|
| 315 |
+
"data": {
|
| 316 |
+
"project_id": "proj_xxxx",
|
| 317 |
+
"task_id": "task_xxxx",
|
| 318 |
+
"message": "ๅพ่ฐฑๆๅปบไปปๅกๅทฒๅฏๅจ"
|
| 319 |
+
}
|
| 320 |
+
}
|
| 321 |
+
"""
|
| 322 |
+
try:
|
| 323 |
+
logger.info("=== ๅผๅงๆ๏ฟฝ๏ฟฝๅพ่ฐฑ ===")
|
| 324 |
+
|
| 325 |
+
# ๆฃๆฅ้
็ฝฎ
|
| 326 |
+
errors = []
|
| 327 |
+
if not Config.ZEP_API_KEY:
|
| 328 |
+
errors.append("ZEP_API_KEYๆช้
็ฝฎ")
|
| 329 |
+
if errors:
|
| 330 |
+
logger.error(f"้
็ฝฎ้่ฏฏ: {errors}")
|
| 331 |
+
return jsonify({
|
| 332 |
+
"success": False,
|
| 333 |
+
"error": "้
็ฝฎ้่ฏฏ: " + "; ".join(errors)
|
| 334 |
+
}), 500
|
| 335 |
+
|
| 336 |
+
# ่งฃๆ่ฏทๆฑ
|
| 337 |
+
data = request.get_json() or {}
|
| 338 |
+
project_id = data.get('project_id')
|
| 339 |
+
logger.debug(f"่ฏทๆฑๅๆฐ: project_id={project_id}")
|
| 340 |
+
|
| 341 |
+
if not project_id:
|
| 342 |
+
return jsonify({
|
| 343 |
+
"success": False,
|
| 344 |
+
"error": "่ฏทๆไพ project_id"
|
| 345 |
+
}), 400
|
| 346 |
+
|
| 347 |
+
# ่ทๅ้กน็ฎ
|
| 348 |
+
project = ProjectManager.get_project(project_id)
|
| 349 |
+
if not project:
|
| 350 |
+
return jsonify({
|
| 351 |
+
"success": False,
|
| 352 |
+
"error": f"้กน็ฎไธๅญๅจ: {project_id}"
|
| 353 |
+
}), 404
|
| 354 |
+
|
| 355 |
+
# ๆฃๆฅ้กน็ฎ็ถๆ
|
| 356 |
+
force = data.get('force', False) # ๅผบๅถ้ๆฐๆๅปบ
|
| 357 |
+
|
| 358 |
+
if project.status == ProjectStatus.CREATED:
|
| 359 |
+
return jsonify({
|
| 360 |
+
"success": False,
|
| 361 |
+
"error": "้กน็ฎๅฐๆช็ๆๆฌไฝ๏ผ่ฏทๅ
่ฐ็จ /ontology/generate"
|
| 362 |
+
}), 400
|
| 363 |
+
|
| 364 |
+
if project.status == ProjectStatus.GRAPH_BUILDING and not force:
|
| 365 |
+
return jsonify({
|
| 366 |
+
"success": False,
|
| 367 |
+
"error": "ๅพ่ฐฑๆญฃๅจๆๅปบไธญ๏ผ่ฏทๅฟ้ๅคๆไบคใๅฆ้ๅผบๅถ้ๅปบ๏ผ่ฏทๆทปๅ force: true",
|
| 368 |
+
"task_id": project.graph_build_task_id
|
| 369 |
+
}), 400
|
| 370 |
+
|
| 371 |
+
# ๅฆๆๅผบๅถ้ๅปบ๏ผ้็ฝฎ็ถๆ
|
| 372 |
+
if force and project.status in [ProjectStatus.GRAPH_BUILDING, ProjectStatus.FAILED, ProjectStatus.GRAPH_COMPLETED]:
|
| 373 |
+
project.status = ProjectStatus.ONTOLOGY_GENERATED
|
| 374 |
+
project.graph_id = None
|
| 375 |
+
project.graph_build_task_id = None
|
| 376 |
+
project.error = None
|
| 377 |
+
|
| 378 |
+
# ่ทๅ้
็ฝฎ
|
| 379 |
+
graph_name = data.get('graph_name', project.name or 'MiroFish Graph')
|
| 380 |
+
chunk_size = data.get('chunk_size', project.chunk_size or Config.DEFAULT_CHUNK_SIZE)
|
| 381 |
+
chunk_overlap = data.get('chunk_overlap', project.chunk_overlap or Config.DEFAULT_CHUNK_OVERLAP)
|
| 382 |
+
|
| 383 |
+
# ๆดๆฐ้กน็ฎ้
็ฝฎ
|
| 384 |
+
project.chunk_size = chunk_size
|
| 385 |
+
project.chunk_overlap = chunk_overlap
|
| 386 |
+
|
| 387 |
+
# ่ทๅๆๅ็ๆๆฌ
|
| 388 |
+
text = ProjectManager.get_extracted_text(project_id)
|
| 389 |
+
if not text:
|
| 390 |
+
return jsonify({
|
| 391 |
+
"success": False,
|
| 392 |
+
"error": "ๆชๆพๅฐๆๅ็ๆๆฌๅ
ๅฎน"
|
| 393 |
+
}), 400
|
| 394 |
+
|
| 395 |
+
# ่ทๅๆฌไฝ
|
| 396 |
+
ontology = project.ontology
|
| 397 |
+
if not ontology:
|
| 398 |
+
return jsonify({
|
| 399 |
+
"success": False,
|
| 400 |
+
"error": "ๆชๆพๅฐๆฌไฝๅฎไน"
|
| 401 |
+
}), 400
|
| 402 |
+
|
| 403 |
+
# ๅๅปบๅผๆญฅไปปๅก
|
| 404 |
+
task_manager = TaskManager()
|
| 405 |
+
task_id = task_manager.create_task(f"ๆๅปบๅพ่ฐฑ: {graph_name}")
|
| 406 |
+
logger.info(f"ๅๅปบๅพ่ฐฑๆๅปบไปปๅก: task_id={task_id}, project_id={project_id}")
|
| 407 |
+
|
| 408 |
+
# ๆดๆฐ้กน็ฎ็ถๆ
|
| 409 |
+
project.status = ProjectStatus.GRAPH_BUILDING
|
| 410 |
+
project.graph_build_task_id = task_id
|
| 411 |
+
ProjectManager.save_project(project)
|
| 412 |
+
|
| 413 |
+
# ๅฏๅจๅๅฐไปปๅก
|
| 414 |
+
def build_task():
|
| 415 |
+
build_logger = get_logger('mirofish.build')
|
| 416 |
+
try:
|
| 417 |
+
build_logger.info(f"[{task_id}] ๅผๅงๆๅปบๅพ่ฐฑ...")
|
| 418 |
+
task_manager.update_task(
|
| 419 |
+
task_id,
|
| 420 |
+
status=TaskStatus.PROCESSING,
|
| 421 |
+
message="ๅๅงๅๅพ่ฐฑๆๅปบๆๅก..."
|
| 422 |
+
)
|
| 423 |
+
|
| 424 |
+
# ๅๅปบๅพ่ฐฑๆๅปบๆๅก
|
| 425 |
+
builder = GraphBuilderService(api_key=Config.ZEP_API_KEY)
|
| 426 |
+
|
| 427 |
+
# ๅๅ
|
| 428 |
+
task_manager.update_task(
|
| 429 |
+
task_id,
|
| 430 |
+
message="ๆๆฌๅๅไธญ...",
|
| 431 |
+
progress=5
|
| 432 |
+
)
|
| 433 |
+
chunks = TextProcessor.split_text(
|
| 434 |
+
text,
|
| 435 |
+
chunk_size=chunk_size,
|
| 436 |
+
overlap=chunk_overlap
|
| 437 |
+
)
|
| 438 |
+
total_chunks = len(chunks)
|
| 439 |
+
|
| 440 |
+
# ๅๅปบๅพ่ฐฑ
|
| 441 |
+
task_manager.update_task(
|
| 442 |
+
task_id,
|
| 443 |
+
message="ๅๅปบZepๅพ่ฐฑ...",
|
| 444 |
+
progress=10
|
| 445 |
+
)
|
| 446 |
+
graph_id = builder.create_graph(name=graph_name)
|
| 447 |
+
|
| 448 |
+
# ๆดๆฐ้กน็ฎ็graph_id
|
| 449 |
+
project.graph_id = graph_id
|
| 450 |
+
ProjectManager.save_project(project)
|
| 451 |
+
|
| 452 |
+
# ่ฎพ็ฝฎๆฌไฝ
|
| 453 |
+
task_manager.update_task(
|
| 454 |
+
task_id,
|
| 455 |
+
message="่ฎพ็ฝฎๆฌไฝๅฎไน...",
|
| 456 |
+
progress=15
|
| 457 |
+
)
|
| 458 |
+
builder.set_ontology(graph_id, ontology)
|
| 459 |
+
|
| 460 |
+
# ๆทปๅ ๆๆฌ๏ผprogress_callback ็ญพๅๆฏ (msg, progress_ratio)๏ผ
|
| 461 |
+
def add_progress_callback(msg, progress_ratio):
|
| 462 |
+
progress = 15 + int(progress_ratio * 40) # 15% - 55%
|
| 463 |
+
task_manager.update_task(
|
| 464 |
+
task_id,
|
| 465 |
+
message=msg,
|
| 466 |
+
progress=progress
|
| 467 |
+
)
|
| 468 |
+
|
| 469 |
+
task_manager.update_task(
|
| 470 |
+
task_id,
|
| 471 |
+
message=f"ๅผๅงๆทปๅ {total_chunks} ไธชๆๆฌๅ...",
|
| 472 |
+
progress=15
|
| 473 |
+
)
|
| 474 |
+
|
| 475 |
+
episode_uuids = builder.add_text_batches(
|
| 476 |
+
graph_id,
|
| 477 |
+
chunks,
|
| 478 |
+
batch_size=3,
|
| 479 |
+
progress_callback=add_progress_callback
|
| 480 |
+
)
|
| 481 |
+
|
| 482 |
+
# ็ญๅพ
Zepๅค็ๅฎๆ๏ผๆฅ่ฏขๆฏไธชepisode็processed็ถๆ๏ผ
|
| 483 |
+
task_manager.update_task(
|
| 484 |
+
task_id,
|
| 485 |
+
message="็ญๅพ
Zepๅค็ๆฐๆฎ...",
|
| 486 |
+
progress=55
|
| 487 |
+
)
|
| 488 |
+
|
| 489 |
+
def wait_progress_callback(msg, progress_ratio):
|
| 490 |
+
progress = 55 + int(progress_ratio * 35) # 55% - 90%
|
| 491 |
+
task_manager.update_task(
|
| 492 |
+
task_id,
|
| 493 |
+
message=msg,
|
| 494 |
+
progress=progress
|
| 495 |
+
)
|
| 496 |
+
|
| 497 |
+
builder._wait_for_episodes(episode_uuids, wait_progress_callback)
|
| 498 |
+
|
| 499 |
+
# ่ทๅๅพ่ฐฑๆฐๆฎ
|
| 500 |
+
task_manager.update_task(
|
| 501 |
+
task_id,
|
| 502 |
+
message="่ทๅๅพ่ฐฑๆฐๆฎ...",
|
| 503 |
+
progress=95
|
| 504 |
+
)
|
| 505 |
+
graph_data = builder.get_graph_data(graph_id)
|
| 506 |
+
|
| 507 |
+
# ๆดๆฐ้กน็ฎ็ถๆ
|
| 508 |
+
project.status = ProjectStatus.GRAPH_COMPLETED
|
| 509 |
+
ProjectManager.save_project(project)
|
| 510 |
+
|
| 511 |
+
node_count = graph_data.get("node_count", 0)
|
| 512 |
+
edge_count = graph_data.get("edge_count", 0)
|
| 513 |
+
build_logger.info(f"[{task_id}] ๅพ่ฐฑๆๅปบๅฎๆ: graph_id={graph_id}, ่็น={node_count}, ่พน={edge_count}")
|
| 514 |
+
|
| 515 |
+
# ๅฎๆ
|
| 516 |
+
task_manager.update_task(
|
| 517 |
+
task_id,
|
| 518 |
+
status=TaskStatus.COMPLETED,
|
| 519 |
+
message="ๅพ่ฐฑๆๅปบๅฎๆ",
|
| 520 |
+
progress=100,
|
| 521 |
+
result={
|
| 522 |
+
"project_id": project_id,
|
| 523 |
+
"graph_id": graph_id,
|
| 524 |
+
"node_count": node_count,
|
| 525 |
+
"edge_count": edge_count,
|
| 526 |
+
"chunk_count": total_chunks
|
| 527 |
+
}
|
| 528 |
+
)
|
| 529 |
+
|
| 530 |
+
except Exception as e:
|
| 531 |
+
# ๆดๆฐ้กน็ฎ็ถๆไธบๅคฑ่ดฅ
|
| 532 |
+
build_logger.error(f"[{task_id}] ๅพ่ฐฑๆๅปบๅคฑ่ดฅ: {str(e)}")
|
| 533 |
+
build_logger.debug(traceback.format_exc())
|
| 534 |
+
|
| 535 |
+
project.status = ProjectStatus.FAILED
|
| 536 |
+
project.error = str(e)
|
| 537 |
+
ProjectManager.save_project(project)
|
| 538 |
+
|
| 539 |
+
task_manager.update_task(
|
| 540 |
+
task_id,
|
| 541 |
+
status=TaskStatus.FAILED,
|
| 542 |
+
message=f"ๆๅปบๅคฑ่ดฅ: {str(e)}",
|
| 543 |
+
error=traceback.format_exc()
|
| 544 |
+
)
|
| 545 |
+
|
| 546 |
+
# ๅฏๅจๅๅฐ็บฟ็จ
|
| 547 |
+
thread = threading.Thread(target=build_task, daemon=True)
|
| 548 |
+
thread.start()
|
| 549 |
+
|
| 550 |
+
return jsonify({
|
| 551 |
+
"success": True,
|
| 552 |
+
"data": {
|
| 553 |
+
"project_id": project_id,
|
| 554 |
+
"task_id": task_id,
|
| 555 |
+
"message": "ๅพ่ฐฑๆๅปบไปปๅกๅทฒๅฏๅจ๏ผ่ฏท้่ฟ /task/{task_id} ๆฅ่ฏข่ฟๅบฆ"
|
| 556 |
+
}
|
| 557 |
+
})
|
| 558 |
+
|
| 559 |
+
except Exception as e:
|
| 560 |
+
return jsonify({
|
| 561 |
+
"success": False,
|
| 562 |
+
"error": str(e),
|
| 563 |
+
"traceback": traceback.format_exc()
|
| 564 |
+
}), 500
|
| 565 |
+
|
| 566 |
+
|
| 567 |
+
# ============== ไปปๅกๆฅ่ฏขๆฅๅฃ ==============
|
| 568 |
+
|
| 569 |
+
@graph_bp.route('/task/<task_id>', methods=['GET'])
|
| 570 |
+
def get_task(task_id: str):
|
| 571 |
+
"""
|
| 572 |
+
ๆฅ่ฏขไปปๅก็ถๆ
|
| 573 |
+
"""
|
| 574 |
+
task = TaskManager().get_task(task_id)
|
| 575 |
+
|
| 576 |
+
if not task:
|
| 577 |
+
return jsonify({
|
| 578 |
+
"success": False,
|
| 579 |
+
"error": f"ไปปๅกไธๅญๅจ: {task_id}"
|
| 580 |
+
}), 404
|
| 581 |
+
|
| 582 |
+
return jsonify({
|
| 583 |
+
"success": True,
|
| 584 |
+
"data": task.to_dict()
|
| 585 |
+
})
|
| 586 |
+
|
| 587 |
+
|
| 588 |
+
@graph_bp.route('/tasks', methods=['GET'])
|
| 589 |
+
def list_tasks():
|
| 590 |
+
"""
|
| 591 |
+
ๅๅบๆๆไปปๅก
|
| 592 |
+
"""
|
| 593 |
+
tasks = TaskManager().list_tasks()
|
| 594 |
+
|
| 595 |
+
return jsonify({
|
| 596 |
+
"success": True,
|
| 597 |
+
"data": [t.to_dict() for t in tasks],
|
| 598 |
+
"count": len(tasks)
|
| 599 |
+
})
|
| 600 |
+
|
| 601 |
+
|
| 602 |
+
# ============== ๅพ่ฐฑๆฐๆฎๆฅๅฃ ==============
|
| 603 |
+
|
| 604 |
+
@graph_bp.route('/data/<graph_id>', methods=['GET'])
|
| 605 |
+
def get_graph_data(graph_id: str):
|
| 606 |
+
"""
|
| 607 |
+
่ทๅๅพ่ฐฑๆฐๆฎ๏ผ่็นๅ่พน๏ผ
|
| 608 |
+
"""
|
| 609 |
+
try:
|
| 610 |
+
if not Config.ZEP_API_KEY:
|
| 611 |
+
return jsonify({
|
| 612 |
+
"success": False,
|
| 613 |
+
"error": "ZEP_API_KEYๆช้
็ฝฎ"
|
| 614 |
+
}), 500
|
| 615 |
+
|
| 616 |
+
builder = GraphBuilderService(api_key=Config.ZEP_API_KEY)
|
| 617 |
+
graph_data = builder.get_graph_data(graph_id)
|
| 618 |
+
|
| 619 |
+
return jsonify({
|
| 620 |
+
"success": True,
|
| 621 |
+
"data": graph_data
|
| 622 |
+
})
|
| 623 |
+
|
| 624 |
+
except Exception as e:
|
| 625 |
+
return jsonify({
|
| 626 |
+
"success": False,
|
| 627 |
+
"error": str(e),
|
| 628 |
+
"traceback": traceback.format_exc()
|
| 629 |
+
}), 500
|
| 630 |
+
|
| 631 |
+
|
| 632 |
+
@graph_bp.route('/delete/<graph_id>', methods=['DELETE'])
|
| 633 |
+
def delete_graph(graph_id: str):
|
| 634 |
+
"""
|
| 635 |
+
ๅ ้คZepๅพ่ฐฑ
|
| 636 |
+
"""
|
| 637 |
+
try:
|
| 638 |
+
if not Config.ZEP_API_KEY:
|
| 639 |
+
return jsonify({
|
| 640 |
+
"success": False,
|
| 641 |
+
"error": "ZEP_API_KEYๆช้
็ฝฎ"
|
| 642 |
+
}), 500
|
| 643 |
+
|
| 644 |
+
builder = GraphBuilderService(api_key=Config.ZEP_API_KEY)
|
| 645 |
+
builder.delete_graph(graph_id)
|
| 646 |
+
|
| 647 |
+
return jsonify({
|
| 648 |
+
"success": True,
|
| 649 |
+
"message": f"ๅพ่ฐฑๅทฒๅ ้ค: {graph_id}"
|
| 650 |
+
})
|
| 651 |
+
|
| 652 |
+
except Exception as e:
|
| 653 |
+
return jsonify({
|
| 654 |
+
"success": False,
|
| 655 |
+
"error": str(e),
|
| 656 |
+
"traceback": traceback.format_exc()
|
| 657 |
+
}), 500
|
backend/app/config.py
ADDED
|
@@ -0,0 +1,53 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
้
็ฝฎ็ฎก็
|
| 3 |
+
็ปไธไป้กน็ฎๆ น็ฎๅฝ็ .env ๆไปถๅ ่ฝฝ้
็ฝฎ
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import os
|
| 7 |
+
from dotenv import load_dotenv
|
| 8 |
+
|
| 9 |
+
# ๅ ่ฝฝ้กน็ฎๆ น็ฎๅฝ็ .env ๆไปถ
|
| 10 |
+
# ่ทฏๅพ: MiroFish/.env (็ธๅฏนไบ backend/app/config.py)
|
| 11 |
+
project_root_env = os.path.join(os.path.dirname(__file__), '../../.env')
|
| 12 |
+
|
| 13 |
+
if os.path.exists(project_root_env):
|
| 14 |
+
load_dotenv(project_root_env)
|
| 15 |
+
else:
|
| 16 |
+
# ๅฆๆๆ น็ฎๅฝๆฒกๆ .env๏ผๅฐ่ฏๅ ่ฝฝ็ฏๅขๅ้๏ผ็จไบ็ไบง็ฏๅข๏ผ
|
| 17 |
+
load_dotenv()
|
| 18 |
+
|
| 19 |
+
|
| 20 |
+
class Config:
|
| 21 |
+
"""Flask้
็ฝฎ็ฑป"""
|
| 22 |
+
|
| 23 |
+
# Flask้
็ฝฎ
|
| 24 |
+
SECRET_KEY = os.environ.get('SECRET_KEY', 'mirofish-secret-key')
|
| 25 |
+
DEBUG = os.environ.get('FLASK_DEBUG', 'True').lower() == 'true'
|
| 26 |
+
|
| 27 |
+
# LLM้
็ฝฎ๏ผ็ปไธไฝฟ็จOpenAIๆ ผๅผ๏ผ
|
| 28 |
+
LLM_API_KEY = os.environ.get('LLM_API_KEY')
|
| 29 |
+
LLM_BASE_URL = os.environ.get('LLM_BASE_URL', 'https://api.openai.com/v1')
|
| 30 |
+
LLM_MODEL_NAME = os.environ.get('LLM_MODEL_NAME', 'gpt-4o-mini')
|
| 31 |
+
|
| 32 |
+
# Zep้
็ฝฎ
|
| 33 |
+
ZEP_API_KEY = os.environ.get('ZEP_API_KEY')
|
| 34 |
+
|
| 35 |
+
# ๆไปถไธไผ ้
็ฝฎ
|
| 36 |
+
MAX_CONTENT_LENGTH = 50 * 1024 * 1024 # 50MB
|
| 37 |
+
UPLOAD_FOLDER = os.path.join(os.path.dirname(__file__), '../uploads')
|
| 38 |
+
ALLOWED_EXTENSIONS = {'pdf', 'md', 'txt', 'markdown'}
|
| 39 |
+
|
| 40 |
+
# ๆๆฌๅค็้
็ฝฎ
|
| 41 |
+
DEFAULT_CHUNK_SIZE = 500 # ้ป่ฎคๅๅๅคงๅฐ
|
| 42 |
+
DEFAULT_CHUNK_OVERLAP = 50 # ้ป่ฎค้ๅ ๅคงๅฐ
|
| 43 |
+
|
| 44 |
+
@classmethod
|
| 45 |
+
def validate(cls):
|
| 46 |
+
"""้ช่ฏๅฟ
่ฆ้
็ฝฎ"""
|
| 47 |
+
errors = []
|
| 48 |
+
if not cls.LLM_API_KEY:
|
| 49 |
+
errors.append("LLM_API_KEY ๆช้
็ฝฎ")
|
| 50 |
+
if not cls.ZEP_API_KEY:
|
| 51 |
+
errors.append("ZEP_API_KEY ๆช้
็ฝฎ")
|
| 52 |
+
return errors
|
| 53 |
+
|
backend/app/models/__init__.py
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๆฐๆฎๆจกๅๆจกๅ
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
from .task import TaskManager, TaskStatus
|
| 6 |
+
from .project import Project, ProjectStatus, ProjectManager
|
| 7 |
+
|
| 8 |
+
__all__ = ['TaskManager', 'TaskStatus', 'Project', 'ProjectStatus', 'ProjectManager']
|
| 9 |
+
|
backend/app/models/project.py
ADDED
|
@@ -0,0 +1,305 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
้กน็ฎไธไธๆ็ฎก็
|
| 3 |
+
็จไบๅจๆๅก็ซฏๆไน
ๅ้กน็ฎ็ถๆ๏ผ้ฟๅ
ๅ็ซฏๅจๆฅๅฃ้ดไผ ้ๅคง้ๆฐๆฎ
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import os
|
| 7 |
+
import json
|
| 8 |
+
import uuid
|
| 9 |
+
import shutil
|
| 10 |
+
from datetime import datetime
|
| 11 |
+
from typing import Dict, Any, List, Optional
|
| 12 |
+
from enum import Enum
|
| 13 |
+
from dataclasses import dataclass, field, asdict
|
| 14 |
+
from ..config import Config
|
| 15 |
+
|
| 16 |
+
|
| 17 |
+
class ProjectStatus(str, Enum):
|
| 18 |
+
"""้กน็ฎ็ถๆ"""
|
| 19 |
+
CREATED = "created" # ๅๅๅปบ๏ผๆไปถๅทฒไธไผ
|
| 20 |
+
ONTOLOGY_GENERATED = "ontology_generated" # ๆฌไฝๅทฒ็ๆ
|
| 21 |
+
GRAPH_BUILDING = "graph_building" # ๅพ่ฐฑๆๅปบไธญ
|
| 22 |
+
GRAPH_COMPLETED = "graph_completed" # ๅพ่ฐฑๆๅปบๅฎๆ
|
| 23 |
+
FAILED = "failed" # ๅคฑ่ดฅ
|
| 24 |
+
|
| 25 |
+
|
| 26 |
+
@dataclass
|
| 27 |
+
class Project:
|
| 28 |
+
"""้กน็ฎๆฐๆฎๆจกๅ"""
|
| 29 |
+
project_id: str
|
| 30 |
+
name: str
|
| 31 |
+
status: ProjectStatus
|
| 32 |
+
created_at: str
|
| 33 |
+
updated_at: str
|
| 34 |
+
|
| 35 |
+
# ๆไปถไฟกๆฏ
|
| 36 |
+
files: List[Dict[str, str]] = field(default_factory=list) # [{filename, path, size}]
|
| 37 |
+
total_text_length: int = 0
|
| 38 |
+
|
| 39 |
+
# ๆฌไฝไฟกๆฏ๏ผๆฅๅฃ1็ๆๅๅกซๅ
๏ผ
|
| 40 |
+
ontology: Optional[Dict[str, Any]] = None
|
| 41 |
+
analysis_summary: Optional[str] = None
|
| 42 |
+
|
| 43 |
+
# ๅพ่ฐฑไฟกๆฏ๏ผๆฅๅฃ2ๅฎๆๅๅกซๅ
๏ผ
|
| 44 |
+
graph_id: Optional[str] = None
|
| 45 |
+
graph_build_task_id: Optional[str] = None
|
| 46 |
+
|
| 47 |
+
# ้
็ฝฎ
|
| 48 |
+
simulation_requirement: Optional[str] = None
|
| 49 |
+
chunk_size: int = 500
|
| 50 |
+
chunk_overlap: int = 50
|
| 51 |
+
|
| 52 |
+
# ้่ฏฏไฟกๆฏ
|
| 53 |
+
error: Optional[str] = None
|
| 54 |
+
|
| 55 |
+
def to_dict(self) -> Dict[str, Any]:
|
| 56 |
+
"""่ฝฌๆขไธบๅญๅ
ธ"""
|
| 57 |
+
return {
|
| 58 |
+
"project_id": self.project_id,
|
| 59 |
+
"name": self.name,
|
| 60 |
+
"status": self.status.value if isinstance(self.status, ProjectStatus) else self.status,
|
| 61 |
+
"created_at": self.created_at,
|
| 62 |
+
"updated_at": self.updated_at,
|
| 63 |
+
"files": self.files,
|
| 64 |
+
"total_text_length": self.total_text_length,
|
| 65 |
+
"ontology": self.ontology,
|
| 66 |
+
"analysis_summary": self.analysis_summary,
|
| 67 |
+
"graph_id": self.graph_id,
|
| 68 |
+
"graph_build_task_id": self.graph_build_task_id,
|
| 69 |
+
"simulation_requirement": self.simulation_requirement,
|
| 70 |
+
"chunk_size": self.chunk_size,
|
| 71 |
+
"chunk_overlap": self.chunk_overlap,
|
| 72 |
+
"error": self.error
|
| 73 |
+
}
|
| 74 |
+
|
| 75 |
+
@classmethod
|
| 76 |
+
def from_dict(cls, data: Dict[str, Any]) -> 'Project':
|
| 77 |
+
"""ไปๅญๅ
ธๅๅปบ"""
|
| 78 |
+
status = data.get('status', 'created')
|
| 79 |
+
if isinstance(status, str):
|
| 80 |
+
status = ProjectStatus(status)
|
| 81 |
+
|
| 82 |
+
return cls(
|
| 83 |
+
project_id=data['project_id'],
|
| 84 |
+
name=data.get('name', 'Unnamed Project'),
|
| 85 |
+
status=status,
|
| 86 |
+
created_at=data.get('created_at', ''),
|
| 87 |
+
updated_at=data.get('updated_at', ''),
|
| 88 |
+
files=data.get('files', []),
|
| 89 |
+
total_text_length=data.get('total_text_length', 0),
|
| 90 |
+
ontology=data.get('ontology'),
|
| 91 |
+
analysis_summary=data.get('analysis_summary'),
|
| 92 |
+
graph_id=data.get('graph_id'),
|
| 93 |
+
graph_build_task_id=data.get('graph_build_task_id'),
|
| 94 |
+
simulation_requirement=data.get('simulation_requirement'),
|
| 95 |
+
chunk_size=data.get('chunk_size', 500),
|
| 96 |
+
chunk_overlap=data.get('chunk_overlap', 50),
|
| 97 |
+
error=data.get('error')
|
| 98 |
+
)
|
| 99 |
+
|
| 100 |
+
|
| 101 |
+
class ProjectManager:
|
| 102 |
+
"""้กน็ฎ็ฎก็ๅจ - ่ด่ดฃ้กน็ฎ็ๆไน
ๅๅญๅจๅๆฃ็ดข"""
|
| 103 |
+
|
| 104 |
+
# ้กน็ฎๅญๅจๆ น็ฎๅฝ
|
| 105 |
+
PROJECTS_DIR = os.path.join(Config.UPLOAD_FOLDER, 'projects')
|
| 106 |
+
|
| 107 |
+
@classmethod
|
| 108 |
+
def _ensure_projects_dir(cls):
|
| 109 |
+
"""็กฎไฟ้กน็ฎ็ฎๅฝๅญๅจ"""
|
| 110 |
+
os.makedirs(cls.PROJECTS_DIR, exist_ok=True)
|
| 111 |
+
|
| 112 |
+
@classmethod
|
| 113 |
+
def _get_project_dir(cls, project_id: str) -> str:
|
| 114 |
+
"""่ทๅ้กน็ฎ็ฎๅฝ่ทฏๅพ"""
|
| 115 |
+
return os.path.join(cls.PROJECTS_DIR, project_id)
|
| 116 |
+
|
| 117 |
+
@classmethod
|
| 118 |
+
def _get_project_meta_path(cls, project_id: str) -> str:
|
| 119 |
+
"""่ทๅ้กน็ฎๅ
ๆฐๆฎๆไปถ่ทฏๅพ"""
|
| 120 |
+
return os.path.join(cls._get_project_dir(project_id), 'project.json')
|
| 121 |
+
|
| 122 |
+
@classmethod
|
| 123 |
+
def _get_project_files_dir(cls, project_id: str) -> str:
|
| 124 |
+
"""่ทๅ้กน็ฎๆไปถๅญๅจ็ฎๅฝ"""
|
| 125 |
+
return os.path.join(cls._get_project_dir(project_id), 'files')
|
| 126 |
+
|
| 127 |
+
@classmethod
|
| 128 |
+
def _get_project_text_path(cls, project_id: str) -> str:
|
| 129 |
+
"""่ทๅ้กน็ฎๆๅๆๆฌๅญๅจ่ทฏๅพ"""
|
| 130 |
+
return os.path.join(cls._get_project_dir(project_id), 'extracted_text.txt')
|
| 131 |
+
|
| 132 |
+
@classmethod
|
| 133 |
+
def create_project(cls, name: str = "Unnamed Project") -> Project:
|
| 134 |
+
"""
|
| 135 |
+
ๅๅปบๆฐ้กน็ฎ
|
| 136 |
+
|
| 137 |
+
Args:
|
| 138 |
+
name: ้กน็ฎๅ็งฐ
|
| 139 |
+
|
| 140 |
+
Returns:
|
| 141 |
+
ๆฐๅๅปบ็Projectๅฏน่ฑก
|
| 142 |
+
"""
|
| 143 |
+
cls._ensure_projects_dir()
|
| 144 |
+
|
| 145 |
+
project_id = f"proj_{uuid.uuid4().hex[:12]}"
|
| 146 |
+
now = datetime.now().isoformat()
|
| 147 |
+
|
| 148 |
+
project = Project(
|
| 149 |
+
project_id=project_id,
|
| 150 |
+
name=name,
|
| 151 |
+
status=ProjectStatus.CREATED,
|
| 152 |
+
created_at=now,
|
| 153 |
+
updated_at=now
|
| 154 |
+
)
|
| 155 |
+
|
| 156 |
+
# ๅๅปบ้กน็ฎ็ฎๅฝ็ปๆ
|
| 157 |
+
project_dir = cls._get_project_dir(project_id)
|
| 158 |
+
files_dir = cls._get_project_files_dir(project_id)
|
| 159 |
+
os.makedirs(project_dir, exist_ok=True)
|
| 160 |
+
os.makedirs(files_dir, exist_ok=True)
|
| 161 |
+
|
| 162 |
+
# ไฟๅญ้กน็ฎๅ
ๆฐๆฎ
|
| 163 |
+
cls.save_project(project)
|
| 164 |
+
|
| 165 |
+
return project
|
| 166 |
+
|
| 167 |
+
@classmethod
|
| 168 |
+
def save_project(cls, project: Project) -> None:
|
| 169 |
+
"""ไฟๅญ้กน็ฎๅ
ๆฐๆฎ"""
|
| 170 |
+
project.updated_at = datetime.now().isoformat()
|
| 171 |
+
meta_path = cls._get_project_meta_path(project.project_id)
|
| 172 |
+
|
| 173 |
+
with open(meta_path, 'w', encoding='utf-8') as f:
|
| 174 |
+
json.dump(project.to_dict(), f, ensure_ascii=False, indent=2)
|
| 175 |
+
|
| 176 |
+
@classmethod
|
| 177 |
+
def get_project(cls, project_id: str) -> Optional[Project]:
|
| 178 |
+
"""
|
| 179 |
+
่ทๅ้กน็ฎ
|
| 180 |
+
|
| 181 |
+
Args:
|
| 182 |
+
project_id: ้กน็ฎID
|
| 183 |
+
|
| 184 |
+
Returns:
|
| 185 |
+
Projectๅฏน่ฑก๏ผๅฆๆไธๅญๅจ่ฟๅNone
|
| 186 |
+
"""
|
| 187 |
+
meta_path = cls._get_project_meta_path(project_id)
|
| 188 |
+
|
| 189 |
+
if not os.path.exists(meta_path):
|
| 190 |
+
return None
|
| 191 |
+
|
| 192 |
+
with open(meta_path, 'r', encoding='utf-8') as f:
|
| 193 |
+
data = json.load(f)
|
| 194 |
+
|
| 195 |
+
return Project.from_dict(data)
|
| 196 |
+
|
| 197 |
+
@classmethod
|
| 198 |
+
def list_projects(cls, limit: int = 50) -> List[Project]:
|
| 199 |
+
"""
|
| 200 |
+
ๅๅบๆๆ้กน็ฎ
|
| 201 |
+
|
| 202 |
+
Args:
|
| 203 |
+
limit: ่ฟๅๆฐ้้ๅถ
|
| 204 |
+
|
| 205 |
+
Returns:
|
| 206 |
+
้กน็ฎๅ่กจ๏ผๆๅๅปบๆถ้ดๅๅบ
|
| 207 |
+
"""
|
| 208 |
+
cls._ensure_projects_dir()
|
| 209 |
+
|
| 210 |
+
projects = []
|
| 211 |
+
for project_id in os.listdir(cls.PROJECTS_DIR):
|
| 212 |
+
project = cls.get_project(project_id)
|
| 213 |
+
if project:
|
| 214 |
+
projects.append(project)
|
| 215 |
+
|
| 216 |
+
# ๆๅๅปบๆถ้ดๅๅบๆๅบ
|
| 217 |
+
projects.sort(key=lambda p: p.created_at, reverse=True)
|
| 218 |
+
|
| 219 |
+
return projects[:limit]
|
| 220 |
+
|
| 221 |
+
@classmethod
|
| 222 |
+
def delete_project(cls, project_id: str) -> bool:
|
| 223 |
+
"""
|
| 224 |
+
ๅ ้ค้กน็ฎๅๅ
ถๆๆๆไปถ
|
| 225 |
+
|
| 226 |
+
Args:
|
| 227 |
+
project_id: ้กน็ฎID
|
| 228 |
+
|
| 229 |
+
Returns:
|
| 230 |
+
ๆฏๅฆๅ ้คๆๅ
|
| 231 |
+
"""
|
| 232 |
+
project_dir = cls._get_project_dir(project_id)
|
| 233 |
+
|
| 234 |
+
if not os.path.exists(project_dir):
|
| 235 |
+
return False
|
| 236 |
+
|
| 237 |
+
shutil.rmtree(project_dir)
|
| 238 |
+
return True
|
| 239 |
+
|
| 240 |
+
@classmethod
|
| 241 |
+
def save_file_to_project(cls, project_id: str, file_storage, original_filename: str) -> Dict[str, str]:
|
| 242 |
+
"""
|
| 243 |
+
ไฟๅญไธไผ ็ๆไปถๅฐ้กน็ฎ็ฎๅฝ
|
| 244 |
+
|
| 245 |
+
Args:
|
| 246 |
+
project_id: ้กน็ฎID
|
| 247 |
+
file_storage: Flask็FileStorageๅฏน่ฑก
|
| 248 |
+
original_filename: ๅๅงๆไปถๅ
|
| 249 |
+
|
| 250 |
+
Returns:
|
| 251 |
+
ๆไปถไฟกๆฏๅญๅ
ธ {filename, path, size}
|
| 252 |
+
"""
|
| 253 |
+
files_dir = cls._get_project_files_dir(project_id)
|
| 254 |
+
os.makedirs(files_dir, exist_ok=True)
|
| 255 |
+
|
| 256 |
+
# ็ๆๅฎๅ
จ็ๆไปถๅ
|
| 257 |
+
ext = os.path.splitext(original_filename)[1].lower()
|
| 258 |
+
safe_filename = f"{uuid.uuid4().hex[:8]}{ext}"
|
| 259 |
+
file_path = os.path.join(files_dir, safe_filename)
|
| 260 |
+
|
| 261 |
+
# ไฟๅญๆไปถ
|
| 262 |
+
file_storage.save(file_path)
|
| 263 |
+
|
| 264 |
+
# ่ทๅๆไปถๅคงๅฐ
|
| 265 |
+
file_size = os.path.getsize(file_path)
|
| 266 |
+
|
| 267 |
+
return {
|
| 268 |
+
"original_filename": original_filename,
|
| 269 |
+
"saved_filename": safe_filename,
|
| 270 |
+
"path": file_path,
|
| 271 |
+
"size": file_size
|
| 272 |
+
}
|
| 273 |
+
|
| 274 |
+
@classmethod
|
| 275 |
+
def save_extracted_text(cls, project_id: str, text: str) -> None:
|
| 276 |
+
"""ไฟๅญๆๅ็ๆๆฌ"""
|
| 277 |
+
text_path = cls._get_project_text_path(project_id)
|
| 278 |
+
with open(text_path, 'w', encoding='utf-8') as f:
|
| 279 |
+
f.write(text)
|
| 280 |
+
|
| 281 |
+
@classmethod
|
| 282 |
+
def get_extracted_text(cls, project_id: str) -> Optional[str]:
|
| 283 |
+
"""่ทๅๆๅ็ๆๆฌ"""
|
| 284 |
+
text_path = cls._get_project_text_path(project_id)
|
| 285 |
+
|
| 286 |
+
if not os.path.exists(text_path):
|
| 287 |
+
return None
|
| 288 |
+
|
| 289 |
+
with open(text_path, 'r', encoding='utf-8') as f:
|
| 290 |
+
return f.read()
|
| 291 |
+
|
| 292 |
+
@classmethod
|
| 293 |
+
def get_project_files(cls, project_id: str) -> List[str]:
|
| 294 |
+
"""่ทๅ้กน็ฎ็ๆๆๆไปถ่ทฏๅพ"""
|
| 295 |
+
files_dir = cls._get_project_files_dir(project_id)
|
| 296 |
+
|
| 297 |
+
if not os.path.exists(files_dir):
|
| 298 |
+
return []
|
| 299 |
+
|
| 300 |
+
return [
|
| 301 |
+
os.path.join(files_dir, f)
|
| 302 |
+
for f in os.listdir(files_dir)
|
| 303 |
+
if os.path.isfile(os.path.join(files_dir, f))
|
| 304 |
+
]
|
| 305 |
+
|
backend/app/models/task.py
ADDED
|
@@ -0,0 +1,178 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ไปปๅก็ถๆ็ฎก็
|
| 3 |
+
็จไบ่ท่ธช้ฟๆถ้ด่ฟ่ก็ไปปๅก๏ผๅฆๅพ่ฐฑๆๅปบ๏ผ
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import uuid
|
| 7 |
+
import threading
|
| 8 |
+
from datetime import datetime
|
| 9 |
+
from enum import Enum
|
| 10 |
+
from typing import Dict, Any, Optional
|
| 11 |
+
from dataclasses import dataclass, field
|
| 12 |
+
|
| 13 |
+
|
| 14 |
+
class TaskStatus(str, Enum):
|
| 15 |
+
"""ไปปๅก็ถๆๆไธพ"""
|
| 16 |
+
PENDING = "pending" # ็ญๅพ
ไธญ
|
| 17 |
+
PROCESSING = "processing" # ๅค็ไธญ
|
| 18 |
+
COMPLETED = "completed" # ๅทฒๅฎๆ
|
| 19 |
+
FAILED = "failed" # ๅคฑ่ดฅ
|
| 20 |
+
|
| 21 |
+
|
| 22 |
+
@dataclass
|
| 23 |
+
class Task:
|
| 24 |
+
"""ไปปๅกๆฐๆฎ็ฑป"""
|
| 25 |
+
task_id: str
|
| 26 |
+
task_type: str
|
| 27 |
+
status: TaskStatus
|
| 28 |
+
created_at: datetime
|
| 29 |
+
updated_at: datetime
|
| 30 |
+
progress: int = 0 # ่ฟๅบฆ็พๅๆฏ 0-100
|
| 31 |
+
message: str = "" # ็ถๆๆถๆฏ
|
| 32 |
+
result: Optional[Dict] = None # ไปปๅก็ปๆ
|
| 33 |
+
error: Optional[str] = None # ้่ฏฏไฟกๆฏ
|
| 34 |
+
metadata: Dict = field(default_factory=dict) # ้ขๅคๅ
ๆฐๆฎ
|
| 35 |
+
|
| 36 |
+
def to_dict(self) -> Dict[str, Any]:
|
| 37 |
+
"""่ฝฌๆขไธบๅญๅ
ธ"""
|
| 38 |
+
return {
|
| 39 |
+
"task_id": self.task_id,
|
| 40 |
+
"task_type": self.task_type,
|
| 41 |
+
"status": self.status.value,
|
| 42 |
+
"created_at": self.created_at.isoformat(),
|
| 43 |
+
"updated_at": self.updated_at.isoformat(),
|
| 44 |
+
"progress": self.progress,
|
| 45 |
+
"message": self.message,
|
| 46 |
+
"result": self.result,
|
| 47 |
+
"error": self.error,
|
| 48 |
+
"metadata": self.metadata,
|
| 49 |
+
}
|
| 50 |
+
|
| 51 |
+
|
| 52 |
+
class TaskManager:
|
| 53 |
+
"""
|
| 54 |
+
ไปปๅก็ฎก็ๅจ
|
| 55 |
+
็บฟ็จๅฎๅ
จ็ไปปๅก็ถๆ็ฎก็
|
| 56 |
+
"""
|
| 57 |
+
|
| 58 |
+
_instance = None
|
| 59 |
+
_lock = threading.Lock()
|
| 60 |
+
|
| 61 |
+
def __new__(cls):
|
| 62 |
+
"""ๅไพๆจกๅผ"""
|
| 63 |
+
if cls._instance is None:
|
| 64 |
+
with cls._lock:
|
| 65 |
+
if cls._instance is None:
|
| 66 |
+
cls._instance = super().__new__(cls)
|
| 67 |
+
cls._instance._tasks: Dict[str, Task] = {}
|
| 68 |
+
cls._instance._task_lock = threading.Lock()
|
| 69 |
+
return cls._instance
|
| 70 |
+
|
| 71 |
+
def create_task(self, task_type: str, metadata: Optional[Dict] = None) -> str:
|
| 72 |
+
"""
|
| 73 |
+
ๅๅปบๆฐไปปๅก
|
| 74 |
+
|
| 75 |
+
Args:
|
| 76 |
+
task_type: ไปปๅก็ฑปๅ
|
| 77 |
+
metadata: ้ขๅคๅ
ๆฐๆฎ
|
| 78 |
+
|
| 79 |
+
Returns:
|
| 80 |
+
ไปปๅกID
|
| 81 |
+
"""
|
| 82 |
+
task_id = str(uuid.uuid4())
|
| 83 |
+
now = datetime.now()
|
| 84 |
+
|
| 85 |
+
task = Task(
|
| 86 |
+
task_id=task_id,
|
| 87 |
+
task_type=task_type,
|
| 88 |
+
status=TaskStatus.PENDING,
|
| 89 |
+
created_at=now,
|
| 90 |
+
updated_at=now,
|
| 91 |
+
metadata=metadata or {}
|
| 92 |
+
)
|
| 93 |
+
|
| 94 |
+
with self._task_lock:
|
| 95 |
+
self._tasks[task_id] = task
|
| 96 |
+
|
| 97 |
+
return task_id
|
| 98 |
+
|
| 99 |
+
def get_task(self, task_id: str) -> Optional[Task]:
|
| 100 |
+
"""่ทๅไปปๅก"""
|
| 101 |
+
with self._task_lock:
|
| 102 |
+
return self._tasks.get(task_id)
|
| 103 |
+
|
| 104 |
+
def update_task(
|
| 105 |
+
self,
|
| 106 |
+
task_id: str,
|
| 107 |
+
status: Optional[TaskStatus] = None,
|
| 108 |
+
progress: Optional[int] = None,
|
| 109 |
+
message: Optional[str] = None,
|
| 110 |
+
result: Optional[Dict] = None,
|
| 111 |
+
error: Optional[str] = None
|
| 112 |
+
):
|
| 113 |
+
"""
|
| 114 |
+
ๆดๆฐไปปๅก็ถๆ
|
| 115 |
+
|
| 116 |
+
Args:
|
| 117 |
+
task_id: ไปปๅกID
|
| 118 |
+
status: ๆฐ็ถๆ
|
| 119 |
+
progress: ่ฟๅบฆ
|
| 120 |
+
message: ๆถๆฏ
|
| 121 |
+
result: ็ปๆ
|
| 122 |
+
error: ้่ฏฏไฟกๆฏ
|
| 123 |
+
"""
|
| 124 |
+
with self._task_lock:
|
| 125 |
+
task = self._tasks.get(task_id)
|
| 126 |
+
if task:
|
| 127 |
+
task.updated_at = datetime.now()
|
| 128 |
+
if status is not None:
|
| 129 |
+
task.status = status
|
| 130 |
+
if progress is not None:
|
| 131 |
+
task.progress = progress
|
| 132 |
+
if message is not None:
|
| 133 |
+
task.message = message
|
| 134 |
+
if result is not None:
|
| 135 |
+
task.result = result
|
| 136 |
+
if error is not None:
|
| 137 |
+
task.error = error
|
| 138 |
+
|
| 139 |
+
def complete_task(self, task_id: str, result: Dict):
|
| 140 |
+
"""ๆ ่ฎฐไปปๅกๅฎๆ"""
|
| 141 |
+
self.update_task(
|
| 142 |
+
task_id,
|
| 143 |
+
status=TaskStatus.COMPLETED,
|
| 144 |
+
progress=100,
|
| 145 |
+
message="ไปปๅกๅฎๆ",
|
| 146 |
+
result=result
|
| 147 |
+
)
|
| 148 |
+
|
| 149 |
+
def fail_task(self, task_id: str, error: str):
|
| 150 |
+
"""ๆ ่ฎฐไปปๅกๅคฑ่ดฅ"""
|
| 151 |
+
self.update_task(
|
| 152 |
+
task_id,
|
| 153 |
+
status=TaskStatus.FAILED,
|
| 154 |
+
message="ไปปๅกๅคฑ่ดฅ",
|
| 155 |
+
error=error
|
| 156 |
+
)
|
| 157 |
+
|
| 158 |
+
def list_tasks(self, task_type: Optional[str] = None) -> list:
|
| 159 |
+
"""ๅๅบไปปๅก"""
|
| 160 |
+
with self._task_lock:
|
| 161 |
+
tasks = list(self._tasks.values())
|
| 162 |
+
if task_type:
|
| 163 |
+
tasks = [t for t in tasks if t.task_type == task_type]
|
| 164 |
+
return [t.to_dict() for t in sorted(tasks, key=lambda x: x.created_at, reverse=True)]
|
| 165 |
+
|
| 166 |
+
def cleanup_old_tasks(self, max_age_hours: int = 24):
|
| 167 |
+
"""ๆธ
็ๆงไปปๅก"""
|
| 168 |
+
from datetime import timedelta
|
| 169 |
+
cutoff = datetime.now() - timedelta(hours=max_age_hours)
|
| 170 |
+
|
| 171 |
+
with self._task_lock:
|
| 172 |
+
old_ids = [
|
| 173 |
+
tid for tid, task in self._tasks.items()
|
| 174 |
+
if task.created_at < cutoff and task.status in [TaskStatus.COMPLETED, TaskStatus.FAILED]
|
| 175 |
+
]
|
| 176 |
+
for tid in old_ids:
|
| 177 |
+
del self._tasks[tid]
|
| 178 |
+
|
backend/app/services/__init__.py
ADDED
|
@@ -0,0 +1,10 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ไธๅกๆๅกๆจกๅ
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
from .ontology_generator import OntologyGenerator
|
| 6 |
+
from .graph_builder import GraphBuilderService
|
| 7 |
+
from .text_processor import TextProcessor
|
| 8 |
+
|
| 9 |
+
__all__ = ['OntologyGenerator', 'GraphBuilderService', 'TextProcessor']
|
| 10 |
+
|
backend/app/services/graph_builder.py
ADDED
|
@@ -0,0 +1,457 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๅพ่ฐฑๆๅปบๆๅก
|
| 3 |
+
ๆฅๅฃ2๏ผไฝฟ็จZep APIๆๅปบStandalone Graph
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import os
|
| 7 |
+
import uuid
|
| 8 |
+
import time
|
| 9 |
+
import threading
|
| 10 |
+
from typing import Dict, Any, List, Optional, Callable
|
| 11 |
+
from dataclasses import dataclass
|
| 12 |
+
|
| 13 |
+
from zep_cloud.client import Zep
|
| 14 |
+
from zep_cloud import EpisodeData, EntityEdgeSourceTarget
|
| 15 |
+
|
| 16 |
+
from ..config import Config
|
| 17 |
+
from ..models.task import TaskManager, TaskStatus
|
| 18 |
+
from .text_processor import TextProcessor
|
| 19 |
+
|
| 20 |
+
|
| 21 |
+
@dataclass
|
| 22 |
+
class GraphInfo:
|
| 23 |
+
"""ๅพ่ฐฑไฟกๆฏ"""
|
| 24 |
+
graph_id: str
|
| 25 |
+
node_count: int
|
| 26 |
+
edge_count: int
|
| 27 |
+
entity_types: List[str]
|
| 28 |
+
|
| 29 |
+
def to_dict(self) -> Dict[str, Any]:
|
| 30 |
+
return {
|
| 31 |
+
"graph_id": self.graph_id,
|
| 32 |
+
"node_count": self.node_count,
|
| 33 |
+
"edge_count": self.edge_count,
|
| 34 |
+
"entity_types": self.entity_types,
|
| 35 |
+
}
|
| 36 |
+
|
| 37 |
+
|
| 38 |
+
class GraphBuilderService:
|
| 39 |
+
"""
|
| 40 |
+
ๅพ่ฐฑๆๅปบๆๅก
|
| 41 |
+
่ด่ดฃ่ฐ็จZep APIๆๅปบ็ฅ่ฏๅพ่ฐฑ
|
| 42 |
+
"""
|
| 43 |
+
|
| 44 |
+
def __init__(self, api_key: Optional[str] = None):
|
| 45 |
+
self.api_key = api_key or Config.ZEP_API_KEY
|
| 46 |
+
if not self.api_key:
|
| 47 |
+
raise ValueError("ZEP_API_KEY ๆช้
็ฝฎ")
|
| 48 |
+
|
| 49 |
+
self.client = Zep(api_key=self.api_key)
|
| 50 |
+
self.task_manager = TaskManager()
|
| 51 |
+
|
| 52 |
+
def build_graph_async(
|
| 53 |
+
self,
|
| 54 |
+
text: str,
|
| 55 |
+
ontology: Dict[str, Any],
|
| 56 |
+
graph_name: str = "MiroFish Graph",
|
| 57 |
+
chunk_size: int = 500,
|
| 58 |
+
chunk_overlap: int = 50,
|
| 59 |
+
batch_size: int = 3
|
| 60 |
+
) -> str:
|
| 61 |
+
"""
|
| 62 |
+
ๅผๆญฅๆๅปบๅพ่ฐฑ
|
| 63 |
+
|
| 64 |
+
Args:
|
| 65 |
+
text: ่พๅ
ฅๆๆฌ
|
| 66 |
+
ontology: ๆฌไฝๅฎไน๏ผๆฅ่ชๆฅๅฃ1็่พๅบ๏ผ
|
| 67 |
+
graph_name: ๅพ่ฐฑๅ็งฐ
|
| 68 |
+
chunk_size: ๆๆฌๅๅคงๅฐ
|
| 69 |
+
chunk_overlap: ๅ้ๅ ๅคงๅฐ
|
| 70 |
+
batch_size: ๆฏๆนๅ้็ๅๆฐ้
|
| 71 |
+
|
| 72 |
+
Returns:
|
| 73 |
+
ไปปๅกID
|
| 74 |
+
"""
|
| 75 |
+
# ๅๅปบไปปๅก
|
| 76 |
+
task_id = self.task_manager.create_task(
|
| 77 |
+
task_type="graph_build",
|
| 78 |
+
metadata={
|
| 79 |
+
"graph_name": graph_name,
|
| 80 |
+
"chunk_size": chunk_size,
|
| 81 |
+
"text_length": len(text),
|
| 82 |
+
}
|
| 83 |
+
)
|
| 84 |
+
|
| 85 |
+
# ๅจๅๅฐ็บฟ็จไธญๆง่กๆๅปบ
|
| 86 |
+
thread = threading.Thread(
|
| 87 |
+
target=self._build_graph_worker,
|
| 88 |
+
args=(task_id, text, ontology, graph_name, chunk_size, chunk_overlap, batch_size)
|
| 89 |
+
)
|
| 90 |
+
thread.daemon = True
|
| 91 |
+
thread.start()
|
| 92 |
+
|
| 93 |
+
return task_id
|
| 94 |
+
|
| 95 |
+
def _build_graph_worker(
|
| 96 |
+
self,
|
| 97 |
+
task_id: str,
|
| 98 |
+
text: str,
|
| 99 |
+
ontology: Dict[str, Any],
|
| 100 |
+
graph_name: str,
|
| 101 |
+
chunk_size: int,
|
| 102 |
+
chunk_overlap: int,
|
| 103 |
+
batch_size: int
|
| 104 |
+
):
|
| 105 |
+
"""ๅพ่ฐฑๆๅปบๅทฅไฝ็บฟ็จ"""
|
| 106 |
+
try:
|
| 107 |
+
self.task_manager.update_task(
|
| 108 |
+
task_id,
|
| 109 |
+
status=TaskStatus.PROCESSING,
|
| 110 |
+
progress=5,
|
| 111 |
+
message="ๅผๅงๆๅปบๅพ่ฐฑ..."
|
| 112 |
+
)
|
| 113 |
+
|
| 114 |
+
# 1. ๅๅปบๅพ่ฐฑ
|
| 115 |
+
graph_id = self.create_graph(graph_name)
|
| 116 |
+
self.task_manager.update_task(
|
| 117 |
+
task_id,
|
| 118 |
+
progress=10,
|
| 119 |
+
message=f"ๅพ่ฐฑๅทฒๅๅปบ: {graph_id}"
|
| 120 |
+
)
|
| 121 |
+
|
| 122 |
+
# 2. ่ฎพ็ฝฎๆฌไฝ
|
| 123 |
+
self.set_ontology(graph_id, ontology)
|
| 124 |
+
self.task_manager.update_task(
|
| 125 |
+
task_id,
|
| 126 |
+
progress=15,
|
| 127 |
+
message="ๆฌไฝๅทฒ่ฎพ็ฝฎ"
|
| 128 |
+
)
|
| 129 |
+
|
| 130 |
+
# 3. ๆๆฌๅๅ
|
| 131 |
+
chunks = TextProcessor.split_text(text, chunk_size, chunk_overlap)
|
| 132 |
+
total_chunks = len(chunks)
|
| 133 |
+
self.task_manager.update_task(
|
| 134 |
+
task_id,
|
| 135 |
+
progress=20,
|
| 136 |
+
message=f"ๆๆฌๅทฒๅๅฒไธบ {total_chunks} ไธชๅ"
|
| 137 |
+
)
|
| 138 |
+
|
| 139 |
+
# 4. ๅๆนๅ้ๆฐๆฎ
|
| 140 |
+
episode_uuids = self.add_text_batches(
|
| 141 |
+
graph_id, chunks, batch_size,
|
| 142 |
+
lambda msg, prog: self.task_manager.update_task(
|
| 143 |
+
task_id,
|
| 144 |
+
progress=20 + int(prog * 0.4), # 20-60%
|
| 145 |
+
message=msg
|
| 146 |
+
)
|
| 147 |
+
)
|
| 148 |
+
|
| 149 |
+
# 5. ็ญๅพ
Zepๅค็ๅฎๆ
|
| 150 |
+
self.task_manager.update_task(
|
| 151 |
+
task_id,
|
| 152 |
+
progress=60,
|
| 153 |
+
message="็ญๅพ
Zepๅค็ๆฐๆฎ..."
|
| 154 |
+
)
|
| 155 |
+
|
| 156 |
+
self._wait_for_episodes(
|
| 157 |
+
episode_uuids,
|
| 158 |
+
lambda msg, prog: self.task_manager.update_task(
|
| 159 |
+
task_id,
|
| 160 |
+
progress=60 + int(prog * 0.3), # 60-90%
|
| 161 |
+
message=msg
|
| 162 |
+
)
|
| 163 |
+
)
|
| 164 |
+
|
| 165 |
+
# 6. ่ทๅๅพ่ฐฑไฟกๆฏ
|
| 166 |
+
self.task_manager.update_task(
|
| 167 |
+
task_id,
|
| 168 |
+
progress=90,
|
| 169 |
+
message="่ทๅๅพ่ฐฑไฟกๆฏ..."
|
| 170 |
+
)
|
| 171 |
+
|
| 172 |
+
graph_info = self._get_graph_info(graph_id)
|
| 173 |
+
|
| 174 |
+
# ๅฎๆ
|
| 175 |
+
self.task_manager.complete_task(task_id, {
|
| 176 |
+
"graph_id": graph_id,
|
| 177 |
+
"graph_info": graph_info.to_dict(),
|
| 178 |
+
"chunks_processed": total_chunks,
|
| 179 |
+
})
|
| 180 |
+
|
| 181 |
+
except Exception as e:
|
| 182 |
+
import traceback
|
| 183 |
+
error_msg = f"{str(e)}\n{traceback.format_exc()}"
|
| 184 |
+
self.task_manager.fail_task(task_id, error_msg)
|
| 185 |
+
|
| 186 |
+
def create_graph(self, name: str) -> str:
|
| 187 |
+
"""ๅๅปบZepๅพ่ฐฑ๏ผๅ
ฌๅผๆนๆณ๏ผ"""
|
| 188 |
+
graph_id = f"mirofish_{uuid.uuid4().hex[:16]}"
|
| 189 |
+
|
| 190 |
+
self.client.graph.create(
|
| 191 |
+
graph_id=graph_id,
|
| 192 |
+
name=name,
|
| 193 |
+
description="MiroFish Social Simulation Graph"
|
| 194 |
+
)
|
| 195 |
+
|
| 196 |
+
return graph_id
|
| 197 |
+
|
| 198 |
+
def set_ontology(self, graph_id: str, ontology: Dict[str, Any]):
|
| 199 |
+
"""่ฎพ็ฝฎๅพ่ฐฑๆฌไฝ๏ผๅ
ฌๅผๆนๆณ๏ผ"""
|
| 200 |
+
from typing import Optional
|
| 201 |
+
from pydantic import Field
|
| 202 |
+
from zep_cloud.external_clients.ontology import EntityModel, EntityText, EdgeModel
|
| 203 |
+
|
| 204 |
+
# Zep ไฟ็ๅ็งฐ๏ผไธ่ฝไฝไธบๅฑๆงๅ
|
| 205 |
+
RESERVED_NAMES = {'uuid', 'name', 'group_id', 'name_embedding', 'summary', 'created_at'}
|
| 206 |
+
|
| 207 |
+
def safe_attr_name(attr_name: str) -> str:
|
| 208 |
+
"""ๅฐไฟ็ๅ็งฐ่ฝฌๆขไธบๅฎๅ
จๅ็งฐ"""
|
| 209 |
+
if attr_name.lower() in RESERVED_NAMES:
|
| 210 |
+
return f"entity_{attr_name}"
|
| 211 |
+
return attr_name
|
| 212 |
+
|
| 213 |
+
# ๅจๆๅๅปบๅฎไฝ็ฑปๅ
|
| 214 |
+
entity_types = {}
|
| 215 |
+
for entity_def in ontology.get("entity_types", []):
|
| 216 |
+
name = entity_def["name"]
|
| 217 |
+
description = entity_def.get("description", f"A {name} entity.")
|
| 218 |
+
|
| 219 |
+
# ๅๅปบๅฑๆงๅญๅ
ธๅ็ฑปๅๆณจ่งฃ๏ผPydantic v2 ้่ฆ๏ผ
|
| 220 |
+
attrs = {"__doc__": description}
|
| 221 |
+
annotations = {}
|
| 222 |
+
|
| 223 |
+
for attr_def in entity_def.get("attributes", []):
|
| 224 |
+
attr_name = safe_attr_name(attr_def["name"]) # ไฝฟ็จๅฎๅ
จๅ็งฐ
|
| 225 |
+
attr_desc = attr_def.get("description", attr_name)
|
| 226 |
+
attrs[attr_name] = Field(description=attr_desc, default=None)
|
| 227 |
+
annotations[attr_name] = Optional[EntityText] # ็ฑปๅๆณจ่งฃ
|
| 228 |
+
|
| 229 |
+
attrs["__annotations__"] = annotations
|
| 230 |
+
|
| 231 |
+
# ๅจๆๅๅปบ็ฑป
|
| 232 |
+
entity_class = type(name, (EntityModel,), attrs)
|
| 233 |
+
entity_class.__doc__ = description
|
| 234 |
+
entity_types[name] = entity_class
|
| 235 |
+
|
| 236 |
+
# ๅจๆๅๅปบ่พน็ฑปๅ
|
| 237 |
+
edge_definitions = {}
|
| 238 |
+
for edge_def in ontology.get("edge_types", []):
|
| 239 |
+
name = edge_def["name"]
|
| 240 |
+
description = edge_def.get("description", f"A {name} relationship.")
|
| 241 |
+
|
| 242 |
+
# ๅๅปบๅฑๆงๅญๅ
ธๅ็ฑปๅๆณจ่งฃ
|
| 243 |
+
attrs = {"__doc__": description}
|
| 244 |
+
annotations = {}
|
| 245 |
+
|
| 246 |
+
for attr_def in edge_def.get("attributes", []):
|
| 247 |
+
attr_name = safe_attr_name(attr_def["name"]) # ไฝฟ็จๅฎๅ
จๅ็งฐ
|
| 248 |
+
attr_desc = attr_def.get("description", attr_name)
|
| 249 |
+
attrs[attr_name] = Field(description=attr_desc, default=None)
|
| 250 |
+
annotations[attr_name] = Optional[str] # ่พนๅฑๆง็จstr็ฑปๅ
|
| 251 |
+
|
| 252 |
+
attrs["__annotations__"] = annotations
|
| 253 |
+
|
| 254 |
+
# ๅจๆๅๅปบ็ฑป
|
| 255 |
+
class_name = ''.join(word.capitalize() for word in name.split('_'))
|
| 256 |
+
edge_class = type(class_name, (EdgeModel,), attrs)
|
| 257 |
+
edge_class.__doc__ = description
|
| 258 |
+
|
| 259 |
+
# ๆๅปบsource_targets
|
| 260 |
+
source_targets = []
|
| 261 |
+
for st in edge_def.get("source_targets", []):
|
| 262 |
+
source_targets.append(
|
| 263 |
+
EntityEdgeSourceTarget(
|
| 264 |
+
source=st.get("source", "Entity"),
|
| 265 |
+
target=st.get("target", "Entity")
|
| 266 |
+
)
|
| 267 |
+
)
|
| 268 |
+
|
| 269 |
+
if source_targets:
|
| 270 |
+
edge_definitions[name] = (edge_class, source_targets)
|
| 271 |
+
|
| 272 |
+
# ่ฐ็จZep API่ฎพ็ฝฎๆฌไฝ
|
| 273 |
+
if entity_types or edge_definitions:
|
| 274 |
+
self.client.graph.set_ontology(
|
| 275 |
+
graph_ids=[graph_id],
|
| 276 |
+
entities=entity_types if entity_types else None,
|
| 277 |
+
edges=edge_definitions if edge_definitions else None,
|
| 278 |
+
)
|
| 279 |
+
|
| 280 |
+
def add_text_batches(
|
| 281 |
+
self,
|
| 282 |
+
graph_id: str,
|
| 283 |
+
chunks: List[str],
|
| 284 |
+
batch_size: int = 3,
|
| 285 |
+
progress_callback: Optional[Callable] = None
|
| 286 |
+
) -> List[str]:
|
| 287 |
+
"""ๅๆนๆทปๅ ๆๆฌๅฐๅพ่ฐฑ๏ผ่ฟๅๆๆ episode ็ uuid ๅ่กจ"""
|
| 288 |
+
episode_uuids = []
|
| 289 |
+
total_chunks = len(chunks)
|
| 290 |
+
|
| 291 |
+
for i in range(0, total_chunks, batch_size):
|
| 292 |
+
batch_chunks = chunks[i:i + batch_size]
|
| 293 |
+
batch_num = i // batch_size + 1
|
| 294 |
+
total_batches = (total_chunks + batch_size - 1) // batch_size
|
| 295 |
+
|
| 296 |
+
if progress_callback:
|
| 297 |
+
progress = (i + len(batch_chunks)) / total_chunks
|
| 298 |
+
progress_callback(
|
| 299 |
+
f"ๅ้็ฌฌ {batch_num}/{total_batches} ๆนๆฐๆฎ ({len(batch_chunks)} ๅ)...",
|
| 300 |
+
progress
|
| 301 |
+
)
|
| 302 |
+
|
| 303 |
+
# ๆๅปบepisodeๆฐๆฎ
|
| 304 |
+
episodes = [
|
| 305 |
+
EpisodeData(data=chunk, type="text")
|
| 306 |
+
for chunk in batch_chunks
|
| 307 |
+
]
|
| 308 |
+
|
| 309 |
+
# ๅ้ๅฐZep
|
| 310 |
+
try:
|
| 311 |
+
batch_result = self.client.graph.add_batch(
|
| 312 |
+
graph_id=graph_id,
|
| 313 |
+
episodes=episodes
|
| 314 |
+
)
|
| 315 |
+
|
| 316 |
+
# ๆถ้่ฟๅ็ episode uuid
|
| 317 |
+
if batch_result and isinstance(batch_result, list):
|
| 318 |
+
for ep in batch_result:
|
| 319 |
+
ep_uuid = getattr(ep, 'uuid_', None) or getattr(ep, 'uuid', None)
|
| 320 |
+
if ep_uuid:
|
| 321 |
+
episode_uuids.append(ep_uuid)
|
| 322 |
+
|
| 323 |
+
# ้ฟๅ
่ฏทๆฑ่ฟๅฟซ
|
| 324 |
+
time.sleep(1)
|
| 325 |
+
|
| 326 |
+
except Exception as e:
|
| 327 |
+
if progress_callback:
|
| 328 |
+
progress_callback(f"ๆนๆฌก {batch_num} ๅ้ๅคฑ่ดฅ: {str(e)}", 0)
|
| 329 |
+
raise
|
| 330 |
+
|
| 331 |
+
return episode_uuids
|
| 332 |
+
|
| 333 |
+
def _wait_for_episodes(
|
| 334 |
+
self,
|
| 335 |
+
episode_uuids: List[str],
|
| 336 |
+
progress_callback: Optional[Callable] = None,
|
| 337 |
+
timeout: int = 600
|
| 338 |
+
):
|
| 339 |
+
"""็ญๅพ
ๆๆ episode ๅค็ๅฎๆ๏ผ้่ฟๆฅ่ฏขๆฏไธช episode ็ processed ็ถๆ๏ผ"""
|
| 340 |
+
if not episode_uuids:
|
| 341 |
+
if progress_callback:
|
| 342 |
+
progress_callback("ๆ ้็ญๅพ
๏ผๆฒกๆ episode๏ผ", 1.0)
|
| 343 |
+
return
|
| 344 |
+
|
| 345 |
+
start_time = time.time()
|
| 346 |
+
pending_episodes = set(episode_uuids)
|
| 347 |
+
completed_count = 0
|
| 348 |
+
total_episodes = len(episode_uuids)
|
| 349 |
+
|
| 350 |
+
if progress_callback:
|
| 351 |
+
progress_callback(f"ๅผๅง็ญๅพ
{total_episodes} ไธชๆๆฌๅๅค็...", 0)
|
| 352 |
+
|
| 353 |
+
while pending_episodes:
|
| 354 |
+
if time.time() - start_time > timeout:
|
| 355 |
+
if progress_callback:
|
| 356 |
+
progress_callback(
|
| 357 |
+
f"้จๅๆๆฌๅ่ถ
ๆถ๏ผๅทฒๅฎๆ {completed_count}/{total_episodes}",
|
| 358 |
+
completed_count / total_episodes
|
| 359 |
+
)
|
| 360 |
+
break
|
| 361 |
+
|
| 362 |
+
# ๆฃๆฅๆฏไธช episode ็ๅค็็ถๆ
|
| 363 |
+
for ep_uuid in list(pending_episodes):
|
| 364 |
+
try:
|
| 365 |
+
episode = self.client.graph.episode.get(uuid_=ep_uuid)
|
| 366 |
+
is_processed = getattr(episode, 'processed', False)
|
| 367 |
+
|
| 368 |
+
if is_processed:
|
| 369 |
+
pending_episodes.remove(ep_uuid)
|
| 370 |
+
completed_count += 1
|
| 371 |
+
|
| 372 |
+
except Exception as e:
|
| 373 |
+
# ๅฟฝ็ฅๅไธชๆฅ่ฏข้่ฏฏ๏ผ็ปง็ปญ
|
| 374 |
+
pass
|
| 375 |
+
|
| 376 |
+
elapsed = int(time.time() - start_time)
|
| 377 |
+
if progress_callback:
|
| 378 |
+
progress_callback(
|
| 379 |
+
f"Zepๅค็ไธญ... {completed_count}/{total_episodes} ๅฎๆ, {len(pending_episodes)} ๅพ
ๅค็ ({elapsed}็ง)",
|
| 380 |
+
completed_count / total_episodes if total_episodes > 0 else 0
|
| 381 |
+
)
|
| 382 |
+
|
| 383 |
+
if pending_episodes:
|
| 384 |
+
time.sleep(3) # ๆฏ3็งๆฃๆฅไธๆฌก
|
| 385 |
+
|
| 386 |
+
if progress_callback:
|
| 387 |
+
progress_callback(f"ๅค็ๅฎๆ: {completed_count}/{total_episodes}", 1.0)
|
| 388 |
+
|
| 389 |
+
def _get_graph_info(self, graph_id: str) -> GraphInfo:
|
| 390 |
+
"""่ทๅๅพ่ฐฑไฟกๆฏ"""
|
| 391 |
+
# ่ทๅ่็น
|
| 392 |
+
nodes = self.client.graph.node.get_by_graph_id(graph_id=graph_id)
|
| 393 |
+
|
| 394 |
+
# ่ทๅ่พน
|
| 395 |
+
edges = self.client.graph.edge.get_by_graph_id(graph_id=graph_id)
|
| 396 |
+
|
| 397 |
+
# ็ป่ฎกๅฎไฝ็ฑปๅ
|
| 398 |
+
entity_types = set()
|
| 399 |
+
for node in nodes:
|
| 400 |
+
if node.labels:
|
| 401 |
+
for label in node.labels:
|
| 402 |
+
if label not in ["Entity", "Node"]:
|
| 403 |
+
entity_types.add(label)
|
| 404 |
+
|
| 405 |
+
return GraphInfo(
|
| 406 |
+
graph_id=graph_id,
|
| 407 |
+
node_count=len(nodes),
|
| 408 |
+
edge_count=len(edges),
|
| 409 |
+
entity_types=list(entity_types)
|
| 410 |
+
)
|
| 411 |
+
|
| 412 |
+
def get_graph_data(self, graph_id: str) -> Dict[str, Any]:
|
| 413 |
+
"""
|
| 414 |
+
่ทๅๅฎๆดๅพ่ฐฑๆฐๆฎ
|
| 415 |
+
|
| 416 |
+
Args:
|
| 417 |
+
graph_id: ๅพ่ฐฑID
|
| 418 |
+
|
| 419 |
+
Returns:
|
| 420 |
+
ๅ
ๅซnodesๅedges็ๅญๅ
ธ
|
| 421 |
+
"""
|
| 422 |
+
nodes = self.client.graph.node.get_by_graph_id(graph_id=graph_id)
|
| 423 |
+
edges = self.client.graph.edge.get_by_graph_id(graph_id=graph_id)
|
| 424 |
+
|
| 425 |
+
nodes_data = []
|
| 426 |
+
for node in nodes:
|
| 427 |
+
nodes_data.append({
|
| 428 |
+
"uuid": node.uuid_,
|
| 429 |
+
"name": node.name,
|
| 430 |
+
"labels": node.labels or [],
|
| 431 |
+
"summary": node.summary or "",
|
| 432 |
+
"attributes": node.attributes or {},
|
| 433 |
+
})
|
| 434 |
+
|
| 435 |
+
edges_data = []
|
| 436 |
+
for edge in edges:
|
| 437 |
+
edges_data.append({
|
| 438 |
+
"uuid": edge.uuid_,
|
| 439 |
+
"name": edge.name or "",
|
| 440 |
+
"fact": edge.fact or "",
|
| 441 |
+
"source_node_uuid": edge.source_node_uuid,
|
| 442 |
+
"target_node_uuid": edge.target_node_uuid,
|
| 443 |
+
"attributes": edge.attributes or {},
|
| 444 |
+
})
|
| 445 |
+
|
| 446 |
+
return {
|
| 447 |
+
"graph_id": graph_id,
|
| 448 |
+
"nodes": nodes_data,
|
| 449 |
+
"edges": edges_data,
|
| 450 |
+
"node_count": len(nodes_data),
|
| 451 |
+
"edge_count": len(edges_data),
|
| 452 |
+
}
|
| 453 |
+
|
| 454 |
+
def delete_graph(self, graph_id: str):
|
| 455 |
+
"""ๅ ้คๅพ่ฐฑ"""
|
| 456 |
+
self.client.graph.delete(graph_id=graph_id)
|
| 457 |
+
|
backend/app/services/ontology_generator.py
ADDED
|
@@ -0,0 +1,361 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๆฌไฝ็ๆๆๅก
|
| 3 |
+
ๆฅๅฃ1๏ผๅๆๆๆฌๅ
ๅฎน๏ผ็ๆ้ๅ็คพไผๆจกๆ็ๅฎไฝๅๅ
ณ็ณป็ฑปๅๅฎไน
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import json
|
| 7 |
+
from typing import Dict, Any, List, Optional
|
| 8 |
+
from ..utils.llm_client import LLMClient
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
# ๆฌไฝ็ๆ็็ณป็ปๆ็คบ่ฏ
|
| 12 |
+
ONTOLOGY_SYSTEM_PROMPT = """ไฝ ๆฏไธไธชไธไธ็็ฅ่ฏๅพ่ฐฑๆฌไฝ่ฎพ่ฎกไธๅฎถใไฝ ็ไปปๅกๆฏๅๆ็ปๅฎ็ๆๆฌๅ
ๅฎนๅๆจกๆ้ๆฑ๏ผ่ฎพ่ฎก้ๅ**็คพไบคๅชไฝ่่ฎบๆจกๆ**็ๅฎไฝ็ฑปๅๅๅ
ณ็ณป็ฑปๅใ
|
| 13 |
+
|
| 14 |
+
**้่ฆ๏ผไฝ ๅฟ
้กป่พๅบๆๆ็JSONๆ ผๅผๆฐๆฎ๏ผไธ่ฆ่พๅบไปปไฝๅ
ถไปๅ
ๅฎนใ**
|
| 15 |
+
|
| 16 |
+
## ๆ ธๅฟไปปๅก่ๆฏ
|
| 17 |
+
|
| 18 |
+
ๆไปฌๆญฃๅจๆๅปบไธไธช**็คพไบคๅชไฝ่่ฎบๆจกๆ็ณป็ป**ใๅจ่ฟไธช็ณป็ปไธญ๏ผ
|
| 19 |
+
- ๆฏไธชๅฎไฝ้ฝๆฏไธไธชๅฏไปฅๅจ็คพไบคๅชไฝไธๅๅฃฐใไบๅจใไผ ๆญไฟกๆฏ็"่ดฆๅท"ๆ"ไธปไฝ"
|
| 20 |
+
- ๅฎไฝไน้ดไผ็ธไบๅฝฑๅใ่ฝฌๅใ่ฏ่ฎบใๅๅบ
|
| 21 |
+
- ๆไปฌ้่ฆๆจกๆ่่ฎบไบไปถไธญๅๆน็ๅๅบๅไฟกๆฏไผ ๆญ่ทฏๅพ
|
| 22 |
+
|
| 23 |
+
ๅ ๆญค๏ผ**ๅฎไฝๅฟ
้กปๆฏ็ฐๅฎไธญ็ๅฎๅญๅจ็ใๅฏไปฅๅจ็คพๅชไธๅๅฃฐๅไบๅจ็ไธปไฝ**๏ผ
|
| 24 |
+
|
| 25 |
+
**ๅฏไปฅๆฏ๏ผ้ผๅฑๅคๆ ทๅๅๅ๏ผ**๏ผ
|
| 26 |
+
- ๅ
ทไฝ็ไธชไบบ๏ผๅ
ฌไผไบบ็ฉใๅฝไบไบบใๆ่ง้ข่ขใไธๅฎถๅญฆ่
๏ผ
|
| 27 |
+
- ๅ
ฌๅธใไผไธ๏ผๅ
ๆฌๅ
ถๅฎๆน่ดฆๅท๏ผ
|
| 28 |
+
- ็ป็ปๆบๆ๏ผๅคงๅญฆใๅไผใNGOใๅทฅไผ็ญ๏ผ
|
| 29 |
+
- ๆฟๅบ้จ้จใ็็ฎกๆบๆ
|
| 30 |
+
- ๅชไฝๆบๆ๏ผๆฅ็บธใ็ต่งๅฐใ่ชๅชไฝใ็ฝ็ซ๏ผ
|
| 31 |
+
- ็คพไบคๅชไฝๅนณๅฐๆฌ่บซ
|
| 32 |
+
- ็นๅฎ็พคไฝไปฃ่กจ๏ผๅฆๆ กๅไผใ็ฒไธๅขใ็ปดๆ็พคไฝ็ญ๏ผ
|
| 33 |
+
|
| 34 |
+
**ไธๅฏไปฅๆฏ**๏ผ
|
| 35 |
+
- ๆฝ่ฑกๆฆๅฟต๏ผๅฆ"่่ฎบ"ใ"ๆ
็ปช"ใ"่ถๅฟ"๏ผ
|
| 36 |
+
- ไธป้ข/่ฏ้ข๏ผๅฆ"ๅญฆๆฏ่ฏไฟก"ใ"ๆ่ฒๆน้ฉ"๏ผ
|
| 37 |
+
- ่ง็น/ๆๅบฆ๏ผๅฆ"ๆฏๆๆน"ใ"ๅๅฏนๆน"๏ผ
|
| 38 |
+
- ๆณๆ็พคไฝ๏ผๅฆ"็ฝๅ"ใ"ๅ
ฌไผ"ใ"ๅญฆ็็พคไฝ"๏ผ
|
| 39 |
+
|
| 40 |
+
## ่พๅบๆ ผๅผ
|
| 41 |
+
|
| 42 |
+
่ฏท่พๅบJSONๆ ผๅผ๏ผๅ
ๅซไปฅไธ็ปๆ๏ผ
|
| 43 |
+
|
| 44 |
+
```json
|
| 45 |
+
{
|
| 46 |
+
"entity_types": [
|
| 47 |
+
{
|
| 48 |
+
"name": "ๅฎไฝ็ฑปๅๅ็งฐ๏ผ่ฑๆ๏ผPascalCase๏ผ",
|
| 49 |
+
"description": "็ฎ็ญๆ่ฟฐ๏ผ่ฑๆ๏ผไธ่ถ
่ฟ100ๅญ็ฌฆ๏ผ",
|
| 50 |
+
"attributes": [
|
| 51 |
+
{
|
| 52 |
+
"name": "ๅฑๆงๅ๏ผ่ฑๆ๏ผsnake_case๏ผ",
|
| 53 |
+
"type": "text",
|
| 54 |
+
"description": "ๅฑๆงๆ่ฟฐ"
|
| 55 |
+
}
|
| 56 |
+
],
|
| 57 |
+
"examples": ["็คบไพๅฎไฝ1", "็คบไพๅฎไฝ2"]
|
| 58 |
+
}
|
| 59 |
+
],
|
| 60 |
+
"edge_types": [
|
| 61 |
+
{
|
| 62 |
+
"name": "ๅ
ณ็ณป็ฑปๅๅ็งฐ๏ผ่ฑๆ๏ผUPPER_SNAKE_CASE๏ผ",
|
| 63 |
+
"description": "็ฎ็ญๆ่ฟฐ๏ผ่ฑๆ๏ผไธ่ถ
่ฟ100ๅญ็ฌฆ๏ผ",
|
| 64 |
+
"source_targets": [
|
| 65 |
+
{"source": "ๆบๅฎไฝ็ฑปๅ", "target": "็ฎๆ ๅฎไฝ็ฑปๅ"}
|
| 66 |
+
],
|
| 67 |
+
"attributes": []
|
| 68 |
+
}
|
| 69 |
+
],
|
| 70 |
+
"analysis_summary": "ๅฏนๆๆฌๅ
ๅฎน็็ฎ่ฆๅๆ่ฏดๆ๏ผไธญๆ๏ผ"
|
| 71 |
+
}
|
| 72 |
+
```
|
| 73 |
+
|
| 74 |
+
## ่ฎพ่ฎกๆๅ
|
| 75 |
+
|
| 76 |
+
1. **ๅฎไฝ็ฑปๅ่ฎพ่ฎก๏ผ้่ฆ๏ผ่ฏทๅฐฝ้ๅคๅๅ๏ผ**๏ผ
|
| 77 |
+
- **ๆฐ้่ฆๆฑ๏ผ่ณๅฐ5ไธช๏ผๆๅค10ไธชๅฎไฝ็ฑปๅ**
|
| 78 |
+
- ๆฏไธชๅฎไฝ็ฑปๅไปฃ่กจไธ็ฑปๅฏไปฅๅจ็คพๅชไธๅๅฃฐ็ไธปไฝ
|
| 79 |
+
- ๅฐฝ้็ปๅไธๅ่ง่ฒ๏ผไพๅฆ๏ผ
|
| 80 |
+
- ไธ่ฆๅช็จ"Person"๏ผๅฏไปฅ็ปๅไธบ"PublicFigure"ใ"Expert"ใ"Whistleblower"็ญ
|
| 81 |
+
- ไธ่ฆๅช็จ"Organization"๏ผๅฏไปฅ็ปๅไธบ"University"ใ"Company"ใ"NGO"็ญ
|
| 82 |
+
- descriptionๅฟ
้กปๆธ
ๆฐ่ฏดๆไปไนๆ ท็ๅฎไฝๅบ่ฏฅ่ขซๆๅ
|
| 83 |
+
- ๆฏไธช็ฑปๅๆไพ2-3ไธชๅ
ทไฝ็คบไพ
|
| 84 |
+
|
| 85 |
+
2. **ๅ
ณ็ณป็ฑปๅ่ฎพ่ฎก**๏ผ
|
| 86 |
+
- ๅ
ณ็ณปๅบ่ฏฅๅๆ ็คพๅชไบๅจไธญ็็ๅฎ่็ณป
|
| 87 |
+
- ๅ
ณๆณจๅฏ่ฝๅฝฑๅ่่ฎบไผ ๆญ็ๅ
ณ็ณป๏ผ
|
| 88 |
+
- ไฟกๆฏไผ ๆญ๏ผREPORTS_ON, COMMENTS_ON, SHARES
|
| 89 |
+
- ็ป็ปๅ
ณ็ณป๏ผAFFILIATED_WITH, WORKS_FOR, REPRESENTS
|
| 90 |
+
- ไบๅจๅ
ณ็ณป๏ผRESPONDS_TO, SUPPORTS, OPPOSES
|
| 91 |
+
- ๅ
ณ็ณป็ฑปๅ๏ผ5-10ไธชไธบๅฎ
|
| 92 |
+
|
| 93 |
+
3. **ๅฑๆง่ฎพ่ฎก**๏ผ
|
| 94 |
+
- ๆฏไธชๅฎไฝ็ฑปๅ1-3ไธชๅ
ณ้ฎๅฑๆง
|
| 95 |
+
- ๅฑๆงๅบๆๅฉไบ่ฏๅซๅฎไฝ็็คพๅชๅฝฑๅๅ๏ผๅฆroleใinfluence_level็ญ๏ผ
|
| 96 |
+
|
| 97 |
+
## ๅฎไฝ็ฑปๅๅ่๏ผ่ฏทๆ นๆฎๆๆฌๅ
ๅฎน็ตๆดป้ๆฉๅๆฉๅฑ๏ผ
|
| 98 |
+
|
| 99 |
+
- Person: ๆฎ้ไธชไบบ
|
| 100 |
+
- PublicFigure: ๅ
ฌไผไบบ็ฉ๏ผๆๆใ็ฝ็บขใๆ่ง้ข่ข๏ผ
|
| 101 |
+
- Expert: ไธๅฎถๅญฆ่
|
| 102 |
+
- Journalist: ่ฎฐ่
|
| 103 |
+
- Company: ๅ
ฌๅธไผไธ
|
| 104 |
+
- University: ้ซๆ ก
|
| 105 |
+
- GovernmentAgency: ๆฟๅบๆบๆ
|
| 106 |
+
- MediaOutlet: ไผ ็ปๅชไฝ
|
| 107 |
+
- SelfMedia: ่ชๅชไฝ่ดฆๅท
|
| 108 |
+
- SocialPlatform: ็คพไบคๅชไฝๅนณๅฐ
|
| 109 |
+
- NGO: ้ๆฟๅบ็ป็ป
|
| 110 |
+
- IndustryAssociation: ่กไธๅไผ
|
| 111 |
+
- AlumniAssociation: ๆ กๅไผ
|
| 112 |
+
- FanGroup: ็ฒไธ็พคไฝ/ๆฏๆ็พคไฝ
|
| 113 |
+
|
| 114 |
+
## ๅ
ณ็ณป็ฑปๅๅ่
|
| 115 |
+
|
| 116 |
+
- WORKS_FOR: ๅทฅไฝไบ
|
| 117 |
+
- AFFILIATED_WITH: ้ถๅฑไบ
|
| 118 |
+
- REPRESENTS: ไปฃ่กจ
|
| 119 |
+
- REGULATES: ็็ฎก
|
| 120 |
+
- REPORTS_ON: ๆฅ้
|
| 121 |
+
- COMMENTS_ON: ่ฏ่ฎบ
|
| 122 |
+
- RESPONDS_TO: ๅๅบ
|
| 123 |
+
- SUPPORTS: ๆฏๆ
|
| 124 |
+
- OPPOSES: ๅๅฏน
|
| 125 |
+
- COLLABORATES_WITH: ๅไฝ
|
| 126 |
+
- COMPETES_WITH: ็ซไบ
|
| 127 |
+
"""
|
| 128 |
+
|
| 129 |
+
|
| 130 |
+
class OntologyGenerator:
|
| 131 |
+
"""
|
| 132 |
+
ๆฌไฝ็ๆๅจ
|
| 133 |
+
ๅๆๆๆฌๅ
ๅฎน๏ผ็ๆๅฎไฝๅๅ
ณ็ณป็ฑปๅๅฎไน
|
| 134 |
+
"""
|
| 135 |
+
|
| 136 |
+
def __init__(self, llm_client: Optional[LLMClient] = None):
|
| 137 |
+
self.llm_client = llm_client or LLMClient()
|
| 138 |
+
|
| 139 |
+
def generate(
|
| 140 |
+
self,
|
| 141 |
+
document_texts: List[str],
|
| 142 |
+
simulation_requirement: str,
|
| 143 |
+
additional_context: Optional[str] = None
|
| 144 |
+
) -> Dict[str, Any]:
|
| 145 |
+
"""
|
| 146 |
+
็ๆๆฌไฝๅฎไน
|
| 147 |
+
|
| 148 |
+
Args:
|
| 149 |
+
document_texts: ๆๆกฃๆๆฌๅ่กจ
|
| 150 |
+
simulation_requirement: ๆจกๆ้ๆฑๆ่ฟฐ
|
| 151 |
+
additional_context: ้ขๅคไธไธๆ
|
| 152 |
+
|
| 153 |
+
Returns:
|
| 154 |
+
ๆฌไฝๅฎไน๏ผentity_types, edge_types็ญ๏ผ
|
| 155 |
+
"""
|
| 156 |
+
# ๆๅปบ็จๆทๆถๆฏ
|
| 157 |
+
user_message = self._build_user_message(
|
| 158 |
+
document_texts,
|
| 159 |
+
simulation_requirement,
|
| 160 |
+
additional_context
|
| 161 |
+
)
|
| 162 |
+
|
| 163 |
+
messages = [
|
| 164 |
+
{"role": "system", "content": ONTOLOGY_SYSTEM_PROMPT},
|
| 165 |
+
{"role": "user", "content": user_message}
|
| 166 |
+
]
|
| 167 |
+
|
| 168 |
+
# ่ฐ็จLLM
|
| 169 |
+
result = self.llm_client.chat_json(
|
| 170 |
+
messages=messages,
|
| 171 |
+
temperature=0.3,
|
| 172 |
+
max_tokens=4096
|
| 173 |
+
)
|
| 174 |
+
|
| 175 |
+
# ้ช่ฏๅๅๅค็
|
| 176 |
+
result = self._validate_and_process(result)
|
| 177 |
+
|
| 178 |
+
return result
|
| 179 |
+
|
| 180 |
+
# ไผ ็ป LLM ็ๆๆฌๆๅคง้ฟๅบฆ๏ผ5ไธๅญ๏ผ
|
| 181 |
+
MAX_TEXT_LENGTH_FOR_LLM = 50000
|
| 182 |
+
|
| 183 |
+
def _build_user_message(
|
| 184 |
+
self,
|
| 185 |
+
document_texts: List[str],
|
| 186 |
+
simulation_requirement: str,
|
| 187 |
+
additional_context: Optional[str]
|
| 188 |
+
) -> str:
|
| 189 |
+
"""ๆๅปบ็จๆทๆถๆฏ"""
|
| 190 |
+
|
| 191 |
+
# ๅๅนถๆๆฌ
|
| 192 |
+
combined_text = "\n\n---\n\n".join(document_texts)
|
| 193 |
+
original_length = len(combined_text)
|
| 194 |
+
|
| 195 |
+
# ๅฆๆๆๆฌ่ถ
่ฟ5ไธๅญ๏ผๆชๆญ๏ผไป
ๅฝฑๅไผ ็ปLLM็ๅ
ๅฎน๏ผไธๅฝฑๅๅพ่ฐฑๆๅปบ๏ผ
|
| 196 |
+
if len(combined_text) > self.MAX_TEXT_LENGTH_FOR_LLM:
|
| 197 |
+
combined_text = combined_text[:self.MAX_TEXT_LENGTH_FOR_LLM]
|
| 198 |
+
combined_text += f"\n\n...(ๅๆๅ
ฑ{original_length}ๅญ๏ผๅทฒๆชๅๅ{self.MAX_TEXT_LENGTH_FOR_LLM}ๅญ็จไบๆฌไฝๅๆ)..."
|
| 199 |
+
|
| 200 |
+
message = f"""## ๆจกๆ้ๆฑ
|
| 201 |
+
|
| 202 |
+
{simulation_requirement}
|
| 203 |
+
|
| 204 |
+
## ๆๆกฃๅ
ๅฎน
|
| 205 |
+
|
| 206 |
+
{combined_text}
|
| 207 |
+
"""
|
| 208 |
+
|
| 209 |
+
if additional_context:
|
| 210 |
+
message += f"""
|
| 211 |
+
## ้ขๅค่ฏดๆ
|
| 212 |
+
|
| 213 |
+
{additional_context}
|
| 214 |
+
"""
|
| 215 |
+
|
| 216 |
+
message += """
|
| 217 |
+
่ฏทๆ นๆฎไปฅไธๅ
ๅฎน๏ผ่ฎพ่ฎก้ๅ็คพไผ่่ฎบๆจกๆ็ๅฎไฝ็ฑปๅๅๅ
ณ็ณป็ฑปๅใ
|
| 218 |
+
่ฎฐไฝ๏ผๆๆๅฎไฝ็ฑปๅๅฟ
้กปๆฏ็ฐๅฎไธญๅฏไปฅๅๅฃฐ็ไธปไฝ๏ผไธ่ฝๆฏๆฝ่ฑกๆฆๅฟตใ
|
| 219 |
+
"""
|
| 220 |
+
|
| 221 |
+
return message
|
| 222 |
+
|
| 223 |
+
def _validate_and_process(self, result: Dict[str, Any]) -> Dict[str, Any]:
|
| 224 |
+
"""้ช่ฏๅๅๅค็็ปๆ"""
|
| 225 |
+
|
| 226 |
+
# ็กฎไฟๅฟ
่ฆๅญๆฎตๅญๅจ
|
| 227 |
+
if "entity_types" not in result:
|
| 228 |
+
result["entity_types"] = []
|
| 229 |
+
if "edge_types" not in result:
|
| 230 |
+
result["edge_types"] = []
|
| 231 |
+
if "analysis_summary" not in result:
|
| 232 |
+
result["analysis_summary"] = ""
|
| 233 |
+
|
| 234 |
+
# ้ช่ฏๅฎไฝ็ฑปๅ
|
| 235 |
+
for entity in result["entity_types"]:
|
| 236 |
+
if "attributes" not in entity:
|
| 237 |
+
entity["attributes"] = []
|
| 238 |
+
if "examples" not in entity:
|
| 239 |
+
entity["examples"] = []
|
| 240 |
+
# ็กฎไฟdescriptionไธ่ถ
่ฟ100ๅญ็ฌฆ
|
| 241 |
+
if len(entity.get("description", "")) > 100:
|
| 242 |
+
entity["description"] = entity["description"][:97] + "..."
|
| 243 |
+
|
| 244 |
+
# ้ช่ฏๅ
ณ็ณป็ฑปๅ
|
| 245 |
+
for edge in result["edge_types"]:
|
| 246 |
+
if "source_targets" not in edge:
|
| 247 |
+
edge["source_targets"] = []
|
| 248 |
+
if "attributes" not in edge:
|
| 249 |
+
edge["attributes"] = []
|
| 250 |
+
if len(edge.get("description", "")) > 100:
|
| 251 |
+
edge["description"] = edge["description"][:97] + "..."
|
| 252 |
+
|
| 253 |
+
return result
|
| 254 |
+
|
| 255 |
+
def generate_python_code(self, ontology: Dict[str, Any]) -> str:
|
| 256 |
+
"""
|
| 257 |
+
ๅฐๆฌไฝๅฎไน่ฝฌๆขไธบPythonไปฃ็ ๏ผ็ฑปไผผontology.py๏ผ
|
| 258 |
+
|
| 259 |
+
Args:
|
| 260 |
+
ontology: ๆฌไฝๅฎไน
|
| 261 |
+
|
| 262 |
+
Returns:
|
| 263 |
+
Pythonไปฃ็ ๅญ็ฌฆไธฒ
|
| 264 |
+
"""
|
| 265 |
+
code_lines = [
|
| 266 |
+
'"""',
|
| 267 |
+
'่ชๅฎไนๅฎไฝ็ฑปๅๅฎไน',
|
| 268 |
+
'็ฑMiroFish่ชๅจ็ๆ๏ผ็จไบ็คพไผ่่ฎบๆจกๆ',
|
| 269 |
+
'"""',
|
| 270 |
+
'',
|
| 271 |
+
'from pydantic import Field',
|
| 272 |
+
'from zep_cloud.external_clients.ontology import EntityModel, EntityText, EdgeModel',
|
| 273 |
+
'',
|
| 274 |
+
'',
|
| 275 |
+
'# ============== ๅฎไฝ็ฑปๅๅฎไน ==============',
|
| 276 |
+
'',
|
| 277 |
+
]
|
| 278 |
+
|
| 279 |
+
# ็ๆๅฎไฝ็ฑปๅ
|
| 280 |
+
for entity in ontology.get("entity_types", []):
|
| 281 |
+
name = entity["name"]
|
| 282 |
+
desc = entity.get("description", f"A {name} entity.")
|
| 283 |
+
|
| 284 |
+
code_lines.append(f'class {name}(EntityModel):')
|
| 285 |
+
code_lines.append(f' """{desc}"""')
|
| 286 |
+
|
| 287 |
+
attrs = entity.get("attributes", [])
|
| 288 |
+
if attrs:
|
| 289 |
+
for attr in attrs:
|
| 290 |
+
attr_name = attr["name"]
|
| 291 |
+
attr_desc = attr.get("description", attr_name)
|
| 292 |
+
code_lines.append(f' {attr_name}: EntityText = Field(')
|
| 293 |
+
code_lines.append(f' description="{attr_desc}",')
|
| 294 |
+
code_lines.append(f' default=None')
|
| 295 |
+
code_lines.append(f' )')
|
| 296 |
+
else:
|
| 297 |
+
code_lines.append(' pass')
|
| 298 |
+
|
| 299 |
+
code_lines.append('')
|
| 300 |
+
code_lines.append('')
|
| 301 |
+
|
| 302 |
+
code_lines.append('# ============== ๅ
ณ็ณป็ฑปๅๅฎไน ==============')
|
| 303 |
+
code_lines.append('')
|
| 304 |
+
|
| 305 |
+
# ็ๆๅ
ณ็ณป็ฑปๅ
|
| 306 |
+
for edge in ontology.get("edge_types", []):
|
| 307 |
+
name = edge["name"]
|
| 308 |
+
# ่ฝฌๆขไธบPascalCase็ฑปๅ
|
| 309 |
+
class_name = ''.join(word.capitalize() for word in name.split('_'))
|
| 310 |
+
desc = edge.get("description", f"A {name} relationship.")
|
| 311 |
+
|
| 312 |
+
code_lines.append(f'class {class_name}(EdgeModel):')
|
| 313 |
+
code_lines.append(f' """{desc}"""')
|
| 314 |
+
|
| 315 |
+
attrs = edge.get("attributes", [])
|
| 316 |
+
if attrs:
|
| 317 |
+
for attr in attrs:
|
| 318 |
+
attr_name = attr["name"]
|
| 319 |
+
attr_desc = attr.get("description", attr_name)
|
| 320 |
+
code_lines.append(f' {attr_name}: EntityText = Field(')
|
| 321 |
+
code_lines.append(f' description="{attr_desc}",')
|
| 322 |
+
code_lines.append(f' default=None')
|
| 323 |
+
code_lines.append(f' )')
|
| 324 |
+
else:
|
| 325 |
+
code_lines.append(' pass')
|
| 326 |
+
|
| 327 |
+
code_lines.append('')
|
| 328 |
+
code_lines.append('')
|
| 329 |
+
|
| 330 |
+
# ็ๆ็ฑปๅๅญๅ
ธ
|
| 331 |
+
code_lines.append('# ============== ็ฑปๅ้
็ฝฎ ==============')
|
| 332 |
+
code_lines.append('')
|
| 333 |
+
code_lines.append('ENTITY_TYPES = {')
|
| 334 |
+
for entity in ontology.get("entity_types", []):
|
| 335 |
+
name = entity["name"]
|
| 336 |
+
code_lines.append(f' "{name}": {name},')
|
| 337 |
+
code_lines.append('}')
|
| 338 |
+
code_lines.append('')
|
| 339 |
+
code_lines.append('EDGE_TYPES = {')
|
| 340 |
+
for edge in ontology.get("edge_types", []):
|
| 341 |
+
name = edge["name"]
|
| 342 |
+
class_name = ''.join(word.capitalize() for word in name.split('_'))
|
| 343 |
+
code_lines.append(f' "{name}": {class_name},')
|
| 344 |
+
code_lines.append('}')
|
| 345 |
+
code_lines.append('')
|
| 346 |
+
|
| 347 |
+
# ็ๆ่พน็source_targetsๆ ๅฐ
|
| 348 |
+
code_lines.append('EDGE_SOURCE_TARGETS = {')
|
| 349 |
+
for edge in ontology.get("edge_types", []):
|
| 350 |
+
name = edge["name"]
|
| 351 |
+
source_targets = edge.get("source_targets", [])
|
| 352 |
+
if source_targets:
|
| 353 |
+
st_list = ', '.join([
|
| 354 |
+
f'{{"source": "{st.get("source", "Entity")}", "target": "{st.get("target", "Entity")}"}}'
|
| 355 |
+
for st in source_targets
|
| 356 |
+
])
|
| 357 |
+
code_lines.append(f' "{name}": [{st_list}],')
|
| 358 |
+
code_lines.append('}')
|
| 359 |
+
|
| 360 |
+
return '\n'.join(code_lines)
|
| 361 |
+
|
backend/app/services/text_processor.py
ADDED
|
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๆๆฌๅค็ๆๅก
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
from typing import List, Optional
|
| 6 |
+
from ..utils.file_parser import FileParser, split_text_into_chunks
|
| 7 |
+
|
| 8 |
+
|
| 9 |
+
class TextProcessor:
|
| 10 |
+
"""ๆๆฌๅค็ๅจ"""
|
| 11 |
+
|
| 12 |
+
@staticmethod
|
| 13 |
+
def extract_from_files(file_paths: List[str]) -> str:
|
| 14 |
+
"""ไปๅคไธชๆไปถๆๅๆๆฌ"""
|
| 15 |
+
return FileParser.extract_from_multiple(file_paths)
|
| 16 |
+
|
| 17 |
+
@staticmethod
|
| 18 |
+
def split_text(
|
| 19 |
+
text: str,
|
| 20 |
+
chunk_size: int = 500,
|
| 21 |
+
overlap: int = 50
|
| 22 |
+
) -> List[str]:
|
| 23 |
+
"""
|
| 24 |
+
ๅๅฒๆๆฌ
|
| 25 |
+
|
| 26 |
+
Args:
|
| 27 |
+
text: ๅๅงๆๆฌ
|
| 28 |
+
chunk_size: ๅๅคงๅฐ
|
| 29 |
+
overlap: ้ๅ ๅคงๅฐ
|
| 30 |
+
|
| 31 |
+
Returns:
|
| 32 |
+
ๆๆฌๅๅ่กจ
|
| 33 |
+
"""
|
| 34 |
+
return split_text_into_chunks(text, chunk_size, overlap)
|
| 35 |
+
|
| 36 |
+
@staticmethod
|
| 37 |
+
def preprocess_text(text: str) -> str:
|
| 38 |
+
"""
|
| 39 |
+
้ขๅค็ๆๆฌ
|
| 40 |
+
- ็งป้คๅคไฝ็ฉบ็ฝ
|
| 41 |
+
- ๆ ๅๅๆข่ก
|
| 42 |
+
|
| 43 |
+
Args:
|
| 44 |
+
text: ๅๅงๆๆฌ
|
| 45 |
+
|
| 46 |
+
Returns:
|
| 47 |
+
ๅค็ๅ็ๆๆฌ
|
| 48 |
+
"""
|
| 49 |
+
import re
|
| 50 |
+
|
| 51 |
+
# ๆ ๅๅๆข่ก
|
| 52 |
+
text = text.replace('\r\n', '\n').replace('\r', '\n')
|
| 53 |
+
|
| 54 |
+
# ็งป้ค่ฟ็ปญ็ฉบ่ก๏ผไฟ็ๆๅคไธคไธชๆข่ก๏ผ
|
| 55 |
+
text = re.sub(r'\n{3,}', '\n\n', text)
|
| 56 |
+
|
| 57 |
+
# ็งป้ค่ก้ฆ่กๅฐพ็ฉบ็ฝ
|
| 58 |
+
lines = [line.strip() for line in text.split('\n')]
|
| 59 |
+
text = '\n'.join(lines)
|
| 60 |
+
|
| 61 |
+
return text.strip()
|
| 62 |
+
|
| 63 |
+
@staticmethod
|
| 64 |
+
def get_text_stats(text: str) -> dict:
|
| 65 |
+
"""่ทๅๆๆฌ็ป่ฎกไฟกๆฏ"""
|
| 66 |
+
return {
|
| 67 |
+
"total_chars": len(text),
|
| 68 |
+
"total_lines": text.count('\n') + 1,
|
| 69 |
+
"total_words": len(text.split()),
|
| 70 |
+
}
|
| 71 |
+
|
backend/app/utils/__init__.py
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๅทฅๅ
ทๆจกๅ
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
from .file_parser import FileParser
|
| 6 |
+
from .llm_client import LLMClient
|
| 7 |
+
|
| 8 |
+
__all__ = ['FileParser', 'LLMClient']
|
| 9 |
+
|
backend/app/utils/file_parser.py
ADDED
|
@@ -0,0 +1,141 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๆไปถ่งฃๆๅทฅๅ
ท
|
| 3 |
+
ๆฏๆPDFใMarkdownใTXTๆไปถ็ๆๆฌๆๅ
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import os
|
| 7 |
+
from pathlib import Path
|
| 8 |
+
from typing import List, Optional
|
| 9 |
+
|
| 10 |
+
|
| 11 |
+
class FileParser:
|
| 12 |
+
"""ๆไปถ่งฃๆๅจ"""
|
| 13 |
+
|
| 14 |
+
SUPPORTED_EXTENSIONS = {'.pdf', '.md', '.markdown', '.txt'}
|
| 15 |
+
|
| 16 |
+
@classmethod
|
| 17 |
+
def extract_text(cls, file_path: str) -> str:
|
| 18 |
+
"""
|
| 19 |
+
ไปๆไปถไธญๆๅๆๆฌ
|
| 20 |
+
|
| 21 |
+
Args:
|
| 22 |
+
file_path: ๆไปถ่ทฏๅพ
|
| 23 |
+
|
| 24 |
+
Returns:
|
| 25 |
+
ๆๅ็ๆๆฌๅ
ๅฎน
|
| 26 |
+
"""
|
| 27 |
+
path = Path(file_path)
|
| 28 |
+
|
| 29 |
+
if not path.exists():
|
| 30 |
+
raise FileNotFoundError(f"ๆไปถไธๅญๅจ: {file_path}")
|
| 31 |
+
|
| 32 |
+
suffix = path.suffix.lower()
|
| 33 |
+
|
| 34 |
+
if suffix not in cls.SUPPORTED_EXTENSIONS:
|
| 35 |
+
raise ValueError(f"ไธๆฏๆ็ๆไปถๆ ผๅผ: {suffix}")
|
| 36 |
+
|
| 37 |
+
if suffix == '.pdf':
|
| 38 |
+
return cls._extract_from_pdf(file_path)
|
| 39 |
+
elif suffix in {'.md', '.markdown'}:
|
| 40 |
+
return cls._extract_from_md(file_path)
|
| 41 |
+
elif suffix == '.txt':
|
| 42 |
+
return cls._extract_from_txt(file_path)
|
| 43 |
+
|
| 44 |
+
raise ValueError(f"ๆ ๆณๅค็็ๆไปถๆ ผๅผ: {suffix}")
|
| 45 |
+
|
| 46 |
+
@staticmethod
|
| 47 |
+
def _extract_from_pdf(file_path: str) -> str:
|
| 48 |
+
"""ไปPDFๆๅๆๆฌ"""
|
| 49 |
+
try:
|
| 50 |
+
import fitz # PyMuPDF
|
| 51 |
+
except ImportError:
|
| 52 |
+
raise ImportError("้่ฆๅฎ่ฃ
PyMuPDF: pip install PyMuPDF")
|
| 53 |
+
|
| 54 |
+
text_parts = []
|
| 55 |
+
with fitz.open(file_path) as doc:
|
| 56 |
+
for page in doc:
|
| 57 |
+
text = page.get_text()
|
| 58 |
+
if text.strip():
|
| 59 |
+
text_parts.append(text)
|
| 60 |
+
|
| 61 |
+
return "\n\n".join(text_parts)
|
| 62 |
+
|
| 63 |
+
@staticmethod
|
| 64 |
+
def _extract_from_md(file_path: str) -> str:
|
| 65 |
+
"""ไปMarkdownๆๅๆๆฌ"""
|
| 66 |
+
with open(file_path, 'r', encoding='utf-8') as f:
|
| 67 |
+
return f.read()
|
| 68 |
+
|
| 69 |
+
@staticmethod
|
| 70 |
+
def _extract_from_txt(file_path: str) -> str:
|
| 71 |
+
"""ไปTXTๆๅๆๆฌ"""
|
| 72 |
+
with open(file_path, 'r', encoding='utf-8') as f:
|
| 73 |
+
return f.read()
|
| 74 |
+
|
| 75 |
+
@classmethod
|
| 76 |
+
def extract_from_multiple(cls, file_paths: List[str]) -> str:
|
| 77 |
+
"""
|
| 78 |
+
ไปๅคไธชๆไปถๆๅๆๆฌๅนถๅๅนถ
|
| 79 |
+
|
| 80 |
+
Args:
|
| 81 |
+
file_paths: ๆไปถ่ทฏๅพๅ่กจ
|
| 82 |
+
|
| 83 |
+
Returns:
|
| 84 |
+
ๅๅนถๅ็ๆๆฌ
|
| 85 |
+
"""
|
| 86 |
+
all_texts = []
|
| 87 |
+
|
| 88 |
+
for i, file_path in enumerate(file_paths, 1):
|
| 89 |
+
try:
|
| 90 |
+
text = cls.extract_text(file_path)
|
| 91 |
+
filename = Path(file_path).name
|
| 92 |
+
all_texts.append(f"=== ๆๆกฃ {i}: {filename} ===\n{text}")
|
| 93 |
+
except Exception as e:
|
| 94 |
+
all_texts.append(f"=== ๆๆกฃ {i}: {file_path} (ๆๅๅคฑ่ดฅ: {str(e)}) ===")
|
| 95 |
+
|
| 96 |
+
return "\n\n".join(all_texts)
|
| 97 |
+
|
| 98 |
+
|
| 99 |
+
def split_text_into_chunks(
|
| 100 |
+
text: str,
|
| 101 |
+
chunk_size: int = 500,
|
| 102 |
+
overlap: int = 50
|
| 103 |
+
) -> List[str]:
|
| 104 |
+
"""
|
| 105 |
+
ๅฐๆๆฌๅๅฒๆๅฐๅ
|
| 106 |
+
|
| 107 |
+
Args:
|
| 108 |
+
text: ๅๅงๆๆฌ
|
| 109 |
+
chunk_size: ๆฏๅ็ๅญ็ฌฆๆฐ
|
| 110 |
+
overlap: ้ๅ ๅญ็ฌฆๆฐ
|
| 111 |
+
|
| 112 |
+
Returns:
|
| 113 |
+
ๆๆฌๅๅ่กจ
|
| 114 |
+
"""
|
| 115 |
+
if len(text) <= chunk_size:
|
| 116 |
+
return [text] if text.strip() else []
|
| 117 |
+
|
| 118 |
+
chunks = []
|
| 119 |
+
start = 0
|
| 120 |
+
|
| 121 |
+
while start < len(text):
|
| 122 |
+
end = start + chunk_size
|
| 123 |
+
|
| 124 |
+
# ๅฐ่ฏๅจๅฅๅญ่พน็ๅคๅๅฒ
|
| 125 |
+
if end < len(text):
|
| 126 |
+
# ๆฅๆพๆ่ฟ็ๅฅๅญ็ปๆ็ฌฆ
|
| 127 |
+
for sep in ['ใ', '๏ผ', '๏ผ', '.\n', '!\n', '?\n', '\n\n', '. ', '! ', '? ']:
|
| 128 |
+
last_sep = text[start:end].rfind(sep)
|
| 129 |
+
if last_sep != -1 and last_sep > chunk_size * 0.3:
|
| 130 |
+
end = start + last_sep + len(sep)
|
| 131 |
+
break
|
| 132 |
+
|
| 133 |
+
chunk = text[start:end].strip()
|
| 134 |
+
if chunk:
|
| 135 |
+
chunks.append(chunk)
|
| 136 |
+
|
| 137 |
+
# ไธไธไธชๅไป้ๅ ไฝ็ฝฎๅผๅง
|
| 138 |
+
start = end - overlap if end < len(text) else len(text)
|
| 139 |
+
|
| 140 |
+
return chunks
|
| 141 |
+
|
backend/app/utils/llm_client.py
ADDED
|
@@ -0,0 +1,91 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
LLMๅฎขๆท็ซฏๅฐ่ฃ
|
| 3 |
+
็ปไธไฝฟ็จOpenAIๆ ผๅผ่ฐ็จ
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import json
|
| 7 |
+
from typing import Optional, Dict, Any, List
|
| 8 |
+
from openai import OpenAI
|
| 9 |
+
|
| 10 |
+
from ..config import Config
|
| 11 |
+
|
| 12 |
+
|
| 13 |
+
class LLMClient:
|
| 14 |
+
"""LLMๅฎขๆท็ซฏ"""
|
| 15 |
+
|
| 16 |
+
def __init__(
|
| 17 |
+
self,
|
| 18 |
+
api_key: Optional[str] = None,
|
| 19 |
+
base_url: Optional[str] = None,
|
| 20 |
+
model: Optional[str] = None
|
| 21 |
+
):
|
| 22 |
+
self.api_key = api_key or Config.LLM_API_KEY
|
| 23 |
+
self.base_url = base_url or Config.LLM_BASE_URL
|
| 24 |
+
self.model = model or Config.LLM_MODEL_NAME
|
| 25 |
+
|
| 26 |
+
if not self.api_key:
|
| 27 |
+
raise ValueError("LLM_API_KEY ๆช้
็ฝฎ")
|
| 28 |
+
|
| 29 |
+
self.client = OpenAI(
|
| 30 |
+
api_key=self.api_key,
|
| 31 |
+
base_url=self.base_url
|
| 32 |
+
)
|
| 33 |
+
|
| 34 |
+
def chat(
|
| 35 |
+
self,
|
| 36 |
+
messages: List[Dict[str, str]],
|
| 37 |
+
temperature: float = 0.7,
|
| 38 |
+
max_tokens: int = 4096,
|
| 39 |
+
response_format: Optional[Dict] = None
|
| 40 |
+
) -> str:
|
| 41 |
+
"""
|
| 42 |
+
ๅ้่ๅคฉ่ฏทๆฑ
|
| 43 |
+
|
| 44 |
+
Args:
|
| 45 |
+
messages: ๆถๆฏๅ่กจ
|
| 46 |
+
temperature: ๆธฉๅบฆๅๆฐ
|
| 47 |
+
max_tokens: ๆๅคงtokenๆฐ
|
| 48 |
+
response_format: ๅๅบๆ ผๅผ๏ผๅฆJSONๆจกๅผ๏ผ
|
| 49 |
+
|
| 50 |
+
Returns:
|
| 51 |
+
ๆจกๅๅๅบๆๆฌ
|
| 52 |
+
"""
|
| 53 |
+
kwargs = {
|
| 54 |
+
"model": self.model,
|
| 55 |
+
"messages": messages,
|
| 56 |
+
"temperature": temperature,
|
| 57 |
+
"max_tokens": max_tokens,
|
| 58 |
+
}
|
| 59 |
+
|
| 60 |
+
if response_format:
|
| 61 |
+
kwargs["response_format"] = response_format
|
| 62 |
+
|
| 63 |
+
response = self.client.chat.completions.create(**kwargs)
|
| 64 |
+
return response.choices[0].message.content
|
| 65 |
+
|
| 66 |
+
def chat_json(
|
| 67 |
+
self,
|
| 68 |
+
messages: List[Dict[str, str]],
|
| 69 |
+
temperature: float = 0.3,
|
| 70 |
+
max_tokens: int = 4096
|
| 71 |
+
) -> Dict[str, Any]:
|
| 72 |
+
"""
|
| 73 |
+
ๅ้่ๅคฉ่ฏทๆฑๅนถ่ฟๅJSON
|
| 74 |
+
|
| 75 |
+
Args:
|
| 76 |
+
messages: ๆถๆฏๅ่กจ
|
| 77 |
+
temperature: ๆธฉๅบฆๅๆฐ
|
| 78 |
+
max_tokens: ๆๅคงtokenๆฐ
|
| 79 |
+
|
| 80 |
+
Returns:
|
| 81 |
+
่งฃๆๅ็JSONๅฏน่ฑก
|
| 82 |
+
"""
|
| 83 |
+
response = self.chat(
|
| 84 |
+
messages=messages,
|
| 85 |
+
temperature=temperature,
|
| 86 |
+
max_tokens=max_tokens,
|
| 87 |
+
response_format={"type": "json_object"}
|
| 88 |
+
)
|
| 89 |
+
|
| 90 |
+
return json.loads(response)
|
| 91 |
+
|
backend/app/utils/logger.py
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
ๆฅๅฟ้
็ฝฎๆจกๅ
|
| 3 |
+
ๆไพ็ปไธ็ๆฅๅฟ็ฎก็๏ผๅๆถ่พๅบๅฐๆงๅถๅฐๅๆไปถ
|
| 4 |
+
"""
|
| 5 |
+
|
| 6 |
+
import os
|
| 7 |
+
import logging
|
| 8 |
+
from datetime import datetime
|
| 9 |
+
from logging.handlers import RotatingFileHandler
|
| 10 |
+
|
| 11 |
+
|
| 12 |
+
# ๆฅๅฟ็ฎๅฝ
|
| 13 |
+
LOG_DIR = os.path.join(os.path.dirname(os.path.dirname(os.path.dirname(__file__))), 'logs')
|
| 14 |
+
|
| 15 |
+
|
| 16 |
+
def setup_logger(name: str = 'mirofish', level: int = logging.DEBUG) -> logging.Logger:
|
| 17 |
+
"""
|
| 18 |
+
่ฎพ็ฝฎๆฅๅฟๅจ
|
| 19 |
+
|
| 20 |
+
Args:
|
| 21 |
+
name: ๆฅๅฟๅจๅ็งฐ
|
| 22 |
+
level: ๆฅๅฟ็บงๅซ
|
| 23 |
+
|
| 24 |
+
Returns:
|
| 25 |
+
้
็ฝฎๅฅฝ็ๆฅๅฟๅจ
|
| 26 |
+
"""
|
| 27 |
+
# ็กฎไฟๆฅๅฟ็ฎๅฝๅญๅจ
|
| 28 |
+
os.makedirs(LOG_DIR, exist_ok=True)
|
| 29 |
+
|
| 30 |
+
# ๅๅปบๆฅๅฟๅจ
|
| 31 |
+
logger = logging.getLogger(name)
|
| 32 |
+
logger.setLevel(level)
|
| 33 |
+
|
| 34 |
+
# ๅฆๆๅทฒ็ปๆๅค็ๅจ๏ผไธ้ๅคๆทปๅ
|
| 35 |
+
if logger.handlers:
|
| 36 |
+
return logger
|
| 37 |
+
|
| 38 |
+
# ๆฅๅฟๆ ผๅผ
|
| 39 |
+
detailed_formatter = logging.Formatter(
|
| 40 |
+
'[%(asctime)s] %(levelname)s [%(name)s.%(funcName)s:%(lineno)d] %(message)s',
|
| 41 |
+
datefmt='%Y-%m-%d %H:%M:%S'
|
| 42 |
+
)
|
| 43 |
+
|
| 44 |
+
simple_formatter = logging.Formatter(
|
| 45 |
+
'[%(asctime)s] %(levelname)s: %(message)s',
|
| 46 |
+
datefmt='%H:%M:%S'
|
| 47 |
+
)
|
| 48 |
+
|
| 49 |
+
# 1. ๆไปถๅค็ๅจ - ่ฏฆ็ปๆฅๅฟ๏ผๆๆฅๆๅฝๅ๏ผๅธฆ่ฝฎ่ฝฌ๏ผ
|
| 50 |
+
log_filename = datetime.now().strftime('%Y-%m-%d') + '.log'
|
| 51 |
+
file_handler = RotatingFileHandler(
|
| 52 |
+
os.path.join(LOG_DIR, log_filename),
|
| 53 |
+
maxBytes=10 * 1024 * 1024, # 10MB
|
| 54 |
+
backupCount=5,
|
| 55 |
+
encoding='utf-8'
|
| 56 |
+
)
|
| 57 |
+
file_handler.setLevel(logging.DEBUG)
|
| 58 |
+
file_handler.setFormatter(detailed_formatter)
|
| 59 |
+
|
| 60 |
+
# 2. ๆงๅถๅฐๅค็ๅจ - ็ฎๆดๆฅๅฟ๏ผINFOๅไปฅไธ๏ผ
|
| 61 |
+
console_handler = logging.StreamHandler()
|
| 62 |
+
console_handler.setLevel(logging.INFO)
|
| 63 |
+
console_handler.setFormatter(simple_formatter)
|
| 64 |
+
|
| 65 |
+
# ๆทปๅ ๅค็ๅจ
|
| 66 |
+
logger.addHandler(file_handler)
|
| 67 |
+
logger.addHandler(console_handler)
|
| 68 |
+
|
| 69 |
+
return logger
|
| 70 |
+
|
| 71 |
+
|
| 72 |
+
def get_logger(name: str = 'mirofish') -> logging.Logger:
|
| 73 |
+
"""
|
| 74 |
+
่ทๅๆฅๅฟๅจ๏ผๅฆๆไธๅญๅจๅๅๅปบ๏ผ
|
| 75 |
+
|
| 76 |
+
Args:
|
| 77 |
+
name: ๆฅๅฟๅจๅ็งฐ
|
| 78 |
+
|
| 79 |
+
Returns:
|
| 80 |
+
ๆฅๅฟๅจๅฎไพ
|
| 81 |
+
"""
|
| 82 |
+
logger = logging.getLogger(name)
|
| 83 |
+
if not logger.handlers:
|
| 84 |
+
return setup_logger(name)
|
| 85 |
+
return logger
|
| 86 |
+
|
| 87 |
+
|
| 88 |
+
# ๅๅปบ้ป่ฎคๆฅๅฟๅจ
|
| 89 |
+
logger = setup_logger()
|
| 90 |
+
|
| 91 |
+
|
| 92 |
+
# ไพฟๆทๆนๆณ
|
| 93 |
+
def debug(msg, *args, **kwargs):
|
| 94 |
+
logger.debug(msg, *args, **kwargs)
|
| 95 |
+
|
| 96 |
+
def info(msg, *args, **kwargs):
|
| 97 |
+
logger.info(msg, *args, **kwargs)
|
| 98 |
+
|
| 99 |
+
def warning(msg, *args, **kwargs):
|
| 100 |
+
logger.warning(msg, *args, **kwargs)
|
| 101 |
+
|
| 102 |
+
def error(msg, *args, **kwargs):
|
| 103 |
+
logger.error(msg, *args, **kwargs)
|
| 104 |
+
|
| 105 |
+
def critical(msg, *args, **kwargs):
|
| 106 |
+
logger.critical(msg, *args, **kwargs)
|
| 107 |
+
|
backend/requirements.txt
ADDED
|
@@ -0,0 +1,22 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# Flaskๆกๆถ
|
| 2 |
+
flask>=3.0.0
|
| 3 |
+
flask-cors>=4.0.0
|
| 4 |
+
|
| 5 |
+
# Zep Cloud SDK
|
| 6 |
+
zep-cloud>=2.0.0
|
| 7 |
+
|
| 8 |
+
# OpenAI SDK๏ผ็จไบLLM่ฐ็จ๏ผ
|
| 9 |
+
openai>=1.0.0
|
| 10 |
+
|
| 11 |
+
# PDFๅค็
|
| 12 |
+
PyMuPDF>=1.24.0
|
| 13 |
+
|
| 14 |
+
# ็ฏๅขๅ้
|
| 15 |
+
python-dotenv>=1.0.0
|
| 16 |
+
|
| 17 |
+
# ๆฐๆฎ้ช่ฏ
|
| 18 |
+
pydantic>=2.0.0
|
| 19 |
+
|
| 20 |
+
# ๆไปถๅค็
|
| 21 |
+
werkzeug>=3.0.0
|
| 22 |
+
|
backend/run.py
ADDED
|
@@ -0,0 +1,57 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
"""
|
| 2 |
+
MiroFish Backend ๅฏๅจๅ
ฅๅฃ
|
| 3 |
+
"""
|
| 4 |
+
|
| 5 |
+
import os
|
| 6 |
+
import sys
|
| 7 |
+
|
| 8 |
+
# ๆทปๅ ้กน็ฎๆ น็ฎๅฝๅฐ่ทฏๅพ
|
| 9 |
+
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
|
| 10 |
+
|
| 11 |
+
from app import create_app
|
| 12 |
+
from app.config import Config
|
| 13 |
+
|
| 14 |
+
|
| 15 |
+
def main():
|
| 16 |
+
"""ไธปๅฝๆฐ"""
|
| 17 |
+
# ้ช่ฏ้
็ฝฎ
|
| 18 |
+
errors = Config.validate()
|
| 19 |
+
if errors:
|
| 20 |
+
print("้
็ฝฎ้่ฏฏ:")
|
| 21 |
+
for err in errors:
|
| 22 |
+
print(f" - {err}")
|
| 23 |
+
print("\n่ฏทๆฃๆฅ .env ๆไปถไธญ็้
็ฝฎ")
|
| 24 |
+
sys.exit(1)
|
| 25 |
+
|
| 26 |
+
# ๅๅปบๅบ็จ
|
| 27 |
+
app = create_app()
|
| 28 |
+
|
| 29 |
+
# ่ทๅ่ฟ่ก้
็ฝฎ
|
| 30 |
+
host = os.environ.get('FLASK_HOST', '0.0.0.0')
|
| 31 |
+
port = int(os.environ.get('FLASK_PORT', 5001))
|
| 32 |
+
debug = Config.DEBUG
|
| 33 |
+
|
| 34 |
+
print(f"""
|
| 35 |
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 36 |
+
โ MiroFish Backend Server โ
|
| 37 |
+
โ โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฃ
|
| 38 |
+
โ Running on: http://{host}:{port}
|
| 39 |
+
โ Debug mode: {debug}
|
| 40 |
+
โ
|
| 41 |
+
โ API Endpoints:
|
| 42 |
+
โ POST /api/graph/ontology/generate - ็ๆๆฌไฝ
|
| 43 |
+
โ POST /api/graph/build - ๆๅปบๅพ่ฐฑ
|
| 44 |
+
โ GET /api/graph/task/<task_id> - ๆฅ่ฏขไปปๅก
|
| 45 |
+
โ GET /api/graph/tasks - ๅๅบไปปๅก
|
| 46 |
+
โ GET /api/graph/data/<graph_id> - ่ทๅๅพๆฐๆฎ
|
| 47 |
+
โ DELETE /api/graph/delete/<graph_id>- ๅ ้คๅพ่ฐฑ
|
| 48 |
+
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ
|
| 49 |
+
""")
|
| 50 |
+
|
| 51 |
+
# ๅฏๅจๆๅก
|
| 52 |
+
app.run(host=host, port=port, debug=debug, threaded=True)
|
| 53 |
+
|
| 54 |
+
|
| 55 |
+
if __name__ == '__main__':
|
| 56 |
+
main()
|
| 57 |
+
|