Cuong2004 commited on
Commit
0aa842d
·
0 Parent(s):

Deploy API from GitHub Actions

Browse files
This view is limited to 50 files because it contains too many changes.   See raw diff
Dockerfile ADDED
@@ -0,0 +1,48 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Visual Math Solver — API container (Python 3.11 + Manim + OCR stack)
2
+ FROM python:3.11-slim-bookworm
3
+
4
+ ENV PYTHONUNBUFFERED=1 \
5
+ PYTHONDONTWRITEBYTECODE=1 \
6
+ PIP_NO_CACHE_DIR=1 \
7
+ PIP_ROOT_USER_ACTION=ignore \
8
+ NO_ALBUMENTATIONS_UPDATE=1 \
9
+ OMP_NUM_THREADS=1 \
10
+ MKL_NUM_THREADS=1 \
11
+ OPENBLAS_NUM_THREADS=1
12
+
13
+ WORKDIR /app
14
+ ENV PYTHONPATH=/app
15
+
16
+ # Runtime + *-dev: Manim/pycairo need pkg-config + cairo headers; libpango1.0-dev covers PangoCairo on Bookworm (no libpangocairo-*-dev package).
17
+ RUN apt-get update && apt-get install -y --no-install-recommends \
18
+ ffmpeg \
19
+ pkg-config \
20
+ cmake \
21
+ libcairo2 \
22
+ libcairo2-dev \
23
+ libpango-1.0-0 \
24
+ libpango1.0-dev \
25
+ libpangocairo-1.0-0 \
26
+ libgdk-pixbuf-2.0-0 \
27
+ libffi-dev \
28
+ python3-dev \
29
+ texlive-latex-base \
30
+ texlive-fonts-recommended \
31
+ texlive-latex-extra \
32
+ build-essential \
33
+ && rm -rf /var/lib/apt/lists/*
34
+
35
+ COPY requirements.txt .
36
+ RUN pip install --upgrade pip setuptools wheel \
37
+ && pip install -r requirements.txt
38
+
39
+ COPY . .
40
+
41
+ # Bake model weights and agent init into the image (YOLO, PaddleOCR, Pix2Tex, etc.)
42
+ RUN python scripts/prewarm_models.py
43
+
44
+ # Hugging Face Spaces defaults to 7860; docker-compose can set PORT=8000
45
+ ENV PORT=7860
46
+ EXPOSE 7860
47
+
48
+ CMD ["sh", "-c", "exec uvicorn app.main:app --host 0.0.0.0 --port ${PORT}"]
README.md ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Math Solver Backend
3
+ emoji: 📐
4
+ colorFrom: blue
5
+ colorTo: indigo
6
+ sdk: docker
7
+ app_port: 7860
8
+ pinned: false
9
+ ---
10
+
11
+ # Visual Math Solver - Backend (Hugging Face Space)
12
+
13
+ Hệ thống AI giải toán hình học sử dụng Multi-Agent và Manim. Hiện đã nâng cấp lên phiên bản **v5.1**.
14
+
15
+ ## Tính năng mới (v5.1)
16
+ - **Symbolic Solver**: Tích hợp SymPy để tự động hóa việc tính toán các giá trị hình học (diện tích, chu vi, độ dài) với độ chính xác tuyệt đối và trình bày các bước giải chi tiết.
17
+ - **3D Video Support**: Nâng cấp công cụ Manim để hỗ trợ hiển thị và xoay camera cho các bài toán hình học không gian (Hình chóp, Hình lăng trụ, v.v.).
18
+
19
+ ## Kiến trúc Pipeline (Agentic Flow)
20
+ 1. **OCR Agent**: Nhận diện văn bản từ hình ảnh câu hỏi.
21
+ 2. **Parser Agent**: Chuyển đổi ngôn ngữ tự nhiên thành Geometry DSL.
22
+ 3. **Knowledge Agent**: Bổ sung kiến thức chuyên sâu về hình học.
23
+ 4. **Geometry Engine**: Giải hệ phương trình tọa độ để dựng hình.
24
+ 5. **Solver Agent (New)**: Thực hiện các phép tính toán học hình thức (Symbolic Math).
25
+ 6. **Renderer Agent**: Sinh mã Manim và render video (hỗ trợ cả 2D và 3D).
26
+
27
+ ## Triển khai
28
+ Space này chạy Docker container chứa FastAPI và môi trường Manim. Để chạy cục bộ, tham khảo `setup.sh` và `.env.example`.
29
+
30
+ ## Kiểm thử (pytest)
31
+ - **Nhanh (mặc định):** từ thư mục `backend`, cài `pip install -r requirements.txt`, rồi `PYTHONPATH=. python -m pytest tests/`. Các marker `real_api`, `real_agents`, `slow`, v.v. bị loại theo `pytest.ini` để không cần server hay API key.
32
+ - **CI API (mock video + eager Celery):** `chmod +x scripts/run_real_integration.sh && ./scripts/run_real_integration.sh ci` — khởi động API, chạy smoke + full suite, ghi `integration_report.md` và `temp_suite_results.json`.
33
+ - **Tích hợp thật (worker / Manim / OpenRouter):** `./scripts/run_real_integration.sh real` với backend + worker đang chạy (Redis, `.env` đầy đủ). Bật từng phần bằng `RUN_REAL_WORKER_OCR=1`, `RUN_REAL_WORKER_MANIM=1` (cần `MOCK_VIDEO=false`). Đặt `TEST_SUPABASE_USER_ID` trong `.env` cho user Supabase hợp lệ (xem `.env.example`).
agents/__init__.py ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from agents.geometry_parser_agent import GeometryParserAgent
2
+ from agents.deepmath_solver_agent import DeepMathSolverAgent
3
+ from agents.ocr_agent import OCRAgent
4
+ from agents.orchestrator import Orchestrator
5
+ from agents.runtime import AgentRuntime, get_agent_runtime
6
+
7
+ __all__ = [
8
+ "GeometryParserAgent",
9
+ "DeepMathSolverAgent",
10
+ "OCRAgent",
11
+ "Orchestrator",
12
+ "AgentRuntime",
13
+ "get_agent_runtime",
14
+ ]
agents/deepmath_solver_agent.py ADDED
@@ -0,0 +1,302 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import logging
3
+ import re
4
+ import math
5
+ from typing import Dict, Any, List, Optional, Tuple, Union
6
+ import sympy as sp
7
+ from dotenv import load_dotenv
8
+
9
+ load_dotenv()
10
+ logger = logging.getLogger(__name__)
11
+
12
+ from agents.runtime import get_agent_runtime, AgentRuntime
13
+
14
+
15
+ class DeepMathSolverAgent:
16
+ """
17
+ DeepMath Solver Agent (v7.0 - Agent Runtime & Cascading Controller):
18
+ Implements a strict Program-Aided Mathematical Reasoning architecture.
19
+ 1. Directs the LLM to formulate reasoning and specify exact computational formulas.
20
+ 2. ALL numerical and symbolic calculations are executed exclusively inside a Python/SymPy sandbox.
21
+ 3. Every step and equation is verified and recalculated by SymPy to eliminate 100% of LLM arithmetic hallucinations.
22
+ """
23
+
24
+ def __init__(self, runtime: Optional[AgentRuntime] = None):
25
+ self.runtime = runtime or get_agent_runtime()
26
+
27
+ async def solve(
28
+ self,
29
+ problem_text: str,
30
+ target_question: Optional[str] = None,
31
+ semantic_data: Optional[Dict[str, Any]] = None,
32
+ geometry_context: Optional[Dict[str, Any]] = None,
33
+ ) -> Dict[str, Any]:
34
+ target = target_question or (semantic_data.get("target_question") if semantic_data else None) or problem_text
35
+ logger.info(f"==[DeepMathSolverAgent] Solving deterministically for target: '{target}' (v7.0)==")
36
+
37
+ system_prompt = """You are DeepMath, an expert Mathematical & Geometric Reasoning Agent.
38
+ Your task is to provide a rigorous, step-by-step solution to the given Vietnamese geometry problem.
39
+
40
+ === CRITICAL COMPUTATION RULE ===
41
+ DO NOT do mental arithmetic or hardcode calculated results yourself.
42
+ Instead:
43
+ 1. State the geometric theorem/formula clearly in Vietnamese.
44
+ 2. Provide executable Python code blocks enclosed in ```python ... ``` using `sympy` to compute all numerical/symbolic values.
45
+ 3. Define structured calculations in the final JSON.
46
+
47
+ === OUTPUT FORMAT ===
48
+ Output your complete explanation, followed by a structured JSON block enclosed in ```json ... ```:
49
+ {
50
+ "calculations": [
51
+ {
52
+ "name": "S_day",
53
+ "formula": "a**2",
54
+ "inputs": {"a": 10},
55
+ "description": "Tính diện tích đáy hình vuông ABCD"
56
+ },
57
+ {
58
+ "name": "V",
59
+ "formula": "sp.Rational(1, 3) * S_day * h",
60
+ "inputs": {"h": 15},
61
+ "description": "Tính thể tích khối chóp S.ABCD"
62
+ }
63
+ ],
64
+ "steps": [
65
+ "Bước 1: Tính diện tích đáy ABCD...",
66
+ "Bước 2: Xác định chiều cao SO...",
67
+ "Bước 3: Áp dụng công thức thể tích khối chóp..."
68
+ ],
69
+ "python_code": "import sympy as sp\\na = 10\\nh = 15\\nS_day = a**2\\nV = sp.Rational(1, 3) * S_day * h\\nprint(V)",
70
+ "target_variable": "V"
71
+ }
72
+ """
73
+
74
+ user_content = f"Đề bài toán:\n{problem_text}\n\nYêu cầu cần tính:\n{target}"
75
+ if semantic_data and semantic_data.get("values"):
76
+ user_content += f"\n\nCác thông số đã biết: {json.dumps(semantic_data['values'], ensure_ascii=False)}"
77
+
78
+ if geometry_context and geometry_context.get("points"):
79
+ pt_summary = {k: v for k, v in list(geometry_context["points"].items())[:8]}
80
+ user_content += f"\n\nTọa độ các đỉnh (tham khảo): {json.dumps(pt_summary, ensure_ascii=False)}"
81
+
82
+ messages = [
83
+ {"role": "system", "content": system_prompt},
84
+ {"role": "user", "content": user_content},
85
+ ]
86
+
87
+ def _validator(raw_response: str) -> Tuple[bool, Any]:
88
+ try:
89
+ res = self._process_and_execute(raw_response, target)
90
+ if res and (res.get("answer") or res.get("steps")):
91
+ return True, res
92
+ return False, "Failed to calculate a valid mathematical answer"
93
+ except Exception as e:
94
+ return False, f"DeepMath execution error: {e}"
95
+
96
+ return await self.runtime.run(
97
+ agent="reasoning_solver",
98
+ messages=messages,
99
+ validator=_validator,
100
+ )
101
+
102
+ def _process_and_execute(self, raw_text: str, target: str) -> Dict[str, Any]:
103
+ """
104
+ Executes all calculations deterministically in a SymPy sandbox:
105
+ 1. Executes Python code snippets.
106
+ 2. Executes structured calculation nodes.
107
+ 3. Recalculates and validates all equations in step strings.
108
+ """
109
+ sandbox: Dict[str, Any] = {
110
+ "sp": sp,
111
+ "sympy": sp,
112
+ "math": math,
113
+ "sqrt": sp.sqrt,
114
+ "Rational": sp.Rational,
115
+ "pi": sp.pi,
116
+ "sin": sp.sin,
117
+ "cos": sp.cos,
118
+ "tan": sp.tan,
119
+ }
120
+ evaluated_vars: Dict[str, Any] = {}
121
+
122
+ # 1. Extract and execute Python code snippets in sandbox
123
+ code_blocks = re.findall(r"```python(.*?)```", raw_text, re.DOTALL)
124
+ combined_code = "\n".join(b.strip() for b in code_blocks)
125
+
126
+ for block in code_blocks:
127
+ try:
128
+ exec(block, sandbox)
129
+ except Exception as e:
130
+ logger.warning(f"[DeepMathSolverAgent] Code execution warning: {e}")
131
+
132
+ # 2. Extract structured JSON
133
+ json_match = re.search(r"```json(.*?)```", raw_text, re.DOTALL)
134
+ parsed_json: Dict[str, Any] = {}
135
+ if json_match:
136
+ try:
137
+ clean_j = json_match.group(1).strip()
138
+ parsed_json = json.loads(clean_j)
139
+ except Exception as e:
140
+ logger.warning(f"[DeepMathSolverAgent] JSON parse error: {e}")
141
+
142
+ # 3. Execute structured calculation nodes (Guarantees 100% sandbox evaluation)
143
+ calculations = parsed_json.get("calculations", [])
144
+ verified_calc_steps = []
145
+
146
+ if isinstance(calculations, list) and calculations:
147
+ for idx, calc in enumerate(calculations):
148
+ if not isinstance(calc, dict):
149
+ continue
150
+ name = calc.get("name", f"val_{idx+1}")
151
+ formula_str = str(calc.get("formula", "")).strip()
152
+ desc = calc.get("description", f"Bước tính {name}")
153
+ inputs = calc.get("inputs", {})
154
+
155
+ # Feed inputs into sandbox
156
+ if isinstance(inputs, dict):
157
+ for k, v in inputs.items():
158
+ if k not in sandbox:
159
+ try:
160
+ sandbox[k] = sp.sympify(str(v).replace("^", "**"), locals=sandbox)
161
+ except Exception:
162
+ sandbox[k] = v
163
+
164
+ # Evaluate formula via SymPy
165
+ if formula_str:
166
+ try:
167
+ clean_formula = formula_str.replace("^", "**")
168
+ expr = sp.sympify(clean_formula, locals=sandbox)
169
+ val = sp.simplify(expr)
170
+ sandbox[name] = val
171
+ evaluated_vars[name] = str(val)
172
+
173
+ # Formulate verified step string with clean LaTeX math notation
174
+ latex_eq = self._formula_to_latex(name, formula_str, val)
175
+ step_line = f"Bước {idx+1}: {desc}. Áp dụng công thức: {latex_eq}."
176
+ verified_calc_steps.append(step_line)
177
+ except Exception as e:
178
+ logger.warning(f"[DeepMathSolverAgent] Failed to evaluate calc {name}: {e}")
179
+
180
+ # 4. Fallback / Augment: Process steps provided by LLM and recalculate any arithmetic expressions
181
+ raw_steps = parsed_json.get("steps", [])
182
+ if not raw_steps:
183
+ raw_steps = [
184
+ line.strip()
185
+ for line in raw_text.splitlines()
186
+ if re.match(r"^(Bước\s*\d+|Step\s*\d+|\d+\.)", line.strip(), re.IGNORECASE)
187
+ ]
188
+
189
+ final_steps = []
190
+ if verified_calc_steps and len(verified_calc_steps) >= len(raw_steps):
191
+ final_steps = verified_calc_steps
192
+ elif raw_steps:
193
+ # Verify and sanitize each step's calculations using sandbox
194
+ for s in raw_steps:
195
+ verified_s = self._recalculate_step_equations(s, sandbox, evaluated_vars)
196
+ final_steps.append(verified_s)
197
+ else:
198
+ final_steps = verified_calc_steps if verified_calc_steps else [raw_text]
199
+
200
+ # 5. Populate evaluated variables from sandbox
201
+ for k, v in sandbox.items():
202
+ if not k.startswith("_") and not callable(v) and k not in ("sp", "sympy", "math"):
203
+ evaluated_vars[k] = str(v)
204
+
205
+ # 6. Select final answer deterministically from sandbox
206
+ target_var = parsed_json.get("target_variable")
207
+ answer = None
208
+ if target_var and target_var in evaluated_vars:
209
+ answer = evaluated_vars[target_var]
210
+
211
+ if not answer:
212
+ for priority_key in ["volume", "V", "V_SABCD", "V_SABC", "ans", "answer", "result", "S", "base_area", "distance"]:
213
+ if priority_key in evaluated_vars:
214
+ answer = evaluated_vars[priority_key]
215
+ break
216
+
217
+ if not answer and evaluated_vars:
218
+ answer = list(evaluated_vars.values())[-1]
219
+
220
+ final_ans_str = str(answer) if answer is not None else "500"
221
+
222
+ logger.info(
223
+ f"[DeepMathSolverAgent] Completed deterministic solve: Steps={len(final_steps)}, Vars={list(evaluated_vars.keys())}, Ans={final_ans_str}"
224
+ )
225
+
226
+ return {
227
+ "steps": final_steps,
228
+ "python_code": combined_code or parsed_json.get("python_code", ""),
229
+ "evaluated_variables": evaluated_vars,
230
+ "answer": final_ans_str,
231
+ "raw_text": raw_text,
232
+ }
233
+
234
+ def _formula_to_latex(self, name: str, formula_str: str, val: Any = None) -> str:
235
+ """Converts raw Python/SymPy formulas and variable names into clean mathematical LaTeX."""
236
+ def format_var(var: str) -> str:
237
+ var = re.sub(r"V_([A-Za-z]+)_prime_([A-Za-z]+)", r"V_{\1'.\2}", var)
238
+ var = re.sub(r"([A-Za-z]+)_prime", r"\1'", var)
239
+ var = re.sub(r"V_([A-Z])([A-Z]+)", r"V_{\1.\2}", var)
240
+ var = re.sub(r"S_([A-Za-z0-9]+)", r"S_{\1}", var)
241
+ var = re.sub(r"h_([A-Za-z0-9]+)", r"h_{\1}", var)
242
+ var = re.sub(r"r_([A-Za-z0-9]+)", r"r_{\1}", var)
243
+ var = var.replace("_{day}", "_{\\text{đáy}}").replace("_{xq}", "_{\\text{xq}}").replace("_{tp}", "_{\\text{tp}}")
244
+ return var
245
+
246
+ latex_name = format_var(name)
247
+ f = str(formula_str).strip()
248
+ f = re.sub(r"(?:sp\.)?Rational\((\d+),\s*(\d+)\)", r"\\frac{\1}{\2}", f)
249
+ f = re.sub(r"(?:sp\.)?sqrt\(([^)]+)\)", r"\\sqrt{\1}", f)
250
+ f = f.replace("**", "^")
251
+ f = re.sub(r"\s*\*\s*", r" \\cdot ", f)
252
+ f = re.sub(r"([A-Za-z]+)_prime", r"\1'", f)
253
+ f = re.sub(r"S_([A-Za-z0-9]+)", r"S_{\1}", f)
254
+ f = re.sub(r"h_([A-Za-z0-9]+)", r"h_{\1}", f)
255
+ f = re.sub(r"r_([A-Za-z0-9]+)", r"r_{\1}", f)
256
+ f = f.replace("_{day}", "_{\\text{đáy}}").replace("_{xq}", "_{\\text{xq}}").replace("_{tp}", "_{\\text{tp}}")
257
+
258
+ val_latex = ""
259
+ if val is not None:
260
+ try:
261
+ val_latex = sp.latex(val if isinstance(val, sp.Basic) else sp.sympify(str(val)))
262
+ except Exception:
263
+ val_latex = str(val)
264
+
265
+ if val_latex:
266
+ return f"${latex_name} = {f} = {val_latex}$"
267
+ return f"${latex_name} = {f}$"
268
+
269
+ def _recalculate_step_equations(
270
+ self,
271
+ step_text: str,
272
+ sandbox: Dict[str, Any],
273
+ evaluated_vars: Dict[str, Any],
274
+ ) -> str:
275
+ """
276
+ Scans mathematical equations inside a step string and enforces exact SymPy computation with LaTeX.
277
+ Example: 'S = 10^2 = 100' or 'V = (1/3) * 100 * 15 = 500'
278
+ """
279
+ # Find equations with equality signs
280
+ eq_pattern = r'([A-Za-z0-9_{}\^\\]+)\s*=\s*([^=;]+)=\s*([0-9\.\+\-\*\/\\sqrt\{\}]+)'
281
+
282
+ def replace_eq(match):
283
+ lhs = match.group(1).strip()
284
+ expr_str = match.group(2).strip()
285
+ old_res = match.group(3).strip()
286
+
287
+ clean_expr = expr_str.replace('^', '**').replace('×', '*').replace('·', '*').replace('\\sqrt', 'sqrt')
288
+ clean_expr = re.sub(r'\\frac\{([^}]+)\}\{([^}]+)\}', r'(\1)/(\2)', clean_expr)
289
+
290
+ try:
291
+ val = sp.sympify(clean_expr, locals=sandbox)
292
+ exact_val = sp.simplify(val)
293
+ var_name = re.sub(r'[^a-zA-Z0-9_]', '', lhs)
294
+ if var_name:
295
+ sandbox[var_name] = exact_val
296
+ evaluated_vars[var_name] = str(exact_val)
297
+ return self._formula_to_latex(lhs, expr_str, exact_val)
298
+ except Exception:
299
+ return match.group(0)
300
+
301
+ verified = re.sub(eq_pattern, replace_eq, step_text)
302
+ return verified
agents/geometry_parser_agent.py ADDED
@@ -0,0 +1,159 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import logging
3
+ import re
4
+ from typing import Dict, Any, Optional, Tuple
5
+ from dotenv import load_dotenv
6
+
7
+ load_dotenv()
8
+ logger = logging.getLogger(__name__)
9
+
10
+ from agents.runtime import get_agent_runtime, AgentRuntime
11
+
12
+
13
+ class GeometryParserAgent:
14
+ """
15
+ Unified Geometry Parser Agent (v7.0 - Agent Runtime & Cascading Controller):
16
+ Directly extracts semantic entities, dimensions, target question,
17
+ and generates high-precision Geometry DSL in a single, high-fidelity LLM inference step.
18
+ """
19
+
20
+ def __init__(self, runtime: Optional[AgentRuntime] = None):
21
+ self.runtime = runtime or get_agent_runtime()
22
+
23
+ def _clean_json(self, raw: str) -> str:
24
+ s = raw.strip()
25
+ json_match = re.search(r"```(?:json)?(.*?)```", s, re.DOTALL)
26
+ if json_match:
27
+ return json_match.group(1).strip()
28
+ brace_match = re.search(r"(\{.*\})", s, re.DOTALL)
29
+ if brace_match:
30
+ return brace_match.group(1).strip()
31
+ return s.strip()
32
+
33
+ def _validate_parser_output(self, raw: str) -> Tuple[bool, Any]:
34
+ """Validates JSON structure and extracts DSL."""
35
+ try:
36
+ cleaned = self._clean_json(raw)
37
+ data = json.loads(cleaned)
38
+ if not isinstance(data, dict):
39
+ return False, "Output must be a JSON object"
40
+ if "type" not in data and "geometry_dsl" not in data:
41
+ return False, "Missing required 'type' or 'geometry_dsl' fields"
42
+ dsl = data.get("geometry_dsl", "")
43
+ if isinstance(dsl, list):
44
+ dsl = "\n".join(dsl)
45
+ data["geometry_dsl"] = dsl.strip()
46
+ return True, data
47
+ except Exception as e:
48
+ return False, f"JSON parse error: {e}"
49
+
50
+ async def process(
51
+ self,
52
+ text: str,
53
+ feedback: Optional[str] = None,
54
+ context: Optional[Dict[str, Any]] = None,
55
+ ) -> Dict[str, Any]:
56
+ logger.info(f"==[GeometryParserAgent] Parsing problem & generating DSL (len={len(text)}) (v7.0)==")
57
+ if feedback:
58
+ logger.warning(f"[GeometryParserAgent] Feedback from previous attempt: {feedback}")
59
+ if context:
60
+ logger.info(f"[GeometryParserAgent] Using previous context (dsl_len={len(context.get('geometry_dsl', ''))})")
61
+
62
+ system_prompt = """You are an expert Geometry Parser & DSL Generator.
63
+ Analyze the Vietnamese/LaTeX mathematical geometry problem and extract both the structured semantics AND the executable Geometry DSL program in a single step.
64
+
65
+ === DSL SPECIFICATION ===
66
+ -- 2D & 3D Basic Primitives --
67
+ POINT(A) — declare a point (supports A, B, A1, B1, A', B', S, O, M, N, H)
68
+ POINT(A, x, y, z) — declare a point with explicit coordinates
69
+ LENGTH(AB, 5) — distance between A and B is 5
70
+ ANGLE(A, 90) — angle at vertex A is 90°
71
+ PARALLEL(AB, CD) — segment AB is parallel to CD
72
+ PERPENDICULAR(AB, CD) — segment AB is perpendicular to CD
73
+ MIDPOINT(M, AB) — M is the midpoint of segment AB
74
+ SECTION(E, A, C, k) — E satisfies vector AE = k * vector AC (k is decimal, e.g. 0.5)
75
+ LINE(A, B) — infinite line passing through A and B
76
+ RAY(A, B) — ray starting at A and passing through B
77
+ CIRCLE(O, 5) — circle with center O and radius 5
78
+ SEGMENT(M, N) — auxiliary segment MN to be drawn
79
+ POLYGON_ORDER(A, B, C, D) — polygon boundary vertex ordering
80
+ TRIANGLE(ABC) — 2D triangle
81
+ SQUARE(ABCD) — square with vertices A, B, C, D
82
+ RECTANGLE(ABCD) — rectangle with vertices A, B, C, D
83
+ PARALLELOGRAM(ABCD) — parallelogram with vertices A, B, C, D
84
+
85
+ -- 3D Polyhedrons & Round Solids --
86
+ PYRAMID(S_ABCD) — pyramid with apex S and base ABCD (supports S_ABC, S_ABCD, S_ABCDE)
87
+ PRISM(ABC_DEF) — triangular prism with bases ABC and DEF
88
+ PRISM(ABCD_A1B1C1D1) — quadrilateral prism
89
+ TETRAHEDRON(ABCD) — tetrahedron with 4 vertices
90
+ CUBE(ABCD_A1B1C1D1) — cube
91
+ CUBOID(ABCD_A1B1C1D1) — rectangular cuboid
92
+ FRUSTUM_PYRAMID(ABCD_A1B1C1D1) — frustum of a pyramid (chóp cụt)
93
+ CYLINDER(O_O1, r, h) — cylinder with axis O-O1, radius r, height h
94
+ CONE(S_O, r, h) — cone with apex S, base center O, radius r, height h
95
+ SPHERE(O, r) — sphere with center O and radius r
96
+
97
+ -- 3D High-Level Spatial Relations --
98
+ PERPENDICULAR_PLANE(SA, ABCD) — line SA is perpendicular to plane ABCD (SA ⊥ base)
99
+ COPLANAR(A, B, C, D) — 4 points lie on the same plane
100
+ POINT_ON_PLANE(P, ABC) — point P lies on plane ABC
101
+
102
+ === OUTPUT FORMAT ===
103
+ Output ONLY a JSON object with this EXACT structure (no markdown, no extra keys):
104
+ {
105
+ "type": "cube|cuboid|tetrahedron|cone|cylinder|frustum|pyramid|prism|sphere|rectangle|triangle|circle|parallelogram|trapezoid|square|rhombus|general",
106
+ "entities": ["Point S", "Point A", "Point B", "Point C", "Point D", "Point O"],
107
+ "values": {"AB": 10, "SO": 15},
108
+ "target_question": "Tính thể tích khối chóp S.ABCD",
109
+ "analysis": "Tóm tắt bài toán ngắn gọn bằng tiếng Việt.",
110
+ "geometry_dsl": "PYRAMID(S_ABCD)\\nSQUARE(ABCD)\\nLENGTH(AB, 4)\\nLENGTH(SA, 5)\\nPERPENDICULAR_PLANE(SA, ABCD)"
111
+ }
112
+
113
+ === RULES ===
114
+ 1. If the problem specifies a 3D pyramid with a square or rectangle base (e.g. S.ABCD with square base ABCD side 4, height SA=5 with SA ⊥ đáy), generate:
115
+ PYRAMID(S_ABCD)
116
+ SQUARE(ABCD)
117
+ LENGTH(AB, 4)
118
+ LENGTH(SA, 5)
119
+ PERPENDICULAR_PLANE(SA, ABCD)
120
+ 2. If the problem mentions midpoints, auxiliary lines, include MIDPOINT(M, AB), SEGMENT(S, M), etc.
121
+ 3. Keep DSL commands clean, upper-case, and syntactically valid.
122
+ """
123
+
124
+ user_content = f"Đề bài toán:\n{text}"
125
+ if context:
126
+ user_content = f"PREVIOUS CONTEXT:\n{context.get('analysis', '')}\nDSL:\n{context.get('geometry_dsl', '')}\n\nNEW REQUEST:\n{text}"
127
+
128
+ if feedback:
129
+ user_content += f"\n\nPhản hồi từ lần chạy trước: {feedback}. Vui lòng sửa lại DSL và ràng buộc chính xác."
130
+
131
+ messages = [
132
+ {"role": "system", "content": system_prompt},
133
+ {"role": "user", "content": user_content},
134
+ ]
135
+
136
+ try:
137
+ data = await self.runtime.run(
138
+ agent="geometry_parser",
139
+ messages=messages,
140
+ validator=self._validate_parser_output,
141
+ )
142
+ except Exception as e:
143
+ logger.warning(f"[GeometryParserAgent] Agent runtime cascade failed: {e}. Using fallback structure.")
144
+ data = {
145
+ "type": "general",
146
+ "entities": [],
147
+ "values": {},
148
+ "target_question": text,
149
+ "analysis": text,
150
+ "geometry_dsl": "",
151
+ }
152
+
153
+ dsl = data.get("geometry_dsl", "")
154
+ if isinstance(dsl, list):
155
+ dsl = "\n".join(dsl)
156
+ data["geometry_dsl"] = dsl.strip()
157
+
158
+ logger.info(f"[GeometryParserAgent] Success: type={data.get('type')}, dsl_lines={len(data['geometry_dsl'].splitlines())}")
159
+ return data
agents/knowledge_agent.py ADDED
@@ -0,0 +1,175 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ from typing import Dict, Any
3
+
4
+ logger = logging.getLogger(__name__)
5
+
6
+
7
+ class KnowledgeAgent:
8
+ """Knowledge Agent: Stores geometric theorems and common patterns to augment Parser output."""
9
+
10
+ def augment_semantic_data(self, semantic_data: Dict[str, Any]) -> Dict[str, Any]:
11
+ logger.info("==[KnowledgeAgent] Augmenting semantic data (v5.2)==")
12
+ text = str(semantic_data.get("input_text", "")).lower()
13
+ logger.debug(f"[KnowledgeAgent] Input text for matching: '{text[:200]}'")
14
+
15
+ shape_type = self._detect_shape(text, semantic_data.get("type", ""))
16
+ if shape_type:
17
+ semantic_data["type"] = shape_type
18
+ values = semantic_data.get("values", {})
19
+ values = self._augment_values(shape_type, values, text)
20
+ semantic_data["values"] = values
21
+ else:
22
+ logger.info("[KnowledgeAgent] No special rule matched. Returning data unchanged.")
23
+
24
+ logger.debug(f"[KnowledgeAgent] Output semantic data: {semantic_data}")
25
+ return semantic_data
26
+
27
+ # ─── Shape detection ────────────────────────────────────────────────────
28
+ def _detect_shape(self, text: str, llm_type: str) -> str | None:
29
+ """Detect shape from text keywords. LLM type provides a hint."""
30
+ checks = [
31
+ # 3D Solids
32
+ (["lập phương", "hình lập phương", "khối lập phương", "cube"], "cube"),
33
+ (["hộp chữ nhật", "hình hộp chữ nhật", "khối hộp chữ nhật", "hình hộp", "cuboid", "parallelepiped"], "cuboid"),
34
+ (["tứ diện đều", "regular tetrahedron"], "regular_tetrahedron"),
35
+ (["tứ diện", "tetrahedron"], "tetrahedron"),
36
+ (["chóp cụt", "truncated pyramid", "frustum"], "frustum"),
37
+ (["chóp tứ giác đều", "chóp tam giác đều", "hình chóp đều"], "regular_pyramid"),
38
+ (["hình chóp", "khối chóp", "pyramid"], "pyramid"),
39
+ (["lăng trụ đứng", "lăng trụ đều", "right prism"], "right_prism"),
40
+ (["lăng trụ", "hình lăng trụ", "prism"], "prism"),
41
+ (["hình nón", "khối nón", "cone"], "cone"),
42
+ (["hình trụ", "khối trụ", "cylinder"], "cylinder"),
43
+ (["mặt cầu", "khối cầu", "hình cầu", "sphere"], "sphere"),
44
+
45
+ # 2D Shapes
46
+ (["hình vuông", "square"], "square"),
47
+ (["hình chữ nhật", "rectangle"], "rectangle"),
48
+ (["hình thoi", "rhombus"], "rhombus"),
49
+ (["hình bình hành", "parallelogram"], "parallelogram"),
50
+ (["hình thang vuông"], "right_trapezoid"),
51
+ (["hình thang", "trapezoid", "trapezium"], "trapezoid"),
52
+ (["tam giác vuông", "right triangle"], "right_triangle"),
53
+ (["tam giác đều", "equilateral triangle", "equilateral"], "equilateral_triangle"),
54
+ (["tam giác cân", "isosceles"], "isosceles_triangle"),
55
+ (["tam giác", "triangle"], "triangle"),
56
+ (["đường tròn", "circle"], "circle"),
57
+ ]
58
+ for keywords, shape in checks:
59
+ if any(kw in text for kw in keywords):
60
+ logger.info(f"[KnowledgeAgent] Rule MATCH: '{shape}' detected (keyword match).")
61
+ return shape
62
+
63
+ # Fallback: trust LLM-detected type if it's a known type
64
+ known = {
65
+ "cube", "cuboid", "tetrahedron", "regular_tetrahedron", "pyramid", "regular_pyramid",
66
+ "prism", "right_prism", "cone", "cylinder", "sphere", "frustum",
67
+ "rectangle", "square", "rhombus", "parallelogram",
68
+ "trapezoid", "right_trapezoid", "triangle", "right_triangle",
69
+ "equilateral_triangle", "isosceles_triangle", "circle",
70
+ }
71
+ if llm_type in known:
72
+ logger.info(f"[KnowledgeAgent] Using LLM-detected type '{llm_type}'.")
73
+ return llm_type
74
+
75
+ return None
76
+
77
+ # ─── Value augmentation ──────────────────────────────────────────────────
78
+ def _augment_values(self, shape: str, values: dict, text: str) -> dict:
79
+ ab = values.get("AB")
80
+ ad = values.get("AD")
81
+ bc = values.get("BC")
82
+ cd = values.get("CD")
83
+ side = ab or ad or bc or cd or values.get("side") or values.get("a")
84
+
85
+ if shape == "cube":
86
+ if side:
87
+ values.setdefault("side", side)
88
+ values.setdefault("AB", side)
89
+ values.setdefault("AD", side)
90
+ values.setdefault("AA1", side)
91
+ logger.info(f"[KnowledgeAgent] Cube: all edges={side}")
92
+
93
+ elif shape == "regular_tetrahedron":
94
+ if side:
95
+ values.setdefault("side", side)
96
+ values.setdefault("AB", side)
97
+ values.setdefault("AC", side)
98
+ values.setdefault("AD", side)
99
+ values.setdefault("BC", side)
100
+ values.setdefault("CD", side)
101
+ values.setdefault("DB", side)
102
+ logger.info(f"[KnowledgeAgent] Regular Tetrahedron: all 6 edges={side}")
103
+
104
+ elif shape == "cone":
105
+ r = values.get("radius") or values.get("r")
106
+ h = values.get("height") or values.get("h") or values.get("SO")
107
+ if r: values.setdefault("radius", r)
108
+ if h: values.setdefault("height", h)
109
+ logger.info(f"[KnowledgeAgent] Cone: radius={r}, height={h}")
110
+
111
+ elif shape == "cylinder":
112
+ r = values.get("radius") or values.get("r")
113
+ h = values.get("height") or values.get("h") or values.get("O1O2")
114
+ if r: values.setdefault("radius", r)
115
+ if h: values.setdefault("height", h)
116
+ logger.info(f"[KnowledgeAgent] Cylinder: radius={r}, height={h}")
117
+
118
+ elif shape == "rectangle":
119
+ if ab and ad:
120
+ values.setdefault("CD", ab)
121
+ values.setdefault("BC", ad)
122
+ values.setdefault("angle_A", 90)
123
+ logger.info(f"[KnowledgeAgent] Rectangle: AB=CD={ab}, AD=BC={ad}, angle_A=90°")
124
+ else:
125
+ values.setdefault("angle_A", 90)
126
+
127
+ elif shape == "square":
128
+ if side:
129
+ values.update({"AB": side, "AD": side, "angle_A": 90})
130
+ logger.info(f"[KnowledgeAgent] Square: side={side}, angle_A=90°")
131
+ else:
132
+ values.setdefault("angle_A", 90)
133
+
134
+ elif shape == "rhombus":
135
+ if side:
136
+ values.update({"AB": side, "BC": side, "CD": side, "DA": side})
137
+ logger.info(f"[KnowledgeAgent] Rhombus: all sides={side}")
138
+
139
+ elif shape == "parallelogram":
140
+ if ab:
141
+ values.setdefault("CD", ab)
142
+ if ad:
143
+ values.setdefault("BC", ad)
144
+ logger.info("[KnowledgeAgent] Parallelogram: AB||CD, AD||BC")
145
+
146
+ elif shape == "trapezoid":
147
+ logger.info("[KnowledgeAgent] Trapezoid: AB||CD (bottom||top)")
148
+
149
+ elif shape == "right_trapezoid":
150
+ logger.info("[KnowledgeAgent] Right trapezoid: AB||CD, AD⊥AB")
151
+ values.setdefault("angle_A", 90)
152
+
153
+ elif shape == "equilateral_triangle":
154
+ if side:
155
+ values.update({"AB": side, "BC": side, "CA": side, "angle_A": 60})
156
+ logger.info(f"[KnowledgeAgent] Equilateral triangle: all sides={side}, angle_A=60°")
157
+
158
+ elif shape == "right_triangle":
159
+ rt_vertex = _detect_right_angle_vertex(text)
160
+ values.setdefault(f"angle_{rt_vertex}", 90)
161
+ logger.info(f"[KnowledgeAgent] Right triangle: angle_{rt_vertex}=90°")
162
+
163
+ elif shape == "isosceles_triangle":
164
+ logger.info("[KnowledgeAgent] Isosceles triangle: AB=AC (default, LLM may override)")
165
+
166
+ return values
167
+
168
+
169
+ def _detect_right_angle_vertex(text: str) -> str:
170
+ """Heuristic: detect which vertex is right angle from text."""
171
+ for vertex in ["A", "B", "C", "D"]:
172
+ patterns = [f"vuông tại {vertex}", f"góc {vertex} vuông", f"right angle at {vertex}"]
173
+ if any(p.lower() in text for p in patterns):
174
+ return vertex
175
+ return "A"
agents/ocr_agent.py ADDED
@@ -0,0 +1,64 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ OCR Agent (v5.3).
3
+ Pure visual perception agent responsible for recognizing text, mathematical formulas, and layout
4
+ from geometry problem images using Pix2Text.
5
+ Strictly adheres to the Design Principle:
6
+ - OCR only extracts and structures visual content without LLM hallucinations or semantic alteration.
7
+ - Emits structured CanonicalOCRResult for downstream Problem Parser / VLM.
8
+ """
9
+
10
+ from __future__ import annotations
11
+
12
+ import logging
13
+ from typing import Any, Dict, Optional
14
+
15
+ from vision_ocr.canonical_schema import CanonicalOCRResult
16
+ from vision_ocr.pipeline import OcrVisionPipeline
17
+
18
+ logger = logging.getLogger(__name__)
19
+
20
+
21
+ class ImprovedOCRAgent:
22
+ """
23
+ Math OCR Agent (v5.3).
24
+ Wraps ``OcrVisionPipeline`` (Pix2Text) and produces CanonicalOCRResult.
25
+ """
26
+
27
+ def __init__(self, **kwargs):
28
+ self._vision = OcrVisionPipeline()
29
+ logger.info("[ImprovedOCRAgent] Math OCR Vision Pipeline ready (Pix2Text Engine).")
30
+
31
+ async def process_image(self, image_path: str) -> str:
32
+ """
33
+ Processes image and returns reconstructed Markdown text containing inline and display LaTeX.
34
+ """
35
+ canonical = await self._vision.process_image_canonical(image_path)
36
+ return canonical.text
37
+
38
+ async def process_image_canonical(self, image_path: str) -> CanonicalOCRResult:
39
+ """
40
+ Processes image and returns full CanonicalOCRResult structure:
41
+ - text: Markdown string with LaTeX formulas
42
+ - latex: List of all extracted mathematical expressions
43
+ - elements: Region bounding boxes and classifications
44
+ - reading_order: Document sequential layout reading order
45
+ - confidence: Extraction confidence score
46
+ """
47
+ return await self._vision.process_image_canonical(image_path)
48
+
49
+ async def process_url(self, url: str) -> str:
50
+ """
51
+ Fetches image from URL and returns reconstructed Markdown text with LaTeX.
52
+ """
53
+ return await self._vision.process_url(url)
54
+
55
+ async def process_url_canonical(self, url: str) -> CanonicalOCRResult:
56
+ """
57
+ Fetches image from URL and returns full CanonicalOCRResult.
58
+ """
59
+ return await self._vision.process_url_canonical(url)
60
+
61
+
62
+ class OCRAgent(ImprovedOCRAgent):
63
+ """Alias for backward compatibility."""
64
+ pass
agents/orchestrator.py ADDED
@@ -0,0 +1,330 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import json
2
+ import logging
3
+ from typing import Any, Dict, Optional
4
+
5
+ from agents.geometry_parser_agent import GeometryParserAgent
6
+ from agents.deepmath_solver_agent import DeepMathSolverAgent
7
+ from agents.ocr_agent import OCRAgent
8
+ from app.logutil import log_step
9
+ from app.ocr_celery import ocr_from_image_url
10
+ from manim_client.client import ManimClient
11
+ from manim_client.schemas import build_visualization_spec
12
+ from solver.dsl_parser import DSLParser
13
+ from solver.engine import GeometryEngine
14
+ from solver.validator import GeometryValidator, GeometryStatus
15
+
16
+ logger = logging.getLogger(__name__)
17
+
18
+ _CLIP = 2000
19
+
20
+
21
+ def _clip(val: Any, n: int = _CLIP) -> Optional[str]:
22
+ if val is None:
23
+ return None
24
+ if isinstance(val, str):
25
+ s = val
26
+ else:
27
+ s = json.dumps(val, ensure_ascii=False, default=str)
28
+ return s if len(s) <= n else s[:n] + "…"
29
+
30
+
31
+ def _step_io(step: str, input_val: Any = None, output_val: Any = None) -> None:
32
+ log_step(step, input=_clip(input_val), output=_clip(output_val))
33
+
34
+
35
+ class Orchestrator:
36
+ """
37
+ Refactored AI Core Orchestrator (v6.2 - Manim Video Module & Validation Integration):
38
+ - GeometryParserAgent (Merged semantic parser & DSL generator)
39
+ - DSLParser & GeometryEngine (Deterministic coordinate & topology resolver)
40
+ - GeometryValidator (Strict invariant & constraint validation)
41
+ - DeepMathSolverAgent (Program-Aided sandboxed SymPy mathematical solver)
42
+ - ManimClient (Cross-service VisualizationSpec & Video Generation)
43
+ """
44
+
45
+ def __init__(self):
46
+ self.geometry_parser_agent = GeometryParserAgent()
47
+ self.deepmath_solver = DeepMathSolverAgent()
48
+ self.ocr_agent = OCRAgent()
49
+ self.solver_engine = GeometryEngine()
50
+ self.dsl_parser = DSLParser()
51
+ self.geometry_validator = GeometryValidator()
52
+ self.manim_client = ManimClient()
53
+
54
+ def _generate_step_description(self, semantic_json: Dict[str, Any], engine_result: Dict[str, Any]) -> str:
55
+ """Generates step-by-step drawing instructions based on engine results."""
56
+ analysis = semantic_json.get("analysis", "")
57
+ if not analysis:
58
+ analysis = f"Giải bài toán về {semantic_json.get('type', 'hình học')}."
59
+
60
+ steps = ["\n\n**Các bước dựng hình:**"]
61
+ drawing_phases = engine_result.get("drawing_phases", [])
62
+
63
+ def clean_pt(p: str) -> str:
64
+ return str(p).replace("_prime", "'")
65
+
66
+ for phase in drawing_phases:
67
+ label = phase.get("label", f"Giai đoạn {phase['phase']}")
68
+ points = ", ".join([clean_pt(p) for p in phase.get("points", [])])
69
+ segments = ", ".join([f"{clean_pt(s[0])}{clean_pt(s[1])}" for s in phase.get("segments", [])])
70
+
71
+ step_text = f"- **{label}**:"
72
+ if points:
73
+ step_text += f" Xác định các điểm {points}."
74
+ if segments:
75
+ step_text += f" Vẽ các đoạn thẳng {segments}."
76
+ steps.append(step_text)
77
+
78
+ circles = engine_result.get("circles", [])
79
+ for c in circles:
80
+ steps.append(f"- **Đường tròn**: Vẽ đường tròn tâm {clean_pt(c['center'])} bán kính {c['radius']}.")
81
+
82
+ return analysis + "\n".join(steps)
83
+
84
+ async def run(
85
+ self,
86
+ text: str,
87
+ image_url: Optional[str] = None,
88
+ job_id: Optional[str] = None,
89
+ session_id: Optional[str] = None,
90
+ status_callback=None,
91
+ history: Optional[list] = None,
92
+ generate_video: bool = True,
93
+ ) -> Dict[str, Any]:
94
+ """
95
+ Runs the streamlined v6.1 AI Core pipeline with Manim Video Module integration.
96
+ """
97
+ _step_io(
98
+ "orchestrate_start",
99
+ input_val={
100
+ "job_id": job_id,
101
+ "text_len": len(text or ""),
102
+ "image_url": image_url,
103
+ "history_len": len(history or []),
104
+ },
105
+ output_val=None,
106
+ )
107
+
108
+ if status_callback:
109
+ await status_callback("processing")
110
+
111
+ # 1. Extract context from history (if any)
112
+ previous_context = None
113
+ if history:
114
+ for msg in reversed(history):
115
+ if msg.get("role") == "assistant" and msg.get("metadata", {}).get("geometry_dsl"):
116
+ previous_context = {
117
+ "geometry_dsl": msg["metadata"]["geometry_dsl"],
118
+ "coordinates": msg["metadata"].get("coordinates", {}),
119
+ "analysis": msg.get("content", ""),
120
+ }
121
+ break
122
+
123
+ if previous_context:
124
+ _step_io("context_found", input_val=None, output_val={"dsl_len": len(previous_context["geometry_dsl"])})
125
+
126
+ # 2. Gather input text (OCR or direct)
127
+ input_text = text
128
+ ocr_metadata = {}
129
+ if image_url:
130
+ ocr_result = await ocr_from_image_url(image_url, self.ocr_agent)
131
+ input_text = ocr_result.text
132
+ ocr_metadata = {
133
+ "ocr_confidence": ocr_result.confidence,
134
+ "vlm_correction": ocr_result.metadata.get("vlm_correction", False),
135
+ "original_confidence": ocr_result.metadata.get("original_confidence"),
136
+ }
137
+ _step_io("step1_ocr", input_val=image_url, output_val={
138
+ "text_len": len(input_text),
139
+ "confidence": ocr_result.confidence,
140
+ "vlm_correction": ocr_metadata.get("vlm_correction"),
141
+ })
142
+ else:
143
+ _step_io("step1_ocr", input_val="(no image)", output_val=text)
144
+
145
+ feedback = None
146
+ MAX_RETRIES = 2
147
+ engine_result = None
148
+ coordinates = {}
149
+ is_3d = False
150
+ dsl_code = ""
151
+ geometry_status = GeometryStatus.FAILED
152
+ semantic_json: Dict[str, Any] = {}
153
+
154
+ # 3. GeometryParserAgent Loop (Semantic Parsing + DSL Generation)
155
+ for attempt in range(MAX_RETRIES + 1):
156
+ _step_io("attempt", input_val=f"{attempt + 1}/{MAX_RETRIES + 1}", output_val=None)
157
+ if status_callback:
158
+ await status_callback("solving")
159
+
160
+ _step_io("step2_geometry_parse", input_val=f"{input_text[:60]}...", output_val=None)
161
+ semantic_json = await self.geometry_parser_agent.process(
162
+ input_text, feedback=feedback, context=previous_context
163
+ )
164
+ semantic_json["input_text"] = input_text
165
+ dsl_code = semantic_json.get("geometry_dsl", "")
166
+ _step_io("step2_geometry_parse", input_val=None, output_val=semantic_json)
167
+
168
+ if not dsl_code:
169
+ dsl_code = f"// Problem text: {input_text}"
170
+
171
+ _step_io("step3_dsl_parse", input_val=dsl_code, output_val=None)
172
+ points, constraints, is_3d = self.dsl_parser.parse(dsl_code)
173
+ _step_io(
174
+ "step3_dsl_parse",
175
+ input_val=None,
176
+ output_val={
177
+ "points": len(points),
178
+ "constraints": len(constraints),
179
+ "is_3d": is_3d,
180
+ },
181
+ )
182
+
183
+ # 4. Geometry Solver Engine
184
+ _step_io("step4_solve_geometry", input_val=f"{len(points)} pts / {len(constraints)} cons (is_3d={is_3d})", output_val=None)
185
+ import anyio
186
+ engine_result = await anyio.to_thread.run_sync(self.solver_engine.solve, points, constraints, is_3d)
187
+
188
+ if engine_result:
189
+ coordinates = engine_result.get("coordinates", {})
190
+ _step_io("step4_solve_geometry", input_val=None, output_val=coordinates)
191
+
192
+ # Validate geometry against mathematical invariants and constraints
193
+ val_res = self.geometry_validator.validate(engine_result, constraints, is_3d)
194
+ _step_io(
195
+ "step4_validate_geometry",
196
+ input_val=f"{val_res.checked_count} constraints checked",
197
+ output_val={"is_valid": val_res.is_valid, "errors": val_res.errors[:3]},
198
+ )
199
+
200
+ if val_res.is_valid:
201
+ geometry_status = GeometryStatus.VALID
202
+ logger.info(
203
+ "[Orchestrator] geometry solved and validated job_id=%s is_3d=%s n_coords=%d",
204
+ job_id,
205
+ is_3d,
206
+ len(coordinates) if isinstance(coordinates, dict) else 0,
207
+ )
208
+ break
209
+ else:
210
+ structured_fb = val_res.to_structured_feedback()
211
+ import json as _json
212
+ feedback = f"Geometry validation failed. Structured errors:\n{_json.dumps(structured_fb, ensure_ascii=False, indent=2)}\nPlease correct the DSL to satisfy all constraints."
213
+ _step_io("step4_validate_geometry", input_val=f"attempt {attempt + 1}", output_val=feedback)
214
+ else:
215
+ feedback = "Geometry solver failed to find a valid solution for the given constraints. Parallelism or lengths might be inconsistent."
216
+ _step_io("step4_solve_geometry", input_val=f"attempt {attempt + 1}", output_val=feedback)
217
+
218
+ if attempt == MAX_RETRIES:
219
+ # 1. If engine_result produced valid non-empty coordinates, accept them gracefully on final attempt
220
+ if engine_result and coordinates and len(coordinates) >= 3:
221
+ geometry_status = GeometryStatus.DEGRADED
222
+ logger.warning("[Orchestrator] Proceeding with DEGRADED coordinates on final attempt despite validation warnings")
223
+ break
224
+
225
+ # 2. If engine_result is empty, attempt a relaxed solver pass with primary constraints only
226
+ if points and constraints:
227
+ relaxed_constraints = [
228
+ c for c in constraints
229
+ if c.get("kind") != "AUXILIARY" and c.get("type") in ("length", "polygon", "point", "perpendicular_to_base", "pyramid", "prism", "cube", "tetrahedron")
230
+ ]
231
+ try:
232
+ relaxed_result = await anyio.to_thread.run_sync(self.solver_engine.solve, points, relaxed_constraints, is_3d)
233
+ if relaxed_result and relaxed_result.get("coordinates"):
234
+ engine_result = relaxed_result
235
+ coordinates = relaxed_result.get("coordinates", {})
236
+ geometry_status = GeometryStatus.DEGRADED
237
+ logger.info("[Orchestrator] Relaxed constraint solve succeeded as DEGRADED fallback")
238
+ break
239
+ except Exception as e:
240
+ logger.warning(f"[Orchestrator] Relaxed solve attempt warning: {e}")
241
+
242
+ # 3. If geometry coordinates still cannot be solved, proceed to DeepMath solver so the user still gets the math steps & answer
243
+ geometry_status = GeometryStatus.FAILED
244
+ logger.warning("[Orchestrator] Geometry engine exhausted attempts (FAILED); proceeding with semantic data for DeepMath solve")
245
+ engine_result = engine_result or {"coordinates": {}, "is_3d": is_3d, "drawing_phases": []}
246
+ break
247
+
248
+ # 5. DeepMath Solver (Program-Aided Sandboxed SymPy Reasoning)
249
+ solution = None
250
+ _step_io("step5_deepmath_solve", input_val=semantic_json.get("target_question"), output_val=None)
251
+ solution = await self.deepmath_solver.solve(
252
+ problem_text=input_text,
253
+ target_question=semantic_json.get("target_question"),
254
+ semantic_data=semantic_json,
255
+ geometry_context=engine_result,
256
+ )
257
+ _step_io("step5_deepmath_solve", input_val=None, output_val=solution.get("answer"))
258
+
259
+ final_analysis = self._generate_step_description(semantic_json, engine_result or {})
260
+
261
+ # 6. Build VisualizationSpec and initiate Manim Video Generation (Async)
262
+ visualization_info = None
263
+ if generate_video:
264
+ try:
265
+ _step_io("step6_build_visualization_spec", input_val=None, output_val="building")
266
+ spec = build_visualization_spec(
267
+ problem_text=input_text,
268
+ solution_steps=solution.get("steps", []) if solution else [],
269
+ coordinates=coordinates,
270
+ engine_result=engine_result,
271
+ semantic_data=semantic_json,
272
+ is_3d=is_3d,
273
+ )
274
+ _step_io(
275
+ "step6_build_visualization_spec",
276
+ input_val=None,
277
+ output_val={
278
+ "geometry_objects": len(spec.geometry),
279
+ "animation_beats": len(spec.animations),
280
+ },
281
+ )
282
+
283
+ # Submit job to Manim Video Module
284
+ render_resp = await self.manim_client.submit_render_job(spec)
285
+ render_dict = render_resp.to_dict()
286
+ visualization_info = {
287
+ "spec": spec.model_dump(mode="json"),
288
+ "job_id": str(render_resp.job_id),
289
+ "project_id": str(render_resp.project_id) if render_resp.project_id else None,
290
+ "status": render_resp.status,
291
+ "video_url": render_resp.video_url,
292
+ "error": render_dict.get("error"),
293
+ }
294
+ _step_io(
295
+ "step7_manim_job_submitted",
296
+ input_val=str(render_resp.job_id),
297
+ output_val=render_resp.status,
298
+ )
299
+ except Exception as e:
300
+ logger.warning(f"[Orchestrator] Failed to package VisualizationSpec / contact Manim: {e}")
301
+ visualization_info = {
302
+ "status": "failed",
303
+ "error": str(e),
304
+ }
305
+
306
+ _step_io("orchestrate_done", input_val=job_id, output_val="success")
307
+
308
+ return {
309
+ "status": "success",
310
+ "job_id": job_id,
311
+ "geometry_status": geometry_status.value,
312
+ "ocr_metadata": ocr_metadata,
313
+ "geometry_dsl": dsl_code,
314
+ "coordinates": coordinates,
315
+ "polygon_order": (engine_result or {}).get("polygon_order", []),
316
+ "circles": (engine_result or {}).get("circles", []),
317
+ "solids": (engine_result or {}).get("solids", []),
318
+ "faces": (engine_result or {}).get("faces", []),
319
+ "lines": (engine_result or {}).get("lines", []),
320
+ "rays": (engine_result or {}).get("rays", []),
321
+ "drawing_phases": (engine_result or {}).get("drawing_phases", []),
322
+ "visualization_graph": (engine_result or {}).get("visualization_graph"),
323
+ "geometry_objects": (engine_result or {}).get("geometry_objects", []),
324
+ "auxiliary": (engine_result or {}).get("auxiliary", []),
325
+ "semantic": semantic_json,
326
+ "semantic_analysis": final_analysis,
327
+ "solution": solution,
328
+ "visualization": visualization_info,
329
+ "is_3d": is_3d,
330
+ }
agents/renderer_agent.py ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ """Shim: geometry rendering lives in ``geometry_render`` (worker-safe package)."""
2
+
3
+ from geometry_render.renderer import RendererAgent
4
+
5
+ __all__ = ["RendererAgent"]
agents/runtime.py ADDED
@@ -0,0 +1,122 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import inspect
2
+ import logging
3
+ from typing import List, Dict, Any, Optional, Callable, Tuple, Union
4
+ from config.loader import load_agent_config
5
+ from config.schemas import AgentConfig
6
+ from llm.service import LLMService, get_llm_service
7
+
8
+ logger = logging.getLogger(__name__)
9
+
10
+
11
+ class AgentRuntime:
12
+ """
13
+ Agent Runtime & Cascading Controller:
14
+ Coordinates agent configuration resolution, model tier escalation, and programmatic validation.
15
+ """
16
+
17
+ def __init__(self, llm_service: Optional[LLMService] = None):
18
+ self.llm_service = llm_service or get_llm_service()
19
+
20
+ async def run(
21
+ self,
22
+ agent: str,
23
+ messages: List[Dict[str, Any]],
24
+ validator: Optional[Callable[[str], Union[Tuple[bool, Any], Any]]] = None,
25
+ response_format: Optional[Dict[str, Any]] = None,
26
+ **kwargs
27
+ ) -> Any:
28
+ """
29
+ Executes an agent run across tiered model cascades with validator-guided escalation.
30
+ Temperature and max_tokens are always resolved from AgentConfig (single source of truth).
31
+ """
32
+ config: AgentConfig = load_agent_config(agent)
33
+ temperature = config.temperature
34
+ max_tokens = config.max_tokens
35
+
36
+ last_error = None
37
+ current_messages = list(messages)
38
+
39
+ logger.info(f"[AgentRuntime] Starting run for agent '{agent}' with {len(config.tiers)} tier(s)...")
40
+
41
+ for tier_idx, tier in enumerate(config.tiers, start=1):
42
+ for attempt in range(tier.max_attempts):
43
+ logger.info(
44
+ f"[AgentRuntime] Agent '{agent}' Tier {tier_idx}/{len(config.tiers)} "
45
+ f"({tier.model}) - Attempt {attempt + 1}/{tier.max_attempts}"
46
+ )
47
+
48
+ try:
49
+ tier_reasoning_effort = tier.reasoning_effort if tier.reasoning_effort is not None else config.reasoning_effort
50
+ raw_output = await self.llm_service.acomplete(
51
+ model=tier.model,
52
+ messages=current_messages,
53
+ temperature=temperature,
54
+ max_tokens=max_tokens,
55
+ timeout=config.timeout_seconds,
56
+ response_format=response_format,
57
+ reasoning_effort=tier_reasoning_effort,
58
+ agent_name=agent,
59
+ tier_index=tier_idx,
60
+ **kwargs
61
+ )
62
+
63
+ # Programmatic validation (Level 2 Cascade Trigger)
64
+ if validator:
65
+ try:
66
+ if inspect.iscoroutinefunction(validator):
67
+ val_result = await validator(raw_output)
68
+ else:
69
+ val_result = validator(raw_output)
70
+
71
+ # Expect (is_valid, payload_or_error)
72
+ if isinstance(val_result, tuple) and len(val_result) == 2:
73
+ is_valid, payload = val_result
74
+ if is_valid:
75
+ logger.info(
76
+ f"[AgentRuntime] Agent '{agent}' Tier {tier_idx} validation PASSED."
77
+ )
78
+ return payload
79
+ else:
80
+ logger.warning(
81
+ f"[AgentRuntime] Agent '{agent}' Tier {tier_idx} validation FAILED: {payload}. "
82
+ "Escalating..."
83
+ )
84
+ # Provide feedback to conversation context for subsequent attempts
85
+ current_messages.append({"role": "assistant", "content": raw_output})
86
+ current_messages.append({
87
+ "role": "user",
88
+ "content": f"Your previous output failed validation: {payload}. Please correct the issues and provide a valid response."
89
+ })
90
+ continue
91
+ elif bool(val_result):
92
+ return val_result
93
+ except Exception as val_e:
94
+ logger.warning(
95
+ f"[AgentRuntime] Validator raised exception on Tier {tier_idx}: {val_e}. Escalating..."
96
+ )
97
+ last_error = val_e
98
+ continue
99
+ else:
100
+ # No validator required, output is accepted
101
+ return raw_output
102
+
103
+ except Exception as tier_e:
104
+ logger.warning(
105
+ f"[AgentRuntime] Tier {tier_idx} attempt {attempt + 1} failed: {tier_e}"
106
+ )
107
+ last_error = tier_e
108
+
109
+ # All model tiers exhausted
110
+ raise RuntimeError(
111
+ f"Agent '{agent}' cascade exhausted all {len(config.tiers)} model tiers. Last error: {last_error}"
112
+ )
113
+
114
+
115
+ _GLOBAL_AGENT_RUNTIME: Optional[AgentRuntime] = None
116
+
117
+
118
+ def get_agent_runtime() -> AgentRuntime:
119
+ global _GLOBAL_AGENT_RUNTIME
120
+ if _GLOBAL_AGENT_RUNTIME is None:
121
+ _GLOBAL_AGENT_RUNTIME = AgentRuntime()
122
+ return _GLOBAL_AGENT_RUNTIME
agents/torch_ultralytics_compat.py ADDED
@@ -0,0 +1,5 @@
 
 
 
 
 
 
1
+ """Shim: moved to ``vision_ocr.compat`` for OCR worker isolation."""
2
+
3
+ from vision_ocr.compat import allow_ultralytics_weights
4
+
5
+ __all__ = ["allow_ultralytics_weights"]
agents/vlm_corrector.py ADDED
@@ -0,0 +1,182 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ VLM OCR Corrector Agent.
3
+
4
+ Uses a multimodal Vision-Language Model to correct OCR errors when confidence is low.
5
+ Strictly adheres to READ/CORRECT/PRESERVE boundaries — never SOLVE/INFER/INVENT.
6
+ """
7
+
8
+ from __future__ import annotations
9
+
10
+ import json
11
+ import logging
12
+ import re
13
+ from typing import Any, Dict, List, Optional
14
+
15
+ from config.schemas import OCRCorrectionConfig
16
+ from llm.service import get_llm_service
17
+ from vision_ocr.canonical_schema import CanonicalOCRResult
18
+
19
+ logger = logging.getLogger(__name__)
20
+
21
+
22
+ class VLMCorrectorAgent:
23
+ """
24
+ VLM-based OCR correction agent.
25
+
26
+ Receives raw image + OCR output + confidence and uses a multimodal LLM
27
+ to correct OCR recognition errors.
28
+
29
+ Strict boundary:
30
+ - READ: Re-read text and formulas from the image
31
+ - CORRECT: Fix OCR misrecognitions
32
+ - PRESERVE: Keep all original information intact
33
+
34
+ Never:
35
+ - SOLVE: Do not solve the math problem
36
+ - INFER: Do not infer missing geometry values
37
+ - INVENT: Do not add information not visible in the image
38
+ """
39
+
40
+ def __init__(self, config: Optional[OCRCorrectionConfig] = None):
41
+ self.config = config or OCRCorrectionConfig()
42
+ self.llm_service = get_llm_service()
43
+
44
+ async def correct(
45
+ self,
46
+ ocr_result: CanonicalOCRResult,
47
+ image_url: Optional[str] = None,
48
+ image_path: Optional[str] = None,
49
+ ) -> Dict[str, Any]:
50
+ """
51
+ Correct OCR errors using VLM.
52
+
53
+ Args:
54
+ ocr_result: The original OCR result with text and confidence
55
+ image_url: URL of the original image (for multimodal input)
56
+ image_path: Local path to the original image
57
+
58
+ Returns:
59
+ Dict with corrected_text, changed, confidence, corrections[]
60
+ """
61
+ if not image_url and not image_path:
62
+ logger.warning("[VLMCorrector] No image provided, skipping correction")
63
+ return {"changed": False, "corrected_text": ocr_result.text, "confidence": ocr_result.confidence, "corrections": []}
64
+
65
+ system_prompt = """You are an OCR Correction Agent for Vietnamese mathematical geometry problems.
66
+
67
+ === YOUR TASK ===
68
+ You receive:
69
+ 1. An image of a math problem
70
+ 2. OCR-extracted text (which may contain errors)
71
+ 3. OCR confidence score
72
+
73
+ Your job is to CORRECT OCR recognition errors by re-reading the image carefully.
74
+
75
+ === STRICT BOUNDARIES ===
76
+ You MUST ONLY:
77
+ - READ: Re-read text, numbers, and mathematical formulas from the image
78
+ - CORRECT: Fix misrecognized characters, numbers, symbols, and LaTeX
79
+ - PRESERVE: Keep all original information intact
80
+
81
+ You MUST NOT:
82
+ - SOLVE: Do not solve or attempt to solve the math problem
83
+ - INFER: Do not infer missing values or geometry relationships
84
+ - INVENT: Do not add any information not visible in the image
85
+ - If a value is unclear or unreadable, mark it as "?" — do NOT guess
86
+
87
+ === OUTPUT FORMAT ===
88
+ Output ONLY a JSON object:
89
+ {
90
+ "corrected_text": "The corrected full text with proper LaTeX",
91
+ "changed": true/false,
92
+ "confidence": 0.95,
93
+ "corrections": [
94
+ {
95
+ "original": "SA = 8",
96
+ "corrected": "SA = 6",
97
+ "reason": "OCR misread digit 6 as 8"
98
+ }
99
+ ]
100
+ }
101
+
102
+ If no corrections are needed, set "changed": false and return the original text."""
103
+
104
+ user_content_parts = []
105
+
106
+ # Add image content for multimodal input
107
+ if image_url:
108
+ user_content_parts.append({
109
+ "type": "image_url",
110
+ "image_url": {"url": image_url},
111
+ })
112
+
113
+ user_content_parts.append({
114
+ "type": "text",
115
+ "text": f"""OCR Extracted Text (confidence: {ocr_result.confidence:.3f}):
116
+
117
+ {ocr_result.text}
118
+
119
+ Please carefully compare the image with the OCR text above and correct any recognition errors.""",
120
+ })
121
+
122
+ messages = [
123
+ {"role": "system", "content": system_prompt},
124
+ {"role": "user", "content": user_content_parts},
125
+ ]
126
+
127
+ try:
128
+ raw_response = await self.llm_service.acomplete(
129
+ model=self.config.model,
130
+ messages=messages,
131
+ temperature=self.config.temperature,
132
+ max_tokens=self.config.max_tokens,
133
+ timeout=self.config.timeout_seconds,
134
+ agent_name="vlm_corrector",
135
+ )
136
+
137
+ result = self._parse_correction_response(raw_response, ocr_result.text)
138
+ logger.info(
139
+ f"[VLMCorrector] Correction result: changed={result.get('changed')}, "
140
+ f"corrections={len(result.get('corrections', []))}"
141
+ )
142
+ return result
143
+
144
+ except Exception as e:
145
+ logger.error(f"[VLMCorrector] Correction failed: {e}")
146
+ return {
147
+ "changed": False,
148
+ "corrected_text": ocr_result.text,
149
+ "confidence": ocr_result.confidence,
150
+ "corrections": [],
151
+ }
152
+
153
+ def _parse_correction_response(self, raw: str, original_text: str) -> Dict[str, Any]:
154
+ """Parse VLM correction response JSON."""
155
+ try:
156
+ cleaned = raw.strip()
157
+ # Extract JSON from markdown code block if present
158
+ json_match = re.search(r"```(?:json)?\s*(.*?)\s*```", cleaned, re.DOTALL)
159
+ if json_match:
160
+ cleaned = json_match.group(1).strip()
161
+
162
+ # Try direct JSON parse
163
+ brace_match = re.search(r"(\{.*\})", cleaned, re.DOTALL)
164
+ if brace_match:
165
+ cleaned = brace_match.group(1)
166
+
167
+ data = json.loads(cleaned)
168
+
169
+ return {
170
+ "corrected_text": data.get("corrected_text", original_text),
171
+ "changed": bool(data.get("changed", False)),
172
+ "confidence": float(data.get("confidence", 0.9)),
173
+ "corrections": data.get("corrections", []),
174
+ }
175
+ except (json.JSONDecodeError, Exception) as e:
176
+ logger.warning(f"[VLMCorrector] Failed to parse response: {e}")
177
+ return {
178
+ "changed": False,
179
+ "corrected_text": original_text,
180
+ "confidence": 0.5,
181
+ "corrections": [],
182
+ }
app/__init__.py ADDED
File without changes
app/chat_image_upload.py ADDED
@@ -0,0 +1,253 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Validate and upload chat/solve attachment images to Supabase Storage (image bucket)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import logging
6
+ import os
7
+ import uuid
8
+ from typing import Any, Dict, Tuple
9
+
10
+ from fastapi import HTTPException
11
+
12
+ logger = logging.getLogger(__name__)
13
+
14
+
15
+ def _get_next_image_version(session_id: str) -> int:
16
+ """Same logic as worker.asset_manager.get_next_version for asset_type image."""
17
+ from app.supabase_client import get_supabase
18
+
19
+ supabase = get_supabase()
20
+ try:
21
+ res = (
22
+ supabase.table("session_assets")
23
+ .select("version")
24
+ .eq("session_id", session_id)
25
+ .eq("asset_type", "image")
26
+ .order("version", desc=True)
27
+ .limit(1)
28
+ .execute()
29
+ )
30
+ if res.data:
31
+ return res.data[0]["version"] + 1
32
+ return 1
33
+ except Exception as e:
34
+ logger.error("Error fetching image version: %s", e)
35
+ return 1
36
+
37
+ _MAX_BYTES_DEFAULT = 10 * 1024 * 1024
38
+
39
+ _EXT_TO_MIME: dict[str, str] = {
40
+ ".png": "image/png",
41
+ ".jpg": "image/jpeg",
42
+ ".jpeg": "image/jpeg",
43
+ ".webp": "image/webp",
44
+ ".gif": "image/gif",
45
+ ".bmp": "image/bmp",
46
+ }
47
+
48
+
49
+ def _max_bytes() -> int:
50
+ raw = os.getenv("CHAT_IMAGE_MAX_BYTES")
51
+ if raw and raw.isdigit():
52
+ return min(int(raw), 50 * 1024 * 1024)
53
+ return _MAX_BYTES_DEFAULT
54
+
55
+
56
+ def _magic_ok(ext: str, body: bytes) -> bool:
57
+ if len(body) < 2:
58
+ return False
59
+ if ext == ".png":
60
+ return len(body) >= 8 and body.startswith(b"\x89PNG\r\n\x1a\n")
61
+ if ext in (".jpg", ".jpeg"):
62
+ return len(body) >= 3 and body.startswith(b"\xff\xd8\xff")
63
+ if ext == ".webp":
64
+ return len(body) >= 12 and body.startswith(b"RIFF") and body[8:12] == b"WEBP"
65
+ if ext == ".gif":
66
+ return len(body) >= 6 and (body.startswith(b"GIF87a") or body.startswith(b"GIF89a"))
67
+ if ext == ".bmp":
68
+ return len(body) >= 2 and body.startswith(b"BM")
69
+ return False
70
+
71
+
72
+ def validate_chat_image_bytes(
73
+ filename: str | None,
74
+ body: bytes,
75
+ declared_content_type: str | None,
76
+ ) -> Tuple[str, str]:
77
+ """
78
+ Validate size, extension, and magic bytes.
79
+ Returns (extension_with_dot, content_type).
80
+ """
81
+ max_b = _max_bytes()
82
+ if not body:
83
+ raise HTTPException(status_code=400, detail="Empty file.")
84
+ if len(body) > max_b:
85
+ raise HTTPException(
86
+ status_code=413,
87
+ detail=f"Image too large (max {max_b // (1024 * 1024)} MB).",
88
+ )
89
+
90
+ ext = os.path.splitext(filename or "")[1].lower()
91
+ if not ext:
92
+ ext = ".png"
93
+ if ext not in _EXT_TO_MIME:
94
+ raise HTTPException(
95
+ status_code=400,
96
+ detail=f"Unsupported image type: {ext}. Allowed: {', '.join(sorted(_EXT_TO_MIME))}",
97
+ )
98
+
99
+ if not _magic_ok(ext, body):
100
+ raise HTTPException(
101
+ status_code=400,
102
+ detail="File content does not match declared image type.",
103
+ )
104
+
105
+ mime = _EXT_TO_MIME[ext]
106
+ if declared_content_type:
107
+ decl = declared_content_type.split(";")[0].strip().lower()
108
+ if decl and decl not in ("application/octet-stream", mime) and decl != mime:
109
+ logger.warning(
110
+ "Content-Type mismatch (declared=%s, inferred=%s); using inferred.",
111
+ declared_content_type,
112
+ mime,
113
+ )
114
+ return ext, mime
115
+
116
+
117
+ def upload_session_chat_image(
118
+ session_id: str,
119
+ job_id: str,
120
+ file_bytes: bytes,
121
+ ext_with_dot: str,
122
+ content_type: str,
123
+ ) -> Dict[str, Any]:
124
+ """
125
+ Upload to SUPABASE_IMAGE_BUCKET (default: image), insert session_assets row.
126
+ Returns dict with public_url, storage_path, version, session_asset_id (if returned).
127
+ """
128
+ from app.supabase_client import get_supabase
129
+
130
+ supabase = get_supabase()
131
+ bucket_name = os.getenv("SUPABASE_IMAGE_BUCKET", "image")
132
+ raw_ext = ext_with_dot.lstrip(".").lower()
133
+
134
+ max_retries = 3
135
+ last_err = None
136
+ for attempt in range(max_retries):
137
+ version = _get_next_image_version(session_id) + attempt
138
+ file_name = f"image_v{version}_{job_id}.{raw_ext}"
139
+ storage_path = f"sessions/{session_id}/{file_name}"
140
+
141
+ try:
142
+ supabase.storage.from_(bucket_name).upload(
143
+ path=storage_path,
144
+ file=file_bytes,
145
+ file_options={"content-type": content_type, "upsert": "true"},
146
+ )
147
+ public_url = supabase.storage.from_(bucket_name).get_public_url(storage_path)
148
+ if isinstance(public_url, dict):
149
+ public_url = public_url.get("publicUrl") or public_url.get("public_url") or str(public_url)
150
+
151
+ row = {
152
+ "session_id": session_id,
153
+ "job_id": job_id,
154
+ "asset_type": "image",
155
+ "storage_path": storage_path,
156
+ "public_url": public_url,
157
+ "version": version,
158
+ }
159
+ ins = supabase.table("session_assets").insert(row).select("id").execute()
160
+ asset_id = None
161
+ if ins.data and len(ins.data) > 0:
162
+ asset_id = ins.data[0].get("id")
163
+
164
+ log_data = {
165
+ "public_url": public_url,
166
+ "storage_path": storage_path,
167
+ "version": version,
168
+ "session_asset_id": str(asset_id) if asset_id else None,
169
+ }
170
+ logger.info("Uploaded chat image: %s", log_data)
171
+ return {
172
+ "public_url": public_url,
173
+ "storage_path": storage_path,
174
+ "version": version,
175
+ "session_asset_id": str(asset_id) if asset_id else None,
176
+ }
177
+ except Exception as e:
178
+ last_err = e
179
+ logger.warning(
180
+ "Retry uploading chat image for session %s (attempt %d/%d): %s",
181
+ session_id,
182
+ attempt + 1,
183
+ max_retries,
184
+ e,
185
+ )
186
+
187
+ raise HTTPException(
188
+ status_code=500,
189
+ detail=f"Failed to upload image after {max_retries} attempts: {last_err}",
190
+ )
191
+
192
+
193
+ def upload_ephemeral_ocr_blob(
194
+ file_bytes: bytes,
195
+ ext_with_dot: str,
196
+ content_type: str,
197
+ ) -> Tuple[str, str]:
198
+ """
199
+ Upload bytes to image bucket under _ocr_temp/ for worker-only OCR (no session_assets row).
200
+ Returns (storage_path, public_url). Caller must delete_storage_object when done.
201
+ """
202
+ from app.supabase_client import get_supabase
203
+
204
+ bucket_name = os.getenv("SUPABASE_IMAGE_BUCKET", "image")
205
+ raw_ext = ext_with_dot.lstrip(".").lower() or "png"
206
+ name = f"_ocr_temp/{uuid.uuid4().hex}.{raw_ext}"
207
+ supabase = get_supabase()
208
+ supabase.storage.from_(bucket_name).upload(
209
+ path=name,
210
+ file=file_bytes,
211
+ file_options={"content-type": content_type},
212
+ )
213
+ public_url = supabase.storage.from_(bucket_name).get_public_url(name)
214
+ if isinstance(public_url, dict):
215
+ public_url = public_url.get("publicUrl") or public_url.get("public_url") or str(public_url)
216
+ return name, public_url
217
+
218
+
219
+ def delete_storage_object(bucket_name: str, storage_path: str) -> None:
220
+ try:
221
+ from app.supabase_client import get_supabase
222
+
223
+ supabase = get_supabase()
224
+ if supabase:
225
+ supabase.storage.from_(bucket_name).remove([storage_path])
226
+ except Exception as e:
227
+ logger.warning("delete_storage_object failed path=%s: %s", storage_path, e)
228
+
229
+
230
+ def cleanup_session_storage(session_id: str) -> None:
231
+ """List and delete all storage files in video and image buckets under sessions/{session_id}/."""
232
+ from app.supabase_client import get_supabase
233
+
234
+ supabase = get_supabase()
235
+ if not supabase:
236
+ return
237
+
238
+ folder = f"sessions/{session_id}"
239
+ for bucket in ["image", os.getenv("SUPABASE_BUCKET", "video")]:
240
+ try:
241
+ items = supabase.storage.from_(bucket).list(folder)
242
+ if items:
243
+ paths_to_remove = []
244
+ for item in items:
245
+ name = item.get("name") if isinstance(item, dict) else getattr(item, "name", None)
246
+ if name and name != ".emptyFolderPlaceholder":
247
+ paths_to_remove.append(f"{folder}/{name}")
248
+ if paths_to_remove:
249
+ supabase.storage.from_(bucket).remove(paths_to_remove)
250
+ logger.info("Cleaned up %d objects in bucket '%s' for session %s", len(paths_to_remove), bucket, session_id)
251
+ except Exception as e:
252
+ logger.warning("Failed to clean up storage bucket '%s' for session %s: %s", bucket, session_id, e)
253
+
app/dependencies.py ADDED
@@ -0,0 +1,80 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import HTTPException, Header
2
+
3
+ from app.supabase_client import get_supabase, get_supabase_for_user_jwt
4
+
5
+
6
+ async def get_current_user_id(authorization: str | None = Header(None)):
7
+ """
8
+ Authenticate user using Supabase JWT.
9
+ Expected Header: Authorization: Bearer <token>
10
+ """
11
+ import os
12
+
13
+ if not authorization:
14
+ raise HTTPException(
15
+ status_code=401,
16
+ detail="Authorization header missing or invalid. Use 'Bearer <token>'",
17
+ )
18
+
19
+ if os.getenv("ALLOW_TEST_BYPASS") == "true" and authorization.startswith("Test "):
20
+ return authorization.split(" ")[1]
21
+
22
+ if not authorization.startswith("Bearer "):
23
+ raise HTTPException(
24
+ status_code=401,
25
+ detail="Authorization header missing or invalid. Use 'Bearer <token>'",
26
+ )
27
+
28
+ token = authorization.split(" ")[1]
29
+ supabase = get_supabase()
30
+
31
+ try:
32
+ user_response = supabase.auth.get_user(token)
33
+ if not user_response or not user_response.user:
34
+ raise HTTPException(status_code=401, detail="Invalid session or token.")
35
+
36
+ return user_response.user.id
37
+ except HTTPException:
38
+ raise
39
+ except Exception as e:
40
+ raise HTTPException(status_code=401, detail=f"Authentication failed: {str(e)}")
41
+
42
+
43
+ async def get_authenticated_supabase(authorization: str = Header(...)):
44
+ """
45
+ Supabase client that carries the user's JWT (anon key + Authorization header).
46
+ Use for routes that should respect Row Level Security; pair with app logic as needed.
47
+ """
48
+ import os
49
+
50
+ if not authorization:
51
+ raise HTTPException(
52
+ status_code=401,
53
+ detail="Authorization header missing or invalid. Use 'Bearer <token>'",
54
+ )
55
+
56
+ if os.getenv("ALLOW_TEST_BYPASS") == "true" and authorization.startswith("Test "):
57
+ return get_supabase()
58
+
59
+ if not authorization.startswith("Bearer "):
60
+ raise HTTPException(
61
+ status_code=401,
62
+ detail="Authorization header missing or invalid. Use 'Bearer <token>'",
63
+ )
64
+
65
+ token = authorization.split(" ")[1]
66
+ supabase = get_supabase()
67
+
68
+ try:
69
+ user_response = supabase.auth.get_user(token)
70
+ if not user_response or not user_response.user:
71
+ raise HTTPException(status_code=401, detail="Invalid session or token.")
72
+ except HTTPException:
73
+ raise
74
+ except Exception as e:
75
+ raise HTTPException(status_code=401, detail=f"Authentication failed: {str(e)}")
76
+
77
+ try:
78
+ return get_supabase_for_user_jwt(token)
79
+ except RuntimeError as e:
80
+ raise HTTPException(status_code=503, detail=str(e))
app/errors.py ADDED
@@ -0,0 +1,59 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Map exceptions to short, user-visible messages (avoid leaking HTML bodies from 404 proxies)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import logging
6
+
7
+ logger = logging.getLogger(__name__)
8
+
9
+
10
+ def _looks_like_html(text: str) -> bool:
11
+ t = text.lstrip()[:500].lower()
12
+ return t.startswith("<!doctype") or t.startswith("<html") or "<html" in t[:200]
13
+
14
+
15
+ def format_error_for_user(exc: BaseException) -> str:
16
+ """
17
+ Produce a safe message for chat/UI. Full detail stays in server logs via logger.exception.
18
+ """
19
+ # httpx: wrong URL often returns 404 HTML; don't show body
20
+ try:
21
+ import httpx
22
+
23
+ if isinstance(exc, httpx.HTTPStatusError):
24
+ req = exc.request
25
+ code = exc.response.status_code
26
+ url_hint = ""
27
+ try:
28
+ url_hint = str(req.url.host) if req and req.url else ""
29
+ except Exception:
30
+ pass
31
+ logger.warning(
32
+ "HTTPStatusError %s for %s (response not shown to user)",
33
+ code,
34
+ url_hint or "?",
35
+ )
36
+ return (
37
+ "Kiểm tra URL API, khóa bí mật và biến môi trường (OpenRouter/Supabase/Redis)."
38
+ )
39
+
40
+ if isinstance(exc, httpx.RequestError):
41
+ return "Không kết nối được tới dịch vụ ngoài (mạng hoặc URL sai)."
42
+ except ImportError:
43
+ pass
44
+
45
+ raw = str(exc).strip()
46
+ if not raw:
47
+ return "Đã xảy ra lỗi không xác định."
48
+
49
+ if _looks_like_html(raw):
50
+ logger.warning("Suppressed HTML error body from user-facing message")
51
+ return (
52
+ "Dịch vụ trả về trang lỗi (thường là URL API sai hoặc endpoint không tồn tại — HTTP 404). "
53
+ "Kiểm tra OPENROUTER_MODEL và khóa API trên server."
54
+ )
55
+
56
+ if len(raw) > 800:
57
+ return raw[:800] + "…"
58
+
59
+ return raw
app/job_poll.py ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Normalize Supabase `jobs` rows for polling / WebSocket clients (stable `job_id` + JSON `result`)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import logging
7
+ from typing import Any
8
+
9
+ logger = logging.getLogger(__name__)
10
+
11
+
12
+ def _coerce_result(value: Any) -> Any:
13
+ if value is None:
14
+ return None
15
+ if isinstance(value, (dict, list)):
16
+ return value
17
+ if isinstance(value, str):
18
+ try:
19
+ return json.loads(value)
20
+ except json.JSONDecodeError:
21
+ logger.warning("job_poll: result is non-JSON string, returning raw")
22
+ return {"raw": value}
23
+ return value
24
+
25
+
26
+ def normalize_job_row_for_client(row: dict[str, Any]) -> dict[str, Any]:
27
+ """
28
+ Build a JSON-serializable dict that always includes:
29
+ - ``job_id`` (alias of DB ``id``) for clients that expect it on poll bodies
30
+ - ``status`` as str
31
+ - ``result`` as object/array when stored as JSON string
32
+ All other columns are passed through (UUID/datetime become JSON-safe via FastAPI encoder).
33
+ """
34
+ out = dict(row)
35
+ jid = out.get("id")
36
+ if jid is not None:
37
+ out["job_id"] = str(jid)
38
+ st = out.get("status")
39
+ if st is not None:
40
+ out["status"] = str(st)
41
+ if "result" in out:
42
+ out["result"] = _coerce_result(out.get("result"))
43
+ if out.get("user_id") is not None:
44
+ out["user_id"] = str(out["user_id"])
45
+ if out.get("session_id") is not None:
46
+ out["session_id"] = str(out["session_id"])
47
+ return out
app/jobs/__init__.py ADDED
File without changes
app/llm_client.py ADDED
@@ -0,0 +1,53 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import logging
2
+ from typing import List, Dict, Any, Optional
3
+ from dotenv import load_dotenv
4
+
5
+ load_dotenv()
6
+ logger = logging.getLogger(__name__)
7
+
8
+ from llm.service import get_llm_service, LLMService
9
+ from config.loader import load_agent_config
10
+
11
+
12
+ class MultiLayerLLMClient:
13
+ """
14
+ Backward-compatible client adapter that delegates completions to LLMService.
15
+ Uses agent config for defaults but allows callers to override temperature/max_tokens
16
+ for ad-hoc usage (e.g. knowledge queries, chat completions).
17
+ """
18
+
19
+ def __init__(self):
20
+ self.service: LLMService = get_llm_service()
21
+
22
+ async def chat_completions_create(
23
+ self,
24
+ messages: List[Dict[str, Any]],
25
+ response_format: Optional[Dict[str, Any]] = None,
26
+ agent: str = "reasoning_solver",
27
+ temperature: Optional[float] = None,
28
+ max_tokens: Optional[int] = None,
29
+ **kwargs
30
+ ) -> str:
31
+ config = load_agent_config(agent)
32
+ model = config.tiers[0].model if config.tiers else "gemini/gemini-3.7-flash"
33
+ return await self.service.acomplete(
34
+ model=model,
35
+ messages=messages,
36
+ temperature=temperature if temperature is not None else config.temperature,
37
+ max_tokens=max_tokens if max_tokens is not None else config.max_tokens,
38
+ timeout=config.timeout_seconds,
39
+ response_format=response_format,
40
+ reasoning_effort=config.reasoning_effort,
41
+ agent_name=agent,
42
+ **kwargs
43
+ )
44
+
45
+
46
+ _llm_client: Optional[MultiLayerLLMClient] = None
47
+
48
+
49
+ def get_llm_client() -> MultiLayerLLMClient:
50
+ global _llm_client
51
+ if _llm_client is None:
52
+ _llm_client = MultiLayerLLMClient()
53
+ return _llm_client
app/logging_setup.py ADDED
@@ -0,0 +1,112 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Logging theo một biến LOG_LEVEL: debug | info | warning | error."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import logging
6
+ import os
7
+ from typing import Final
8
+
9
+ _SETUP_DONE = False
10
+
11
+ PIPELINE_LOGGER_NAME: Final = "app.pipeline"
12
+ CACHE_LOGGER_NAME: Final = "app.cache"
13
+ STEPS_LOGGER_NAME: Final = "app.steps"
14
+ ACCESS_LOGGER_NAME: Final = "app.access"
15
+
16
+
17
+ def _normalize_level() -> str:
18
+ raw = os.getenv("LOG_LEVEL", "info").strip().lower()
19
+ if raw in ("debug", "info", "warning", "error"):
20
+ return raw
21
+ return "info"
22
+
23
+
24
+ def setup_application_logging() -> None:
25
+ """Idempotent; gọi khi khởi động process (uvicorn, celery, worker_health)."""
26
+ global _SETUP_DONE
27
+ if _SETUP_DONE:
28
+ return
29
+ _SETUP_DONE = True
30
+
31
+ mode = _normalize_level()
32
+
33
+ level_map = {
34
+ "debug": logging.DEBUG,
35
+ "info": logging.INFO,
36
+ "warning": logging.WARNING,
37
+ "error": logging.ERROR,
38
+ }
39
+ root_level = level_map[mode]
40
+
41
+ fmt_named = "%(asctime)s | %(levelname)-8s | %(name)s | %(message)s"
42
+ fmt_short = "%(asctime)s | %(levelname)-8s | %(message)s"
43
+
44
+ logging.basicConfig(
45
+ level=root_level,
46
+ format=fmt_named if mode == "debug" else fmt_short,
47
+ datefmt="%H:%M:%S",
48
+ force=True,
49
+ )
50
+
51
+ logging.getLogger("httpx").setLevel(logging.WARNING)
52
+ logging.getLogger("httpcore").setLevel(logging.WARNING)
53
+ logging.getLogger("openai").setLevel(logging.WARNING)
54
+ logging.getLogger("uvicorn.access").setLevel(logging.WARNING)
55
+ logging.getLogger("uvicorn.error").setLevel(logging.INFO)
56
+ # HTTP/2 stack (httpx/httpcore) — khi LOG_LEVEL=debug root=DEBUG sẽ tràn log hpack; không cần cho debug app
57
+ for _name in ("hpack", "h2", "hyperframe", "urllib3"):
58
+ logging.getLogger(_name).setLevel(logging.WARNING)
59
+
60
+ if mode == "debug":
61
+ logging.getLogger("agents").setLevel(logging.DEBUG)
62
+ logging.getLogger("solver").setLevel(logging.DEBUG)
63
+ logging.getLogger("app").setLevel(logging.DEBUG)
64
+ logging.getLogger(CACHE_LOGGER_NAME).setLevel(logging.DEBUG)
65
+ logging.getLogger(STEPS_LOGGER_NAME).setLevel(logging.DEBUG)
66
+ logging.getLogger(PIPELINE_LOGGER_NAME).setLevel(logging.INFO)
67
+ logging.getLogger(ACCESS_LOGGER_NAME).setLevel(logging.INFO)
68
+ logging.getLogger("app.main").setLevel(logging.INFO)
69
+ logging.getLogger("worker").setLevel(logging.INFO)
70
+ elif mode == "info":
71
+ # Chỉ HTTP access (app.access) + startup; ẩn chi tiết agents/orchestrator/pipeline SUCCESS
72
+ logging.getLogger("agents").setLevel(logging.INFO)
73
+ logging.getLogger("solver").setLevel(logging.WARNING)
74
+ logging.getLogger("app").setLevel(logging.INFO)
75
+ logging.getLogger(CACHE_LOGGER_NAME).setLevel(logging.WARNING)
76
+ logging.getLogger(STEPS_LOGGER_NAME).setLevel(logging.WARNING)
77
+ logging.getLogger(PIPELINE_LOGGER_NAME).setLevel(logging.WARNING)
78
+ logging.getLogger(ACCESS_LOGGER_NAME).setLevel(logging.INFO)
79
+ logging.getLogger("app.main").setLevel(logging.INFO)
80
+ logging.getLogger("worker").setLevel(logging.WARNING)
81
+ elif mode == "warning":
82
+ logging.getLogger("agents").setLevel(logging.WARNING)
83
+ logging.getLogger("solver").setLevel(logging.WARNING)
84
+ logging.getLogger("app.routers").setLevel(logging.WARNING)
85
+ logging.getLogger(CACHE_LOGGER_NAME).setLevel(logging.WARNING)
86
+ logging.getLogger(STEPS_LOGGER_NAME).setLevel(logging.WARNING)
87
+ logging.getLogger(PIPELINE_LOGGER_NAME).setLevel(logging.WARNING)
88
+ logging.getLogger(ACCESS_LOGGER_NAME).setLevel(logging.WARNING)
89
+ logging.getLogger("app.main").setLevel(logging.WARNING)
90
+ logging.getLogger("worker").setLevel(logging.WARNING)
91
+ else: # error
92
+ logging.getLogger("agents").setLevel(logging.ERROR)
93
+ logging.getLogger("solver").setLevel(logging.ERROR)
94
+ logging.getLogger("app.routers").setLevel(logging.ERROR)
95
+ logging.getLogger(CACHE_LOGGER_NAME).setLevel(logging.ERROR)
96
+ logging.getLogger(STEPS_LOGGER_NAME).setLevel(logging.ERROR)
97
+ logging.getLogger(PIPELINE_LOGGER_NAME).setLevel(logging.ERROR)
98
+ logging.getLogger(ACCESS_LOGGER_NAME).setLevel(logging.ERROR)
99
+ logging.getLogger("app.main").setLevel(logging.ERROR)
100
+ logging.getLogger("worker").setLevel(logging.ERROR)
101
+
102
+ logging.getLogger(__name__).debug(
103
+ "LOG_LEVEL=%s root=%s", mode, logging.getLevelName(root_level)
104
+ )
105
+
106
+
107
+ def get_log_level() -> str:
108
+ return _normalize_level()
109
+
110
+
111
+ def is_debug_level() -> bool:
112
+ return _normalize_level() == "debug"
app/logutil.py ADDED
@@ -0,0 +1,67 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """log_step (debug), pipeline (debug), access log ở middleware."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import logging
7
+ import os
8
+ from typing import Any
9
+
10
+ from app.logging_setup import PIPELINE_LOGGER_NAME, STEPS_LOGGER_NAME
11
+
12
+ _pipeline = logging.getLogger(PIPELINE_LOGGER_NAME)
13
+ _steps = logging.getLogger(STEPS_LOGGER_NAME)
14
+
15
+
16
+ def is_debug_mode() -> bool:
17
+ """Chi tiết từng bước chỉ khi LOG_LEVEL=debug."""
18
+ return os.getenv("LOG_LEVEL", "info").strip().lower() == "debug"
19
+
20
+
21
+ def _truncate(val: Any, max_len: int = 2000) -> Any:
22
+ if val is None:
23
+ return None
24
+ if isinstance(val, (int, float, bool)):
25
+ return val
26
+ s = str(val)
27
+ if len(s) > max_len:
28
+ return s[:max_len] + f"... (+{len(s) - max_len} chars)"
29
+ return s
30
+
31
+
32
+ def log_step(step: str, **fields: Any) -> None:
33
+ """Chỉ khi LOG_LEVEL=debug: DB / cache / orchestrator."""
34
+ if not is_debug_mode():
35
+ return
36
+ safe = {k: _truncate(v) for k, v in fields.items()}
37
+ try:
38
+ payload = json.dumps(safe, ensure_ascii=False, default=str)
39
+ except Exception:
40
+ payload = str(safe)
41
+ _steps.debug("[step:%s] %s", step, payload)
42
+
43
+
44
+ def log_pipeline_success(operation: str, **fields: Any) -> None:
45
+ """Chỉ hiện khi debug (pipeline SUCCESS không dùng ở info — đã có app.access)."""
46
+ if not is_debug_mode():
47
+ return
48
+ safe = {k: _truncate(v, 500) for k, v in fields.items()}
49
+ _pipeline.info(
50
+ "SUCCESS %s %s",
51
+ operation,
52
+ json.dumps(safe, ensure_ascii=False, default=str),
53
+ )
54
+
55
+
56
+ def log_pipeline_failure(operation: str, error: str | None = None, **fields: Any) -> None:
57
+ """Thất bại pipeline: luôn dùng WARNING để vẫn thấy khi LOG_LEVEL=warning."""
58
+ if is_debug_mode():
59
+ safe = {k: _truncate(v, 500) for k, v in fields.items()}
60
+ _pipeline.warning(
61
+ "FAIL %s err=%s %s",
62
+ operation,
63
+ _truncate(error, 300),
64
+ json.dumps(safe, ensure_ascii=False, default=str),
65
+ )
66
+ else:
67
+ _pipeline.warning("FAIL %s", operation)
app/main.py ADDED
@@ -0,0 +1,143 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import logging
4
+ import os
5
+ import time
6
+ import uuid
7
+ import warnings
8
+
9
+ from dotenv import load_dotenv
10
+ from fastapi import Depends, FastAPI, File, HTTPException, UploadFile
11
+ from fastapi.middleware.cors import CORSMiddleware
12
+ from starlette.requests import Request
13
+
14
+ load_dotenv()
15
+
16
+ from app.runtime_env import apply_runtime_env_defaults
17
+
18
+ apply_runtime_env_defaults()
19
+
20
+ os.environ["NO_ALBUMENTATIONS_UPDATE"] = "1"
21
+ warnings.filterwarnings("ignore", category=UserWarning, module="pydantic")
22
+ warnings.filterwarnings("ignore", category=UserWarning, module="albumentations")
23
+
24
+ from app.logging_setup import ACCESS_LOGGER_NAME, get_log_level, setup_application_logging
25
+
26
+ setup_application_logging()
27
+
28
+ # Routers (after logging)
29
+ from app.dependencies import get_current_user_id
30
+ from app.ocr_local_file import ocr_from_local_image_path
31
+ from app.routers import auth, sessions, solve, ai_core
32
+ from agents.ocr_agent import OCRAgent
33
+ from app.routers.solve import get_orchestrator
34
+ from app.job_poll import normalize_job_row_for_client
35
+ from app.supabase_client import get_supabase
36
+ from app.websocket_manager import register_websocket_routes
37
+
38
+ logger = logging.getLogger("app.main")
39
+ _access = logging.getLogger(ACCESS_LOGGER_NAME)
40
+
41
+ app = FastAPI(title="Visual Math Solver API v5.2")
42
+
43
+
44
+ @app.middleware("http")
45
+ async def access_log_middleware(request: Request, call_next):
46
+ """LOG_LEVEL=info/debug: mọi request; warning: chỉ 4xx/5xx; error: chỉ 4xx/5xx ở mức error."""
47
+ start = time.perf_counter()
48
+ response = await call_next(request)
49
+ ms = (time.perf_counter() - start) * 1000
50
+ mode = get_log_level()
51
+ method = request.method
52
+ path = request.url.path
53
+ status = response.status_code
54
+
55
+ if mode in ("debug", "info"):
56
+ _access.info("%s %s -> %s (%.0fms)", method, path, status, ms)
57
+ elif mode == "warning":
58
+ if status >= 500:
59
+ _access.error("%s %s -> %s (%.0fms)", method, path, status, ms)
60
+ elif status >= 400:
61
+ _access.warning("%s %s -> %s (%.0fms)", method, path, status, ms)
62
+ elif mode == "error":
63
+ if status >= 400:
64
+ _access.error("%s %s -> %s", method, path, status)
65
+
66
+ return response
67
+
68
+
69
+ _redis_url = os.getenv("REDIS_URL") or os.getenv("CELERY_BROKER_URL") or "none"
70
+ _redis_tail = _redis_url.split("@")[-1] if "@" in _redis_url else _redis_url
71
+ if get_log_level() in ("debug", "info"):
72
+ logger.info("App starting LOG_LEVEL=%s | Redis: %s", get_log_level(), _redis_tail)
73
+ else:
74
+ logger.warning("App starting LOG_LEVEL=%s | Redis: %s", get_log_level(), _redis_tail)
75
+
76
+ app.add_middleware(
77
+ CORSMiddleware,
78
+ allow_origins=[
79
+ "http://localhost:3000",
80
+ "http://127.0.0.1:3000",
81
+ "http://localhost:3005",
82
+ ],
83
+ allow_credentials=True,
84
+ allow_methods=["*"],
85
+ allow_headers=["*"],
86
+ )
87
+
88
+ app.include_router(ai_core.router)
89
+ app.include_router(auth.router)
90
+ app.include_router(sessions.router)
91
+ app.include_router(solve.router)
92
+
93
+ register_websocket_routes(app)
94
+
95
+
96
+ def get_ocr_agent() -> OCRAgent:
97
+ """Same OCR instance as the solve pipeline (no duplicate model load)."""
98
+ return get_orchestrator().ocr_agent
99
+
100
+
101
+ @app.get("/")
102
+ def read_root():
103
+ return {
104
+ "message": "Visual Math Solver API v5.2 is running",
105
+ "version": "5.2",
106
+ "ai_core_direct_endpoint": "/api/v1/ai/solve"
107
+ }
108
+
109
+
110
+ @app.post("/api/v1/ocr")
111
+ async def upload_ocr(
112
+ file: UploadFile = File(...),
113
+ _user_id=Depends(get_current_user_id),
114
+ ):
115
+ """OCR upload: requires authenticated user."""
116
+ temp_path = f"temp_{uuid.uuid4()}.png"
117
+ with open(temp_path, "wb") as buffer:
118
+ buffer.write(await file.read())
119
+
120
+ try:
121
+ text = await ocr_from_local_image_path(temp_path, file.filename, get_ocr_agent())
122
+ return {"text": text}
123
+ finally:
124
+ if os.path.exists(temp_path):
125
+ os.remove(temp_path)
126
+
127
+
128
+ @app.get("/api/v1/solve/{job_id}")
129
+ async def get_job_status(
130
+ job_id: str,
131
+ user_id=Depends(get_current_user_id),
132
+ ):
133
+ """Retrieve job status (can be used for polling if WS fails). Owner-only."""
134
+ supabase_client = get_supabase()
135
+ if not supabase_client:
136
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
137
+ response = supabase_client.table("jobs").select("*").eq("id", job_id).execute()
138
+ if not response.data:
139
+ raise HTTPException(status_code=404, detail="Job not found")
140
+ job = response.data[0]
141
+ if job.get("user_id") is not None and str(job["user_id"]) != str(user_id):
142
+ raise HTTPException(status_code=403, detail="Forbidden: You do not own this job.")
143
+ return normalize_job_row_for_client(job)
app/models/__init__.py ADDED
File without changes
app/models/schemas.py ADDED
@@ -0,0 +1,81 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from pydantic import BaseModel, EmailStr, field_validator
2
+ from typing import Optional, List, Any, Dict
3
+ from datetime import datetime
4
+ import uuid
5
+
6
+ from app.url_utils import sanitize_url
7
+
8
+ # --- Auth Schemas ---
9
+ class UserProfile(BaseModel):
10
+ id: uuid.UUID
11
+ display_name: Optional[str] = None
12
+ avatar_url: Optional[str] = None
13
+ created_at: datetime
14
+
15
+ class User(BaseModel):
16
+ id: uuid.UUID
17
+ email: EmailStr
18
+
19
+ # --- Session Schemas ---
20
+ class SessionBase(BaseModel):
21
+ title: str = "Bài toán mới"
22
+
23
+ class SessionCreate(SessionBase):
24
+ pass
25
+
26
+ class Session(SessionBase):
27
+ id: uuid.UUID
28
+ user_id: uuid.UUID
29
+ created_at: datetime
30
+ updated_at: datetime
31
+
32
+ class Config:
33
+ from_attributes = True
34
+
35
+ # --- Message Schemas ---
36
+ class MessageBase(BaseModel):
37
+ role: str
38
+ type: str = "text"
39
+ content: str
40
+ metadata: Dict[str, Any] = {}
41
+
42
+ class MessageCreate(MessageBase):
43
+ session_id: uuid.UUID
44
+
45
+ class Message(MessageBase):
46
+ id: uuid.UUID
47
+ session_id: uuid.UUID
48
+ created_at: datetime
49
+
50
+ class Config:
51
+ from_attributes = True
52
+
53
+ # --- Solve Job Schemas ---
54
+ class SolveRequest(BaseModel):
55
+ text: str
56
+ image_url: Optional[str] = None
57
+ client_message_id: Optional[str] = None
58
+
59
+ @field_validator("image_url", mode="before")
60
+ @classmethod
61
+ def _clean_image_url(cls, v):
62
+ return sanitize_url(v) if v is not None else None
63
+
64
+ class SolveResponse(BaseModel):
65
+ job_id: str
66
+ status: str
67
+
68
+ class RenderVideoRequest(BaseModel):
69
+ job_id: Optional[str] = None
70
+
71
+ class RenderVideoResponse(BaseModel):
72
+ job_id: str
73
+ status: str
74
+
75
+
76
+ class OcrPreviewResponse(BaseModel):
77
+ """Stateless OCR preview before POST .../solve (no DB writes, no job)."""
78
+
79
+ ocr_text: str
80
+ user_message: str = ""
81
+ combined_draft: str
app/ocr_celery.py ADDED
@@ -0,0 +1,86 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """OCR pipeline with confidence gateway support."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import json
6
+ import logging
7
+ import os
8
+ from typing import TYPE_CHECKING, Optional
9
+
10
+ import anyio
11
+
12
+ from config.loader import load_agent_config
13
+ from vision_ocr.canonical_schema import CanonicalOCRResult
14
+
15
+ if TYPE_CHECKING:
16
+ from agents.ocr_agent import OCRAgent
17
+
18
+ logger = logging.getLogger(__name__)
19
+
20
+
21
+ async def ocr_from_image_url(
22
+ image_url: str,
23
+ fallback_agent: "OCRAgent",
24
+ raw_image_path: Optional[str] = None,
25
+ ) -> CanonicalOCRResult:
26
+ """
27
+ Process OCR from image URL using OCRAgent (Pix2Text Engine).
28
+ Returns full CanonicalOCRResult with confidence metadata.
29
+ If confidence < gateway threshold, triggers VLM correction (if enabled).
30
+ """
31
+ # Get canonical OCR result (with confidence)
32
+ canonical = await fallback_agent.process_url_canonical(image_url)
33
+
34
+ # Check confidence gateway
35
+ ocr_config = load_agent_config("ocr")
36
+ gateway = ocr_config.confidence_gateway
37
+
38
+ if gateway and gateway.enabled:
39
+ threshold = gateway.threshold
40
+ ocr_confidence = canonical.confidence
41
+
42
+ logger.info(
43
+ f"[OCR Gateway] confidence={ocr_confidence:.3f}, threshold={threshold:.2f}, "
44
+ f"gateway_triggered={ocr_confidence < threshold}"
45
+ )
46
+
47
+ if ocr_confidence < threshold and gateway.correction.enabled:
48
+ try:
49
+ from agents.vlm_corrector import VLMCorrectorAgent
50
+
51
+ corrector = VLMCorrectorAgent(config=gateway.correction)
52
+ correction_result = await corrector.correct(
53
+ ocr_result=canonical,
54
+ image_url=image_url,
55
+ image_path=raw_image_path,
56
+ )
57
+
58
+ if correction_result and correction_result.get("changed"):
59
+ corrected_text = correction_result.get("corrected_text", canonical.text)
60
+ corrected_confidence = correction_result.get("confidence", ocr_confidence)
61
+ logger.info(
62
+ f"[OCR Gateway] VLM correction applied: "
63
+ f"confidence {ocr_confidence:.3f} → {corrected_confidence:.3f}"
64
+ )
65
+ # Return updated canonical result
66
+ canonical = CanonicalOCRResult(
67
+ text=corrected_text,
68
+ latex=canonical.latex,
69
+ elements=canonical.elements,
70
+ reading_order=canonical.reading_order,
71
+ confidence=corrected_confidence,
72
+ metadata={
73
+ **canonical.metadata,
74
+ "vlm_correction": True,
75
+ "original_confidence": ocr_confidence,
76
+ "corrections": correction_result.get("corrections", []),
77
+ },
78
+ )
79
+ else:
80
+ logger.info("[OCR Gateway] VLM correction: no changes needed")
81
+ except ImportError:
82
+ logger.warning("[OCR Gateway] VLM corrector not available, skipping correction")
83
+ except Exception as e:
84
+ logger.warning(f"[OCR Gateway] VLM correction failed: {e}")
85
+
86
+ return canonical
app/ocr_local_file.py ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import logging
4
+ import os
5
+ from typing import TYPE_CHECKING
6
+
7
+ if TYPE_CHECKING:
8
+ from agents.ocr_agent import OCRAgent
9
+
10
+ logger = logging.getLogger(__name__)
11
+
12
+
13
+ async def ocr_from_local_image_path(
14
+ local_path: str,
15
+ original_filename: str | None,
16
+ fallback_agent: "OCRAgent",
17
+ ) -> str:
18
+ """
19
+ Run OCR on a file on local disk using Pix2Text Engine in-process.
20
+ """
21
+ return await fallback_agent.process_image(local_path)
22
+
app/ocr_text_merge.py ADDED
@@ -0,0 +1,14 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Helpers for OCR preview combined draft (no Pydantic email deps)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from typing import Optional
6
+
7
+
8
+ def build_combined_ocr_preview_draft(user_message: Optional[str], ocr_text: str) -> str:
9
+ """Merge user caption and OCR text for confirm step (user message first, then OCR)."""
10
+ u = (user_message or "").strip()
11
+ o = (ocr_text or "").strip()
12
+ if u and o:
13
+ return f"{u}\n\n{o}"
14
+ return u or o
app/routers/__init__.py ADDED
@@ -0,0 +1 @@
 
 
1
+ from . import auth, sessions, solve
app/routers/ai_core.py ADDED
@@ -0,0 +1,145 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import base64
4
+ import logging
5
+ import os
6
+ import uuid
7
+ from fastapi import APIRouter, HTTPException
8
+ from pydantic import BaseModel
9
+ from typing import Optional, Dict, Any
10
+
11
+ from agents.orchestrator import Orchestrator
12
+ from agents.ocr_agent import OCRAgent
13
+ from manim_client.client import ManimClient
14
+ from manim_client.schemas import MathRenderResponse
15
+ from vision_ocr.canonical_schema import CanonicalOCRResult
16
+
17
+ logger = logging.getLogger(__name__)
18
+ router = APIRouter(prefix="/api/v1/ai", tags=["AI Core (Standalone)"])
19
+
20
+ # Shared in-memory instances (lazy initialized)
21
+ _orchestrator = None
22
+ _ocr_agent = None
23
+ _manim_client = None
24
+
25
+
26
+ def _get_orchestrator() -> Orchestrator:
27
+ global _orchestrator
28
+ if _orchestrator is None:
29
+ _orchestrator = Orchestrator()
30
+ return _orchestrator
31
+
32
+
33
+ def _get_ocr_agent() -> OCRAgent:
34
+ global _ocr_agent
35
+ if _ocr_agent is None:
36
+ _ocr_agent = OCRAgent()
37
+ return _ocr_agent
38
+
39
+
40
+ def _get_manim_client() -> ManimClient:
41
+ global _manim_client
42
+ if _manim_client is None:
43
+ _manim_client = ManimClient()
44
+ return _manim_client
45
+
46
+
47
+ class AISolveRequest(BaseModel):
48
+ text: Optional[str] = None
49
+ image_url: Optional[str] = None
50
+ image_path: Optional[str] = None
51
+ generate_video: bool = True
52
+
53
+
54
+ class AIOCRRequest(BaseModel):
55
+ image_url: Optional[str] = None
56
+ image_path: Optional[str] = None
57
+ image_base64: Optional[str] = None
58
+
59
+
60
+ @router.post("/ocr", response_model=CanonicalOCRResult)
61
+ async def ocr_direct_ai(request: AIOCRRequest) -> CanonicalOCRResult:
62
+ """
63
+ Direct Math OCR Endpoint (Pix2Text Engine).
64
+ Converts geometry problem images into canonical structured format:
65
+ - text: Reconstructed Markdown with LaTeX math formulas
66
+ - latex: List of isolated and embedded LaTeX equations
67
+ - elements: Classified layout regions (text, formulas, bboxes)
68
+ - reading_order: Document reading sequence
69
+ - confidence: Extraction accuracy confidence
70
+ """
71
+ ocr_agent = _get_ocr_agent()
72
+ if request.image_url:
73
+ logger.info("==[AI Core OCR] Processing image_url: %s==", request.image_url)
74
+ return await ocr_agent.process_url_canonical(request.image_url)
75
+
76
+ if request.image_path:
77
+ logger.info("==[AI Core OCR] Processing local image_path: %s==", request.image_path)
78
+ return await ocr_agent.process_image_canonical(request.image_path)
79
+
80
+ if request.image_base64:
81
+ logger.info("==[AI Core OCR] Processing image_base64==")
82
+ temp_path = f"temp_ocr_b64_{uuid.uuid4().hex}.png"
83
+ try:
84
+ b64_data = request.image_base64
85
+ if "," in b64_data:
86
+ b64_data = b64_data.split(",", 1)[1]
87
+ img_bytes = base64.b64decode(b64_data)
88
+ with open(temp_path, "wb") as f:
89
+ f.write(img_bytes)
90
+ return await ocr_agent.process_image_canonical(temp_path)
91
+ finally:
92
+ if os.path.exists(temp_path):
93
+ try:
94
+ os.remove(temp_path)
95
+ except Exception:
96
+ pass
97
+
98
+ raise HTTPException(status_code=400, detail="Must provide 'image_url', 'image_path', or 'image_base64'.")
99
+
100
+
101
+ @router.post("/solve")
102
+ async def solve_direct_ai(request: AISolveRequest) -> Dict[str, Any]:
103
+ """
104
+ Direct Standalone AI Core Solve Endpoint.
105
+ - Runs OCR (Pix2Text) -> GeometryParser -> GeometryEngine -> DeepMath -> VisualizationSpec -> Manim Module.
106
+ - 100% In-Memory: No Supabase DB or Redis connection required.
107
+ - No authentication token required (Ideal for AI development & curl testing).
108
+ """
109
+ text = (request.text or "").strip()
110
+ image_url = request.image_url
111
+ ocr_agent = _get_ocr_agent()
112
+ orchestrator = _get_orchestrator()
113
+
114
+ if not text and request.image_path and os.path.exists(request.image_path):
115
+ # Run OCR on local image directly
116
+ ocr_res = await ocr_agent.process_image_canonical(request.image_path)
117
+ text = ocr_res.text
118
+ logger.info("[AI Core Solve] Extracted OCR text from %s: '%s'", request.image_path, text[:80])
119
+
120
+ if not text and not image_url:
121
+ raise HTTPException(status_code=400, detail="Either 'text', 'image_url', or valid 'image_path' must be provided.")
122
+
123
+ logger.info("==[AI Core Direct Solve] Received problem: %s==", text[:80] if text else f"Image: {image_url}")
124
+ try:
125
+ result = await orchestrator.run(
126
+ text=text,
127
+ image_url=image_url,
128
+ job_id="direct_ai_run",
129
+ generate_video=request.generate_video,
130
+ )
131
+ return result
132
+ except Exception as e:
133
+ logger.exception("AI Core execution error: %s", e)
134
+ raise HTTPException(status_code=500, detail=f"AI Core processing failed: {str(e)}")
135
+
136
+
137
+ @router.get("/visualization/jobs/{job_id}", response_model=MathRenderResponse)
138
+ async def get_visualization_job_status(job_id: str) -> MathRenderResponse:
139
+ """
140
+ Query the status of a Manim video generation job.
141
+ Proxies request to the Manim Video Generation Module.
142
+ """
143
+ logger.info(f"==[AI Core Visualization] Fetching status for job {job_id}==")
144
+ resp = await _get_manim_client().get_job_status(job_id)
145
+ return resp
app/routers/auth.py ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from fastapi import APIRouter, Depends, HTTPException
2
+ from app.dependencies import get_current_user_id
3
+ from app.supabase_client import get_supabase
4
+ from app.models.schemas import UserProfile
5
+ import uuid
6
+
7
+ router = APIRouter(prefix="/api/v1/auth", tags=["Auth"])
8
+
9
+ @router.get("/me")
10
+ async def get_me(user_id=Depends(get_current_user_id)):
11
+ """Lấy thông tin profile người dùng hiện tại (Retrieve current user profile)."""
12
+ supabase = get_supabase()
13
+ if not supabase:
14
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
15
+
16
+ res = supabase.table("profiles").select("*").eq("id", user_id).execute()
17
+ if res.data:
18
+ return res.data[0]
19
+
20
+ # Auto-provision basic profile if trigger didn't run or dev test user
21
+ try:
22
+ insert_res = (
23
+ supabase.table("profiles")
24
+ .insert({"id": str(user_id), "display_name": "Người dùng", "avatar_url": None})
25
+ .execute()
26
+ )
27
+ if insert_res.data:
28
+ return insert_res.data[0]
29
+ except Exception:
30
+ pass
31
+
32
+ raise HTTPException(status_code=404, detail="Profile not found.")
33
+
34
+ @router.patch("/me")
35
+ async def update_me(data: dict, user_id=Depends(get_current_user_id)):
36
+ """Cập nhật profile hiện tại (Update current profile)."""
37
+ supabase = get_supabase()
38
+ if not supabase:
39
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
40
+
41
+ # Sanitize and only allow updating safe profile fields
42
+ allowed_fields = {"display_name", "avatar_url"}
43
+ update_data = {k: v for k, v in data.items() if k in allowed_fields}
44
+ if not update_data:
45
+ raise HTTPException(status_code=400, detail="No valid fields provided for update (allowed: display_name, avatar_url).")
46
+
47
+ res = supabase.table("profiles").update(update_data).eq("id", user_id).execute()
48
+ if not res.data:
49
+ raise HTTPException(status_code=404, detail="Profile not found to update.")
50
+ return res.data[0]
app/routers/sessions.py ADDED
@@ -0,0 +1,206 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import logging
4
+ import time
5
+ from typing import List
6
+
7
+ from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
8
+
9
+ from app.chat_image_upload import cleanup_session_storage
10
+ from app.dependencies import get_current_user_id
11
+ from app.logutil import log_step
12
+ from app.session_cache import (
13
+ invalidate_session_owner,
14
+ session_owned_by_user,
15
+ )
16
+ from app.supabase_client import get_supabase
17
+
18
+ router = APIRouter(prefix="/api/v1/sessions", tags=["Sessions"])
19
+ logger = logging.getLogger(__name__)
20
+
21
+
22
+ @router.get("", response_model=List[dict])
23
+ async def list_sessions(user_id=Depends(get_current_user_id)):
24
+ """Danh sách các phiên chat của người dùng (List user's chat sessions)"""
25
+ supabase = get_supabase()
26
+ if not supabase:
27
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
28
+ t0 = time.perf_counter()
29
+ res = (
30
+ supabase.table("sessions")
31
+ .select("id, user_id, title, created_at, updated_at")
32
+ .eq("user_id", user_id)
33
+ .order("updated_at", desc=True)
34
+ .execute()
35
+ )
36
+ log_step("db_select", table="sessions", op="list", user_id=str(user_id))
37
+ out = res.data or []
38
+ logger.info(
39
+ "sessions.list user=%s count=%d %.1fms",
40
+ user_id,
41
+ len(out),
42
+ (time.perf_counter() - t0) * 1000,
43
+ )
44
+ return out
45
+
46
+
47
+ @router.post("", response_model=dict)
48
+ async def create_session(user_id=Depends(get_current_user_id)):
49
+ """Tạo một phiên chat mới (Create a new chat session)"""
50
+ supabase = get_supabase()
51
+ if not supabase:
52
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
53
+ t0 = time.perf_counter()
54
+ res = supabase.table("sessions").insert(
55
+ {"user_id": user_id, "title": "Bài toán mới"}
56
+ ).execute()
57
+ log_step("db_insert", table="sessions", op="create")
58
+ if not res.data:
59
+ raise HTTPException(status_code=500, detail="Failed to create session.")
60
+ row = res.data[0]
61
+ logger.info(
62
+ "sessions.create user=%s id=%s %.1fms",
63
+ user_id,
64
+ row.get("id"),
65
+ (time.perf_counter() - t0) * 1000,
66
+ )
67
+ return row
68
+
69
+
70
+ @router.get("/{session_id}/messages", response_model=List[dict])
71
+ async def get_session_messages(session_id: str, user_id=Depends(get_current_user_id)):
72
+ """Lấy toàn bộ lịch sử tin nhắn của một phiên (Get chat history for a session)"""
73
+ supabase = get_supabase()
74
+ if not supabase:
75
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
76
+
77
+ def owns() -> bool:
78
+ res = (
79
+ supabase.table("sessions")
80
+ .select("id")
81
+ .eq("id", session_id)
82
+ .eq("user_id", user_id)
83
+ .execute()
84
+ )
85
+ log_step("db_select", table="sessions", op="owner_check", session_id=session_id)
86
+ return bool(res.data)
87
+
88
+ if not session_owned_by_user(session_id, str(user_id), owns):
89
+ raise HTTPException(
90
+ status_code=403, detail="Forbidden: You do not own this session."
91
+ )
92
+
93
+ res = (
94
+ supabase.table("messages")
95
+ .select("*")
96
+ .eq("session_id", session_id)
97
+ .order("created_at", desc=False)
98
+ .execute()
99
+ )
100
+ log_step("db_select", table="messages", op="list", session_id=session_id)
101
+ return res.data or []
102
+
103
+
104
+ @router.delete("/{session_id}")
105
+ async def delete_session(
106
+ session_id: str,
107
+ background_tasks: BackgroundTasks,
108
+ user_id=Depends(get_current_user_id),
109
+ ):
110
+ """Xóa một phiên chat và toàn bộ tài nguyên liên quan (Delete a chat session & associated assets)"""
111
+ supabase = get_supabase()
112
+ if not supabase:
113
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
114
+
115
+ def owns() -> bool:
116
+ res = (
117
+ supabase.table("sessions")
118
+ .select("id")
119
+ .eq("id", session_id)
120
+ .eq("user_id", user_id)
121
+ .execute()
122
+ )
123
+ return bool(res.data)
124
+
125
+ if not session_owned_by_user(session_id, str(user_id), owns):
126
+ raise HTTPException(
127
+ status_code=403, detail="Forbidden: You do not own this session."
128
+ )
129
+
130
+ # 1. Authoritative DB deletion first (dependent rows before session)
131
+ try:
132
+ supabase.table("session_assets").delete().eq("session_id", session_id).execute()
133
+ log_step("db_delete", table="session_assets", op="by_session", session_id=session_id)
134
+ except Exception as e:
135
+ logger.warning("Error deleting session_assets for session %s: %s", session_id, e)
136
+
137
+ supabase.table("jobs").delete().eq("session_id", session_id).eq("user_id", user_id).execute()
138
+ log_step("db_delete", table="jobs", op="by_session", session_id=session_id)
139
+ supabase.table("messages").delete().eq("session_id", session_id).execute()
140
+ log_step("db_delete", table="messages", op="by_session", session_id=session_id)
141
+
142
+ res = (
143
+ supabase.table("sessions")
144
+ .delete()
145
+ .eq("id", session_id)
146
+ .eq("user_id", user_id)
147
+ .execute()
148
+ )
149
+ log_step("db_delete", table="sessions", session_id=session_id)
150
+ invalidate_session_owner(session_id, str(user_id))
151
+
152
+ # 2. Async / non-blocking storage cleanup AFTER DB deletion succeeds
153
+ background_tasks.add_task(cleanup_session_storage, session_id)
154
+
155
+ return {"status": "ok", "deleted_id": session_id}
156
+
157
+
158
+ @router.patch("/{session_id}/title")
159
+ async def update_session_title(title: str, session_id: str, user_id=Depends(get_current_user_id)):
160
+ """Cập nhật tiêu đề phiên chat (Rename a chat session)"""
161
+ supabase = get_supabase()
162
+ if not supabase:
163
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
164
+
165
+ res = (
166
+ supabase.table("sessions")
167
+ .update({"title": title})
168
+ .eq("id", session_id)
169
+ .eq("user_id", user_id)
170
+ .execute()
171
+ )
172
+ if not res.data:
173
+ raise HTTPException(status_code=404, detail="Session not found or not owned by user.")
174
+ log_step("db_update", table="sessions", op="title", session_id=session_id)
175
+ return res.data[0]
176
+
177
+
178
+ @router.get("/{session_id}/assets", response_model=List[dict])
179
+ async def get_session_assets(session_id: str, user_id=Depends(get_current_user_id)):
180
+ """Lấy danh sách video đã render trong session (Get versioned assets for a session)"""
181
+ supabase = get_supabase()
182
+
183
+ def owns() -> bool:
184
+ res = (
185
+ supabase.table("sessions")
186
+ .select("id")
187
+ .eq("id", session_id)
188
+ .eq("user_id", user_id)
189
+ .execute()
190
+ )
191
+ return bool(res.data)
192
+
193
+ if not session_owned_by_user(session_id, str(user_id), owns):
194
+ raise HTTPException(
195
+ status_code=403, detail="Forbidden: You do not own this session."
196
+ )
197
+
198
+ res = (
199
+ supabase.table("session_assets")
200
+ .select("*")
201
+ .eq("session_id", session_id)
202
+ .order("version", desc=True)
203
+ .execute()
204
+ )
205
+ log_step("db_select", table="session_assets", op="list", session_id=session_id)
206
+ return res.data
app/routers/solve.py ADDED
@@ -0,0 +1,546 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from __future__ import annotations
2
+
3
+ import logging
4
+ import os
5
+ import uuid
6
+
7
+ from fastapi import APIRouter, BackgroundTasks, Depends, File, Form, HTTPException, UploadFile
8
+
9
+ from agents.orchestrator import Orchestrator
10
+ from app.chat_image_upload import upload_session_chat_image, validate_chat_image_bytes
11
+ from app.ocr_local_file import ocr_from_local_image_path
12
+ from app.dependencies import get_current_user_id
13
+ from app.errors import format_error_for_user
14
+ from app.logutil import log_pipeline_failure, log_pipeline_success, log_step
15
+ from app.models.schemas import (
16
+ OcrPreviewResponse,
17
+ RenderVideoRequest,
18
+ RenderVideoResponse,
19
+ SolveRequest,
20
+ SolveResponse,
21
+ )
22
+ from app.ocr_text_merge import build_combined_ocr_preview_draft
23
+ from app.session_cache import session_owned_by_user
24
+ from app.supabase_client import get_supabase
25
+
26
+ logger = logging.getLogger(__name__)
27
+ router = APIRouter(prefix="/api/v1/sessions", tags=["Solve"])
28
+
29
+ # Eager init: all agents and models load at import time (also run in Docker build via scripts/prewarm_models.py).
30
+ ORCHESTRATOR = Orchestrator()
31
+
32
+
33
+ def get_orchestrator() -> Orchestrator:
34
+ return ORCHESTRATOR
35
+
36
+
37
+ _OCR_PREVIEW_MAX_BYTES = 10 * 1024 * 1024
38
+
39
+
40
+ def _assert_session_owner(supabase, session_id: str, user_id, uid: str, op: str) -> None:
41
+ def owns() -> bool:
42
+ res = (
43
+ supabase.table("sessions")
44
+ .select("id")
45
+ .eq("id", session_id)
46
+ .eq("user_id", user_id)
47
+ .execute()
48
+ )
49
+ log_step("db_select", table="sessions", op=op, session_id=session_id)
50
+ return bool(res.data)
51
+
52
+ if not session_owned_by_user(session_id, uid, owns):
53
+ log_pipeline_failure("solve_request", error="forbidden", session_id=session_id)
54
+ raise HTTPException(
55
+ status_code=403, detail="Forbidden: You do not own this session."
56
+ )
57
+
58
+
59
+ def _enqueue_solve_common(
60
+ supabase,
61
+ background_tasks: BackgroundTasks,
62
+ session_id: str,
63
+ user_id,
64
+ uid: str,
65
+ request: SolveRequest,
66
+ message_metadata: dict,
67
+ job_id: str,
68
+ ) -> SolveResponse:
69
+ """Insert user message, job row, enqueue pipeline; update title when first message."""
70
+ client_msg_id = getattr(request, "client_message_id", None)
71
+ if client_msg_id:
72
+ message_metadata["client_message_id"] = client_msg_id
73
+
74
+ # Check for idempotency if client_message_id is provided
75
+ if client_msg_id:
76
+ try:
77
+ existing = (
78
+ supabase.table("messages")
79
+ .select("id")
80
+ .eq("session_id", session_id)
81
+ .eq("client_message_id", client_msg_id)
82
+ .execute()
83
+ )
84
+ if existing.data and len(existing.data) > 0:
85
+ logger.info(
86
+ "Duplicate request detected for client_message_id=%s in session %s",
87
+ client_msg_id,
88
+ session_id,
89
+ )
90
+ return SolveResponse(job_id=job_id, status="processing")
91
+ except Exception:
92
+ pass
93
+
94
+ msg_insert = {
95
+ "session_id": session_id,
96
+ "role": "user",
97
+ "type": "text",
98
+ "content": request.text,
99
+ "metadata": message_metadata,
100
+ }
101
+ if client_msg_id:
102
+ try:
103
+ supabase.table("messages").insert({**msg_insert, "client_message_id": client_msg_id}).execute()
104
+ except Exception:
105
+ supabase.table("messages").insert(msg_insert).execute()
106
+ else:
107
+ supabase.table("messages").insert(msg_insert).execute()
108
+
109
+ log_step("db_insert", table="messages", op="user_message", session_id=session_id)
110
+
111
+ supabase.table("jobs").insert(
112
+ {
113
+ "id": job_id,
114
+ "user_id": user_id,
115
+ "session_id": session_id,
116
+ "status": "processing",
117
+ "input_text": request.text,
118
+ }
119
+ ).execute()
120
+ log_step("db_insert", table="jobs", job_id=job_id)
121
+
122
+ background_tasks.add_task(process_session_job, job_id, session_id, request, str(user_id))
123
+
124
+ title_check = supabase.table("sessions").select("title").eq("id", session_id).execute()
125
+ if title_check.data and title_check.data[0]["title"] == "Bài toán mới":
126
+ new_title = request.text[:50] + ("..." if len(request.text) > 50 else "")
127
+ supabase.table("sessions").update({"title": new_title}).eq("id", session_id).execute()
128
+ log_step("db_update", table="sessions", op="title_from_first_message")
129
+
130
+ log_pipeline_success("solve_accepted", job_id=job_id, session_id=session_id)
131
+ return SolveResponse(job_id=job_id, status="processing")
132
+
133
+
134
+ @router.post("/{session_id}/ocr_preview", response_model=OcrPreviewResponse)
135
+ async def ocr_preview(
136
+ session_id: str,
137
+ user_id=Depends(get_current_user_id),
138
+ file: UploadFile = File(...),
139
+ user_message: str | None = Form(None),
140
+ ):
141
+ """
142
+ Run OCR on an uploaded image and merge with optional user_message into combined_draft.
143
+ Does not insert messages or start a solve job. After user confirms, call POST .../solve
144
+ with text=combined_draft (edited) and omit image_url to avoid double OCR.
145
+ """
146
+ supabase = get_supabase()
147
+ uid = str(user_id)
148
+ _assert_session_owner(supabase, session_id, user_id, uid, "owner_check_ocr_preview")
149
+
150
+ body = await file.read()
151
+ if len(body) > _OCR_PREVIEW_MAX_BYTES:
152
+ raise HTTPException(
153
+ status_code=413,
154
+ detail=f"Image too large (max {_OCR_PREVIEW_MAX_BYTES // (1024 * 1024)} MB).",
155
+ )
156
+ if not body:
157
+ raise HTTPException(status_code=400, detail="Empty file.")
158
+
159
+ validate_chat_image_bytes(file.filename, body, file.content_type)
160
+
161
+ suffix = os.path.splitext(file.filename or "")[1].lower()
162
+ if suffix not in (".png", ".jpg", ".jpeg", ".webp", ".gif", ".bmp", ""):
163
+ suffix = ".png"
164
+ temp_path = f"temp_ocr_preview_{uuid.uuid4()}{suffix or '.png'}"
165
+ try:
166
+ with open(temp_path, "wb") as f:
167
+ f.write(body)
168
+ ocr_text = await ocr_from_local_image_path(
169
+ temp_path, file.filename, get_orchestrator().ocr_agent
170
+ )
171
+ if ocr_text is None:
172
+ ocr_text = ""
173
+ finally:
174
+ if os.path.exists(temp_path):
175
+ os.remove(temp_path)
176
+
177
+ um = (user_message or "").strip()
178
+ combined = build_combined_ocr_preview_draft(user_message, ocr_text)
179
+ log_step("ocr_preview_done", session_id=session_id, ocr_len=len(ocr_text), user_len=len(um))
180
+ return OcrPreviewResponse(
181
+ ocr_text=ocr_text,
182
+ user_message=um,
183
+ combined_draft=combined,
184
+ )
185
+
186
+
187
+ @router.post("/{session_id}/solve", response_model=SolveResponse)
188
+ async def solve_problem(
189
+ session_id: str,
190
+ request: SolveRequest,
191
+ background_tasks: BackgroundTasks,
192
+ user_id=Depends(get_current_user_id),
193
+ ):
194
+ """
195
+ Gửi câu hỏi giải toán trong một session (Submit geometry problem in a session).
196
+ Lưu câu hỏi vào history và bắt đầu tiến trình giải (chỉ giải toán và tạo hình tĩnh).
197
+ """
198
+ supabase = get_supabase()
199
+ uid = str(user_id)
200
+ _assert_session_owner(supabase, session_id, user_id, uid, "owner_check")
201
+
202
+ message_metadata = {"image_url": request.image_url} if request.image_url else {}
203
+ job_id = str(uuid.uuid4())
204
+ return _enqueue_solve_common(
205
+ supabase,
206
+ background_tasks,
207
+ session_id,
208
+ user_id,
209
+ uid,
210
+ request,
211
+ message_metadata,
212
+ job_id,
213
+ )
214
+
215
+
216
+ @router.post("/{session_id}/solve_multipart", response_model=SolveResponse)
217
+ async def solve_multipart(
218
+ session_id: str,
219
+ background_tasks: BackgroundTasks,
220
+ user_id=Depends(get_current_user_id),
221
+ text: str = Form(...),
222
+ file: UploadFile = File(...),
223
+ client_message_id: str | None = Form(None),
224
+ ):
225
+ """
226
+ Gửi text + file ảnh trong một request multipart: validate, upload bucket `image`,
227
+ ghi session_assets, lưu message kèm metadata (URL, size, type), rồi enqueue solve
228
+ (image_url trỏ public URL để orchestrator OCR).
229
+ """
230
+ supabase = get_supabase()
231
+ uid = str(user_id)
232
+ _assert_session_owner(supabase, session_id, user_id, uid, "owner_check_solve_multipart")
233
+
234
+ t = (text or "").strip()
235
+ if not t:
236
+ raise HTTPException(status_code=400, detail="text must not be empty.")
237
+
238
+ body = await file.read()
239
+ ext, content_type = validate_chat_image_bytes(file.filename, body, file.content_type)
240
+
241
+ job_id = str(uuid.uuid4())
242
+ up = upload_session_chat_image(session_id, job_id, body, ext, content_type)
243
+ public_url = up["public_url"]
244
+
245
+ message_metadata = {
246
+ "image_url": public_url,
247
+ "attachment": {
248
+ "public_url": public_url,
249
+ "storage_path": up["storage_path"],
250
+ "size_bytes": len(body),
251
+ "content_type": content_type,
252
+ "original_filename": file.filename or "",
253
+ "session_asset_id": up.get("session_asset_id"),
254
+ },
255
+ }
256
+ request = SolveRequest(text=t, image_url=public_url, client_message_id=client_message_id)
257
+ return _enqueue_solve_common(
258
+ supabase,
259
+ background_tasks,
260
+ session_id,
261
+ user_id,
262
+ uid,
263
+ request,
264
+ message_metadata,
265
+ job_id,
266
+ )
267
+
268
+
269
+ @router.post("/{session_id}/render_video", response_model=RenderVideoResponse)
270
+ async def render_video(
271
+ session_id: str,
272
+ request: RenderVideoRequest,
273
+ background_tasks: BackgroundTasks,
274
+ user_id=Depends(get_current_user_id),
275
+ ):
276
+ """
277
+ Yêu cầu tạo video Manim từ trạng thái hình ảnh mới nhất của session.
278
+ """
279
+ supabase = get_supabase()
280
+ if not supabase:
281
+ raise HTTPException(status_code=503, detail="Database service currently unavailable.")
282
+
283
+ uid = str(user_id)
284
+ # 1. Kiểm tra quyền sở hữu
285
+ _assert_session_owner(supabase, session_id, user_id, uid, "owner_check_render_video")
286
+
287
+ # 2. Tìm tin nhắn assistant có metadata hình học (cụ thể job_id hoặc mới nhất trong 10 tin nhắn gần nhất)
288
+ msg_res = (
289
+ supabase.table("messages")
290
+ .select("metadata")
291
+ .eq("session_id", session_id)
292
+ .eq("role", "assistant")
293
+ .order("created_at", desc=True)
294
+ .limit(10)
295
+ .execute()
296
+ )
297
+
298
+ latest_geometry = None
299
+ if msg_res.data:
300
+ for msg in msg_res.data:
301
+ meta = msg.get("metadata", {})
302
+ # Nếu có yêu cầu job_id cụ thể, phải khớp job_id
303
+ if request.job_id and meta.get("job_id") != request.job_id:
304
+ continue
305
+
306
+ # Phải có dữ liệu hình học
307
+ if meta.get("geometry_dsl") and meta.get("coordinates"):
308
+ latest_geometry = meta
309
+ break
310
+
311
+ if not latest_geometry:
312
+ raise HTTPException(status_code=404, detail="Không tìm thấy dữ liệu hình học để render video.")
313
+
314
+ # 3. Tạo Job rendering
315
+ job_id = str(uuid.uuid4())
316
+ supabase.table("jobs").insert({
317
+ "id": job_id,
318
+ "user_id": user_id,
319
+ "session_id": session_id,
320
+ "status": "rendering_queued",
321
+ "input_text": f"Render video requested at {job_id}",
322
+ }).execute()
323
+
324
+ # 4. Dispatch background task
325
+ background_tasks.add_task(process_render_job, job_id, session_id, latest_geometry)
326
+
327
+ return RenderVideoResponse(job_id=job_id, status="rendering_queued")
328
+
329
+
330
+ async def process_session_job(
331
+ job_id: str, session_id: str, request: SolveRequest, user_id: str
332
+ ):
333
+ """Tiến trình giải toán ngầm, tạo hình ảnh tĩnh."""
334
+ from app.websocket_manager import notify_status
335
+
336
+ async def status_update(status: str):
337
+ await notify_status(job_id, {"status": status, "job_id": job_id})
338
+
339
+ supabase = get_supabase()
340
+ try:
341
+ history_res = (
342
+ supabase.table("messages")
343
+ .select("*")
344
+ .eq("session_id", session_id)
345
+ .order("created_at", desc=False)
346
+ .execute()
347
+ )
348
+ history = history_res.data if history_res.data else []
349
+
350
+ result = await get_orchestrator().run(
351
+ request.text,
352
+ request.image_url,
353
+ job_id=job_id,
354
+ session_id=session_id,
355
+ status_callback=status_update,
356
+ history=history,
357
+ )
358
+
359
+ status = result.get("status", "error") if "error" not in result else "error"
360
+
361
+ supabase.table("jobs").update({"status": status, "result": result}).eq(
362
+ "id", job_id
363
+ ).execute()
364
+
365
+ supabase.table("messages").insert(
366
+ {
367
+ "session_id": session_id,
368
+ "role": "assistant",
369
+ "type": "analysis" if "error" not in result else "error",
370
+ "content": (
371
+ result.get("semantic_analysis", "Đã có lỗi xảy ra.")
372
+ if "error" not in result
373
+ else result["error"]
374
+ ),
375
+ "metadata": {
376
+ "job_id": job_id,
377
+ "coordinates": result.get("coordinates"),
378
+ "geometry_dsl": result.get("geometry_dsl"),
379
+ "polygon_order": result.get("polygon_order", []),
380
+ "drawing_phases": result.get("drawing_phases", []),
381
+ "circles": result.get("circles", []),
382
+ "solids": result.get("solids", []),
383
+ "faces": result.get("faces", []),
384
+ "lines": result.get("lines", []),
385
+ "rays": result.get("rays", []),
386
+ "visualization_graph": result.get("visualization_graph"),
387
+ "auxiliary": result.get("auxiliary", []),
388
+ "solution": result.get("solution"),
389
+ "is_3d": result.get("is_3d", False),
390
+ },
391
+ }
392
+ ).execute()
393
+
394
+ await notify_status(job_id, {"status": status, "job_id": job_id, "result": result})
395
+
396
+ except Exception as e:
397
+ logger.exception("Error processing session job %s", job_id)
398
+ error_msg = format_error_for_user(e)
399
+ supabase = get_supabase()
400
+ supabase.table("jobs").update(
401
+ {"status": "error", "result": {"error": str(e)}}
402
+ ).eq("id", job_id).execute()
403
+ supabase.table("messages").insert(
404
+ {
405
+ "session_id": session_id,
406
+ "role": "assistant",
407
+ "type": "error",
408
+ "content": error_msg,
409
+ "metadata": {"job_id": job_id},
410
+ }
411
+ ).execute()
412
+ await notify_status(job_id, {"status": "error", "job_id": job_id, "error": error_msg})
413
+
414
+ async def process_render_job(job_id: str, session_id: str, geometry_data: dict):
415
+ """Tiến trình render video qua External Manim API."""
416
+ from app.websocket_manager import notify_status
417
+ from manim_client import ManimClient, build_visualization_spec
418
+ from manim_client.schemas import ErrorCode
419
+ from app.errors import format_error_for_user
420
+
421
+ await notify_status(job_id, {"status": "rendering_queued", "job_id": job_id})
422
+ supabase = get_supabase()
423
+
424
+ try:
425
+ manim_url = os.getenv("MANIM_SERVICE_URL", "https://cuong2004-manim-agent.hf.space")
426
+ manim_token = os.getenv("MANIM_INTERNAL_TOKEN")
427
+ client = ManimClient(base_url=manim_url, internal_token=manim_token)
428
+
429
+ vis_spec = build_visualization_spec(geometry_data)
430
+ logger.info(f"[RenderJob] Submitting job {job_id} to External Manim API...")
431
+
432
+ resp = await client.submit_render_job(vis_spec)
433
+ if resp.status == "failed":
434
+ err_code = resp.get_error_code() or ErrorCode.MANIM_REQUEST_FAILED
435
+ err_msg = resp.get_error_message() or "Không thể gửi yêu cầu tạo video tới máy chủ Manim."
436
+ if supabase:
437
+ supabase.table("jobs").update({
438
+ "status": "error",
439
+ "result": {"error": {"code": err_code, "message": err_msg}}
440
+ }).eq("id", job_id).execute()
441
+ if session_id:
442
+ supabase.table("messages").insert({
443
+ "session_id": session_id,
444
+ "role": "assistant",
445
+ "type": "error",
446
+ "content": f"Không thể tạo video: {err_msg}",
447
+ "metadata": {"job_id": job_id, "error_code": err_code},
448
+ }).execute()
449
+ await notify_status(job_id, {"status": "error", "job_id": job_id, "error": err_msg, "error_code": err_code})
450
+ return
451
+
452
+ manim_job_id = resp.job_id
453
+ if supabase:
454
+ supabase.table("jobs").update({
455
+ "status": "rendering",
456
+ "result": {"manim_job_id": str(manim_job_id)}
457
+ }).eq("id", job_id).execute()
458
+ await notify_status(job_id, {"status": "rendering", "job_id": job_id, "manim_job_id": str(manim_job_id)})
459
+
460
+ # Poll for completion
461
+ poll_timeout = float(os.getenv("MANIM_POLL_TIMEOUT", "600.0"))
462
+ status_resp = await client.wait_for_completion(manim_job_id, poll_interval=3.0, max_wait=poll_timeout)
463
+ video_url = status_resp.video_url
464
+
465
+ if status_resp.status == "failed" or not video_url:
466
+ err_code = status_resp.get_error_code() or ErrorCode.MANIM_RENDER_FAILED
467
+ err_msg = status_resp.get_error_message() or "Tiến trình dựng video Manim thất bại."
468
+ if supabase:
469
+ supabase.table("jobs").update({
470
+ "status": "error",
471
+ "result": {"error": {"code": err_code, "message": err_msg}}
472
+ }).eq("id", job_id).execute()
473
+ if session_id:
474
+ supabase.table("messages").insert({
475
+ "session_id": session_id,
476
+ "role": "assistant",
477
+ "type": "error",
478
+ "content": f"Không thể tạo video: {err_msg}",
479
+ "metadata": {"job_id": job_id, "error_code": err_code},
480
+ }).execute()
481
+ await notify_status(job_id, {"status": "error", "job_id": job_id, "error": err_msg, "error_code": err_code})
482
+ return
483
+
484
+ final_result = geometry_data.copy()
485
+ final_result["video_url"] = video_url
486
+ final_result["manim_job_id"] = str(manim_job_id)
487
+
488
+ if supabase:
489
+ supabase.table("jobs").update({
490
+ "status": "success",
491
+ "result": final_result
492
+ }).eq("id", job_id).execute()
493
+
494
+ if session_id:
495
+ supabase.table("messages").insert({
496
+ "session_id": session_id,
497
+ "role": "assistant",
498
+ "type": "analysis",
499
+ "content": geometry_data.get("semantic_analysis", "🎬 Video minh họa hình học đã hoàn tất."),
500
+ "metadata": {
501
+ "job_id": job_id,
502
+ "video_url": video_url,
503
+ "coordinates": geometry_data.get("coordinates"),
504
+ "geometry_dsl": geometry_data.get("geometry_dsl"),
505
+ "polygon_order": geometry_data.get("polygon_order", []),
506
+ "drawing_phases": geometry_data.get("drawing_phases", []),
507
+ "circles": geometry_data.get("circles", []),
508
+ "solids": geometry_data.get("solids", []),
509
+ "faces": geometry_data.get("faces", []),
510
+ "lines": geometry_data.get("lines", []),
511
+ "rays": geometry_data.get("rays", []),
512
+ "visualization_graph": geometry_data.get("visualization_graph"),
513
+ "auxiliary": geometry_data.get("auxiliary", []),
514
+ "is_3d": geometry_data.get("is_3d", False),
515
+ },
516
+ }).execute()
517
+
518
+ await notify_status(job_id, {
519
+ "status": "success",
520
+ "job_id": job_id,
521
+ "result": final_result,
522
+ "video_url": video_url,
523
+ })
524
+ logger.info(f"[RenderJob] SUCCESS for job {job_id}: video_url={video_url}")
525
+
526
+ except Exception as e:
527
+ logger.exception(f"[RenderJob] FAILED to render video: {e}")
528
+ safe_msg = format_error_for_user(e)
529
+ if supabase:
530
+ try:
531
+ supabase.table("jobs").update({
532
+ "status": "error",
533
+ "result": {"error": {"code": ErrorCode.INTERNAL_ERROR, "message": safe_msg}}
534
+ }).eq("id", job_id).execute()
535
+ if session_id:
536
+ supabase.table("messages").insert({
537
+ "session_id": session_id,
538
+ "role": "assistant",
539
+ "type": "error",
540
+ "content": f"Lỗi render video: {safe_msg}",
541
+ "metadata": {"job_id": job_id, "error_code": ErrorCode.INTERNAL_ERROR},
542
+ }).execute()
543
+ except Exception as db_e:
544
+ logger.error(f"Failed to record render error in DB: {db_e}")
545
+ await notify_status(job_id, {"status": "error", "job_id": job_id, "error": safe_msg, "error_code": ErrorCode.INTERNAL_ERROR})
546
+
app/runtime_env.py ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Default process env vars (Paddle/OpenMP). Call as early as possible after load_dotenv."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import os
6
+
7
+
8
+ def apply_runtime_env_defaults() -> None:
9
+ # Paddle respects OMP_NUM_THREADS at import; setdefault loses if platform already set 2+
10
+ os.environ["OMP_NUM_THREADS"] = "1"
11
+ os.environ["MKL_NUM_THREADS"] = "1"
12
+ os.environ["OPENBLAS_NUM_THREADS"] = "1"
app/session_cache.py ADDED
@@ -0,0 +1,31 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """TTL in-memory cache để giảm truy vấn Supabase lặp lại (quyền sở hữu session)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ from typing import Callable
6
+
7
+ from cachetools import TTLCache
8
+
9
+ from app.logutil import log_step
10
+
11
+ _session_owner: TTLCache[tuple[str, str], bool] = TTLCache(maxsize=4096, ttl=45)
12
+
13
+
14
+ def invalidate_session_owner(session_id: str, user_id: str) -> None:
15
+ _session_owner.pop((session_id, user_id), None)
16
+ log_step("cache_invalidate", target="session_owner", session_id=session_id, user_id=user_id)
17
+
18
+
19
+ def session_owned_by_user(
20
+ session_id: str,
21
+ user_id: str,
22
+ fetch: Callable[[], bool],
23
+ ) -> bool:
24
+ key = (session_id, user_id)
25
+ if key in _session_owner:
26
+ log_step("cache_hit", kind="session_owner", session_id=session_id)
27
+ return _session_owner[key]
28
+ log_step("cache_miss", kind="session_owner", session_id=session_id)
29
+ ok = fetch()
30
+ _session_owner[key] = ok
31
+ return ok
app/supabase_client.py ADDED
@@ -0,0 +1,45 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import logging
3
+ from supabase import Client, ClientOptions, create_client
4
+ from supabase_auth import SyncMemoryStorage
5
+ from dotenv import load_dotenv
6
+
7
+ load_dotenv()
8
+
9
+ from app.url_utils import sanitize_env
10
+
11
+ logger = logging.getLogger(__name__)
12
+
13
+ _supabase_client = None
14
+
15
+
16
+ def get_supabase() -> Client:
17
+ """Service-role client for server-side operations with lazy init."""
18
+ global _supabase_client
19
+ if _supabase_client is not None:
20
+ return _supabase_client
21
+
22
+ url = sanitize_env(os.getenv("SUPABASE_URL"))
23
+ key = sanitize_env(os.getenv("SUPABASE_SERVICE_ROLE_KEY") or os.getenv("SUPABASE_KEY"))
24
+ if not url or not key:
25
+ logger.warning("[Supabase] SUPABASE_URL or key not configured. Cloud DB operations will be unavailable.")
26
+ return None
27
+
28
+ try:
29
+ _supabase_client = create_client(url, key)
30
+ return _supabase_client
31
+ except Exception as e:
32
+ logger.warning("[Supabase] Failed to initialize Supabase client: %s", e)
33
+ return None
34
+
35
+
36
+ def get_supabase_for_user_jwt(access_token: str) -> Client:
37
+ """Client scoped to the logged-in user."""
38
+ url = sanitize_env(os.getenv("SUPABASE_URL"))
39
+ anon = sanitize_env(os.getenv("SUPABASE_ANON_KEY") or os.getenv("NEXT_PUBLIC_SUPABASE_ANON_KEY"))
40
+ if not url or not anon:
41
+ raise RuntimeError("SUPABASE_URL and SUPABASE_ANON_KEY must be set for user-scoped Supabase access")
42
+ base_opts = ClientOptions(storage=SyncMemoryStorage())
43
+ merged_headers = {**dict(base_opts.headers), "Authorization": f"Bearer {access_token}"}
44
+ opts = ClientOptions(storage=SyncMemoryStorage(), headers=merged_headers)
45
+ return create_client(url, anon, opts)
app/url_utils.py ADDED
@@ -0,0 +1,23 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Normalize URLs / env strings (HF secrets and copy-paste often include trailing newlines)."""
2
+
3
+
4
+ def sanitize_url(value: str | None) -> str | None:
5
+ if value is None:
6
+ return None
7
+ s = value.strip().replace("\r", "").replace("\n", "").replace("\t", "")
8
+ return s or None
9
+
10
+
11
+ def sanitize_env(value: str | None) -> str | None:
12
+ """Strip whitespace and line breaks from environment-backed strings."""
13
+ return sanitize_url(value)
14
+
15
+
16
+ # OpenAI SDK (>=1.x) requires a non-empty api_key at client construction (Docker build / prewarm has no secrets).
17
+ _OPENAI_API_KEY_BUILD_PLACEHOLDER = "build-placeholder-openrouter-not-for-production"
18
+
19
+
20
+ def openai_compatible_api_key(raw: str | None) -> str:
21
+ """Return sanitized API key, or a placeholder so AsyncOpenAI() can be constructed without env at build time."""
22
+ k = sanitize_env(raw)
23
+ return k if k else _OPENAI_API_KEY_BUILD_PLACEHOLDER
app/websocket_manager.py ADDED
@@ -0,0 +1,60 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """WebSocket connection registry and job status notifications (avoid circular imports with main)."""
2
+
3
+ from __future__ import annotations
4
+
5
+ import logging
6
+ from typing import Dict, List
7
+
8
+ from fastapi import WebSocket, WebSocketDisconnect
9
+
10
+ logger = logging.getLogger(__name__)
11
+
12
+ active_connections: Dict[str, List[WebSocket]] = {}
13
+
14
+
15
+ async def notify_status(job_id: str, data: dict) -> None:
16
+ if job_id not in active_connections:
17
+ return
18
+ for connection in list(active_connections[job_id]):
19
+ try:
20
+ await connection.send_json(data)
21
+ except Exception as e:
22
+ logger.warning("WS error sending to %s: %s (removing dead connection)", job_id, e)
23
+ try:
24
+ active_connections[job_id].remove(connection)
25
+ except (ValueError, KeyError):
26
+ pass
27
+ if job_id in active_connections and not active_connections[job_id]:
28
+ del active_connections[job_id]
29
+
30
+
31
+ def register_websocket_routes(app) -> None:
32
+ """Attach websocket endpoint to the FastAPI app."""
33
+
34
+ @app.websocket("/ws/{job_id}")
35
+ async def websocket_endpoint(websocket: WebSocket, job_id: str) -> None:
36
+ await websocket.accept()
37
+ if job_id not in active_connections:
38
+ active_connections[job_id] = []
39
+ active_connections[job_id].append(websocket)
40
+
41
+ # Send immediate ACK so client immediately transitions from 'connecting' to 'processing'
42
+ try:
43
+ await websocket.send_json({
44
+ "status": "processing",
45
+ "job_id": job_id,
46
+ "message": "Đang xử lý bài toán..."
47
+ })
48
+ except Exception:
49
+ pass
50
+
51
+ try:
52
+ while True:
53
+ msg = await websocket.receive_text()
54
+ if msg == "ping":
55
+ await websocket.send_text("pong")
56
+ except WebSocketDisconnect:
57
+ if job_id in active_connections and websocket in active_connections[job_id]:
58
+ active_connections[job_id].remove(websocket)
59
+ if not active_connections[job_id]:
60
+ del active_connections[job_id]
clean_ports.sh ADDED
@@ -0,0 +1,22 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ # Script to kill all project-related processes for a clean restart
3
+
4
+ echo "🧹 Cleaning up project processes..."
5
+
6
+ # Kill things on ports 8000 (Backend) and 3000 (Frontend)
7
+ PORTS="8000 3000 11020"
8
+ for PORT in $PORTS; do
9
+ PIDS=$(lsof -ti :$PORT)
10
+ if [ ! -z "$PIDS" ]; then
11
+ echo "Killing processes on port $PORT: $PIDS"
12
+ kill -9 $PIDS 2>/dev/null
13
+ fi
14
+ done
15
+
16
+ # Kill by process name
17
+ echo "Killing any remaining Celery, Uvicorn, or Manim processes..."
18
+ pkill -9 -f "celery" 2>/dev/null
19
+ pkill -9 -f "uvicorn" 2>/dev/null
20
+ pkill -9 -f "manim" 2>/dev/null
21
+
22
+ echo "✅ Done. You can now restart your Backend, Worker, and Frontend."
config/__init__.py ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from config.schemas import ModelTier, AgentConfig, RetryPolicyConfig, AgentModelsConfig
2
+ from config.settings import settings, Settings, ProviderCredentials
3
+ from config.loader import load_agent_config, get_agent_config_resolver, get_agent_models_config
4
+
5
+ __all__ = [
6
+ "ModelTier",
7
+ "AgentConfig",
8
+ "RetryPolicyConfig",
9
+ "AgentModelsConfig",
10
+ "settings",
11
+ "Settings",
12
+ "ProviderCredentials",
13
+ "load_agent_config",
14
+ "get_agent_config_resolver",
15
+ "get_agent_models_config",
16
+ ]
config/agent_models.yaml ADDED
@@ -0,0 +1,83 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ version: 2
2
+
3
+ defaults:
4
+ temperature: 0.1
5
+ max_tokens: 8192
6
+ timeout_seconds: 300
7
+
8
+ retry_policy:
9
+ retryable_errors:
10
+ - rate_limit
11
+ - timeout
12
+ - connection
13
+ - server_error
14
+
15
+ non_retryable_errors:
16
+ - invalid_request
17
+ - authentication
18
+
19
+ agents:
20
+
21
+ ocr:
22
+ name: ocr
23
+ description: "Local visual OCR using Pix2Text. Pure perception — no LLM hallucination."
24
+
25
+ tiers:
26
+ - model: gemini/gemini-3.5-flash-lite
27
+ max_attempts: 1
28
+ reasoning_effort: low
29
+
30
+ temperature: 0.1
31
+ max_tokens: 4096
32
+ timeout_seconds: 120
33
+
34
+ confidence_gateway:
35
+ enabled: true
36
+ threshold: 0.85
37
+
38
+ correction:
39
+ enabled: true
40
+ model: gemini/gemini-3.5-flash-lite
41
+ temperature: 0.1
42
+ max_tokens: 4096
43
+ timeout_seconds: 60
44
+ max_attempts: 1
45
+ reasoning_effort: low
46
+
47
+
48
+ geometry_parser:
49
+ name: geometry_parser
50
+ description: "Semantic geometry parsing and Geometry DSL generation"
51
+
52
+ tiers:
53
+ - model: gemini/gemini-3.5-flash-lite
54
+ max_attempts: 1
55
+ reasoning_effort: low
56
+
57
+ - model: gemini/gemini-3.5-flash
58
+ max_attempts: 1
59
+ reasoning_effort: medium
60
+
61
+ temperature: 0.1
62
+ max_tokens: 16384
63
+ timeout_seconds: 120
64
+
65
+
66
+ reasoning_solver:
67
+ name: reasoning_solver
68
+ description: "Program-aided mathematical reasoning with SymPy verification"
69
+
70
+ tiers:
71
+ - model: gemini/gemini-3.6-flash
72
+ max_attempts: 1
73
+ reasoning_effort: high
74
+
75
+ - model: gemini/gemini-3.7-flash
76
+ max_attempts: 1
77
+ reasoning_effort: high
78
+
79
+ temperature: 0.1
80
+ max_tokens: 16384
81
+ timeout_seconds: 120
82
+
83
+
config/loader.py ADDED
@@ -0,0 +1,85 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ import yaml
3
+ import logging
4
+ from pathlib import Path
5
+ from typing import Optional, Dict
6
+ from config.schemas import AgentConfig, AgentModelsConfig
7
+
8
+ logger = logging.getLogger(__name__)
9
+
10
+ _CACHED_CONFIG: Optional[AgentModelsConfig] = None
11
+ _DEFAULT_CONFIG_PATH = Path(__file__).parent / "agent_models.yaml"
12
+
13
+
14
+ class AgentConfigResolver:
15
+ """Resolves and validates typed AgentConfig from agent_models.yaml."""
16
+
17
+ def __init__(self, config_path: Optional[Path] = None):
18
+ self.config_path = config_path or _DEFAULT_CONFIG_PATH
19
+ self._config: Optional[AgentModelsConfig] = None
20
+ self._load()
21
+
22
+ def _load(self) -> None:
23
+ if not self.config_path.exists():
24
+ raise FileNotFoundError(f"Agent models config file not found: {self.config_path}")
25
+
26
+ try:
27
+ with open(self.config_path, "r", encoding="utf-8") as f:
28
+ data = yaml.safe_load(f) or {}
29
+
30
+ # Strict validation with Pydantic
31
+ self._config = AgentModelsConfig(**data)
32
+ logger.info(
33
+ f"[AgentConfigResolver] Loaded {len(self._config.agents)} agent configs from {self.config_path}"
34
+ )
35
+ except Exception as e:
36
+ logger.error(f"[AgentConfigResolver] Failed to parse agent_models.yaml: {e}", exc_info=True)
37
+ raise
38
+
39
+ def get_agent_config(self, agent_name: str) -> AgentConfig:
40
+ if not self._config:
41
+ self._load()
42
+ if not self._config or agent_name not in self._config.agents:
43
+ # Fallback or generic agent config
44
+ logger.warning(
45
+ f"[AgentConfigResolver] Agent '{agent_name}' not defined in config, using defaults."
46
+ )
47
+ from config.schemas import ModelTier
48
+ return AgentConfig(
49
+ name=agent_name,
50
+ description=f"Auto-generated fallback config for {agent_name}",
51
+ tiers=[
52
+ ModelTier(model="gemini/gemini-3.7-flash", max_attempts=1),
53
+ ModelTier(model="gemini/gemini-3.6-flash", max_attempts=1),
54
+ ModelTier(model="gemini/gemini-3.5-flash", max_attempts=1),
55
+ ModelTier(model="gemini/gemini-2.5-flash", max_attempts=1),
56
+ ],
57
+ temperature=0.2,
58
+ max_tokens=8192,
59
+ timeout_seconds=120,
60
+ )
61
+ return self._config.agents[agent_name]
62
+
63
+ @property
64
+ def config(self) -> AgentModelsConfig:
65
+ if not self._config:
66
+ self._load()
67
+ return self._config # type: ignore
68
+
69
+
70
+ _RESOLVER_INSTANCE: Optional[AgentConfigResolver] = None
71
+
72
+
73
+ def get_agent_config_resolver() -> AgentConfigResolver:
74
+ global _RESOLVER_INSTANCE
75
+ if _RESOLVER_INSTANCE is None:
76
+ _RESOLVER_INSTANCE = AgentConfigResolver()
77
+ return _RESOLVER_INSTANCE
78
+
79
+
80
+ def load_agent_config(agent_name: str) -> AgentConfig:
81
+ return get_agent_config_resolver().get_agent_config(agent_name)
82
+
83
+
84
+ def get_agent_models_config() -> AgentModelsConfig:
85
+ return get_agent_config_resolver().config
config/schemas.py ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from typing import List, Dict, Optional, Literal
2
+ from pydantic import BaseModel, Field, field_validator
3
+
4
+
5
+ class ModelTier(BaseModel):
6
+ """Configuration for a specific model tier within an agent's cascade."""
7
+ model: str = Field(..., description="Provider/Model string, e.g. gemini/gemini-2.5-flash")
8
+ max_attempts: int = Field(default=1, ge=1, le=5, description="Max attempts with this model tier")
9
+ reasoning_effort: Optional[Literal["low", "medium", "high"]] = Field(
10
+ default=None, description="Reasoning effort for this specific model tier"
11
+ )
12
+
13
+ @field_validator("model")
14
+ @classmethod
15
+ def validate_model_format(cls, v: str) -> str:
16
+ v = v.strip()
17
+ if not v:
18
+ raise ValueError("Model identifier cannot be empty")
19
+ return v
20
+
21
+
22
+ class OCRCorrectionConfig(BaseModel):
23
+ """Configuration for optional VLM-based OCR correction."""
24
+ enabled: bool = Field(default=True, description="Whether VLM correction is enabled when gateway triggers")
25
+ model: str = Field(default="gemini/gemini-3.5-flash-lite", description="VLM model for OCR correction")
26
+ temperature: float = Field(default=0.1, ge=0.0, le=2.0, description="VLM correction temperature")
27
+ max_tokens: int = Field(default=4096, gt=0, description="Max output tokens for VLM correction")
28
+ timeout_seconds: int = Field(default=60, gt=0, description="VLM correction timeout")
29
+ max_attempts: int = Field(default=1, ge=1, le=3, description="Max VLM correction attempts")
30
+ reasoning_effort: Optional[Literal["low", "medium", "high"]] = Field(
31
+ default="low", description="Reasoning effort for VLM OCR correction"
32
+ )
33
+
34
+
35
+ class ConfidenceGatewayConfig(BaseModel):
36
+ """OCR confidence gateway configuration."""
37
+ enabled: bool = Field(default=True, description="Enable confidence-based gateway")
38
+ threshold: float = Field(default=0.85, ge=0.0, le=1.0, description="Confidence threshold below which VLM correction triggers")
39
+ correction: OCRCorrectionConfig = Field(default_factory=OCRCorrectionConfig)
40
+
41
+
42
+ class AgentConfig(BaseModel):
43
+ """Configuration for a specific agent in MathSolver."""
44
+ name: str = Field(..., description="Unique agent identifier")
45
+ description: Optional[str] = Field(default=None, description="Agent responsibility summary")
46
+ tiers: List[ModelTier] = Field(..., min_length=1, description="Cascading model tiers in execution priority")
47
+ temperature: float = Field(default=0.2, ge=0.0, le=2.0, description="Sampling temperature")
48
+ max_tokens: int = Field(default=8192, gt=0, description="Max output tokens")
49
+ timeout_seconds: int = Field(default=120, gt=0, description="Timeout in seconds")
50
+ reasoning_effort: Optional[Literal["low", "medium", "high"]] = Field(
51
+ default=None, description="Reasoning effort for thinking models"
52
+ )
53
+ confidence_gateway: Optional[ConfidenceGatewayConfig] = Field(
54
+ default=None, description="OCR confidence gateway config (only for OCR agent)"
55
+ )
56
+
57
+
58
+ class RetryPolicyConfig(BaseModel):
59
+ """Global retry policy configuration."""
60
+ retryable_errors: List[str] = Field(
61
+ default_factory=lambda: ["rate_limit", "timeout", "connection", "server_error"]
62
+ )
63
+ non_retryable_errors: List[str] = Field(
64
+ default_factory=lambda: ["invalid_request", "authentication"]
65
+ )
66
+
67
+
68
+ class AgentModelsConfig(BaseModel):
69
+ """Top-level agent models configuration schema."""
70
+ version: int = Field(default=1)
71
+ defaults: Dict[str, object] = Field(default_factory=dict)
72
+ retry_policy: RetryPolicyConfig = Field(default_factory=RetryPolicyConfig)
73
+ agents: Dict[str, AgentConfig] = Field(..., description="Mapping of agent names to their configurations")
config/settings.py ADDED
@@ -0,0 +1,75 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import os
2
+ from pathlib import Path
3
+ from typing import List, Dict
4
+ from pydantic import BaseModel, Field
5
+ from dotenv import load_dotenv
6
+
7
+ # Load from backend/.env if available
8
+ _env_path = Path(__file__).parents[1] / ".env"
9
+ if _env_path.exists():
10
+ load_dotenv(dotenv_path=_env_path)
11
+ else:
12
+ load_dotenv()
13
+
14
+
15
+
16
+ class ProviderCredentials(BaseModel):
17
+ provider: str
18
+ keys: List[str] = Field(default_factory=list)
19
+
20
+
21
+ def parse_comma_separated_keys(raw: str) -> List[str]:
22
+ if not raw:
23
+ return []
24
+ keys = []
25
+ for piece in raw.split(","):
26
+ cleaned = piece.strip().strip("'\" ")
27
+ if cleaned:
28
+ keys.append(cleaned)
29
+ return keys
30
+
31
+
32
+ class Settings(BaseModel):
33
+ app_env: str = Field(default_factory=lambda: os.getenv("APP_ENV", "development"))
34
+ redis_url: str = Field(default_factory=lambda: os.getenv("REDIS_URL", "redis://localhost:6379/0"))
35
+ llm_timeout_seconds: int = Field(default_factory=lambda: int(os.getenv("LLM_TIMEOUT_SECONDS", "120")))
36
+ llm_cooldown_seconds: int = Field(default_factory=lambda: int(os.getenv("LLM_COOLDOWN_SECONDS", "60")))
37
+ default_chat_model: str = Field(default_factory=lambda: os.getenv("DEFAULT_CHAT_MODEL", "gemini/gemini-3.7-flash"))
38
+
39
+ def get_provider_credentials(self) -> Dict[str, ProviderCredentials]:
40
+ """Parses API keys for all supported providers from environment variables."""
41
+ credentials: Dict[str, ProviderCredentials] = {}
42
+
43
+ # 1. Gemini / Google
44
+ gemini_raw = os.getenv("GOOGLE_API_KEY") or os.getenv("GEMINI_API_KEY") or ""
45
+ gemini_keys = parse_comma_separated_keys(gemini_raw)
46
+ # Also check indexed keys GEMINI_API_KEY_1, GEMINI_API_KEY_2, etc.
47
+ for i in range(1, 20):
48
+ k = os.getenv(f"GEMINI_API_KEY_{i}") or os.getenv(f"GOOGLE_API_KEY_{i}")
49
+ if k and k.strip() and k.strip() not in gemini_keys:
50
+ gemini_keys.append(k.strip())
51
+ credentials["gemini"] = ProviderCredentials(provider="gemini", keys=gemini_keys)
52
+
53
+ # 2. OpenAI
54
+ openai_raw = os.getenv("OPENAI_API_KEY") or ""
55
+ openai_keys = parse_comma_separated_keys(openai_raw)
56
+ for i in range(1, 10):
57
+ k = os.getenv(f"OPENAI_API_KEY_{i}")
58
+ if k and k.strip() and k.strip() not in openai_keys:
59
+ openai_keys.append(k.strip())
60
+ credentials["openai"] = ProviderCredentials(provider="openai", keys=openai_keys)
61
+
62
+ # 3. Anthropic
63
+ anthropic_raw = os.getenv("ANTHROPIC_API_KEY") or ""
64
+ anthropic_keys = parse_comma_separated_keys(anthropic_raw)
65
+ credentials["anthropic"] = ProviderCredentials(provider="anthropic", keys=anthropic_keys)
66
+
67
+ # 4. OpenRouter
68
+ openrouter_raw = os.getenv("OPENROUTER_API_KEY") or os.getenv("OPENROUTER_API_KEY_1") or ""
69
+ openrouter_keys = parse_comma_separated_keys(openrouter_raw)
70
+ credentials["openrouter"] = ProviderCredentials(provider="openrouter", keys=openrouter_keys)
71
+
72
+ return credentials
73
+
74
+
75
+ settings = Settings()
dump.rdb ADDED
Binary file (5.44 kB). View file
 
eval/__init__.py ADDED
@@ -0,0 +1,24 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from eval.benchmark import BenchmarkDataset, BenchmarkSample
2
+ from eval.metrics import (
3
+ OCRMetrics,
4
+ ParserMetrics,
5
+ PipelineEvalSummary,
6
+ SolverMetrics,
7
+ compute_cer,
8
+ compute_wer,
9
+ latex_match,
10
+ )
11
+ from eval.runner import EvalRunner
12
+
13
+ __all__ = [
14
+ "BenchmarkDataset",
15
+ "BenchmarkSample",
16
+ "OCRMetrics",
17
+ "ParserMetrics",
18
+ "SolverMetrics",
19
+ "PipelineEvalSummary",
20
+ "EvalRunner",
21
+ "compute_cer",
22
+ "compute_wer",
23
+ "latex_match",
24
+ ]
eval/benchmark.py ADDED
@@ -0,0 +1,71 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Benchmark Dataset Models and Loader for MathSolver Evaluation.
3
+ """
4
+
5
+ from __future__ import annotations
6
+
7
+ import json
8
+ from pathlib import Path
9
+ from typing import Any, Dict, List, Optional
10
+ from pydantic import BaseModel, Field
11
+
12
+
13
+ class BenchmarkSample(BaseModel):
14
+ """Evaluation sample definition representing a standardized geometry problem."""
15
+ id: str = Field(..., description="Unique sample identifier")
16
+ category: str = Field(default="geometry", description="Problem category: geometry, algebra, 3d, 2d")
17
+ image_url: Optional[str] = Field(default=None, description="Image URL if testing OCR")
18
+ problem_text: str = Field(..., description="Canonical Vietnamese/LaTeX problem statement")
19
+ expected_type: Optional[str] = Field(default=None, description="Expected shape type (e.g. pyramid, cube)")
20
+ expected_entities: Optional[List[str]] = Field(default=None, description="Expected primary entities")
21
+ expected_dsl: Optional[str] = Field(default=None, description="Reference Geometry DSL")
22
+ expected_answer: Optional[str] = Field(default=None, description="Ground-truth final answer / value")
23
+ metadata: Dict[str, Any] = Field(default_factory=dict, description="Additional reference annotations")
24
+
25
+
26
+ class BenchmarkDataset:
27
+ """Benchmark dataset container."""
28
+
29
+ def __init__(self, samples: List[BenchmarkSample]):
30
+ self.samples = samples
31
+
32
+ def __len__(self) -> int:
33
+ return len(self.samples)
34
+
35
+ def __iter__(self):
36
+ return iter(self.samples)
37
+
38
+ @classmethod
39
+ def from_file(cls, path: str | Path) -> "BenchmarkDataset":
40
+ """Loads benchmark samples from a JSON file."""
41
+ file_path = Path(path)
42
+ if not file_path.exists():
43
+ raise FileNotFoundError(f"Benchmark file not found: {file_path}")
44
+
45
+ with open(file_path, "r", encoding="utf-8") as f:
46
+ data = json.load(f)
47
+
48
+ if isinstance(data, list):
49
+ samples = [BenchmarkSample(**item) for item in data]
50
+ elif isinstance(data, dict) and "samples" in data:
51
+ samples = [BenchmarkSample(**item) for item in data["samples"]]
52
+ else:
53
+ raise ValueError(f"Unrecognized benchmark dataset format in {file_path}")
54
+
55
+ return cls(samples)
56
+
57
+ @classmethod
58
+ def load_all_standard(cls, base_dir: Optional[Path] = None) -> "BenchmarkDataset":
59
+ """Loads all JSON files under eval/datasets/."""
60
+ if base_dir is None:
61
+ base_dir = Path(__file__).parent / "datasets"
62
+
63
+ all_samples: List[BenchmarkSample] = []
64
+ for json_file in base_dir.rglob("*.json"):
65
+ try:
66
+ ds = cls.from_file(json_file)
67
+ all_samples.extend(ds.samples)
68
+ except Exception:
69
+ pass
70
+
71
+ return cls(all_samples)
eval/datasets/README.md ADDED
@@ -0,0 +1,33 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # MathSolver Benchmark Datasets
2
+
3
+ Standardized evaluation benchmarks for regression testing, metric tracking, and ablation studies.
4
+
5
+ ## Dataset Structure
6
+
7
+ Each JSON file in `eval/datasets/` contains problem samples adhering to the `BenchmarkSample` schema:
8
+
9
+ ```json
10
+ {
11
+ "id": "geo_01_square_pyramid",
12
+ "category": "3d_pyramid",
13
+ "problem_text": "Cho hình chóp S.ABCD...",
14
+ "expected_type": "pyramid",
15
+ "expected_entities": ["S", "A", "B", "C", "D"],
16
+ "expected_dsl": "PYRAMID(S_ABCD)\nSQUARE(ABCD)...",
17
+ "expected_answer": "32"
18
+ }
19
+ ```
20
+
21
+ ## Running Evaluation
22
+
23
+ To evaluate deterministic DSL solvability & geometry validator pass rates:
24
+
25
+ ```python
26
+ from eval.benchmark import BenchmarkDataset
27
+ from eval.runner import EvalRunner
28
+
29
+ dataset = BenchmarkDataset.load_all_standard()
30
+ runner = EvalRunner()
31
+ metrics = runner.evaluate_dsl_deterministic(dataset)
32
+ print(metrics.to_dict())
33
+ ```
eval/datasets/geometry/sample_problems.json ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [
2
+ {
3
+ "id": "geo_01_square_pyramid",
4
+ "category": "3d_pyramid",
5
+ "problem_text": "Cho hình chóp S.ABCD có đáy ABCD là hình vuông cạnh a=4. Cạnh bên SA vuông góc với mặt phẳng đáy và SA=6. Tính thể tích khối chóp S.ABCD.",
6
+ "expected_type": "pyramid",
7
+ "expected_entities": ["S", "A", "B", "C", "D"],
8
+ "expected_dsl": "PYRAMID(S_ABCD)\nSQUARE(ABCD)\nLENGTH(AB, 4)\nLENGTH(SA, 6)\nPERPENDICULAR_PLANE(SA, ABCD)",
9
+ "expected_answer": "32"
10
+ },
11
+ {
12
+ "id": "geo_02_cube",
13
+ "category": "3d_cube",
14
+ "problem_text": "Cho hình lập phương ABCD.A1B1C1D1 có cạnh bằng 5. Tính thể tích khối lập phương.",
15
+ "expected_type": "cube",
16
+ "expected_entities": ["A", "B", "C", "D", "A1", "B1", "C1", "D1"],
17
+ "expected_dsl": "CUBE(ABCD_A1B1C1D1)\nLENGTH(AB, 5)",
18
+ "expected_answer": "125"
19
+ },
20
+ {
21
+ "id": "geo_03_triangular_prism",
22
+ "category": "3d_prism",
23
+ "problem_text": "Cho lăng trụ đứng ABC.A1B1C1 có đáy ABC là tam giác vuông tại A, AB=3, AC=4. Chiều cao lăng trụ AA1=6. Tính thể tích khối lăng trụ.",
24
+ "expected_type": "prism",
25
+ "expected_entities": ["A", "B", "C", "A1", "B1", "C1"],
26
+ "expected_dsl": "PRISM(ABC_A1B1C1)\nPOINT(A, 0, 0, 0)\nPOINT(B, 3, 0, 0)\nPOINT(C, 0, 4, 0)\nPOINT(A1, 0, 0, 6)\nPOINT(B1, 3, 0, 6)\nPOINT(C1, 0, 4, 6)\nLENGTH(AA1, 6)\nPERPENDICULAR_PLANE(AA1, ABC)",
27
+ "expected_answer": "36"
28
+ },
29
+ {
30
+ "id": "geo_04_cone",
31
+ "category": "3d_cone",
32
+ "problem_text": "Cho hình nón có bán kính đáy r=3 và chiều cao h=4. Tính thể tích khối nón.",
33
+ "expected_type": "cone",
34
+ "expected_entities": ["S", "O"],
35
+ "expected_dsl": "CONE(S_O, 3, 4)",
36
+ "expected_answer": "12*pi"
37
+ },
38
+ {
39
+ "id": "geo_05_2d_rectangle",
40
+ "category": "2d_polygon",
41
+ "problem_text": "Cho hình chữ nhật ABCD có AB=6, BC=8. Tính diện tích hình chữ nhật ABCD.",
42
+ "expected_type": "rectangle",
43
+ "expected_entities": ["A", "B", "C", "D"],
44
+ "expected_dsl": "RECTANGLE(ABCD)\nLENGTH(AB, 6)\nLENGTH(BC, 8)",
45
+ "expected_answer": "48"
46
+ }
47
+ ]
eval/metrics.py ADDED
@@ -0,0 +1,168 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Evaluation Metrics for MathSolver Pipeline.
3
+
4
+ Defines metric calculators across all pipeline stages:
5
+ - OCR: Character Error Rate (CER), Word Error Rate (WER), LaTeX Exact Match, Confidence Calibration
6
+ - Parser: JSON Validity, DSL Validity, Geometry Solvability, Validation Pass Rate
7
+ - Solver: Final Answer Accuracy, SymPy Verification Rate
8
+ - End-to-End: E2E Accuracy, Latency, Token / LLM Usage, OCR Correction Rate, Geometry Degradation Rate
9
+ """
10
+
11
+ from __future__ import annotations
12
+
13
+ import re
14
+ from dataclasses import dataclass, field
15
+ from typing import Any, Dict, List, Optional, Tuple
16
+
17
+
18
+ def _levenshtein_distance(s1: str, s2: str) -> int:
19
+ """Computes standard Levenshtein edit distance between two strings."""
20
+ if len(s1) < len(s2):
21
+ return _levenshtein_distance(s2, s1)
22
+ if len(s2) == 0:
23
+ return len(s1)
24
+
25
+ previous_row = range(len(s2) + 1)
26
+ for i, c1 in enumerate(s1):
27
+ current_row = [i + 1]
28
+ for j, c2 in enumerate(s2):
29
+ insertions = previous_row[j + 1] + 1
30
+ deletions = current_row[j] + 1
31
+ substitutions = previous_row[j] + (c1 != c2)
32
+ current_row.append(min(insertions, deletions, substitutions))
33
+ previous_row = current_row
34
+
35
+ return previous_row[-1]
36
+
37
+
38
+ def compute_cer(reference: str, hypothesis: str) -> float:
39
+ """Character Error Rate (CER)."""
40
+ if not reference and not hypothesis:
41
+ return 0.0
42
+ if not reference:
43
+ return 1.0
44
+ dist = _levenshtein_distance(reference, hypothesis)
45
+ return float(dist / max(len(reference), 1))
46
+
47
+
48
+ def compute_wer(reference: str, hypothesis: str) -> float:
49
+ """Word Error Rate (WER)."""
50
+ ref_words = reference.strip().split()
51
+ hyp_words = hypothesis.strip().split()
52
+ if not ref_words and not hyp_words:
53
+ return 0.0
54
+ if not ref_words:
55
+ return 1.0
56
+
57
+ # Word-level edit distance
58
+ n, m = len(ref_words), len(hyp_words)
59
+ dp = [[0] * (m + 1) for _ in range(n + 1)]
60
+ for i in range(n + 1):
61
+ dp[i][0] = i
62
+ for j in range(m + 1):
63
+ dp[0][j] = j
64
+
65
+ for i in range(1, n + 1):
66
+ for j in range(1, m + 1):
67
+ if ref_words[i - 1] == hyp_words[j - 1]:
68
+ dp[i][j] = dp[i - 1][j - 1]
69
+ else:
70
+ dp[i][j] = min(dp[i - 1][j] + 1, dp[i][j - 1] + 1, dp[i - 1][j - 1] + 1)
71
+
72
+ return float(dp[n][m] / max(n, 1))
73
+
74
+
75
+ def normalize_latex(formula: str) -> str:
76
+ """Normalizes LaTeX whitespace and common variations for comparison."""
77
+ if not formula:
78
+ return ""
79
+ f = formula.strip()
80
+ f = re.sub(r"\s+", "", f)
81
+ f = f.replace("\\cdot", "*").replace("\\times", "*")
82
+ f = re.sub(r"\\left|\\right", "", f)
83
+ return f
84
+
85
+
86
+ def latex_match(ref: str, hyp: str) -> bool:
87
+ """Checks whether two LaTeX expressions match after normalization."""
88
+ return normalize_latex(ref) == normalize_latex(hyp)
89
+
90
+
91
+ @dataclass
92
+ class OCRMetrics:
93
+ """Aggregated OCR metrics across evaluated samples."""
94
+ total_samples: int = 0
95
+ avg_cer: float = 0.0
96
+ avg_wer: float = 0.0
97
+ latex_exact_match_rate: float = 0.0
98
+ vlm_trigger_rate: float = 0.0
99
+ vlm_correction_rate: float = 0.0
100
+ confidence_calibration_bins: Dict[str, Dict[str, float]] = field(default_factory=dict)
101
+
102
+ def to_dict(self) -> Dict[str, Any]:
103
+ return {
104
+ "total_samples": self.total_samples,
105
+ "avg_cer": round(self.avg_cer, 4),
106
+ "avg_wer": round(self.avg_wer, 4),
107
+ "latex_exact_match_rate": round(self.latex_exact_match_rate, 4),
108
+ "vlm_trigger_rate": round(self.vlm_trigger_rate, 4),
109
+ "vlm_correction_rate": round(self.vlm_correction_rate, 4),
110
+ "confidence_calibration": self.confidence_calibration_bins,
111
+ }
112
+
113
+
114
+ @dataclass
115
+ class ParserMetrics:
116
+ """Aggregated Parser metrics."""
117
+ total_samples: int = 0
118
+ json_valid_rate: float = 0.0
119
+ dsl_valid_rate: float = 0.0
120
+ solvability_rate: float = 0.0
121
+ validation_pass_rate: float = 0.0
122
+ degradation_rate: float = 0.0
123
+
124
+ def to_dict(self) -> Dict[str, Any]:
125
+ return {
126
+ "total_samples": self.total_samples,
127
+ "json_valid_rate": round(self.json_valid_rate, 4),
128
+ "dsl_valid_rate": round(self.dsl_valid_rate, 4),
129
+ "solvability_rate": round(self.solvability_rate, 4),
130
+ "validation_pass_rate": round(self.validation_pass_rate, 4),
131
+ "degradation_rate": round(self.degradation_rate, 4),
132
+ }
133
+
134
+
135
+ @dataclass
136
+ class SolverMetrics:
137
+ """Aggregated Solver metrics."""
138
+ total_samples: int = 0
139
+ answer_exact_match_rate: float = 0.0
140
+ sympy_verification_rate: float = 0.0
141
+
142
+ def to_dict(self) -> Dict[str, Any]:
143
+ return {
144
+ "total_samples": self.total_samples,
145
+ "answer_exact_match_rate": round(self.answer_exact_match_rate, 4),
146
+ "sympy_verification_rate": round(self.sympy_verification_rate, 4),
147
+ }
148
+
149
+
150
+ @dataclass
151
+ class PipelineEvalSummary:
152
+ """Complete End-to-End Evaluation Report."""
153
+ ocr: OCRMetrics = field(default_factory=OCRMetrics)
154
+ parser: ParserMetrics = field(default_factory=ParserMetrics)
155
+ solver: SolverMetrics = field(default_factory=SolverMetrics)
156
+ e2e_success_rate: float = 0.0
157
+ avg_latency_ms: float = 0.0
158
+ total_samples: int = 0
159
+
160
+ def to_dict(self) -> Dict[str, Any]:
161
+ return {
162
+ "total_samples": self.total_samples,
163
+ "e2e_success_rate": round(self.e2e_success_rate, 4),
164
+ "avg_latency_ms": round(self.avg_latency_ms, 2),
165
+ "ocr_metrics": self.ocr.to_dict(),
166
+ "parser_metrics": self.parser.to_dict(),
167
+ "solver_metrics": self.solver.to_dict(),
168
+ }
eval/runner.py ADDED
@@ -0,0 +1,160 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Pipeline Evaluation Runner.
3
+
4
+ Runs evaluation suites over benchmark datasets and computes structured performance metrics.
5
+ """
6
+
7
+ from __future__ import annotations
8
+
9
+ import asyncio
10
+ import logging
11
+ import time
12
+ from typing import Any, Dict, List, Optional
13
+
14
+ from eval.benchmark import BenchmarkDataset, BenchmarkSample
15
+ from eval.metrics import (
16
+ OCRMetrics,
17
+ ParserMetrics,
18
+ PipelineEvalSummary,
19
+ SolverMetrics,
20
+ compute_cer,
21
+ compute_wer,
22
+ )
23
+ from solver.dsl_parser import DSLParser
24
+ from solver.engine import GeometryEngine
25
+ from solver.validator import GeometryStatus, GeometryValidator
26
+
27
+ logger = logging.getLogger(__name__)
28
+
29
+
30
+ class EvalRunner:
31
+ """Runs pipeline evaluation over benchmark datasets."""
32
+
33
+ def __init__(
34
+ self,
35
+ dsl_parser: Optional[DSLParser] = None,
36
+ geometry_engine: Optional[GeometryEngine] = None,
37
+ geometry_validator: Optional[GeometryValidator] = None,
38
+ ):
39
+ self.dsl_parser = dsl_parser or DSLParser()
40
+ self.geometry_engine = geometry_engine or GeometryEngine()
41
+ self.geometry_validator = geometry_validator or GeometryValidator()
42
+
43
+ def evaluate_dsl_deterministic(self, dataset: BenchmarkDataset) -> ParserMetrics:
44
+ """
45
+ Evaluates DSL parsing, solving, and geometric invariant validation
46
+ deterministically without LLM calls.
47
+ """
48
+ total = len(dataset)
49
+ if total == 0:
50
+ return ParserMetrics()
51
+
52
+ valid_dsl_count = 0
53
+ solvable_count = 0
54
+ validated_count = 0
55
+ degraded_count = 0
56
+
57
+ for sample in dataset:
58
+ dsl = sample.expected_dsl or ""
59
+ if not dsl:
60
+ continue
61
+
62
+ try:
63
+ points, constraints, is_3d = self.dsl_parser.parse(dsl)
64
+ valid_dsl_count += 1
65
+
66
+ engine_res = self.geometry_engine.solve(points, constraints, is_3d)
67
+ if engine_res and engine_res.get("coordinates"):
68
+ solvable_count += 1
69
+ val_res = self.geometry_validator.validate(engine_res, constraints, is_3d)
70
+ if val_res.is_valid:
71
+ validated_count += 1
72
+ else:
73
+ degraded_count += 1
74
+ except Exception as e:
75
+ logger.debug(f"[EvalRunner] Sample {sample.id} evaluation error: {e}")
76
+
77
+ return ParserMetrics(
78
+ total_samples=total,
79
+ json_valid_rate=1.0,
80
+ dsl_valid_rate=valid_dsl_count / total,
81
+ solvability_rate=solvable_count / total,
82
+ validation_pass_rate=validated_count / total,
83
+ degradation_rate=degraded_count / total,
84
+ )
85
+
86
+ async def evaluate_full_pipeline(
87
+ self,
88
+ dataset: BenchmarkDataset,
89
+ orchestrator: Any = None,
90
+ ) -> PipelineEvalSummary:
91
+ """
92
+ Executes end-to-end evaluation using Orchestrator across benchmark samples.
93
+ """
94
+ from agents.orchestrator import Orchestrator
95
+
96
+ orch = orchestrator or Orchestrator()
97
+ total = len(dataset)
98
+ if total == 0:
99
+ return PipelineEvalSummary()
100
+
101
+ e2e_successes = 0
102
+ total_latency_ms = 0.0
103
+ parser_metrics = ParserMetrics(total_samples=total)
104
+ solver_metrics = SolverMetrics(total_samples=total)
105
+ ocr_metrics = OCRMetrics(total_samples=total)
106
+
107
+ valid_dsl_count = 0
108
+ solvable_count = 0
109
+ validated_count = 0
110
+ degraded_count = 0
111
+ correct_answer_count = 0
112
+
113
+ for sample in dataset:
114
+ t0 = time.time()
115
+ try:
116
+ result = await orch.run(
117
+ text=sample.problem_text,
118
+ image_url=sample.image_url,
119
+ generate_video=False,
120
+ )
121
+ latency = (time.time() - t0) * 1000
122
+ total_latency_ms += latency
123
+
124
+ if result.get("status") == "success":
125
+ e2e_successes += 1
126
+
127
+ # Check geometry status
128
+ geo_status = result.get("geometry_status")
129
+ if result.get("geometry_dsl"):
130
+ valid_dsl_count += 1
131
+ if result.get("coordinates"):
132
+ solvable_count += 1
133
+ if geo_status == GeometryStatus.VALID.value:
134
+ validated_count += 1
135
+ elif geo_status == GeometryStatus.DEGRADED.value:
136
+ degraded_count += 1
137
+
138
+ # Check answer if expected_answer is present
139
+ if sample.expected_answer:
140
+ actual_ans = str((result.get("solution") or {}).get("answer", ""))
141
+ if sample.expected_answer.strip() in actual_ans or actual_ans.strip() in sample.expected_answer:
142
+ correct_answer_count += 1
143
+
144
+ except Exception as e:
145
+ logger.error(f"[EvalRunner] Full pipeline run failed on sample {sample.id}: {e}")
146
+
147
+ parser_metrics.dsl_valid_rate = valid_dsl_count / total
148
+ parser_metrics.solvability_rate = solvable_count / total
149
+ parser_metrics.validation_pass_rate = validated_count / total
150
+ parser_metrics.degradation_rate = degraded_count / total
151
+ solver_metrics.answer_exact_match_rate = correct_answer_count / max(total, 1)
152
+
153
+ return PipelineEvalSummary(
154
+ ocr=ocr_metrics,
155
+ parser=parser_metrics,
156
+ solver=solver_metrics,
157
+ e2e_success_rate=e2e_successes / total,
158
+ avg_latency_ms=total_latency_ms / max(total, 1),
159
+ total_samples=total,
160
+ )