ktsn-ud commited on
Commit
8779d33
·
1 Parent(s): dafa8fa

codex: 更新点の洗い出し

Browse files
docs/project_schema_change_points.md ADDED
@@ -0,0 +1,41 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Project Schema / Data Source Migration – Change Points
2
+
3
+ ## Background
4
+ - `schemas/projects.py` now defines DB-backed models whose core fields are `description` (企画内容), `prSummary` (PRコメント), `prDetail` (PRコメント詳細) together with `notes`, `images`, and `urls`.
5
+ - Batch scripts under `scripts/` must keep generating search assets from the database export (`projects.json`) instead of the legacy CSV.
6
+ - The API layer (`app/main.py`) serves the DB-derived models directly, while the search engine (`app/search/engine.py`) consumes the statically generated artefacts.
7
+
8
+ ## Config (`config/files.json`)
9
+ - `config/files.json:11-12` still references `projects.original_csv`; this key can be retired once the CSV path is unused.
10
+ - Add an explicit output mapping (e.g. `projects.output_json`) so scripts resolve the generated JSON path without relying on hard-coded fallbacks. Update all callers to reference the new key.
11
+
12
+ ## Batch Scripts (`scripts/`)
13
+ ### `scripts/2_create_projects_data.py`
14
+ - Lines `20-32` call `ProjectsRepository.list_projects()` and then pass the resulting Pydantic models straight into `json_dumps`; convert them to serialisable dictionaries via `model_dump(mode="json")` (or equivalent) before writing.
15
+ - Ensure `ProjectsRepository` and `app/repositories/query.sql` surface `prSummary`, `prDetail`, and `notes` so the exported JSON aligns with `schemas.projects.Project`.
16
+
17
+ ### `scripts/3_build_synonyms_from_sudachi.py`
18
+ - The loader at `scripts/3_build_synonyms_from_sudachi.py:65-112` instantiates `Project` objects and iterates `target_fields`. Verify that the regenerated `projects.json` exposes `name`, `circleName`, `circleNameKana`, `description`, `prSummary`, and `prDetail`; missing keys will degrade vocabulary extraction for synonyms.
19
+
20
+ ### `scripts/4_prepare_bm25f_meta.py`
21
+ - The token loop at `scripts/4_prepare_bm25f_meta.py:52-104` depends on the same field names. Confirm that `projects.json` no longer contains legacy keys such as `organization`/`title`, otherwise IDF statistics will be incorrect.
22
+
23
+ ### `scripts/5_prepare_tf_token.py`
24
+ - Update the `tf_token.Fields` construction at `scripts/5_prepare_tf_token.py:108-115` to use `name`, `circleName`, `circleNameKana`, `description`, `prSummary`, and `prDetail`. This keeps the TF schema consistent with the updated models.
25
+ - Continue serialising with `model_dump()` at `scripts/5_prepare_tf_token.py:117-127` once the field names match.
26
+
27
+ ### `scripts/7_prepare_circle_names.py`
28
+ - The mapper at `scripts/7_prepare_circle_names.py:68-74` still relies on `Project.organization` / `Project.reading`. Replace these with `circleName` / `circleNameKana` so the generated circle name and substring assets remain accurate.
29
+
30
+ ## API Layer (`app/main.py`)
31
+ - `app/main.py:94-105` currently dumps `ProjectSummary` instances directly. Convert to `model_dump()` (or `jsonable_encoder`) before gzipping to avoid serialisation errors.
32
+ - `ProjectSummary.image` should pick the `Image` entry whose `order == 0`. If multiple images share `order == 0`, use the *last* one encountered to mirror repository behaviour.
33
+ - Ensure the summary payload exposes `notes` (renamed from `note`) and the PR fields `prSummary` / `prDetail` as returned by the repository.
34
+
35
+ ## Search Engine (`app/search/engine.py`)
36
+ - In `SearchEngine.initialize` (`app/search/engine.py:164-175`) we populate kana norms into `self.circleNameKana_norms`, but the boosting logic reads `self.reading_norms` (`app/search/engine.py:545`). Assign the kana norms into `self.reading_norms` so substring boosts still consider kana.
37
+ - Debug responses at `app/search/engine.py:643-645` reference `organization` / `title`; switch these to `circleName` / `name` so diagnostics stay meaningful with the new schema.
38
+ - Rebuild all search assets (`projects.json`, `tf_token.json`, `bm25_meta.json`, `substring_index.json`) after applying the script updates above.
39
+
40
+ ## Open Questions / Follow-ups
41
+ 1. None – requirements above clarify the PR field names, notes rename, image selection rule, and config key addition.