Takosaga commited on
Commit
d20505f
Β·
1 Parent(s): 0a6ae97

docs: add CSV export design spec

Browse files
docs/superpowers/specs/2026-06-13-csv-export-design.md ADDED
@@ -0,0 +1,197 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # CSV Export Design
2
+
3
+ ## Overview
4
+
5
+ Replace the stub `.csv` and `.apkg` buttons with a real CSV export that produces a downloadable `.zip` archive containing a CSV file and all associated media files (audio, images). Remove the "Sync to Anki" button from the UI. Export buttons become clickable after Phase 2 (translation + media generation) completes.
6
+
7
+ ## Architecture
8
+
9
+ ```
10
+ User clicks "Export CSV" (after Phase 2)
11
+ β†’ app.py: _handle_export_csv() click handler
12
+ β†’ csv_export.py: export_csv_zip()
13
+ β”œβ”€β”€ Creates folder: {scenario_slug}_{CEFR}_{LANG_ABBREV}/
14
+ β”œβ”€β”€ Writes CSV with columns: scenario, cefr_level, target_language, english_text, translated_text, audio_filename, image_filename
15
+ β”œβ”€β”€ Copies media files into the folder
16
+ └── Zips the folder β†’ returns .zip path
17
+ β†’ Gradio gr.File component updates β†’ user downloads zip
18
+ ```
19
+
20
+ ## Components
21
+
22
+ ### 1. `export/csv_export.py` β€” Real Implementation
23
+
24
+ A single public function:
25
+
26
+ ```python
27
+ def export_csv_zip(
28
+ cards: list[dict],
29
+ scenario: str,
30
+ cefr_level: CEFRLevel,
31
+ target_language: str,
32
+ ) -> str:
33
+ ```
34
+
35
+ **Responsibilities:**
36
+
37
+ - Create an output directory under `{models_dir}/output/export/` (configurable via settings.yaml, default `.local/models/output/export/`)
38
+ - Generate folder name by sanitizing the scenario string (lowercase, spaces β†’ underscores, remove special characters), then append CEFR level and language abbreviation: `ordering_coffee_A2_LV`
39
+ - Write CSV using Python's `csv` module with RFC 4180 double-quote quoting
40
+ - Copy media files into subfolders (`audio/`, `images/`) within the export folder
41
+ - Zip the entire folder and return the `.zip` path
42
+
43
+ **Language abbreviation mapping (ISO 639-1):**
44
+
45
+ | Language | Abbreviation |
46
+ |---|---|
47
+ | Latvian | LV |
48
+ | Spanish | ES |
49
+ | French | FR |
50
+ | German | DE |
51
+ | Polish | PL |
52
+ | Italian | IT |
53
+ | Portuguese | PT |
54
+ | Finnish | FI |
55
+
56
+ **CSV columns:** `scenario,cefr_level,target_language,english_text,translated_text,audio_filename,image_filename`
57
+
58
+ - `audio_filename` / `image_filename`: relative path within the export folder (e.g., `audio/audio_0.wav`) or empty string if no media
59
+ - Fields containing commas, quotes, or newlines are automatically double-quote escaped by the `csv` module
60
+
61
+ **Sanitization rules for folder names:**
62
+
63
+ ```python
64
+ def _sanitize_folder_name(scenario: str) -> str:
65
+ """Convert scenario text to a filesystem-safe folder name slug."""
66
+ slug = scenario.strip().lower()
67
+ slug = re.sub(r'[^a-z0-9\s_]', '', slug) # remove special chars
68
+ slug = re.sub(r'\s+', '_', slug) # spaces β†’ underscores
69
+ slug = re.sub(r'_+', '_', slug) # collapse multiple underscores
70
+ return slug.strip('_')
71
+ ```
72
+
73
+ ### 2. UI Changes in `frontend/ui/widgets.py`
74
+
75
+ **Button row changes:**
76
+
77
+ Before (3 buttons):
78
+ ```
79
+ [.apkg] [.csv] [Sync to Anki]
80
+ ```
81
+
82
+ After (2 buttons + file download):
83
+ ```
84
+ [.apkg] [.csv] [gr.File for zip download β€” hidden until export]
85
+ ```
86
+
87
+ - Remove "Sync to Anki" button entirely
88
+ - `.csv` button: enabled after Phase 2, triggers `export_csv_zip()`
89
+ - `.apkg` button: enabled after Phase 2, stub handler shows "Coming soon" message
90
+ - Add a `gr.File` component (hidden by default) that displays the downloaded zip file
91
+
92
+ **Button enabling logic:**
93
+
94
+ Both export buttons are disabled during idle/Phase 1 state. They become interactive in `_enable_phase2()`:
95
+
96
+ ```python
97
+ def _enable_phase2() -> tuple:
98
+ return (
99
+ gr.Checkbox(interactive=True, value=True),
100
+ gr.Checkbox(interactive=True, value=True),
101
+ gr.Button(interactive=True), # generate_cards_btn
102
+ gr.Dropdown(interactive=True), # voice_dropdown
103
+ "", # phase_css
104
+ gr.Button(interactive=True), # NEW: export_csv_btn
105
+ gr.Button(interactive=True), # NEW: export_apkg_btn
106
+ )
107
+ ```
108
+
109
+ **Event wiring:**
110
+
111
+ ```python
112
+ # Export CSV handler (generator β€” yields progress + file path)
113
+ def _handle_export_csv():
114
+ if not cards_exist():
115
+ yield "No cards to export.", None
116
+ return
117
+ zip_path = csv_export.export_csv_zip(...)
118
+ yield "Export complete!", zip_path
119
+
120
+ # Export APKG stub handler
121
+ def _handle_export_apkg_stub():
122
+ yield "APKG export coming soon.", ""
123
+ ```
124
+
125
+ ### 3. `app.py` β€” Click Handlers
126
+
127
+ Two new generator functions in `app.py`:
128
+
129
+ ```python
130
+ def _handle_export_csv(scenario, cefr_level, target_language):
131
+ """Export current cards as a zipped CSV folder."""
132
+ from frontend.ui.cards import generate_progress_html
133
+ from export.csv_export import export_csv_zip
134
+ # ... validate cards exist, call export_csv_zip, return zip path
135
+
136
+ def _handle_export_apkg_stub():
137
+ """Stub: APKG export not yet implemented."""
138
+ from frontend.ui.cards import generate_progress_html
139
+ yield generate_progress_html(0, "APKG export coming soon."), ""
140
+ ```
141
+
142
+ ### 4. Tests β€” `tests/csv_export_test.py`
143
+
144
+ | Test | What it verifies |
145
+ |---|---|
146
+ | `test_folder_name_generation` | Scenario/CEFR/lang β†’ correct folder name (e.g., `"ordering coffee"` + A2 + Latvian β†’ `ordering_coffee_A2_LV`) |
147
+ | `test_csv_content_columns` | CSV has the 7 expected columns in order |
148
+ | `test_csv_content_row_count` | Row count matches number of cards |
149
+ | `test_csv_quoting` | Fields with commas/accents are properly double-quote escaped |
150
+ | `test_media_file_copying` | Audio/image files are copied into `audio/` and `images/` subfolders |
151
+ | `test_zip_creation` | Zip is created, extractable, contains expected structure |
152
+ | `test_language_abbreviation_mapping` | All 8 languages map to correct ISO 639-1 codes |
153
+ | `test_sanitize_folder_name` | Special chars removed, spaces β†’ underscores, no leading/trailing `_` |
154
+
155
+ ## Data Flow
156
+
157
+ ```
158
+ Phase 2 completes β†’ cards list populated with text, translation, audio_path, image_path
159
+ ↓
160
+ User clicks "Export CSV"
161
+ ↓
162
+ _handle_export_csv() reads current card data from app state
163
+ ↓
164
+ export_csv_zip(cards, scenario, cefr_level, target_language)
165
+ ↓
166
+ Creates: .local/models/output/export/{folder}/
167
+ β”œβ”€β”€ {folder}.csv (7 columns, one row per card)
168
+ β”œβ”€β”€ audio/audio_0.wav (copied from TTS output)
169
+ β”œβ”€β”€ images/image_0.png (copied from image generation)
170
+ └── {folder}.zip (zipped archive of the above)
171
+ ↓
172
+ Gradio gr.File component receives zip path β†’ triggers browser download
173
+ ```
174
+
175
+ ## Error Handling
176
+
177
+ - **No cards to export:** Show warning message, no file generated
178
+ - **Missing media files:** Skip missing files silently (don't fail the export). CSV entries for missing media remain empty strings.
179
+ - **Zip creation failure:** Catch `shutil.make_archive` exceptions, show error message in progress HTML
180
+ - **IO errors (disk full, permissions):** Catch and display user-friendly error
181
+
182
+ ## Files Changed
183
+
184
+ | File | Action |
185
+ |---|---|
186
+ | `export/csv_export.py` | Implement `export_csv_zip()` and helpers |
187
+ | `frontend/ui/widgets.py` | Remove Sync button, add export buttons + gr.File, update `_enable_phase2()` outputs |
188
+ | `app.py` | Add `_handle_export_csv()` and `_handle_export_apkg_stub()` generator functions |
189
+ | `tests/csv_export_test.py` | New test file (8 tests) |
190
+ | `export/anki_tunnel.py` | Keep as stub (unused, not deleted) |
191
+
192
+ ## Out of Scope
193
+
194
+ - Real `.apkg` implementation (button is stub only)
195
+ - CSV import into Anki (user handles manually via Anki's Import feature)
196
+ - Configurable output directory (uses default `models_dir/output/export/`)
197
+ - Incremental export (export always captures current card state)