mohsin-devs commited on
Commit
044baa8
·
verified ·
1 Parent(s): 1677e13

Delete PRODUCTION_TESTING_RUNBOOK.md

Browse files
Files changed (1) hide show
  1. PRODUCTION_TESTING_RUNBOOK.md +0 -432
PRODUCTION_TESTING_RUNBOOK.md DELETED
@@ -1,432 +0,0 @@
1
- # 🚀 PRODUCTION READINESS TESTING GUIDE
2
-
3
- **Status**: Before deployment to HuggingFace Spaces
4
- **Time Required**: 3 hours
5
- **Tools Needed**: Browser, Python 3.7+, requests library
6
-
7
- ---
8
-
9
- ## 📋 WHAT'S CHANGED
10
-
11
- ### Added: Comprehensive Production Logging
12
- - Every API endpoint now logs:
13
- - Operation start/end
14
- - Success/failure status
15
- - Input parameters (paths, filenames)
16
- - Execution time
17
- - Exceptions with stack traces
18
-
19
- **Example log output**:
20
- ```
21
- [UPLOAD_FILE] START | user=default_user | file=document.pdf | folder=reports | size=1048576
22
- [UPLOAD_FILE] SUCCESS | user=default_user | file=document.pdf | folder=reports
23
- [RENAME] START | user=default_user | path=reports | new_name=archive_reports
24
- [RENAME] SUCCESS | user=default_user | old_path=reports | new_name=archive_reports
25
- ```
26
-
27
- ### Added: Automated Stress Test Script
28
- `stress_test.py` - Runs 5 comprehensive stress tests:
29
- 1. Bulk Upload (50 files)
30
- 2. Folder Rename with 30 files
31
- 3. Rapid Operations (upload→delete→rename)
32
- 4. Cache Behavior (TTL validation)
33
- 5. Error Handling (invalid operations)
34
-
35
- ---
36
-
37
- ## 🧪 TESTING EXECUTION PLAN
38
-
39
- ### PHASE 1: Setup (15 minutes)
40
-
41
- #### Step 1: Verify Logging
42
- ```bash
43
- # Check that logging is configured correctly
44
- cd c:\Users\mohat\OneDrive\Desktop\Doc
45
- python -c "from server.utils.logger import setup_logger; logger = setup_logger('test'); logger.info('Logging works')"
46
- ```
47
-
48
- Expected output: Should see log message in console or logs/ directory
49
-
50
- #### Step 2: Install Test Dependencies
51
- ```bash
52
- pip install requests
53
- ```
54
-
55
- ---
56
-
57
- ### PHASE 2: Manual Testing (45 minutes)
58
-
59
- #### Test A: Basic Rename (NEW FEATURE)
60
- **Goal**: Verify rename works end-to-end
61
-
62
- 1. Open: http://localhost:5000 (or HF Spaces URL)
63
- 2. Upload test file: `rename_test.txt`
64
- 3. Right-click file → "Rename" → `renamed_file.txt`
65
- 4. Verify: File appears with new name
66
- 5. Download to verify content intact
67
- 6. Check logs: Should see `[RENAME] SUCCESS`
68
-
69
- **Pass Criteria**:
70
- - ✓ File renamed successfully
71
- - ✓ UI updates immediately
72
- - ✓ Log shows [RENAME] SUCCESS
73
-
74
- #### Test B: Folder Operations
75
- **Goal**: Verify folders work with new rename
76
-
77
- 1. Create folder: `TestFolder`
78
- 2. Upload 5 files into it
79
- 3. Rename folder: `TestFolder` → `RenamedFolder`
80
- 4. Verify: All 5 files still accessible in new folder
81
- 5. Check logs: Should see [RENAME] SUCCESS with all files preserved
82
-
83
- **Pass Criteria**:
84
- - ✓ Folder renamed with contents
85
- - ✓ Files accessible after rename
86
- - ✓ No data loss
87
-
88
- #### Test C: Delete Operations
89
- 1. Upload file: `delete_test.txt`
90
- 2. Delete it
91
- 3. Verify: File removed from list
92
- 4. Check logs: Should see `[DELETE_FILE] SUCCESS`
93
-
94
- **Pass Criteria**:
95
- - ✓ File removed
96
- - ✓ Delete confirmation modal shown
97
- - ✓ Log shows [DELETE_FILE] SUCCESS
98
-
99
- #### Test D: Cache Behavior
100
- **Goal**: Verify 60-second cache works
101
-
102
- 1. Upload file: `cache_test.txt`
103
- 2. Open DevTools (F12) → Network tab
104
- 3. Reload page within 30 seconds
105
- 4. Observe: List call should be instant (using cache)
106
- 5. Wait 65 seconds
107
- 6. Reload page
108
- 7. Observe: Slightly slower (fresh from server)
109
-
110
- **Pass Criteria**:
111
- - ✓ First reload is instant (<100ms)
112
- - ✓ After 60s, fresh data fetched
113
- - ✓ No errors in console
114
-
115
- ---
116
-
117
- ### PHASE 3: Automated Stress Testing (60 minutes)
118
-
119
- #### Step 1: Start Server
120
- ```bash
121
- # Terminal 1: Start Flask backend
122
- cd c:\Users\mohat\OneDrive\Desktop\Doc
123
- python -m server.app
124
- # Should see: Running on http://127.0.0.1:5000
125
- ```
126
-
127
- #### Step 2: Run Stress Tests
128
- ```bash
129
- # Terminal 2: Run stress test suite (LOCAL mode)
130
- cd c:\Users\mohat\OneDrive\Desktop\Doc
131
- python stress_test.py http://localhost:5000
132
- ```
133
-
134
- **Expected Output**:
135
- ```
136
- ============================================================
137
- DOCVAULT STRESS TEST SUITE
138
- ============================================================
139
- Base URL: http://localhost:5000
140
- User: stress_test_user
141
- Timeout: 30s
142
-
143
- Checking server connectivity...
144
- ✓ Server is healthy
145
-
146
- 🔥 TEST 1: BULK UPLOAD (50 files)
147
- ----------------------------------------
148
- ✓ bulk_upload_all_succeed
149
- ✓ bulk_upload_speed
150
- Uploaded: 50/50
151
- Total time: 25.34s
152
- Avg per file: 0.51s
153
-
154
- 🔥 TEST 2: FOLDER RENAME STRESS (30 files in folder)
155
- ----------------------------------------
156
- ✓ folder_rename
157
- ✓ folder_rename_speed
158
- ✓ folder_rename_data_preserved
159
- Renaming folder...
160
- Time: 2.15s
161
-
162
- 🔥 TEST 3: RAPID OPERATIONS (Race Conditions)
163
- ----------------------------------------
164
- ✓ rapid_ops_20_cycles
165
- ✓ rapid_seq_operations
166
-
167
- 🧪 TEST 4: CACHE VALIDATION (60s TTL)
168
- ----------------------------------------
169
- ✓ cache_first_list
170
- ✓ cache_second_list
171
-
172
- 🧪 TEST 5: ERROR HANDLING
173
- ----------------------------------------
174
- ✓ error_handling_delete_nonexistent
175
- ✓ error_handling_rename_nonexistent
176
- ✓ error_handling_list_nonexistent
177
-
178
- ============================================================
179
- STRESS TEST RESULTS
180
- ============================================================
181
- ✓ Passed: 16
182
- ✗ Failed: 0
183
- Total: 16
184
-
185
- PERFORMANCE METRICS:
186
- bulk_upload: avg=0.51s, max=1.23s, count=50
187
- folder_rename: avg=2.15s, max=2.15s, count=1
188
- list_fresh: avg=0.05s, max=0.05s, count=1
189
- list_cached: avg=0.02s, max=0.02s, count=1
190
-
191
- ============================================================
192
- ```
193
-
194
- #### Step 3: Check Logs
195
- ```bash
196
- # Review logs for any warnings or errors
197
- tail -f logs/docvault.log | grep -E "\[UPLOAD\]|\[DELETE\]|\[RENAME\]|\[ERROR\]"
198
- ```
199
-
200
- **Look For**:
201
- - ✓ All operations have START and SUCCESS
202
- - ✓ No EXCEPTION entries
203
- - ⚠️ Any FAIL entries (investigate)
204
-
205
- #### Step 4: Interpret Results
206
-
207
- **Perfect Scenario**:
208
- - All 16 tests pass
209
- - No race conditions detected
210
- - Performance metrics reasonable
211
- - No errors in logs
212
-
213
- **Warning Scenarios**:
214
- - Bulk upload > 30 seconds: May be slow
215
- - Folder rename > 5 seconds: Performance issue
216
- - Failed rapid operations: Race condition risk
217
- - Exceptions in logs: Critical issue
218
-
219
- **Failure Scenario**:
220
- - Tests fail: Do NOT deploy
221
- - Race conditions detected: Do NOT deploy
222
- - Exceptions: Do NOT deploy
223
-
224
- ---
225
-
226
- ## 📊 TEST RESULTS TEMPLATE
227
-
228
- Record results here:
229
-
230
- ```
231
- TEST EXECUTION REPORT
232
- ═════════════════════════════════════════════════════════
233
-
234
- Date: ________________
235
- Environment: [ ] LOCAL [ ] HF_SPACES
236
- Tester: ________________
237
-
238
- MANUAL TESTS:
239
- ─────────────────────────────────────────────────────────
240
- Test A - Rename Feature: [ ] PASS [ ] FAIL
241
- Test B - Folder Operations: [ ] PASS [ ] FAIL
242
- Test C - Delete Operations: [ ] PASS [ ] FAIL
243
- Test D - Cache Behavior: [ ] PASS [ ] FAIL
244
-
245
- AUTOMATED STRESS TESTS:
246
- ─────────────────────────────────────────────────────────
247
- Total Tests: ____
248
- Passed: ____
249
- Failed: ____
250
- Race Conditions: [ ] YES [ ] NO
251
-
252
- PERFORMANCE METRICS (from stress test):
253
- ─────────────────────────────────────────────────────────
254
- Avg Upload Time: ______ seconds
255
- Max Rename Time: ______ seconds
256
- Cache Hit Time: ______ seconds
257
- List Operation Time: ______ seconds
258
-
259
- ISSUES FOUND:
260
- ─────────────────────────────────────────────────────────
261
- [ ] None
262
- [ ] Minor (document & monitor)
263
- [ ] Major (fix before deploy)
264
- [ ] Critical (DO NOT DEPLOY)
265
-
266
- Issues:
267
- 1. ________________________________________
268
- 2. ________________________________________
269
- 3. ________________________________________
270
-
271
- LOGS STATUS:
272
- ─────────────────────────────────────────────────────────
273
- [ ] All operations logged correctly
274
- [ ] No exceptions in logs
275
- [ ] Errors are meaningful
276
- [ ] Performance data captured
277
-
278
- FINAL VERDICT:
279
- ─────────────────────────────────────────────────────────
280
- [ ] READY FOR PRODUCTION
281
- [ ] NEEDS MORE TESTING
282
- [ ] DO NOT DEPLOY
283
-
284
- Comments:
285
- __________________________________________________________________
286
- __________________________________________________________________
287
-
288
- Signed: _________________________ Date: ________________
289
- ```
290
-
291
- ---
292
-
293
- ## 🔥 CRITICAL ISSUES TO WATCH FOR
294
-
295
- ### Red Flag #1: Upload Failures
296
- ```
297
- [UPLOAD_FILE] FAIL | reason=permission_denied
298
- ```
299
- → Check HF token permissions
300
-
301
- ### Red Flag #2: Rename Failures
302
- ```
303
- [RENAME] FAIL | error=atomic_commit_failed
304
- ```
305
- → Check HF API availability
306
-
307
- ### Red Flag #3: Race Conditions
308
- ```
309
- rapid_ops test: 3/20 operations failed
310
- ```
311
- → Investigate cache invalidation timing
312
-
313
- ### Red Flag #4: Slow Performance
314
- ```
315
- bulk_upload total time: 180s (should be <60s)
316
- ```
317
- → Check network/HF API performance
318
-
319
- ### Red Flag #5: Memory Leaks
320
- ```
321
- Processing time increases over 150+ operations
322
- ```
323
- → Check for unclosed file handles
324
-
325
- ---
326
-
327
- ## 🎯 SUCCESS CRITERIA FOR PRODUCTION DEPLOYMENT
328
-
329
- **All Must Be True**:
330
- - ✅ Manual tests: 4/4 passing
331
- - ✅ Stress tests: 16/16 passing
332
- - ✅ No race conditions detected
333
- - ✅ No exceptions in logs
334
- - ✅ Performance acceptable (bulk upload <60s)
335
- - ✅ Cache working (list cached <100ms)
336
- - ✅ All logs readable and structured
337
- - ✅ Zero data loss observed
338
-
339
- **If Any Fails**:
340
- → DO NOT DEPLOY
341
- → Investigate & fix
342
- → Re-test
343
-
344
- ---
345
-
346
- ## 🚀 DEPLOYMENT STEPS (After Passing Tests)
347
-
348
- 1. **Backup Current HF Repo**
349
- ```bash
350
- # Create a snapshot of current state
351
- # Document in deployment log
352
- ```
353
-
354
- 2. **Deploy Code**
355
- - Push fixes to HF Space git repo
356
- - Or: Upload files directly to Space
357
-
358
- 3. **Restart Application**
359
- - HF Spaces: App auto-restarts on code change
360
- - Local: Restart Flask server
361
-
362
- 4. **Smoke Test**
363
- - Upload test file
364
- - Rename it
365
- - Delete it
366
- - Check logs are generated
367
-
368
- 5. **Monitor for 24 Hours**
369
- - Check logs regularly
370
- - Monitor error rates
371
- - Gather user feedback
372
-
373
- ---
374
-
375
- ## 📝 LOGGING REFERENCE
376
-
377
- ### Log Format
378
- ```
379
- [OPERATION] STATUS | user=X | details=Y | duration=Z
380
- ```
381
-
382
- ### Example Logs
383
- ```
384
- [UPLOAD_FILE] START | user=default_user | file=report.pdf | size=2097152
385
- [UPLOAD_FILE] SUCCESS | user=default_user | file=report.pdf
386
- [DELETE_FILE] START | user=default_user | path=report.pdf
387
- [DELETE_FILE] SUCCESS | user=default_user | path=report.pdf
388
- [RENAME] START | user=default_user | path=old_name | new_name=new_name
389
- [RENAME] SUCCESS | user=default_user | old_path=old_name | new_name=new_name
390
- [LIST] START | user=default_user | path=root
391
- [LIST] SUCCESS | user=default_user | path=root | files=42 | folders=5
392
- ```
393
-
394
- ### What Each Field Means
395
- - `[OPERATION]` - What's happening (UPLOAD, DELETE, RENAME, etc.)
396
- - `STATUS` - Result (START, SUCCESS, FAIL, EXCEPTION)
397
- - `user=X` - Which user (always check: should be isolated)
398
- - `details` - Operation context (files, paths, sizes)
399
- - `duration` - How long it took (helps spot performance issues)
400
-
401
- ---
402
-
403
- ## 📞 TROUBLESHOOTING
404
-
405
- ### Stress Test Won't Run
406
- ```bash
407
- # Error: No module named 'requests'
408
- pip install requests
409
-
410
- # Error: Connection refused
411
- # Make sure Flask app is running: python -m server.app
412
- ```
413
-
414
- ### Tests Fail With Timeouts
415
- - Check: Server is responding normally
416
- - Check: Network connectivity
417
- - Increase: TIMEOUT variable in stress_test.py
418
-
419
- ### Logs Not Appearing
420
- - Check: Logs directory exists
421
- - Check: config.py LOG_LEVEL = "INFO"
422
- - Check: Flask app has logging enabled
423
-
424
- ---
425
-
426
- **NEXT STEP**: Execute this entire procedure and record results in the template above.
427
-
428
- **THEN**: If all passes → **YOU'RE PRODUCTION READY** ✅
429
-
430
- ---
431
-
432
- *Ready to test? Let's ensure this app is truly production-grade.*