File size: 3,567 Bytes
e35f67f 61b6204 e35f67f 61b6204 e35f67f 61b6204 e35f67f 61b6204 e35f67f 61b6204 e35f67f 61b6204 e35f67f 61b6204 e35f67f 61b6204 e35f67f 61b6204 e35f67f | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | # Imageboard Grabber - Further Development & Roadmap
This document outlines a roadmap and future development plan for the **Imageboard Grabber** application on Hugging Face Spaces.
---
## 🚀 1. Feature Enhancements
### 1.1 Support for Authenticated APIs (API Keys & Username)
- **Goal:** Allow users to safely search and download from private or rate-limited sites (like Danbooru premium tiers, e621 personal accounts).
- **Implementation:**
- Add credential fields (API Key / Password Hash, Username) to the **Add Site** dialog.
- Securely transmit credentials to the server using HTTP headers or request bodies.
- Store credentials locally in the browser's `localStorage` or session-only cookies so they are never stored globally on the server.
### 1.2 Persistent Site Configuration
- **Goal:** Avoid losing newly added schemas when refreshing or clearing session storage.
- **Implementation:**
- Sync the list of added custom sites with IndexedDB or `localStorage`.
- Provide an **Export/Import Sites** button to allow users to backup their custom configurations as JSON.
### 1.3 Advanced Query Builder
- **Goal:** Help non-technical users build complex booru search queries.
- **Implementation:**
- Add UI controls for searching by rating (Safe, Questionable, Explicit), score range, upload date, or file extension.
- Autocomplete / tag suggestion box by querying the active site's tag API endpoint (e.g., `/tags.json?search[name_matches]=*`).
### 1.4 Enhanced Lightbox & Media Viewer
- **Goal:** Support video playback and metadata exploration.
- **Implementation:**
- Auto-detect video extensions (`.webm`, `.mp4`) in the Lightbox and render a native HTML `<video>` player with volume and speed controls.
- Show side-panel info: Artist, Character, Copyright, and Meta tags. Click a tag to search.
---
## 🛠️ 2. Architectural & Performance Improvements
### 2.1 Concurrent / Streamed Server Downloads
- **Goal:** Speed up ZIP compilation and handle massive file downloads without causing OOM (Out Of Memory) crashes.
- **Implementation:**
- Implement streaming ZIP using `fflate` on the server so bytes are sent back to the client immediately as they download, rather than buffering entire gigabytes in server memory.
- Limit concurrent download workers per request to avoid IP bans from image hosters.
### 2.2 Client-Side Image Caching
- **Goal:** Reduce bandwidth usage and accelerate browsing.
- **Implementation:**
- Register a Service Worker to cache thumbnails and UI assets.
- Use `Cache-Control` headers for downloaded image URLs.
---
## 🧪 3. Quality Assurance & Automated Testing
### 3.1 End-to-End Tests with Playwright
- **Goal:** Ensure code modifications do not break key features like site additions or tag searches.
- **Implementation:**
- Add E2E test suites inside a `tests/` directory.
- Mock API responses from e621/Danbooru to test search, grid rendering, and selection UI without hitting actual boorus.
### 3.2 Unit and API Integration Tests
- **Goal:** Validate parsing logic, schema validation, and URL building helpers.
- **Implementation:**
- Write Vitest suites to verify mapping functions in `src/lib/grabber/sites.ts` and zip streaming endpoints.
---
## 🌐 4. Deployment & Infrastructure Optimization
### 4.1 Hugging Face Storage Integration
- **Goal:** Retain user session data or cache assets on persistent storage.
- **Implementation:**
- Connect a Hugging Face Dataset or persistent Space storage to backup logs or temporary configuration files if requested.
|