| # Imageboard Grabber - Further Development & Roadmap |
|
|
| This document outlines a roadmap and future development plan for the **Imageboard Grabber** application on Hugging Face Spaces. |
|
|
| --- |
|
|
| ## 🚀 1. Feature Enhancements |
|
|
| ### 1.1 Support for Authenticated APIs (API Keys & Username) |
|
|
| - **Goal:** Allow users to safely search and download from private or rate-limited sites (like Danbooru premium tiers, e621 personal accounts). |
| - **Implementation:** |
| - Add credential fields (API Key / Password Hash, Username) to the **Add Site** dialog. |
| - Securely transmit credentials to the server using HTTP headers or request bodies. |
| - Store credentials locally in the browser's `localStorage` or session-only cookies so they are never stored globally on the server. |
|
|
| ### 1.2 Persistent Site Configuration |
|
|
| - **Goal:** Avoid losing newly added schemas when refreshing or clearing session storage. |
| - **Implementation:** |
| - Sync the list of added custom sites with IndexedDB or `localStorage`. |
| - Provide an **Export/Import Sites** button to allow users to backup their custom configurations as JSON. |
|
|
| ### 1.3 Advanced Query Builder |
|
|
| - **Goal:** Help non-technical users build complex booru search queries. |
| - **Implementation:** |
| - Add UI controls for searching by rating (Safe, Questionable, Explicit), score range, upload date, or file extension. |
| - Autocomplete / tag suggestion box by querying the active site's tag API endpoint (e.g., `/tags.json?search[name_matches]=*`). |
|
|
| ### 1.4 Enhanced Lightbox & Media Viewer |
|
|
| - **Goal:** Support video playback and metadata exploration. |
| - **Implementation:** |
| - Auto-detect video extensions (`.webm`, `.mp4`) in the Lightbox and render a native HTML `<video>` player with volume and speed controls. |
| - Show side-panel info: Artist, Character, Copyright, and Meta tags. Click a tag to search. |
|
|
| --- |
|
|
| ## 🛠️ 2. Architectural & Performance Improvements |
|
|
| ### 2.1 Concurrent / Streamed Server Downloads |
|
|
| - **Goal:** Speed up ZIP compilation and handle massive file downloads without causing OOM (Out Of Memory) crashes. |
| - **Implementation:** |
| - Implement streaming ZIP using `fflate` on the server so bytes are sent back to the client immediately as they download, rather than buffering entire gigabytes in server memory. |
| - Limit concurrent download workers per request to avoid IP bans from image hosters. |
|
|
| ### 2.2 Client-Side Image Caching |
|
|
| - **Goal:** Reduce bandwidth usage and accelerate browsing. |
| - **Implementation:** |
| - Register a Service Worker to cache thumbnails and UI assets. |
| - Use `Cache-Control` headers for downloaded image URLs. |
|
|
| --- |
|
|
| ## 🧪 3. Quality Assurance & Automated Testing |
|
|
| ### 3.1 End-to-End Tests with Playwright |
|
|
| - **Goal:** Ensure code modifications do not break key features like site additions or tag searches. |
| - **Implementation:** |
| - Add E2E test suites inside a `tests/` directory. |
| - Mock API responses from e621/Danbooru to test search, grid rendering, and selection UI without hitting actual boorus. |
|
|
| ### 3.2 Unit and API Integration Tests |
|
|
| - **Goal:** Validate parsing logic, schema validation, and URL building helpers. |
| - **Implementation:** |
| - Write Vitest suites to verify mapping functions in `src/lib/grabber/sites.ts` and zip streaming endpoints. |
|
|
| --- |
|
|
| ## 🌐 4. Deployment & Infrastructure Optimization |
|
|
| ### 4.1 Hugging Face Storage Integration |
|
|
| - **Goal:** Retain user session data or cache assets on persistent storage. |
| - **Implementation:** |
| - Connect a Hugging Face Dataset or persistent Space storage to backup logs or temporary configuration files if requested. |
|
|
