Hugging Face's logo

Spaces:
Nymbo
/
MiniSearch
Paused

App Files Community
MiniSearch / agents.md
Nymbo's picture
Nymbo
Sync with upstream felladrin/MiniSearch (2026-10-01)
ff7a361 verified
preview code
|
Raw History Blame Contribute Delete
13.5 kB

MiniSearch Agent Guidelines

This is your navigation hub. Start here, follow the links, and return when you need orientation.

Before You Start

New to this codebase? Read in this order:

  1. docs/quick-start.md - Get it running
  2. docs/overview.md - Understand the system
  3. docs/project-structure.md - Navigate the code

Making changes? Check:

  • docs/coding-conventions.md - Code style
  • docs/development-commands.md - Available commands
  • docs/pull-requests.md - How to submit

Repository Map

Getting Started

  • docs/quick-start.md - Installation, first run, verification
  • docs/overview.md - System architecture and data flow
  • docs/project-structure.md - Directory layout and component organization

Configuration & Setup

  • docs/configuration.md - Environment variables and settings reference
  • docs/configuration.md#tuning-the-search-engines - Replacing the SearXNG settings file at container start: engine selection, engines blocked from an IP, the TMPDIR requirement
  • docs/security.md - Access control, privacy, and security model

Core Functionality

  • docs/api.md - HTTP endpoint reference: parameters, responses, and error codes
  • docs/ai-integration.md - AI inference types (Wllama, OpenAI, AI Horde, Internal)
  • docs/ui-components.md - Component architecture and PubSub patterns
  • docs/search-history.md - History database schema and management
  • docs/conversation-memory.md - Token budgeting and rolling summaries
  • docs/reranking.md - Reranker subsystem and model lifecycle
  • docs/page-content.md - Reading result pages to ground AI answers
  • docs/glossary.md - Codebase-specific terms and domain concepts

Development

  • docs/development-commands.md - Docker, npm, and testing commands
  • docs/failure-injection.md - Graceful-degradation test matrix
  • docs/coding-conventions.md - Style guide and patterns
  • docs/pull-requests.md - PR process and merge philosophy
  • docs/core-technologies.md - Technology stack and dependencies
  • docs/design.md - UI/UX design principles

Agent Decision Tree 

Need to:
├── Add a feature?
│   ├── UI component → docs/ui-components.md
│   ├── AI integration → docs/ai-integration.md
│   ├── Search functionality → client/modules/search.ts
│   └── Settings option → docs/configuration.md
├── Fix a bug?
│   ├── UI issue → Check component + PubSub channels
│   ├── AI not working → docs/ai-integration.md + browser console
│   ├── Search failing → Check SearXNG + server hooks
│   └── Build error → docs/development-commands.md
├── Configure deployment?
│   ├── Environment variables → docs/configuration.md
│   ├── Access control → docs/security.md
│   └── Docker setup → docs/overview.md
└── Understand data flow?
    ├── Search flow → client/modules/search.ts
    ├── AI generation → client/modules/textGeneration.ts
    ├── State management → docs/ui-components.md
    └── History/Chat → docs/search-history.md + docs/conversation-memory.md

Key Files Reference

Entry Points

  • client/index.tsx - React app initialization
  • vite.config.ts - Vite dev server with hooks
  • Dockerfile - Single-stage container build

Business Logic Modules

  • client/modules/search.ts - Search orchestration and caching
  • client/modules/textGeneration.ts - AI response flow
  • client/modules/pubSub.ts - All PubSub channels
  • client/modules/settings.ts - Settings management
  • client/modules/history.ts - Search history database

Server-Side Modules

  • server/searchEndpointServerHook.ts - /search endpoints
  • server/pageContentEndpointServerHook.ts - /page-content endpoint
  • server/internalApiEndpointServerHook.ts - /inference proxy
  • server/webSearchService.ts - SearXNG integration
  • server/rerankerService.ts - Local result reranking

UI Components

  • client/components/DictationButton.tsx - Microphone control shared by the search field and the chat follow-up field
  • client/components/App/ - Application shell with error boundaries
  • client/components/Search/Form/ - Search input and form
  • client/components/Search/Results/ - Textual and graphical results display
  • client/components/Search/History/ - History drawer and button
  • client/components/AiResponse/ - AI response display and chat interface
  • client/components/Pages/Main/ - Main page layout
  • client/components/Pages/Main/Menu/ - Settings drawers (AI, Search, Interface, History, Voice, Actions)
  • client/components/Pages/AccessPage.tsx - Access key validation page
  • client/components/Analytics/SearchStats.tsx - Search analytics cards
  • client/components/Logs/ - Application logging modal
  • client/components/Settings/HistorySettings.tsx - History configuration UI

Client Modules

  • client/modules/pubSub.ts - All PubSub channels for state management
  • client/modules/search.ts - Search orchestration and IndexedDB caching
  • client/modules/textGeneration.ts - AI response generation and chat handling
  • client/modules/textGenerationWithWllama.ts - Browser-based inference
  • client/modules/textGenerationWithOpenAi.ts - OpenAI-compatible API inference
  • client/modules/textGenerationWithHorde.ts - AI Horde distributed inference
  • client/modules/textGenerationWithInternalApi.ts - Internal API proxy inference
  • client/modules/settings.ts - Default settings and inference type definitions
  • client/modules/history.ts - IndexedDB persistence for searches and chat
  • client/modules/wllama.ts - Wllama model configuration and management
  • client/modules/webGpu.ts - WebGPU availability detection ("gpu" in navigator)
  • client/modules/querySuggestions.ts - Search suggestion UI, stored in IndexedDB
  • client/modules/relatedSearchQuery.ts - Generates related search queries
  • client/modules/followUpQuestions.ts - Generates follow-up questions via followUpQuestionPubSub
  • client/modules/accessKey.ts - Validates and stores access keys using argon2id hashing and localStorage
  • client/modules/parentWindow.ts - PostMessage API for embedding in parent windows
  • client/modules/searchTokenHash.ts - CSRF protection token generation
  • client/modules/pageContent.ts - Requests extracted page text from /page-content
  • client/modules/pageExcerptAllocation.ts - Pure page-excerpt budget logic, shared by the worker and the synchronous fallback
  • client/modules/pageExcerptWorker.ts - Tokenizes page bodies off the main thread at the search-to-generation handoff
  • client/modules/pageExcerptWorkerProtocol.ts - Message types shared by both sides of the excerpt worker
  • client/modules/systemPrompt.ts - System prompt templates
  • client/modules/logEntries.ts - Application logging with unique IDs
  • client/modules/appInfo.ts - Application metadata and version info
  • client/modules/textToSpeech.ts - Reads answers aloud: engine selection, fallback, and the playback queue
  • client/modules/textToSpeechWorker.ts - Synthesizes sentences off the main thread
  • client/modules/textToSpeechWorkerProtocol.ts - Message types shared by both sides of the worker
  • client/modules/piper.ts - The only module that imports @diffusionstudio/vits-web
  • client/modules/speechToText.ts - Dictation: microphone capture, the worker's lifecycle, and engine selection between the on-device model and SpeechRecognition
  • client/modules/speechToTextWorker.ts - Runs the Moonshine transcriber off the main thread
  • client/modules/speechToTextWorkerProtocol.ts - Message types shared by both sides of the dictation worker
  • client/modules/moonshine.ts - The only module that imports @moonshine-ai/moonshine-wasm
  • client/modules/keyboard.ts - Keyboard shortcut handling
  • client/modules/stringFormatters.ts - Text formatting utilities
  • client/modules/types.ts - Shared TypeScript type definitions

Server Modules

  • server/searchEndpointServerHook.ts - /search/text and /search/images endpoints
  • server/thumbnailEndpointServerHook.ts - /thumbnail endpoint: one search-result thumbnail at a time, token-gated, with an in-process LRU in front of the upstream host
  • server/internalApiEndpointServerHook.ts - /inference proxy to self-hosted API
  • server/validateAccessKeyServerHook.ts - Access key validation endpoint
  • server/statusEndpointServerHook.ts - /status health check endpoint
  • server/configEndpointServerHook.ts - /api/config runtime client configuration
  • server/rerankerServiceHook.ts - Reranker model lifecycle management
  • server/compressionServerHook.ts - gzip/brotli compression for responses
  • server/crossOriginServerHook.ts - COOP/COEP headers for SharedArrayBuffer
  • server/dictationModelServerHook.ts - /dictation-models/ endpoint: serves the pinned speech-to-text model files from the instance
  • server/cacheServerHook.ts - Cache-Control headers (preview server only)
  • server/webSearchService.ts - SearXNG integration with circuit breaker and retry logic
  • server/pageContentService.ts - Reads result pages and extracts query-relevant passages
  • server/pageReadHostBreaker.ts - Per-host circuit breaker for page reads: skips a host that refused the last three reads, then probes it once
  • server/utils/publicUrl.ts - Blocks private and reserved addresses before a server-side fetch
  • server/rerankerService.ts - Reranker service (ONNX Runtime inference)
  • server/rankSearchResults.ts - Score-based filtering and result reordering
  • server/searchToken.ts - CSRF token generation and storage
  • server/verifiedTokens.ts - In-memory Set<string> of verified session tokens
  • server/verifyTokenAndRateLimit.ts - Token verification and rate limiting
  • server/handleTokenVerification.ts - Search token validation logic
  • server/searchesSinceLastRestart.ts - In-memory search counters for analytics
  • server/downloadFileFromHuggingFaceRepository.ts - Downloads model files from HuggingFace

Hooks

  • client/hooks/useSearchHistory.ts - Search history management from IndexedDB
  • client/hooks/useHistoryRestore.ts - Restores full search state from history
  • client/hooks/useDrawerState.ts - Drawer open/close state with logging
  • client/hooks/useScreenWakeLock.ts - Keeps the screen awake while a response is generating

Common Tasks Quick Reference

Add a new AI model

  1. Add to client/modules/wllama.ts
  2. Update docs/ai-integration.md
  3. Update docs/configuration.md defaults

Add a new setting

  1. Add to client/modules/settings.ts default object
  2. Add UI in client/components/Pages/Main/Menu/
  3. Update docs/configuration.md settings table

Modify search behavior

  1. Edit client/modules/search.ts
  2. Update server/webSearchService.ts if server-side changes needed
  3. Check server/rerankerService.ts if reranking affected

Fix UI state issues

  1. Check PubSub channels in client/modules/pubSub.ts
  2. Verify component subscriptions in docs/ui-components.md
  3. Ensure proper state updates in business logic modules

Analyze test coverage

  1. Run npm run test:coverage to generate reports
  2. Check coverage/coverage-summary.json for quick metrics
  3. See docs/development-commands.md for full coverage analysis guide

Quality Gates

Before any change:

npm run lint

Inside the dev container, the same command is docker compose exec development-server npm run lint.

This runs:

  • Biome (formatting/linting)
  • TypeScript (type checking)
  • knip (dead code detection)
  • jscpd (copy-paste detection)
  • Custom architectural linter
  • Doc gardening and documentation validator scripts

A bare tsc is not a full typecheck

The project is split across three TypeScript configs, and the default one covers less than its name suggests:

Config Covers
tsconfig.json client, shared, test
tsconfig.node.json vite.config.ts, server, shared, e2e, playwright.config.ts
tsconfig.eval.json eval, shared

So npx tsc --noEmit reports success while type errors sit in server/ or eval/, and CI then fails on a change that looked clean locally. Only npm run lint runs all three configs. Never treat a bare tsc as a green typecheck.

Agent-First Principles

Repository as System of Record:

  • All knowledge lives in versioned docs/ structure
  • This file is your entry point - start here
  • Follow links, don't assume - verify in code

Context Efficiency:

  • Use this map to navigate quickly
  • Return to this file when context drifts
  • Follow the decision tree for common tasks

Architecture & Boundaries:

  • Respect PubSub boundaries - don't cross concerns
  • Client vs server - keep them separate
  • Feature-based organization - one folder per feature

Documentation Maintenance:

  • Update these docs when you learn something new
  • Add cross-references when linking concepts
  • Keep examples current with actual code

Technology Stack

React + TypeScript + Mantine UI v9, with privacy-first architecture. See docs/core-technologies.md for complete dependency list and selection criteria.

Need Help?

  1. Check relevant doc in docs/
  2. Read the module code in client/modules/ or server/
  3. Look at similar existing implementations
  4. Run npm run lint to validate changes