docs/glossary.md

Glossary

Codebase-specific terms, jargon, and domain concepts used in MiniSearch.

Core System Concepts

Search Token & Hash

A security mechanism used to authorize communication between the client and the internal search/AI endpoints.

• Search Token: A string generated at build time (VITE_SEARCH_TOKEN). Used to verify that requests to the server originate from a trusted build.
• Search Token Hash: To avoid exposing the raw token in all requests, the client generates a hash of the token. Managed via the lastSearchTokenHashPubSub channel.
• Verification: The server verifies these tokens to prevent unauthorized access to the search API. Stored in server/verifiedTokens.ts as an in-memory Set<string>.

Inference Types

MiniSearch supports multiple backends for Large Language Model (LLM) inference, configured via inferenceType in the application settings.

Type	Description	Implementation
browser	Local inference using WASM (Wllama)	Client-side, privacy-preserving
openai	Connection to any OpenAI-compatible external API	Requires API key
horde	Crowdsourced inference via the AI Horde network	Distributed, anonymous or authenticated
internal	Server-side proxy using pre-configured credentials	API key hidden from client

PubSub (State Management)

Instead of a heavy state management library like Redux, MiniSearch uses a minimalist Publish-Subscribe pattern powered by the create-pubsub library.

• Data Flow: Components subscribe to "channels" (e.g., queryPubSub, responsePubSub)
• Tuple Pattern: Each channel is a 3-element tuple: [update, subscribe, get]
• Persistence: Some channels use createLocalStoragePubSub to automatically sync state with localStorage
• Throttling: UI-heavy updates like AI response streaming are throttled to ~12 updates/sec using throttleit

Reranker

A secondary search stage that takes initial results from SearXNG and re-orders them based on relevance to the query using a cross-encoder model (jina-reranker-v1-tiny-en) running on a local llama-server instance.

• Implementation: Spawns llama-server child process with --reranking and --pooling rank flags
• Health Check: Polls /health endpoint via getRerankerStatus
• Scoring: Results filtered using standard deviation thresholds (kStandardDeviationFactor = 0.3)
• Fallback: If reranker is unhealthy, returns unranked SearXNG results

Wllama

A WebAssembly (WASM) based integration of llama.cpp for running LLMs on the CPU in the browser.

• Initialization: Loads models from HuggingFace using initializeWllama
• Warmup: Includes a warmup phase with a single token completion using n_threads: 1
• OPFS: Uses the Origin Private File System via Wllama's cache manager to store model shards locally
• Models: GGUF format, Q4_K_S or UD-Q4_K_XL quantized, stored at Felladrin/gguf-sharded-* on HuggingFace

AI Horde

A crowdsourced distributed cluster of workers providing AI inference. MiniSearch integrates with it using a polling strategy against the /generate/text/status endpoint.

• Kudos: Virtual currency used by the Horde. Default anonymous key is 0000000000
• Polling: Requests sent to async API, status checked periodically until completion
• Cancellation: Can abort generation via DELETE on the status endpoint

Conversation Memory & Rolling Summary

A mechanism to handle long chats that exceed the LLM context window.

• Summarization: When older messages are dropped, createLlmSummary asks the LLM to condense them under a limit of 800 tokens
• Extractive Fallback: If LLM summarization fails, summarizeDroppedMessages uses a token-counting extractive approach
• Token Budget: Computed based on openAiContextLength setting and current message count

Technical Jargon & Abbreviations

SearXNG

A privacy-respecting metasearch engine that aggregates results from multiple search engines without tracking. Runs locally on port 8888 within the Docker container.

GGUF

GGML Universal File format. Binary format for storing LLM weights, optimized for fast loading and inference. Used by Wllama and llama-server.

Dexie

A minimalist wrapper for IndexedDB used for client-side persistence. MiniSearch uses two Dexie databases:

• SearchCacheDatabase: Temporary cache with TTL-based expiration
• HistoryDatabase: Long-term search history with retention policies

Vite Server Hooks

Middleware registered via Vite plugin hooks (configureServer, configurePreviewServer). All server-side logic in MiniSearch is implemented as hooks:

Hook	Purpose
compressionServerHook	gzip/brotli compression
crossOriginServerHook	COOP/COEP headers for SharedArrayBuffer
searchEndpointServerHook	/search/text and /search/images endpoints
statusEndpointServerHook	/status health check
cacheServerHook	Cache-Control headers
validateAccessKeyServerHook	Access key validation
internalApiEndpointServerHook	/inference proxy
rerankerServiceHook	llama-server lifecycle management

Circuit Breaker

A resilience pattern used in webSearchService.ts to handle SearXNG service degradation. Opens after 5 consecutive failures, blocking requests for 60 seconds before attempting reset.

LRU Pruning

Least Recently Used cache eviction strategy. The search cache prunes oldest entries every 10 writes when MAX_ENTRIES (100) is reached.

Argon2id

A password hashing algorithm used for access key validation. Client hashes the access key before transmission; server verifies against configured keys.

Data Structures

SearchCacheDatabase Schema

Store	Primary Key	Indexed Field	Entry Type
textSearchHistory	key (hash)	timestamp	TextSearchCache
imageSearchHistory	key (hash)	timestamp	ImageSearchCache

HistoryDatabase Schema

Table	Purpose
searches	Canonical log of each query with hydrated results payloads
llmResponses	AI answers tied to their originating search run
chatHistory	Chronological chat turns scoped by conversationId

PubSub Channel Types

Channel	Data Type	Persistence
queryPubSub	string	Memory
responsePubSub	string	Memory (throttled)
settingsPubSub	Settings	localStorage
textSearchResultsPubSub	TextSearchResults	Memory
textGenerationStatePubSub	TextGenerationState	Memory
chatMessagesPubSub	ChatMessage[]	Memory
conversationSummaryPubSub	{id, summary}	Memory

Related Topics

• Overview: docs/overview.md - System architecture
• Configuration: docs/configuration.md - Environment variables and settings
• UI Components: docs/ui-components.md - Component architecture
• Reranking: docs/reranking.md - Reranker subsystem