import React, { useState } from 'react'; import { Search, Zap, Layers, Sparkles } from 'lucide-react'; export const MathInspectorTab: React.FC = () => { const [selectedParadigm, setSelectedParadigm] = useState<'sparse' | 'dense' | 'learned-sparse' | 'cross-encoder'>('dense'); return (
Modern information retrieval maps both lexical and neural models to a unified mathematical objective: Maximum Inner Product Search (MIPS) over vector spaces. Sparse, dense, and learned sparse methods differ primarily in dimensionality, sparsity, and indexing data structures.
{/* Math Equation */}In BM25, query and document vectors lie in high-dimensional lexical vocabulary space. Only non-zero entries correspond to tokens present in the document. The dot product is evaluated via inverted posting lists without scanning non-matching documents.
Dense retrieval projects text into low-dimensional semantic vector spaces via dual transformers:
E_Q(q) = Transformer_Q(q) and
E_D(d) = Transformer_D(d).
Because all coordinates are non-zero, inverted lists cannot be used. Instead, approximate nearest neighbor (ANN) search algorithms like
HNSW traverse multi-layer proximity graphs in logarithmic time.
SPLADE combines the efficiency of inverted indexes with deep contextual vocabulary expansion. A masked language model predicts non-negative weights over the full vocabulary, expanding terms with synonyms and related concepts. In Anserini, floating-point weights are quantized to integers and indexed directly in Lucene as synthetic term frequencies.
Unlike bi-encoders which compute independent representations, cross-encoders concatenate query and passage into a single transformer context. Every token attends directly to every other token, capturing nuanced nuances, negation, and syntactic dependencies at higher inference cost.