dejanseo/200k-wiki-data-embeddings-gemma
Viewer • Updated • 200k • 30 • 1
Dan Petrovic PRO
dejanseo
AI & ML interests
DEJAN - The AI SEO agency.
[DEJAN](https://dejan.ai/)
Recent Activity
posted an update about 18 hours ago
10 Search Subqueries in 200 Microseconds: 1-Bit Consistency Fanout
We built Fanout Diffusion, an ultra-low-latency model that expands a search query into 10 diverse subquery vectors in a single forward pass.
Inspired by the continuous retrieval framework in R4T (arXiv:2603.06397), our goal was to make query expansion fast enough for production search without running language models.
How it works:
• Direct Embedding Space: Takes a query embedded via google/embeddinggemma-300m (768-dim) and predicts 10 distinct subquery vectors simultaneously.
• 1-Step Consistency Denoiser: Generates all 10 slots analytically in a single pass with zero ODE integration loops.
• 1-Bit Hardware Tensor Cores: Quantized ternary weights (-1, 0, +1) running on Ampere/Ada sub-byte PTX instructions with INT4 outer projections.
Key numbers on an RTX 4090:
• Latency: 0.200 ms (ONNX) / 0.329 ms (native C++ CUDA graph)
• Throughput: 417,035 queries/second in batch mode
• Model Size: 1.64 MB (PyTorch QAT) / 32.1 MB (ONNX graph)
• Quality: 0.683 prompt alignment across 540k search queries
Interactive Space:
https://huggingface.co/spaces/dejanseo/fanout-diffusion
Model weights, ONNX graph, and C++ engine:
https://huggingface.co/dejanseo/fanout-diffusion updated a Space about 18 hours ago
dejanseo/fanout-diffusion updated a model about 18 hours ago
dejanseo/fanout-diffusionOrganizations
