ruvector-fixed / README.md
Archie
Fix dimension/dimensions bug and positional insert/search args
40d7073
# ruvector
[![npm version](https://badge.fury.io/js/ruvector.svg)](https://www.npmjs.com/package/ruvector)
[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](https://opensource.org/licenses/MIT)
[![Node Version](https://img.shields.io/node/v/ruvector)](https://nodejs.org)
[![Downloads](https://img.shields.io/npm/dm/ruvector)](https://www.npmjs.com/package/ruvector)
[![Build Status](https://img.shields.io/badge/build-passing-brightgreen.svg)](https://github.com/ruvnet/ruvector)
[![Performance](https://img.shields.io/badge/latency-<0.5ms-green.svg)](https://github.com/ruvnet/ruvector)
[![GitHub Stars](https://img.shields.io/github/stars/ruvnet/ruvector?style=social)](https://github.com/ruvnet/ruvector)
**The fastest vector database for Node.jsβ€”built in Rust, runs everywhere**
Ruvector is a next-generation vector database that brings **enterprise-grade semantic search** to Node.js applications. Unlike cloud-only solutions or Python-first databases, Ruvector is designed specifically for JavaScript/TypeScript developers who need **blazing-fast vector similarity search** without the complexity of external services.
> πŸš€ **Sub-millisecond queries** β€’ 🎯 **52,000+ inserts/sec** β€’ πŸ’Ύ **~50 bytes per vector** β€’ 🌍 **Runs anywhere**
Built by [rUv](https://ruv.io) with production-grade Rust performance and intelligent platform detectionβ€”**automatically uses native bindings when available, falls back to WebAssembly when needed**.
🌐 **[Visit ruv.io](https://ruv.io)** | πŸ“¦ **[GitHub](https://github.com/ruvnet/ruvector)** | πŸ“š **[Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)**
---
## 🧠 Claude Code Intelligence v2.0
**Self-learning intelligence for Claude Code** β€” RuVector provides optimized hooks with ONNX embeddings, AST analysis, and coverage-aware routing.
```bash
# One-command setup with pretrain and agent generation
npx ruvector hooks init --pretrain --build-agents quality
```
### Core Features
- 🎯 **Smart Agent Routing** β€” Q-learning optimized suggestions with 80%+ accuracy
- πŸ“š **9-Phase Pretrain** β€” AST, diff, coverage, neural, and graph analysis
- πŸ€– **Agent Builder** β€” Generates optimized `.claude/agents/` configs
- πŸ”— **Co-edit Patterns** β€” Learns file relationships from git history
- πŸ’Ύ **Vector Memory** β€” HNSW-indexed semantic recall (150x faster)
### New in v2.0
- ⚑ **ONNX WASM Embeddings** β€” all-MiniLM-L6-v2 (384d) runs locally, no API needed
- 🌳 **AST Analysis** β€” Symbol extraction, complexity metrics, import graphs
- πŸ“Š **Diff Embeddings** β€” Semantic change classification with risk scoring
- πŸ§ͺ **Coverage Routing** β€” Test coverage-aware agent selection
- πŸ” **Graph Algorithms** β€” MinCut boundaries, Louvain communities, Spectral clustering
- πŸ›‘οΈ **Security Scanning** β€” Parallel vulnerability pattern detection
- 🎯 **RAG Context** β€” Semantic retrieval with HNSW indexing
### Performance
| Backend | Read Time | Speedup |
|---------|-----------|---------|
| ONNX inference | ~400ms | baseline |
| HNSW search | ~0.045ms | 8,800x |
| Memory cache | ~0.01ms | **40,000x** |
πŸ“– **[Full Hooks Documentation β†’](https://github.com/ruvnet/ruvector/blob/main/npm/packages/ruvector/HOOKS.md)**
### MCP Server Integration
RuVector includes an MCP server for Claude Code with 30+ tools:
```bash
# Add to Claude Code
claude mcp add ruvector -- npx ruvector mcp start
```
**Available MCP Tools:**
- `hooks_route`, `hooks_route_enhanced` β€” Agent routing with signals
- `hooks_ast_analyze`, `hooks_ast_complexity` β€” Code structure analysis
- `hooks_diff_analyze`, `hooks_diff_classify` β€” Change classification
- `hooks_coverage_route`, `hooks_coverage_suggest` β€” Test-aware routing
- `hooks_graph_mincut`, `hooks_graph_cluster` β€” Code boundaries
- `hooks_security_scan` β€” Vulnerability detection
- `hooks_rag_context` β€” Semantic context retrieval
- `hooks_attention_info`, `hooks_gnn_info` β€” Neural capabilities
---
## 🌟 Why Ruvector?
### The Problem with Existing Vector Databases
Most vector databases force you to choose between three painful trade-offs:
1. **Cloud-Only Services** (Pinecone, Weaviate Cloud) - Expensive, vendor lock-in, latency issues, API rate limits
2. **Python-First Solutions** (ChromaDB, Faiss) - Poor Node.js support, require separate Python processes
3. **Self-Hosted Complexity** (Milvus, Qdrant) - Heavy infrastructure, Docker orchestration, operational overhead
**Ruvector eliminates these trade-offs.**
### The Ruvector Advantage
Ruvector is purpose-built for **modern JavaScript/TypeScript applications** that need vector search:
🎯 **Native Node.js Integration**
- Drop-in npm packageβ€”no Docker, no Python, no external services
- Full TypeScript support with complete type definitions
- Automatic platform detection with native Rust bindings
- Seamless WebAssembly fallback for universal compatibility
⚑ **Production-Grade Performance**
- **52,000+ inserts/second** with native Rust (10x faster than Python alternatives)
- **<0.5ms query latency** with HNSW indexing and SIMD optimizations
- **~50 bytes per vector** with advanced memory optimization
- Scales from edge devices to millions of vectors
🧠 **Built for AI Applications**
- Optimized for LLM embeddings (OpenAI, Cohere, Hugging Face)
- Perfect for RAG (Retrieval-Augmented Generation) systems
- Agent memory and semantic caching
- Real-time recommendation engines
🌍 **Universal Deployment**
- **Linux, macOS, Windows** with native performance
- **Browser support** via WebAssembly (experimental)
- **Edge computing** and serverless environments
- **Alpine Linux** and non-glibc systems supported
πŸ’° **Zero Operational Costs**
- No cloud API fees or usage limits
- No infrastructure to manage
- No separate database servers
- Open source MIT license
### Key Advantages
- ⚑ **Blazing Fast**: <0.5ms p50 latency with native Rust, 10-50ms with WASM fallback
- 🎯 **Automatic Platform Detection**: Uses native when available, falls back to WASM seamlessly
- 🧠 **AI-Native**: Built specifically for embeddings, RAG, semantic search, and agent memory
- πŸ”§ **CLI Tools Included**: Full command-line interface for database management
- 🌍 **Universal Deployment**: Works on all platformsβ€”Linux, macOS, Windows, even browsers
- πŸ’Ύ **Memory Efficient**: ~50 bytes per vector with advanced quantization
- πŸš€ **Production Ready**: Battle-tested algorithms with comprehensive benchmarks
- πŸ”“ **Open Source**: MIT licensed, community-driven
## πŸš€ Quick Start Tutorial
### Step 1: Installation
Install Ruvector with a single npm command:
```bash
npm install ruvector
```
**What happens during installation:**
- npm automatically detects your platform (Linux, macOS, Windows)
- Downloads the correct native binary for maximum performance
- Falls back to WebAssembly if native binaries aren't available
- No additional setup, Docker, or external services required
**Windows Installation (without build tools):**
```bash
# Skip native compilation, use WASM fallback
npm install ruvector --ignore-scripts
# The ONNX WASM runtime (7.4MB) works without build tools
# Memory cache provides 40,000x speedup over inference
```
**Verify installation:**
```bash
npx ruvector info
```
You should see your platform and implementation type (native Rust or WASM fallback).
### Step 2: Your First Vector Database
Let's create a simple vector database and perform basic operations. This example demonstrates the complete CRUD (Create, Read, Update, Delete) workflow:
```javascript
const { VectorDb } = require('ruvector');
async function tutorial() {
// Step 2.1: Create a new vector database
// The 'dimensions' parameter must match your embedding model
// Common sizes: 128, 384 (sentence-transformers), 768 (BERT), 1536 (OpenAI)
const db = new VectorDb({
dimensions: 128, // Vector size - MUST match your embeddings
maxElements: 10000, // Maximum vectors (can grow automatically)
storagePath: './my-vectors.db' // Persist to disk (omit for in-memory)
});
console.log('βœ… Database created successfully');
// Step 2.2: Insert vectors
// In real applications, these would come from an embedding model
const documents = [
{ id: 'doc1', text: 'Artificial intelligence and machine learning' },
{ id: 'doc2', text: 'Deep learning neural networks' },
{ id: 'doc3', text: 'Natural language processing' },
];
for (const doc of documents) {
// Generate random vector for demonstration
// In production: use OpenAI, Cohere, or sentence-transformers
const vector = new Float32Array(128).map(() => Math.random());
await db.insert({
id: doc.id,
vector: vector,
metadata: {
text: doc.text,
timestamp: Date.now(),
category: 'AI'
}
});
console.log(`βœ… Inserted: ${doc.id}`);
}
// Step 2.3: Search for similar vectors
// Create a query vector (in production, this would be from your search query)
const queryVector = new Float32Array(128).map(() => Math.random());
const results = await db.search({
vector: queryVector,
k: 5, // Return top 5 most similar vectors
threshold: 0.7 // Only return results with similarity > 0.7
});
console.log('\nπŸ” Search Results:');
results.forEach((result, index) => {
console.log(`${index + 1}. ${result.id} - Score: ${result.score.toFixed(3)}`);
console.log(` Text: ${result.metadata.text}`);
});
// Step 2.4: Retrieve a specific vector
const retrieved = await db.get('doc1');
if (retrieved) {
console.log('\nπŸ“„ Retrieved document:', retrieved.metadata.text);
}
// Step 2.5: Get database statistics
const count = await db.len();
console.log(`\nπŸ“Š Total vectors in database: ${count}`);
// Step 2.6: Delete a vector
const deleted = await db.delete('doc1');
console.log(`\nπŸ—‘οΈ Deleted doc1: ${deleted ? 'Success' : 'Not found'}`);
// Final count
const finalCount = await db.len();
console.log(`πŸ“Š Final count: ${finalCount}`);
}
// Run the tutorial
tutorial().catch(console.error);
```
**Expected Output:**
```
βœ… Database created successfully
βœ… Inserted: doc1
βœ… Inserted: doc2
βœ… Inserted: doc3
πŸ” Search Results:
1. doc2 - Score: 0.892
Text: Deep learning neural networks
2. doc1 - Score: 0.856
Text: Artificial intelligence and machine learning
3. doc3 - Score: 0.801
Text: Natural language processing
πŸ“„ Retrieved document: Artificial intelligence and machine learning
πŸ“Š Total vectors in database: 3
πŸ—‘οΈ Deleted doc1: Success
πŸ“Š Final count: 2
```
### Step 3: TypeScript Tutorial
Ruvector provides full TypeScript support with complete type safety. Here's how to use it:
```typescript
import { VectorDb, VectorEntry, SearchQuery, SearchResult } from 'ruvector';
// Step 3.1: Define your custom metadata type
interface DocumentMetadata {
title: string;
content: string;
author: string;
date: Date;
tags: string[];
}
async function typescriptTutorial() {
// Step 3.2: Create typed database
const db = new VectorDb({
dimensions: 384, // sentence-transformers/all-MiniLM-L6-v2
maxElements: 10000,
storagePath: './typed-vectors.db'
});
// Step 3.3: Type-safe vector entry
const entry: VectorEntry<DocumentMetadata> = {
id: 'article-001',
vector: new Float32Array(384), // Your embedding here
metadata: {
title: 'Introduction to Vector Databases',
content: 'Vector databases enable semantic search...',
author: 'Jane Doe',
date: new Date('2024-01-15'),
tags: ['database', 'AI', 'search']
}
};
// Step 3.4: Insert with type checking
await db.insert(entry);
console.log('βœ… Inserted typed document');
// Step 3.5: Type-safe search
const query: SearchQuery = {
vector: new Float32Array(384),
k: 10,
threshold: 0.8
};
// Step 3.6: Fully typed results
const results: SearchResult<DocumentMetadata>[] = await db.search(query);
// TypeScript knows the exact shape of metadata
results.forEach(result => {
console.log(`Title: ${result.metadata.title}`);
console.log(`Author: ${result.metadata.author}`);
console.log(`Tags: ${result.metadata.tags.join(', ')}`);
console.log(`Similarity: ${result.score.toFixed(3)}\n`);
});
// Step 3.7: Type-safe retrieval
const doc = await db.get('article-001');
if (doc) {
// TypeScript autocomplete works perfectly here
const publishYear = doc.metadata.date.getFullYear();
console.log(`Published in ${publishYear}`);
}
}
typescriptTutorial().catch(console.error);
```
**TypeScript Benefits:**
- βœ… Full autocomplete for all methods and properties
- βœ… Compile-time type checking prevents errors
- βœ… IDE IntelliSense shows documentation
- βœ… Custom metadata types for your use case
- βœ… No `any` types - fully typed throughout
## 🎯 Platform Detection
Ruvector automatically detects the best implementation for your platform:
```javascript
const { getImplementationType, isNative, isWasm } = require('ruvector');
console.log(getImplementationType()); // 'native' or 'wasm'
console.log(isNative()); // true if using native Rust
console.log(isWasm()); // true if using WebAssembly fallback
// Performance varies by implementation:
// Native (Rust): <0.5ms latency, 50K+ ops/sec
// WASM fallback: 10-50ms latency, ~1K ops/sec
```
## πŸ”§ CLI Tools
Ruvector includes a full command-line interface for database management:
### Create Database
```bash
# Create a new vector database
npx ruvector create mydb.vec --dimensions 384 --metric cosine
# Options:
# --dimensions, -d Vector dimensionality (required)
# --metric, -m Distance metric (cosine, euclidean, dot)
# --max-elements Maximum number of vectors (default: 10000)
```
### Insert Vectors
```bash
# Insert vectors from JSON file
npx ruvector insert mydb.vec vectors.json
# JSON format:
# [
# { "id": "doc1", "vector": [0.1, 0.2, ...], "metadata": {...} },
# { "id": "doc2", "vector": [0.3, 0.4, ...], "metadata": {...} }
# ]
```
### Search Vectors
```bash
# Search for similar vectors
npx ruvector search mydb.vec --vector "[0.1,0.2,0.3,...]" --top-k 10
# Options:
# --vector, -v Query vector (JSON array)
# --top-k, -k Number of results (default: 10)
# --threshold Minimum similarity score
```
### Database Statistics
```bash
# Show database statistics
npx ruvector stats mydb.vec
# Output:
# Total vectors: 10,000
# Dimensions: 384
# Metric: cosine
# Memory usage: ~500 KB
# Index type: HNSW
```
### Benchmarking
```bash
# Run performance benchmark
npx ruvector benchmark --num-vectors 10000 --num-queries 1000
# Options:
# --num-vectors Number of vectors to insert
# --num-queries Number of search queries
# --dimensions Vector dimensionality (default: 128)
```
### System Information
```bash
# Show platform and implementation info
npx ruvector info
# Output:
# Platform: linux-x64-gnu
# Implementation: native (Rust)
# GNN Module: Available
# Node.js: v18.17.0
# Performance: <0.5ms p50 latency
```
### Install Optional Packages
Ruvector supports optional packages that extend functionality. Use the `install` command to add them:
```bash
# List available packages
npx ruvector install
# Output:
# Available Ruvector Packages:
#
# gnn not installed
# Graph Neural Network layers, tensor compression, differentiable search
# npm: @ruvector/gnn
#
# core βœ“ installed
# Core vector database with native Rust bindings
# npm: @ruvector/core
# Install specific package
npx ruvector install gnn
# Install all optional packages
npx ruvector install --all
# Interactive selection
npx ruvector install -i
```
The install command auto-detects your package manager (npm, yarn, pnpm, bun).
### GNN Commands
Ruvector includes Graph Neural Network (GNN) capabilities for advanced tensor compression and differentiable search.
#### GNN Info
```bash
# Show GNN module information
npx ruvector gnn info
# Output:
# GNN Module Information
# Status: Available
# Platform: linux
# Architecture: x64
#
# Available Features:
# β€’ RuvectorLayer - GNN layer with multi-head attention
# β€’ TensorCompress - Adaptive tensor compression (5 levels)
# β€’ differentiableSearch - Soft attention-based search
# β€’ hierarchicalForward - Multi-layer GNN processing
```
#### GNN Layer
```bash
# Create and test a GNN layer
npx ruvector gnn layer -i 128 -h 256 --test
# Options:
# -i, --input-dim Input dimension (required)
# -h, --hidden-dim Hidden dimension (required)
# -a, --heads Number of attention heads (default: 4)
# -d, --dropout Dropout rate (default: 0.1)
# --test Run a test forward pass
# -o, --output Save layer config to JSON file
```
#### GNN Compress
```bash
# Compress embeddings using adaptive tensor compression
npx ruvector gnn compress -f embeddings.json -l pq8 -o compressed.json
# Options:
# -f, --file Input JSON file with embeddings (required)
# -l, --level Compression level: none|half|pq8|pq4|binary (default: auto)
# -a, --access-freq Access frequency for auto compression (default: 0.5)
# -o, --output Output file for compressed data
# Compression levels:
# none (freq > 0.8) - Full precision, hot data
# half (freq > 0.4) - ~50% savings, warm data
# pq8 (freq > 0.1) - ~8x compression, cool data
# pq4 (freq > 0.01) - ~16x compression, cold data
# binary (freq <= 0.01) - ~32x compression, archive
```
#### GNN Search
```bash
# Differentiable search with soft attention
npx ruvector gnn search -q "[1.0,0.0,0.0]" -c candidates.json -k 5
# Options:
# -q, --query Query vector as JSON array (required)
# -c, --candidates Candidates file - JSON array of vectors (required)
# -k, --top-k Number of results (default: 5)
# -t, --temperature Softmax temperature (default: 1.0)
```
### Attention Commands
Ruvector includes high-performance attention mechanisms for transformer-based operations, hyperbolic embeddings, and graph attention.
```bash
# Install the attention module (optional)
npm install @ruvector/attention
```
#### Attention Mechanisms Reference
| Mechanism | Type | Complexity | When to Use |
|-----------|------|------------|-------------|
| **DotProductAttention** | Core | O(nΒ²) | Standard scaled dot-product attention for transformers |
| **MultiHeadAttention** | Core | O(nΒ²) | Parallel attention heads for capturing different relationships |
| **FlashAttention** | Core | O(nΒ²) IO-optimized | Memory-efficient attention for long sequences |
| **HyperbolicAttention** | Core | O(nΒ²) | Hierarchical data, tree-like structures, taxonomies |
| **LinearAttention** | Core | O(n) | Very long sequences where O(nΒ²) is prohibitive |
| **MoEAttention** | Core | O(n*k) | Mixture of Experts routing, specialized attention |
| **GraphRoPeAttention** | Graph | O(nΒ²) | Graph data with rotary position embeddings |
| **EdgeFeaturedAttention** | Graph | O(nΒ²) | Graphs with rich edge features/attributes |
| **DualSpaceAttention** | Graph | O(nΒ²) | Combined Euclidean + hyperbolic representation |
| **LocalGlobalAttention** | Graph | O(n*k) | Large graphs with local + global context |
#### Attention Info
```bash
# Show attention module information
npx ruvector attention info
# Output:
# Attention Module Information
# Status: Available
# Version: 0.1.0
# Platform: linux
# Architecture: x64
#
# Core Attention Mechanisms:
# β€’ DotProductAttention - Scaled dot-product attention
# β€’ MultiHeadAttention - Multi-head self-attention
# β€’ FlashAttention - Memory-efficient IO-aware attention
# β€’ HyperbolicAttention - PoincarΓ© ball attention
# β€’ LinearAttention - O(n) linear complexity attention
# β€’ MoEAttention - Mixture of Experts attention
```
#### Attention List
```bash
# List all available attention mechanisms
npx ruvector attention list
# With verbose details
npx ruvector attention list -v
```
#### Attention Benchmark
```bash
# Benchmark attention mechanisms
npx ruvector attention benchmark -d 256 -n 100 -i 100
# Options:
# -d, --dimension Vector dimension (default: 256)
# -n, --num-vectors Number of vectors (default: 100)
# -i, --iterations Benchmark iterations (default: 100)
# -t, --types Attention types to benchmark (default: dot,flash,linear)
# Example output:
# Dimension: 256
# Vectors: 100
# Iterations: 100
#
# dot: 0.012ms/op (84,386 ops/sec)
# flash: 0.012ms/op (82,844 ops/sec)
# linear: 0.066ms/op (15,259 ops/sec)
```
#### Hyperbolic Operations
```bash
# Calculate PoincarΓ© distance between two points
npx ruvector attention hyperbolic -a distance -v "[0.1,0.2,0.3]" -b "[0.4,0.5,0.6]"
# Project vector to PoincarΓ© ball
npx ruvector attention hyperbolic -a project -v "[1.5,2.0,0.8]"
# MΓΆbius addition in hyperbolic space
npx ruvector attention hyperbolic -a mobius-add -v "[0.1,0.2]" -b "[0.3,0.4]"
# Exponential map (tangent space β†’ PoincarΓ© ball)
npx ruvector attention hyperbolic -a exp-map -v "[0.1,0.2,0.3]"
# Options:
# -a, --action Action: distance|project|mobius-add|exp-map|log-map
# -v, --vector Input vector as JSON array (required)
# -b, --vector-b Second vector for binary operations
# -c, --curvature PoincarΓ© ball curvature (default: 1.0)
```
#### When to Use Each Attention Type
| Use Case | Recommended Attention | Reason |
|----------|----------------------|--------|
| **Standard NLP/Transformers** | MultiHeadAttention | Industry standard, well-tested |
| **Long Documents (>4K tokens)** | FlashAttention or LinearAttention | Memory efficient |
| **Hierarchical Classification** | HyperbolicAttention | Captures tree-like structures |
| **Knowledge Graphs** | GraphRoPeAttention | Position-aware graph attention |
| **Multi-Relational Graphs** | EdgeFeaturedAttention | Leverages edge attributes |
| **Taxonomy/Ontology Search** | DualSpaceAttention | Best of both Euclidean + hyperbolic |
| **Large-Scale Graphs** | LocalGlobalAttention | Efficient local + global context |
| **Model Routing/MoE** | MoEAttention | Expert selection and routing |
### ⚑ ONNX WASM Embeddings (v2.0)
RuVector includes a pure JavaScript ONNX runtime for local embeddings - no Python, no API calls, no build tools required.
```bash
# Embeddings work out of the box
npx ruvector hooks remember "important context" -t project
npx ruvector hooks recall "context query"
npx ruvector hooks rag-context "how does auth work"
```
**Model**: all-MiniLM-L6-v2 (384 dimensions, 23MB)
- Downloads automatically on first use
- Cached in `.ruvector/models/`
- SIMD-accelerated when available
**Performance:**
| Operation | Time | Notes |
|-----------|------|-------|
| Model load | ~2s | First use only |
| Embedding | ~50ms | Per text chunk |
| HNSW search | 0.045ms | 150x faster than brute force |
| Cache hit | 0.01ms | 40,000x faster than inference |
**Fallback Chain:**
1. Native SQLite β†’ best persistence
2. WASM SQLite β†’ cross-platform
3. Memory Cache β†’ fastest (no persistence)
### 🧠 Self-Learning Hooks v2.0
Ruvector includes **self-learning intelligence hooks** for Claude Code integration with ONNX embeddings, AST analysis, and coverage-aware routing.
#### Initialize Hooks
```bash
# Initialize hooks in your project
npx ruvector hooks init
# Options:
# --force Overwrite existing configuration
# --minimal Minimal configuration (no optional hooks)
# --pretrain Initialize + pretrain from git history
# --build-agents quality Generate optimized agent configs
```
This creates `.claude/settings.json` with pre-configured hooks and `CLAUDE.md` with comprehensive documentation.
#### Session Management
```bash
# Start a session (load intelligence data)
npx ruvector hooks session-start
# End a session (save learned patterns)
npx ruvector hooks session-end
```
#### Pre/Post Edit Hooks
```bash
# Before editing a file - get agent recommendations
npx ruvector hooks pre-edit src/index.ts
# Output: πŸ€– Recommended: typescript-developer (85% confidence)
# After editing - record success/failure for learning
npx ruvector hooks post-edit src/index.ts --success
npx ruvector hooks post-edit src/index.ts --error "Type error on line 42"
```
#### Pre/Post Command Hooks
```bash
# Before running a command - risk analysis
npx ruvector hooks pre-command "npm test"
# Output: βœ… Risk: LOW, Category: test
# After running - record outcome
npx ruvector hooks post-command "npm test" --success
npx ruvector hooks post-command "npm test" --error "3 tests failed"
```
#### Agent Routing
```bash
# Get agent recommendation for a task
npx ruvector hooks route "fix the authentication bug in login.ts"
# Output: πŸ€– Recommended: security-specialist (92% confidence)
npx ruvector hooks route "add unit tests for the API"
# Output: πŸ€– Recommended: tester (88% confidence)
```
#### Memory Operations
```bash
# Store context in vector memory
npx ruvector hooks remember "API uses JWT tokens with 1h expiry" --type decision
npx ruvector hooks remember "Database schema in docs/schema.md" --type reference
# Semantic search memory
npx ruvector hooks recall "authentication mechanism"
# Returns relevant stored memories
```
#### Context Suggestions
```bash
# Get relevant context for current task
npx ruvector hooks suggest-context
# Output: Based on recent files, suggests relevant context
```
#### Intelligence Statistics
```bash
# Show learned patterns and statistics
npx ruvector hooks stats
# Output:
# Patterns: 156 learned
# Success rate: 87%
# Top agents: rust-developer, tester, reviewer
# Memory entries: 42
```
#### Swarm Recommendations
```bash
# Get agent recommendation for task type
npx ruvector hooks swarm-recommend "code-review"
# Output: Recommended agents for code review task
```
#### AST Analysis (v2.0)
```bash
# Analyze file structure, symbols, imports, complexity
npx ruvector hooks ast-analyze src/index.ts --json
# Get complexity metrics for multiple files
npx ruvector hooks ast-complexity src/*.ts --threshold 15
# Flags files exceeding cyclomatic complexity threshold
```
#### Diff & Risk Analysis (v2.0)
```bash
# Analyze commit with semantic embeddings and risk scoring
npx ruvector hooks diff-analyze HEAD
# Output: risk score, category, affected files
# Classify change type (feature, bugfix, refactor, docs, test)
npx ruvector hooks diff-classify
# Find similar past commits via embeddings
npx ruvector hooks diff-similar -k 5
# Git churn analysis (hot spots)
npx ruvector hooks git-churn --days 30
```
#### Coverage-Aware Routing (v2.0)
```bash
# Get coverage-aware routing for a file
npx ruvector hooks coverage-route src/api.ts
# Output: agent weights based on test coverage
# Suggest tests for files based on coverage gaps
npx ruvector hooks coverage-suggest src/*.ts
```
#### Graph Analysis (v2.0)
```bash
# Find optimal code boundaries (MinCut algorithm)
npx ruvector hooks graph-mincut src/*.ts
# Detect code communities (Louvain/Spectral clustering)
npx ruvector hooks graph-cluster src/*.ts --method louvain
```
#### Security & RAG (v2.0)
```bash
# Parallel security vulnerability scan
npx ruvector hooks security-scan src/*.ts
# RAG-enhanced context retrieval
npx ruvector hooks rag-context "how does auth work"
# Enhanced routing with all signals
npx ruvector hooks route-enhanced "fix bug" --file src/api.ts
```
#### Hooks Configuration
The hooks integrate with Claude Code via `.claude/settings.json`:
```json
{
"env": {
"RUVECTOR_INTELLIGENCE_ENABLED": "true",
"RUVECTOR_LEARNING_RATE": "0.1",
"RUVECTOR_AST_ENABLED": "true",
"RUVECTOR_DIFF_EMBEDDINGS": "true",
"RUVECTOR_COVERAGE_ROUTING": "true",
"RUVECTOR_GRAPH_ALGORITHMS": "true",
"RUVECTOR_SECURITY_SCAN": "true"
},
"hooks": {
"PreToolUse": [
{
"matcher": "Edit|Write|MultiEdit",
"hooks": [{ "type": "command", "command": "npx ruvector hooks pre-edit \"$TOOL_INPUT_file_path\"" }]
},
{
"matcher": "Bash",
"hooks": [{ "type": "command", "command": "npx ruvector hooks pre-command \"$TOOL_INPUT_command\"" }]
}
],
"PostToolUse": [
{
"matcher": "Edit|Write|MultiEdit",
"hooks": [{ "type": "command", "command": "npx ruvector hooks post-edit \"$TOOL_INPUT_file_path\"" }]
}
],
"SessionStart": [{ "hooks": [{ "type": "command", "command": "npx ruvector hooks session-start" }] }],
"Stop": [{ "hooks": [{ "type": "command", "command": "npx ruvector hooks session-end" }] }]
}
}
```
#### How Self-Learning Works
1. **Pattern Recording**: Every edit and command is recorded with context
2. **Q-Learning**: Success/failure updates agent routing weights
3. **AST Analysis**: Code complexity informs agent selection
4. **Diff Embeddings**: Change patterns improve risk assessment
5. **Coverage Routing**: Test coverage guides testing priorities
6. **Vector Memory**: Decisions and references stored for semantic recall (HNSW indexed)
7. **Continuous Improvement**: The more you use it, the smarter it gets
## πŸ“Š Performance Benchmarks
Tested on AMD Ryzen 9 5950X, 128-dimensional vectors:
### Native Performance (Rust)
| Operation | Throughput | Latency (p50) | Latency (p99) |
|-----------|------------|---------------|---------------|
| Insert | 52,341 ops/sec | 0.019 ms | 0.045 ms |
| Search (k=10) | 11,234 ops/sec | 0.089 ms | 0.156 ms |
| Search (k=100) | 8,932 ops/sec | 0.112 ms | 0.203 ms |
| Delete | 45,678 ops/sec | 0.022 ms | 0.051 ms |
**Memory Usage**: ~50 bytes per 128-dim vector (including index)
### Comparison with Alternatives
| Database | Insert (ops/sec) | Search (ops/sec) | Memory per Vector | Node.js | Browser |
|----------|------------------|------------------|-------------------|---------|---------|
| **Ruvector (Native)** | **52,341** | **11,234** | **50 bytes** | βœ… | ❌ |
| **Ruvector (WASM)** | **~1,000** | **~100** | **50 bytes** | βœ… | βœ… |
| Faiss (HNSW) | 38,200 | 9,800 | 68 bytes | ❌ | ❌ |
| Hnswlib | 41,500 | 10,200 | 62 bytes | βœ… | ❌ |
| ChromaDB | ~1,000 | ~20 | 150 bytes | βœ… | ❌ |
*Benchmarks measured with 100K vectors, 128 dimensions, k=10*
## πŸ” Comparison with Other Vector Databases
Comprehensive comparison of Ruvector against popular vector database solutions:
| Feature | Ruvector | Pinecone | Qdrant | Weaviate | Milvus | ChromaDB | Faiss |
|---------|----------|----------|--------|----------|--------|----------|-------|
| **Deployment** |
| Installation | `npm install` βœ… | Cloud API ☁️ | Docker 🐳 | Docker 🐳 | Docker/K8s 🐳 | `pip install` 🐍 | `pip install` 🐍 |
| Node.js Native | βœ… First-class | ❌ API only | ⚠️ HTTP API | ⚠️ HTTP API | ⚠️ HTTP API | ❌ Python | ❌ Python |
| Setup Time | < 1 minute | 5-10 minutes | 10-30 minutes | 15-30 minutes | 30-60 minutes | 5 minutes | 5 minutes |
| Infrastructure | None required | Managed cloud | Self-hosted | Self-hosted | Self-hosted | Embedded | Embedded |
| **Performance** |
| Query Latency (p50) | **<0.5ms** | ~2-5ms | ~1-2ms | ~2-3ms | ~3-5ms | ~50ms | ~1ms |
| Insert Throughput | **52,341 ops/sec** | ~10,000 ops/sec | ~20,000 ops/sec | ~15,000 ops/sec | ~25,000 ops/sec | ~1,000 ops/sec | ~40,000 ops/sec |
| Memory per Vector (128d) | **50 bytes** | ~80 bytes | 62 bytes | ~100 bytes | ~70 bytes | 150 bytes | 68 bytes |
| Recall @ k=10 | 95%+ | 93% | 94% | 92% | 96% | 85% | 97% |
| **Platform Support** |
| Linux | βœ… Native | ☁️ API | βœ… Docker | βœ… Docker | βœ… Docker | βœ… Python | βœ… Python |
| macOS | βœ… Native | ☁️ API | βœ… Docker | βœ… Docker | βœ… Docker | βœ… Python | βœ… Python |
| Windows | βœ… Native | ☁️ API | βœ… Docker | βœ… Docker | ⚠️ WSL2 | βœ… Python | βœ… Python |
| Browser/WASM | βœ… Yes | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No |
| ARM64 | βœ… Native | ☁️ API | βœ… Yes | βœ… Yes | ⚠️ Limited | βœ… Yes | βœ… Yes |
| Alpine Linux | βœ… WASM | ☁️ API | ⚠️ Build from source | ⚠️ Build from source | ❌ No | βœ… Yes | βœ… Yes |
| **Features** |
| Distance Metrics | Cosine, L2, Dot | Cosine, L2, Dot | 11 metrics | 10 metrics | 8 metrics | L2, Cosine, IP | L2, IP, Cosine |
| Filtering | βœ… Metadata | βœ… Advanced | βœ… Advanced | βœ… Advanced | βœ… Advanced | βœ… Basic | ❌ Limited |
| Persistence | βœ… File-based | ☁️ Managed | βœ… Disk | βœ… Disk | βœ… Disk | βœ… DuckDB | ❌ Memory |
| Indexing | HNSW | Proprietary | HNSW | HNSW | IVF/HNSW | HNSW | IVF/HNSW |
| Quantization | βœ… PQ | βœ… Yes | βœ… Scalar | βœ… PQ | βœ… PQ/SQ | ❌ No | βœ… PQ |
| Batch Operations | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Yes |
| **Developer Experience** |
| TypeScript Types | βœ… Full | βœ… Generated | ⚠️ Community | ⚠️ Community | ⚠️ Community | ⚠️ Partial | ❌ No |
| Documentation | βœ… Excellent | βœ… Excellent | βœ… Good | βœ… Good | βœ… Good | βœ… Good | ⚠️ Technical |
| Examples | βœ… Many | βœ… Many | βœ… Good | βœ… Good | βœ… Many | βœ… Good | ⚠️ Limited |
| CLI Tools | βœ… Included | ⚠️ Limited | βœ… Yes | βœ… Yes | βœ… Yes | ⚠️ Basic | ❌ No |
| **Operations** |
| Monitoring | βœ… Metrics | βœ… Dashboard | βœ… Prometheus | βœ… Prometheus | βœ… Prometheus | ⚠️ Basic | ❌ No |
| Backups | βœ… File copy | ☁️ Automatic | βœ… Snapshots | βœ… Snapshots | βœ… Snapshots | βœ… File copy | ❌ Manual |
| High Availability | ⚠️ App-level | βœ… Built-in | βœ… Clustering | βœ… Clustering | βœ… Clustering | ❌ No | ❌ No |
| Auto-Scaling | ⚠️ App-level | βœ… Automatic | ⚠️ Manual | ⚠️ Manual | ⚠️ K8s HPA | ❌ No | ❌ No |
| **Cost** |
| Pricing Model | Free (MIT) | Pay-per-use | Free (Apache) | Free (BSD) | Free (Apache) | Free (Apache) | Free (MIT) |
| Monthly Cost (1M vectors) | **$0** | ~$70-200 | ~$20-50 (infra) | ~$30-60 (infra) | ~$50-100 (infra) | $0 | $0 |
| Monthly Cost (10M vectors) | **$0** | ~$500-1000 | ~$100-200 (infra) | ~$150-300 (infra) | ~$200-400 (infra) | $0 | $0 |
| API Rate Limits | None | Yes | None | None | None | None | None |
| **Use Cases** |
| RAG Systems | βœ… Excellent | βœ… Excellent | βœ… Excellent | βœ… Excellent | βœ… Excellent | βœ… Good | ⚠️ Limited |
| Serverless | βœ… Perfect | βœ… Good | ❌ No | ❌ No | ❌ No | ⚠️ Possible | ⚠️ Possible |
| Edge Computing | βœ… Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ❌ No | ⚠️ Possible |
| Production Scale (100M+) | ⚠️ Single node | βœ… Yes | βœ… Yes | βœ… Yes | βœ… Excellent | ⚠️ Limited | ⚠️ Manual |
| Embedded Apps | βœ… Excellent | ❌ No | ❌ No | ❌ No | ❌ No | ⚠️ Possible | βœ… Good |
### When to Choose Ruvector
βœ… **Perfect for:**
- **Node.js/TypeScript applications** needing embedded vector search
- **Serverless and edge computing** where external services aren't practical
- **Rapid prototyping and development** with minimal setup time
- **RAG systems** with LangChain, LlamaIndex, or custom implementations
- **Cost-sensitive projects** that can't afford cloud API pricing
- **Offline-first applications** requiring local vector search
- **Browser-based AI** with WebAssembly fallback
- **Small to medium scale** (up to 10M vectors per instance)
⚠️ **Consider alternatives for:**
- **Massive scale (100M+ vectors)** - Consider Pinecone, Milvus, or Qdrant clusters
- **Multi-tenancy requirements** - Weaviate or Qdrant offer better isolation
- **Distributed systems** - Milvus provides better horizontal scaling
- **Zero-ops cloud solution** - Pinecone handles all infrastructure
### Why Choose Ruvector Over...
**vs Pinecone:**
- βœ… No API costs (save $1000s/month)
- βœ… No network latency (10x faster queries)
- βœ… No vendor lock-in
- βœ… Works offline and in restricted environments
- ❌ No managed multi-region clusters
**vs ChromaDB:**
- βœ… 50x faster queries (native Rust vs Python)
- βœ… True Node.js support (not HTTP API)
- βœ… Better TypeScript integration
- βœ… Lower memory usage
- ❌ Smaller ecosystem and community
**vs Qdrant:**
- βœ… Zero infrastructure setup
- βœ… Embedded in your app (no Docker)
- βœ… Better for serverless environments
- βœ… Native Node.js bindings
- ❌ No built-in clustering or HA
**vs Faiss:**
- βœ… Full Node.js support (Faiss is Python-only)
- βœ… Easier API and better developer experience
- βœ… Built-in persistence and metadata
- ⚠️ Slightly lower recall at same performance
## 🎯 Real-World Tutorials
### Tutorial 1: Building a RAG System with OpenAI
**What you'll learn:** Create a production-ready Retrieval-Augmented Generation system that enhances LLM responses with relevant context from your documents.
**Prerequisites:**
```bash
npm install ruvector openai
export OPENAI_API_KEY="your-api-key-here"
```
**Complete Implementation:**
```javascript
const { VectorDb } = require('ruvector');
const OpenAI = require('openai');
class RAGSystem {
constructor() {
// Initialize OpenAI client
this.openai = new OpenAI({
apiKey: process.env.OPENAI_API_KEY
});
// Create vector database for OpenAI embeddings
// text-embedding-ada-002 produces 1536-dimensional vectors
this.db = new VectorDb({
dimensions: 1536,
maxElements: 100000,
storagePath: './rag-knowledge-base.db'
});
console.log('βœ… RAG System initialized');
}
// Step 1: Index your knowledge base
async indexDocuments(documents) {
console.log(`πŸ“š Indexing ${documents.length} documents...`);
for (let i = 0; i < documents.length; i++) {
const doc = documents[i];
// Generate embedding for the document
const response = await this.openai.embeddings.create({
model: 'text-embedding-ada-002',
input: doc.content
});
// Store in vector database
await this.db.insert({
id: doc.id || `doc_${i}`,
vector: new Float32Array(response.data[0].embedding),
metadata: {
title: doc.title,
content: doc.content,
source: doc.source,
date: doc.date || new Date().toISOString()
}
});
console.log(` βœ… Indexed: ${doc.title}`);
}
const count = await this.db.len();
console.log(`\nβœ… Indexed ${count} documents total`);
}
// Step 2: Retrieve relevant context for a query
async retrieveContext(query, k = 3) {
console.log(`πŸ” Searching for: "${query}"`);
// Generate embedding for the query
const response = await this.openai.embeddings.create({
model: 'text-embedding-ada-002',
input: query
});
// Search for similar documents
const results = await this.db.search({
vector: new Float32Array(response.data[0].embedding),
k: k,
threshold: 0.7 // Only use highly relevant results
});
console.log(`πŸ“„ Found ${results.length} relevant documents\n`);
return results.map(r => ({
content: r.metadata.content,
title: r.metadata.title,
score: r.score
}));
}
// Step 3: Generate answer with retrieved context
async answer(question) {
// Retrieve relevant context
const context = await this.retrieveContext(question, 3);
if (context.length === 0) {
return "I don't have enough information to answer that question.";
}
// Build prompt with context
const contextText = context
.map((doc, i) => `[${i + 1}] ${doc.title}\n${doc.content}`)
.join('\n\n');
const prompt = `Answer the question based on the following context. If the context doesn't contain the answer, say so.
Context:
${contextText}
Question: ${question}
Answer:`;
console.log('πŸ€– Generating answer...\n');
// Generate completion
const completion = await this.openai.chat.completions.create({
model: 'gpt-4',
messages: [
{ role: 'system', content: 'You are a helpful assistant that answers questions based on provided context.' },
{ role: 'user', content: prompt }
],
temperature: 0.3 // Lower temperature for more factual responses
});
return {
answer: completion.choices[0].message.content,
sources: context.map(c => c.title)
};
}
}
// Example Usage
async function main() {
const rag = new RAGSystem();
// Step 1: Index your knowledge base
const documents = [
{
id: 'doc1',
title: 'Ruvector Introduction',
content: 'Ruvector is a high-performance vector database for Node.js built in Rust. It provides sub-millisecond query latency and supports over 52,000 inserts per second.',
source: 'documentation'
},
{
id: 'doc2',
title: 'Vector Databases Explained',
content: 'Vector databases store data as high-dimensional vectors, enabling semantic similarity search. They are essential for AI applications like RAG systems and recommendation engines.',
source: 'blog'
},
{
id: 'doc3',
title: 'HNSW Algorithm',
content: 'Hierarchical Navigable Small World (HNSW) is a graph-based algorithm for approximate nearest neighbor search. It provides excellent recall with low latency.',
source: 'research'
}
];
await rag.indexDocuments(documents);
// Step 2: Ask questions
console.log('\n' + '='.repeat(60) + '\n');
const result = await rag.answer('What is Ruvector and what are its performance characteristics?');
console.log('πŸ“ Answer:', result.answer);
console.log('\nπŸ“š Sources:', result.sources.join(', '));
}
main().catch(console.error);
```
**Expected Output:**
```
βœ… RAG System initialized
πŸ“š Indexing 3 documents...
βœ… Indexed: Ruvector Introduction
βœ… Indexed: Vector Databases Explained
βœ… Indexed: HNSW Algorithm
βœ… Indexed 3 documents total
============================================================
πŸ” Searching for: "What is Ruvector and what are its performance characteristics?"
πŸ“„ Found 2 relevant documents
πŸ€– Generating answer...
πŸ“ Answer: Ruvector is a high-performance vector database built in Rust for Node.js applications. Its key performance characteristics include:
- Sub-millisecond query latency
- Over 52,000 inserts per second
- Optimized for semantic similarity search
πŸ“š Sources: Ruvector Introduction, Vector Databases Explained
```
**Production Tips:**
- βœ… Use batch embedding for better throughput (OpenAI supports up to 2048 texts)
- βœ… Implement caching for frequently asked questions
- βœ… Add error handling for API rate limits
- βœ… Monitor token usage and costs
- βœ… Regularly update your knowledge base
---
### Tutorial 2: Semantic Search Engine
**What you'll learn:** Build a semantic search engine that understands meaning, not just keywords.
**Prerequisites:**
```bash
npm install ruvector @xenova/transformers
```
**Complete Implementation:**
```javascript
const { VectorDb } = require('ruvector');
const { pipeline } = require('@xenova/transformers');
class SemanticSearchEngine {
constructor() {
this.db = null;
this.embedder = null;
}
// Step 1: Initialize the embedding model
async initialize() {
console.log('πŸš€ Initializing semantic search engine...');
// Load sentence-transformers model (runs locally, no API needed!)
console.log('πŸ“₯ Loading embedding model...');
this.embedder = await pipeline(
'feature-extraction',
'Xenova/all-MiniLM-L6-v2'
);
// Create vector database (384 dimensions for all-MiniLM-L6-v2)
this.db = new VectorDb({
dimensions: 384,
maxElements: 50000,
storagePath: './semantic-search.db'
});
console.log('βœ… Search engine ready!\n');
}
// Step 2: Generate embeddings
async embed(text) {
const output = await this.embedder(text, {
pooling: 'mean',
normalize: true
});
// Convert to Float32Array
return new Float32Array(output.data);
}
// Step 3: Index documents
async indexDocuments(documents) {
console.log(`πŸ“š Indexing ${documents.length} documents...`);
for (const doc of documents) {
const vector = await this.embed(doc.content);
await this.db.insert({
id: doc.id,
vector: vector,
metadata: {
title: doc.title,
content: doc.content,
category: doc.category,
url: doc.url
}
});
console.log(` βœ… ${doc.title}`);
}
const count = await this.db.len();
console.log(`\nβœ… Indexed ${count} documents\n`);
}
// Step 4: Semantic search
async search(query, options = {}) {
const {
k = 5,
category = null,
threshold = 0.3
} = options;
console.log(`πŸ” Searching for: "${query}"`);
// Generate query embedding
const queryVector = await this.embed(query);
// Search vector database
const results = await this.db.search({
vector: queryVector,
k: k * 2, // Get more results for filtering
threshold: threshold
});
// Filter by category if specified
let filtered = results;
if (category) {
filtered = results.filter(r => r.metadata.category === category);
}
// Return top k after filtering
const final = filtered.slice(0, k);
console.log(`πŸ“„ Found ${final.length} results\n`);
return final.map(r => ({
id: r.id,
title: r.metadata.title,
content: r.metadata.content,
category: r.metadata.category,
score: r.score,
url: r.metadata.url
}));
}
// Step 5: Find similar documents
async findSimilar(documentId, k = 5) {
const doc = await this.db.get(documentId);
if (!doc) {
throw new Error(`Document ${documentId} not found`);
}
const results = await this.db.search({
vector: doc.vector,
k: k + 1 // +1 because the document itself will be included
});
// Remove the document itself from results
return results
.filter(r => r.id !== documentId)
.slice(0, k);
}
}
// Example Usage
async function main() {
const engine = new SemanticSearchEngine();
await engine.initialize();
// Sample documents (in production, load from your database)
const documents = [
{
id: '1',
title: 'Understanding Neural Networks',
content: 'Neural networks are computing systems inspired by biological neural networks. They learn to perform tasks by considering examples.',
category: 'AI',
url: '/docs/neural-networks'
},
{
id: '2',
title: 'Introduction to Machine Learning',
content: 'Machine learning is a subset of artificial intelligence that provides systems the ability to learn and improve from experience.',
category: 'AI',
url: '/docs/machine-learning'
},
{
id: '3',
title: 'Web Development Best Practices',
content: 'Modern web development involves responsive design, performance optimization, and accessibility considerations.',
category: 'Web',
url: '/docs/web-dev'
},
{
id: '4',
title: 'Deep Learning Applications',
content: 'Deep learning has revolutionized computer vision, natural language processing, and speech recognition.',
category: 'AI',
url: '/docs/deep-learning'
}
];
// Index documents
await engine.indexDocuments(documents);
// Example 1: Basic semantic search
console.log('Example 1: Basic Search\n' + '='.repeat(60));
const results1 = await engine.search('AI and neural nets');
results1.forEach((result, i) => {
console.log(`${i + 1}. ${result.title} (Score: ${result.score.toFixed(3)})`);
console.log(` ${result.content.slice(0, 80)}...`);
console.log(` Category: ${result.category}\n`);
});
// Example 2: Category-filtered search
console.log('\nExample 2: Category-Filtered Search\n' + '='.repeat(60));
const results2 = await engine.search('learning algorithms', {
category: 'AI',
k: 3
});
results2.forEach((result, i) => {
console.log(`${i + 1}. ${result.title} (Score: ${result.score.toFixed(3)})`);
});
// Example 3: Find similar documents
console.log('\n\nExample 3: Find Similar Documents\n' + '='.repeat(60));
const similar = await engine.findSimilar('1', 2);
console.log('Documents similar to "Understanding Neural Networks":');
similar.forEach((doc, i) => {
console.log(`${i + 1}. ${doc.metadata.title} (Score: ${doc.score.toFixed(3)})`);
});
}
main().catch(console.error);
```
**Key Features:**
- βœ… Runs completely locally (no API keys needed)
- βœ… Understands semantic meaning, not just keywords
- βœ… Category filtering for better results
- βœ… "Find similar" functionality
- βœ… Fast: ~10ms query latency
---
### Tutorial 3: AI Agent Memory System
**What you'll learn:** Implement a memory system for AI agents that remembers past experiences and learns from them.
**Complete Implementation:**
```javascript
const { VectorDb } = require('ruvector');
class AgentMemory {
constructor(agentId) {
this.agentId = agentId;
// Create separate databases for different memory types
this.episodicMemory = new VectorDb({
dimensions: 768,
storagePath: `./memory/${agentId}-episodic.db`
});
this.semanticMemory = new VectorDb({
dimensions: 768,
storagePath: `./memory/${agentId}-semantic.db`
});
console.log(`🧠 Memory system initialized for agent: ${agentId}`);
}
// Step 1: Store an experience (episodic memory)
async storeExperience(experience) {
const {
state,
action,
result,
reward,
embedding
} = experience;
const experienceId = `exp_${Date.now()}_${Math.random()}`;
await this.episodicMemory.insert({
id: experienceId,
vector: new Float32Array(embedding),
metadata: {
state: state,
action: action,
result: result,
reward: reward,
timestamp: Date.now(),
type: 'episodic'
}
});
console.log(`πŸ’Ύ Stored experience: ${action} -> ${result} (reward: ${reward})`);
return experienceId;
}
// Step 2: Store learned knowledge (semantic memory)
async storeKnowledge(knowledge) {
const {
concept,
description,
embedding,
confidence = 1.0
} = knowledge;
const knowledgeId = `know_${Date.now()}`;
await this.semanticMemory.insert({
id: knowledgeId,
vector: new Float32Array(embedding),
metadata: {
concept: concept,
description: description,
confidence: confidence,
learned: Date.now(),
uses: 0,
type: 'semantic'
}
});
console.log(`πŸ“š Learned: ${concept}`);
return knowledgeId;
}
// Step 3: Recall similar experiences
async recallExperiences(currentState, k = 5) {
console.log(`πŸ” Recalling similar experiences...`);
const results = await this.episodicMemory.search({
vector: new Float32Array(currentState.embedding),
k: k,
threshold: 0.6 // Only recall reasonably similar experiences
});
// Sort by reward to prioritize successful experiences
const sorted = results.sort((a, b) => b.metadata.reward - a.metadata.reward);
console.log(`πŸ“ Recalled ${sorted.length} relevant experiences`);
return sorted.map(r => ({
state: r.metadata.state,
action: r.metadata.action,
result: r.metadata.result,
reward: r.metadata.reward,
similarity: r.score
}));
}
// Step 4: Query knowledge base
async queryKnowledge(query, k = 3) {
const results = await this.semanticMemory.search({
vector: new Float32Array(query.embedding),
k: k
});
// Update usage statistics
for (const result of results) {
const knowledge = await this.semanticMemory.get(result.id);
if (knowledge) {
knowledge.metadata.uses += 1;
// In production, update the entry
}
}
return results.map(r => ({
concept: r.metadata.concept,
description: r.metadata.description,
confidence: r.metadata.confidence,
relevance: r.score
}));
}
// Step 5: Reflect and learn from experiences
async reflect() {
console.log('\nπŸ€” Reflecting on experiences...');
// Get all experiences
const totalExperiences = await this.episodicMemory.len();
console.log(`πŸ“Š Total experiences: ${totalExperiences}`);
// Analyze success rate
// In production, you'd aggregate experiences and extract patterns
console.log('πŸ’‘ Analysis complete');
return {
totalExperiences: totalExperiences,
knowledgeItems: await this.semanticMemory.len()
};
}
// Step 6: Get memory statistics
async getStats() {
return {
episodicMemorySize: await this.episodicMemory.len(),
semanticMemorySize: await this.semanticMemory.len(),
agentId: this.agentId
};
}
}
// Example Usage: Simulated agent learning to navigate
async function main() {
const agent = new AgentMemory('agent-001');
// Simulate embedding function (in production, use a real model)
function embed(text) {
return Array(768).fill(0).map(() => Math.random());
}
console.log('\n' + '='.repeat(60));
console.log('PHASE 1: Learning from experiences');
console.log('='.repeat(60) + '\n');
// Store some experiences
await agent.storeExperience({
state: { location: 'room1', goal: 'room3' },
action: 'move_north',
result: 'reached room2',
reward: 0.5,
embedding: embed('navigating from room1 to room2')
});
await agent.storeExperience({
state: { location: 'room2', goal: 'room3' },
action: 'move_east',
result: 'reached room3',
reward: 1.0,
embedding: embed('navigating from room2 to room3')
});
await agent.storeExperience({
state: { location: 'room1', goal: 'room3' },
action: 'move_south',
result: 'hit wall',
reward: -0.5,
embedding: embed('failed navigation attempt')
});
// Store learned knowledge
await agent.storeKnowledge({
concept: 'navigation_strategy',
description: 'Moving north then east is efficient for reaching room3 from room1',
embedding: embed('navigation strategy knowledge'),
confidence: 0.9
});
console.log('\n' + '='.repeat(60));
console.log('PHASE 2: Applying memory');
console.log('='.repeat(60) + '\n');
// Agent encounters a similar situation
const currentState = {
location: 'room1',
goal: 'room3',
embedding: embed('navigating from room1 to room3')
};
// Recall relevant experiences
const experiences = await agent.recallExperiences(currentState, 3);
console.log('\nπŸ“– Recalled experiences:');
experiences.forEach((exp, i) => {
console.log(`${i + 1}. Action: ${exp.action} | Result: ${exp.result} | Reward: ${exp.reward} | Similarity: ${exp.similarity.toFixed(3)}`);
});
// Query relevant knowledge
const knowledge = await agent.queryKnowledge({
embedding: embed('how to navigate efficiently')
}, 2);
console.log('\nπŸ“š Relevant knowledge:');
knowledge.forEach((k, i) => {
console.log(`${i + 1}. ${k.concept}: ${k.description} (confidence: ${k.confidence})`);
});
console.log('\n' + '='.repeat(60));
console.log('PHASE 3: Reflection');
console.log('='.repeat(60) + '\n');
// Reflect on learning
const stats = await agent.reflect();
const memoryStats = await agent.getStats();
console.log('\nπŸ“Š Memory Statistics:');
console.log(` Episodic memories: ${memoryStats.episodicMemorySize}`);
console.log(` Semantic knowledge: ${memoryStats.semanticMemorySize}`);
console.log(` Agent ID: ${memoryStats.agentId}`);
}
main().catch(console.error);
```
**Expected Output:**
```
🧠 Memory system initialized for agent: agent-001
============================================================
PHASE 1: Learning from experiences
============================================================
πŸ’Ύ Stored experience: move_north -> reached room2 (reward: 0.5)
πŸ’Ύ Stored experience: move_east -> reached room3 (reward: 1.0)
πŸ’Ύ Stored experience: move_south -> hit wall (reward: -0.5)
πŸ“š Learned: navigation_strategy
============================================================
PHASE 2: Applying memory
============================================================
πŸ” Recalling similar experiences...
πŸ“ Recalled 3 relevant experiences
πŸ“– Recalled experiences:
1. Action: move_east | Result: reached room3 | Reward: 1.0 | Similarity: 0.892
2. Action: move_north | Result: reached room2 | Reward: 0.5 | Similarity: 0.876
3. Action: move_south | Result: hit wall | Reward: -0.5 | Similarity: 0.654
πŸ“š Relevant knowledge:
1. navigation_strategy: Moving north then east is efficient for reaching room3 from room1 (confidence: 0.9)
============================================================
PHASE 3: Reflection
============================================================
πŸ€” Reflecting on experiences...
πŸ“Š Total experiences: 3
πŸ’‘ Analysis complete
πŸ“Š Memory Statistics:
Episodic memories: 3
Semantic knowledge: 1
Agent ID: agent-001
```
**Use Cases:**
- βœ… Reinforcement learning agents
- βœ… Chatbot conversation history
- βœ… Game AI that learns from gameplay
- βœ… Personal assistant memory
- βœ… Robotic navigation systems
## πŸ—οΈ API Reference
### Constructor
```typescript
new VectorDb(options: {
dimensions: number; // Vector dimensionality (required)
maxElements?: number; // Max vectors (default: 10000)
storagePath?: string; // Persistent storage path
ef_construction?: number; // HNSW construction parameter (default: 200)
m?: number; // HNSW M parameter (default: 16)
distanceMetric?: string; // 'cosine', 'euclidean', or 'dot' (default: 'cosine')
})
```
### Methods
#### insert(entry: VectorEntry): Promise<string>
Insert a vector into the database.
```javascript
const id = await db.insert({
id: 'doc_1',
vector: new Float32Array([0.1, 0.2, 0.3, ...]),
metadata: { title: 'Document 1' }
});
```
#### search(query: SearchQuery): Promise<SearchResult[]>
Search for similar vectors.
```javascript
const results = await db.search({
vector: new Float32Array([0.1, 0.2, 0.3, ...]),
k: 10,
threshold: 0.7
});
```
#### get(id: string): Promise<VectorEntry | null>
Retrieve a vector by ID.
```javascript
const entry = await db.get('doc_1');
if (entry) {
console.log(entry.vector, entry.metadata);
}
```
#### delete(id: string): Promise<boolean>
Remove a vector from the database.
```javascript
const deleted = await db.delete('doc_1');
console.log(deleted ? 'Deleted' : 'Not found');
```
#### len(): Promise<number>
Get the total number of vectors.
```javascript
const count = await db.len();
console.log(`Total vectors: ${count}`);
```
## 🎨 Advanced Configuration
### HNSW Parameters
```javascript
const db = new VectorDb({
dimensions: 384,
maxElements: 1000000,
ef_construction: 200, // Higher = better recall, slower build
m: 16, // Higher = better recall, more memory
storagePath: './large-db.db'
});
```
**Parameter Guidelines:**
- `ef_construction`: 100-400 (higher = better recall, slower indexing)
- `m`: 8-64 (higher = better recall, more memory)
- Default values work well for most use cases
### Distance Metrics
```javascript
// Cosine similarity (default, best for normalized vectors)
const db1 = new VectorDb({
dimensions: 128,
distanceMetric: 'cosine'
});
// Euclidean distance (L2, best for spatial data)
const db2 = new VectorDb({
dimensions: 128,
distanceMetric: 'euclidean'
});
// Dot product (best for pre-normalized vectors)
const db3 = new VectorDb({
dimensions: 128,
distanceMetric: 'dot'
});
```
### Persistence
```javascript
// Auto-save to disk
const persistent = new VectorDb({
dimensions: 128,
storagePath: './persistent.db'
});
// In-memory only (faster, but data lost on exit)
const temporary = new VectorDb({
dimensions: 128
// No storagePath = in-memory
});
```
## πŸ“¦ Platform Support
Automatically installs the correct implementation for:
### Native (Rust) - Best Performance
- **Linux**: x64, ARM64 (GNU libc)
- **macOS**: x64 (Intel), ARM64 (Apple Silicon)
- **Windows**: x64 (MSVC)
Performance: **<0.5ms latency**, **50K+ ops/sec**
### WASM Fallback - Universal Compatibility
- Any platform where native module isn't available
- Browser environments (experimental)
- Alpine Linux (musl) and other non-glibc systems
Performance: **10-50ms latency**, **~1K ops/sec**
**Node.js 18+ required** for all platforms.
## πŸ”§ Building from Source
If you need to rebuild the native module:
```bash
# Install Rust toolchain
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
# Clone repository
git clone https://github.com/ruvnet/ruvector.git
cd ruvector
# Build native module
cd npm/packages/core
npm run build:napi
# Build wrapper package
cd ../ruvector
npm install
npm run build
# Run tests
npm test
```
**Requirements:**
- Rust 1.77+
- Node.js 18+
- Cargo
## 🌍 Ecosystem
### Related Packages
- **[ruvector-core](https://www.npmjs.com/package/ruvector-core)** - Core native bindings (lower-level API)
- **[ruvector-wasm](https://www.npmjs.com/package/ruvector-wasm)** - WebAssembly implementation for browsers
- **[ruvector-cli](https://www.npmjs.com/package/ruvector-cli)** - Standalone CLI tools
- **[@ruvector/rvf](https://www.npmjs.com/package/@ruvector/rvf)** - RVF cognitive container SDK
- **[@ruvector/rvf-wasm](https://www.npmjs.com/package/@ruvector/rvf-wasm)** - RVF WASM build for browsers, Deno, and edge
- **[rvlite](https://www.npmjs.com/package/rvlite)** - Lightweight vector database with SQL, SPARQL, and Cypher
### Platform-Specific Packages (auto-installed)
- **[ruvector-core-linux-x64-gnu](https://www.npmjs.com/package/ruvector-core-linux-x64-gnu)**
- **[ruvector-core-linux-arm64-gnu](https://www.npmjs.com/package/ruvector-core-linux-arm64-gnu)**
- **[ruvector-core-darwin-x64](https://www.npmjs.com/package/ruvector-core-darwin-x64)**
- **[ruvector-core-darwin-arm64](https://www.npmjs.com/package/ruvector-core-darwin-arm64)**
- **[ruvector-core-win32-x64-msvc](https://www.npmjs.com/package/ruvector-core-win32-x64-msvc)**
---
## RVF Cognitive Containers
Ruvector integrates with [RVF (RuVector Format)](https://github.com/ruvnet/ruvector/tree/main/crates/rvf) β€” a universal binary substrate that stores vectors, models, graphs, compute kernels, and attestation in a single `.rvf` file.
### Enable RVF Backend
```bash
# Install the optional RVF package
npm install @ruvector/rvf
# Set backend via environment variable
export RUVECTOR_BACKEND=rvf
# Or detect automatically (native -> rvf -> wasm fallback)
npx ruvector info
```
```typescript
import { getImplementationType, isRvf } from 'ruvector';
console.log(getImplementationType()); // 'native' | 'rvf' | 'wasm'
console.log(isRvf()); // true if RVF backend is active
```
### RVF CLI Commands
8 RVF-specific subcommands are available through the ruvector CLI:
```bash
# Create an RVF store
npx ruvector rvf create mydb.rvf -d 384 --metric cosine
# Ingest vectors from JSON
npx ruvector rvf ingest mydb.rvf --input vectors.json --format json
# Query nearest neighbors
npx ruvector rvf query mydb.rvf --vector "[0.1,0.2,...]" --k 10
# File status and segment listing
npx ruvector rvf status mydb.rvf
npx ruvector rvf segments mydb.rvf
# COW branching β€” derive a child file
npx ruvector rvf derive mydb.rvf --output child.rvf
# Compact and reclaim space
npx ruvector rvf compact mydb.rvf
# Export to JSON
npx ruvector rvf export mydb.rvf --output dump.json
```
### RVF Platform Support
| Platform | Runtime | Backend |
|----------|---------|---------|
| Linux x86_64 / aarch64 | Node.js 18+ | Native (N-API) |
| macOS x86_64 / arm64 | Node.js 18+ | Native (N-API) |
| Windows x86_64 | Node.js 18+ | Native (N-API) |
| Any | Deno | WASM (`@ruvector/rvf-wasm`) |
| Any | Browser | WASM (`@ruvector/rvf-wasm`) |
| Any | Cloudflare Workers | WASM (`@ruvector/rvf-wasm`) |
### Download Example .rvf Files
45 pre-built example files are available (~11 MB total):
```bash
# Download a specific example
curl -LO https://raw.githubusercontent.com/ruvnet/ruvector/main/examples/rvf/output/basic_store.rvf
# Popular examples:
# basic_store.rvf (152 KB) β€” 1,000 vectors, dim 128
# semantic_search.rvf (755 KB) β€” Semantic search with HNSW
# rag_pipeline.rvf (303 KB) β€” RAG pipeline embeddings
# agent_memory.rvf (32 KB) β€” AI agent memory store
# self_booting.rvf (31 KB) β€” Self-booting with kernel
# progressive_index.rvf (2.5 MB) β€” Large-scale HNSW index
# Generate all examples locally
cd crates/rvf && cargo run --example generate_all
```
Full catalog: [examples/rvf/output/](https://github.com/ruvnet/ruvector/tree/main/examples/rvf/output)
### Working Examples: Cognitive Containers
#### Self-Booting Microservice
A single `.rvf` file that contains vectors AND a bootable Linux kernel:
```bash
# Build and run the self-booting example
cd crates/rvf && cargo run --example self_booting
# Output:
# Ingested 50 vectors (128 dims)
# Pre-kernel query: top-5 results OK (nearest ID=25)
# Kernel: 4,640 bytes embedded (x86_64, Hermit)
# Witness chain: 5 entries, all verified
# File: bootable.rvf (31 KB) β€” data + runtime in one file
```
```rust
// The pattern: vectors + kernel + witness in one file
let mut store = RvfStore::create("bootable.rvf", options)?;
store.ingest_batch(&vectors, &ids, None)?;
store.embed_kernel(KernelArch::X86_64 as u8, KernelType::Hermit as u8,
0x0018, &kernel_image, 8080, Some("console=ttyS0 quiet"))?;
// Result: drop on a VM and it boots as a query service
```
#### Linux Microkernel Distribution
20-package Linux distro with SSH keys and kernel in a single file:
```bash
cd crates/rvf && cargo run --example linux_microkernel
# Output:
# Installed 20 packages as vector embeddings
# Kernel embedded: Linux x86_64 (4,640 bytes)
# SSH keys: Ed25519, signed and verified
# Witness chain: 22 entries (1 per package + kernel + SSH)
# File: microkernel.rvf (14 KB) β€” immutable bootable system
```
Features: package search by embedding similarity, Ed25519 signed SSH keys, witness-audited installs, COW-derived child images for atomic updates.
#### Claude Code AI Appliance
A sealed, bootable AI development environment:
```bash
cd crates/rvf && cargo run --example claude_code_appliance
# Output:
# 20 dev packages (rust, node, python, docker, ...)
# Kernel: Linux x86_64 with SSH on port 2222
# eBPF: XDP distance program for fast-path lookups
# Witness chain: 6 entries, all verified
# Crypto: Ed25519 signature
# File: claude_code_appliance.rvf (17 KB)
```
#### CLI Full Lifecycle
```bash
# Create β†’ Ingest β†’ Query β†’ Derive β†’ Inspect
rvf create vectors.rvf --dimension 384
rvf ingest vectors.rvf --input data.json --format json
rvf query vectors.rvf --vector "0.1,0.2,..." --k 10
rvf derive vectors.rvf child.rvf --type filter
rvf inspect vectors.rvf
# Embed kernel and launch as microVM
rvf embed-kernel vectors.rvf --image bzImage
rvf launch vectors.rvf --port 8080
# Verify tamper-evident witness chain
rvf verify-witness vectors.rvf
rvf verify-attestation vectors.rvf
```
#### Integration Tests (46 passing)
```bash
cd crates/rvf
cargo test --workspace
# attestation .............. 6 passed
# crypto ................... 10 passed
# computational_container .. 8 passed
# cow_branching ............ 8 passed
# cross_platform ........... 6 passed
# lineage .................. 4 passed
# smoke .................... 4 passed
# Total: 46/46 passed
```
## πŸ› Troubleshooting
### Native Module Not Loading
If you see "Cannot find module 'ruvector-core-*'":
```bash
# Reinstall with optional dependencies
npm install --include=optional ruvector
# Verify platform
npx ruvector info
# Check Node.js version (18+ required)
node --version
```
### WASM Fallback Performance
If you're using WASM fallback and need better performance:
1. **Install native toolchain** for your platform
2. **Rebuild native module**: `npm rebuild ruvector`
3. **Verify native**: `npx ruvector info` should show "native (Rust)"
### Platform Compatibility
- **Alpine Linux**: Uses WASM fallback (musl not supported)
- **Windows ARM**: Not yet supported, uses WASM fallback
- **Node.js < 18**: Not supported, upgrade to Node.js 18+
## πŸ“š Documentation
- 🏠 [Homepage](https://ruv.io)
- πŸ“¦ [GitHub Repository](https://github.com/ruvnet/ruvector)
- πŸ“š [Full Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)
- πŸš€ [Getting Started Guide](https://github.com/ruvnet/ruvector/blob/main/docs/guide/GETTING_STARTED.md)
- πŸ“– [API Reference](https://github.com/ruvnet/ruvector/blob/main/docs/api/NODEJS_API.md)
- 🎯 [Performance Tuning](https://github.com/ruvnet/ruvector/blob/main/docs/optimization/PERFORMANCE_TUNING_GUIDE.md)
- πŸ› [Issue Tracker](https://github.com/ruvnet/ruvector/issues)
- πŸ’¬ [Discussions](https://github.com/ruvnet/ruvector/discussions)
## 🀝 Contributing
We welcome contributions! See [CONTRIBUTING.md](https://github.com/ruvnet/ruvector/blob/main/docs/development/CONTRIBUTING.md) for guidelines.
### Quick Start
1. Fork the repository
2. Create a feature branch: `git checkout -b feature/amazing-feature`
3. Commit changes: `git commit -m 'Add amazing feature'`
4. Push to branch: `git push origin feature/amazing-feature`
5. Open a Pull Request
## 🌐 Community & Support
- **GitHub**: [github.com/ruvnet/ruvector](https://github.com/ruvnet/ruvector) - ⭐ Star and follow
- **Discord**: [Join our community](https://discord.gg/ruvnet) - Chat with developers
- **Twitter**: [@ruvnet](https://twitter.com/ruvnet) - Follow for updates
- **Issues**: [Report bugs](https://github.com/ruvnet/ruvector/issues)
### Enterprise Support
Need custom development or consulting?
πŸ“§ [enterprise@ruv.io](mailto:enterprise@ruv.io)
## πŸ“œ License
**MIT License** - see [LICENSE](https://github.com/ruvnet/ruvector/blob/main/LICENSE) for details.
Free for commercial and personal use.
## πŸ™ Acknowledgments
Built with battle-tested technologies:
- **HNSW**: Hierarchical Navigable Small World graphs
- **SIMD**: Hardware-accelerated vector operations via simsimd
- **Rust**: Memory-safe, zero-cost abstractions
- **NAPI-RS**: High-performance Node.js bindings
- **WebAssembly**: Universal browser compatibility
---
<div align="center">
**Built with ❀️ by [rUv](https://ruv.io)**
[![npm](https://img.shields.io/npm/v/ruvector.svg)](https://www.npmjs.com/package/ruvector)
[![GitHub Stars](https://img.shields.io/github/stars/ruvnet/ruvector?style=social)](https://github.com/ruvnet/ruvector)
[![Twitter](https://img.shields.io/twitter/follow/ruvnet?style=social)](https://twitter.com/ruvnet)
**[Get Started](https://github.com/ruvnet/ruvector/blob/main/docs/guide/GETTING_STARTED.md)** β€’ **[Documentation](https://github.com/ruvnet/ruvector/tree/main/docs)** β€’ **[API Reference](https://github.com/ruvnet/ruvector/blob/main/docs/api/NODEJS_API.md)** β€’ **[Contributing](https://github.com/ruvnet/ruvector/blob/main/docs/development/CONTRIBUTING.md)**
</div>