Create README.md

README.md

@@ -0,0 +1,74 @@
+---
+tags:
+- vector-database
+- benchmarks
+- faiss
+- weaviate
+- chroma
+- multimodal
+- clip
+- retrieval
+license: apache-2.0
+---
+
+# Vector Database Benchmarks: FAISS vs Chroma vs Weaviate
+
+This repository contains experiments benchmarking popular vector databases on **multimodal embeddings** generated from the [Flickr8k dataset](https://huggingface.co/datasets/jxie/flickr8k).  
+We focused on four key evaluation dimensions:
+
+1. **Latency per query**  
+2. **Recall@5 vs Flat (accuracy tradeoffs)**  
+3. **Queries per second (QPS throughput)**  
+4. **Ingestion scaling performance**
+
+All experiments were run on **Google Colab** (T4 GPU for embedding generation, CPU backend for databases).  
+
+---
+
+## Methodology
+
+- Dataset: 6k images and 30k captions from Flickr8k.  
+- Embeddings: CLIP (OpenAI ViT-B/32).  
+- Workload: Caption-to-image retrieval (cross-modal).  
+- Baseline: FAISS Flat index used as the ground-truth for recall calculations.  
+
+Each vector database was tested under the same conditions for ingestion, search, and recall.  
+
+---
+
+## Results Summary
+
+| Metric                  | FAISS            | Chroma           | Weaviate         |
+|--------------------------|------------------|------------------|------------------|
+| **Avg Latency per Query** | 0.19 ms          | 0.76 ms          | 1.82 ms          |
+| **Recall@5 (Flat Baseline)** | 1.00             | 0.002            | 0.918            |
+| **QPS Throughput**       | 1929.94          | 719.01           | 598.40           |
+| **Ingestion Scaling (20k)** | 0.024s           | 2.806s           | 4.000s           |
+
+---
+
+## Key Takeaways
+
+- **FAISS** is fastest, leveraging in-memory array ingestion and customizable indexing strategies.  
+- **Chroma** offers simplicity and ease of integration but struggles at scale due to batching and internal constraints.  
+- **Weaviate** provides a more feature-rich ecosystem (schema, hybrid search, persistence) but at higher ingestion and query overhead.  
+
+At the million-vector scale, speed alone will not decide your choice; **engineering tradeoffs, developer productivity, and system features** will.  
+Benchmarks tell one part of the story, your use case tells the rest.  
+
+---
+
+## Usage
+
+You can reproduce these experiments using the provided notebook and Hugging Face dataset.  
+See full code here: [rag-experiments/VectorDB-Benchmarks](https://huggingface.co/rag-experiments/VectorDB-Benchmarks).
+Dataset used: Flickr8k (train split — 6k images, 30k captions, multimodal — images and text), CLIP Embeddings. Author: Johnathan Xie
+
+All tests run on Google Colab (T4 GPU, CPU backend for DBs).
+
+---
+
+## Citation
+
+If you find this useful, please cite this repository:
+