🔍 RAG Visualization Dashboard

RAG Categories Distribution

RAG Types by Year

📅 Evolution Timeline: Year → RAG Type

RAG Workflow Flowcharts

Chunking Methods Distribution

Chunking Categories & Methods

📅 Chunking Methods Timeline

Chunking Method Flowcharts

🧮 Indexing Techniques (Algorithmic Level)

Algorithmic methods for organizing and searching vectors efficiently. These are the core algorithms that determine how vectors are structured and retrieved (e.g., Flat Index, ANN, IVF, HNSW, PQ/OPQ).

💾 Vector Databases (System Level)

Complete systems/engines that implement indexing techniques along with additional features like persistence, metadata support, distributed architecture, and APIs. They are the production-grade platforms (e.g., FAISS, Milvus, Pinecone, Weaviate).

Indexing Technique Description Workflow Pros Cons Peer-Reviewed Reference
Flat Index (Brute-Force) Exact similarity search over all vectors using distance metrics. Embed documents → store vectors → compare query with all vectors → return top-k Exact results
Simple implementation		 Not scalable
High latency & memory cost		 📄Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W., Rocktäschel, T., Riedel, S., & Kiela, D. (2020). Retrieval‑augmented generation for knowledge‑intensive NLP tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, & H. T. Lin (Eds.), Advances in Neural Information Processing Systems, 33, 9459–9474. Curran Associates, Inc.
ANN (Approximate Nearest Neighbor) Fast similarity search with controlled approximation. Embed documents → build ANN structure → retrieve approximate neighbors Fast retrieval
Scales to millions/billions		 Approximate (not exact) 📄Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval‑Augmented Generation for Large Language Models: A Survey (arXiv:2312.10997). arXiv Preprint. https://doi.org/10.48550/arXiv.2312.10997
IVF (Inverted File Index) Partitions vector space into clusters; searches only relevant partitions. K-means clustering → assign vectors → query nearest clusters → search inside Good speed-accuracy trade-off
Scalable		 Requires tuning
Cluster-quality sensitive		 📄Johnson, J., Douze, M., & Jégou, H. (2019). Billion‑scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3), 535–547. https://doi.org/10.1109/TBDATA.2019.2921572
HNSW Graph-based ANN using hierarchical proximity graphs. Build layered graph → incremental insertion → top-down graph traversal Very fast
High recall
Dynamic updates		 High memory usage
Complex construction		 📄Malkov, Y. A., & Yashunin, D. A. (2020). Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4), 824–836. https://doi.org/10.1109/TPAMI.2018.2889473
PQ / OPQ Compresses vectors into short codes for memory-efficient ANN. Subspace split → quantization → store codes → approximate distance Memory efficient
Enables billion-scale RAG		 Lossy compression
Lower recall if misconfigured		 📄Jégou, H., Douze, M., & Schmid, C. (2011). Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 117–128. https://doi.org/10.1109/TPAMI.2010.57
Vector DB Description Underlying Index Pros Cons Peer-Reviewed Reference
FAISS Research-oriented vector similarity library widely used in RAG. Flat, IVF, HNSW, PQ, OPQ Highly optimized
Flexible ANN
Academic standard		 Not a full DBMS
Limited metadata		 📄Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval‑Augmented Generation for Large Language Models: A Survey (arXiv:2312.10997). arXiv Preprint. https://doi.org/10.48550/arXiv.2312.10997
Milvus Distributed open-source vector database for large-scale RAG. HNSW, IVF-PQ Scalable
Distributed
Open-source		 Deployment complexity
Operational overhead		 📄Rusum, G. P., & Anasuri, S. (2025). Vector databases in modern applications: Real‑time search, recommendations, and retrieval‑augmented generation (RAG). International Journal of AI, BigData, Computational and Management Studies, 5(4), Article 113. https://doi.org/10.63282/3050‑9416.IJAIBDCMS‑V5I4P113
Pinecone Fully managed vector database for production RAG systems. Proprietary ANN (HNSW-like) Fully managed
High availability
Scalable		 Closed-source
Opaque indexing		 📄Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval‑Augmented Generation for Large Language Models: A Survey (arXiv:2312.10997). arXiv Preprint. https://doi.org/10.48550/arXiv.2312.10997
Weaviate Open-source vector DB with schema & metadata-aware retrieval. HNSW Simple API
Schema support		 High memory usage
Limited ANN tuning		 📄Rusum, G. P., & Anasuri, S. (2025). Vector databases in modern applications: Real‑time search, recommendations, and retrieval‑augmented generation (RAG). International Journal of AI, BigData, Computational and Management Studies, 5(4), Article 113. https://doi.org/10.63282/3050‑9416.IJAIBDCMS‑V5I4P113
Elasticsearch (Vector) Hybrid sparse-dense retrieval using BM25 + vectors. HNSW + BM25 Hybrid retrieval
Mature ecosystem		 Higher latency
Slower pure vector search		 📄Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval‑Augmented Generation for Large Language Models: A Survey (arXiv:2312.10997). arXiv Preprint. https://doi.org/10.48550/arXiv.2312.10997
Chroma Lightweight developer-focused vector store for prototyping. HNSW (ANN backends) Easy integration
Fast prototyping		 Limited scalability
Not enterprise-grade		 📄Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2024). Retrieval‑Augmented Generation for Large Language Models: A Survey (arXiv:2312.10997). arXiv Preprint. https://doi.org/10.48550/arXiv.2312.10997