htaf commited on
Commit
780e6e0
·
1 Parent(s): f927509

added key benefits

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -63,6 +63,16 @@ This structure mirrors the data format most distillation workflows expect.
63
  **Goal:**
64
  Make it easy for researchers and builders to create **high-quality domain-specific AI models**.
65
 
 
 
 
 
 
 
 
 
 
 
66
  ---
67
 
68
  ## 🔧 How It Works (At a Glance)
 
63
  **Goal:**
64
  Make it easy for researchers and builders to create **high-quality domain-specific AI models**.
65
 
66
+ ---
67
+ ### ⚡ Key Benefits: Speed, Simplicity, and Efficiency
68
+ Built entirely in Node.js, `distill_rag` leverages async promises and lightweight concurrency to deliver blazing-fast performance—often 5–10× faster than equivalent Python-based tools like LlamaIndex, LangChain, or Haystack. This makes it ideal for local workflows on consumer hardware, where you can process thousands of chunks in minutes without heavy dependencies or complex setups.
69
+
70
+ - **GPU-Bound Efficiency**: Embeddings (via Ollama) are the bottleneck, but everything else (extraction, chunking, indexing) is lightning-fast, with sustained rates of 30+ chunks/second on a single RTX 3090.
71
+ - **No Overhead Bloat**: Direct HTTP calls to embedding APIs and seamless parallelism mean low CPU/RAM usage and no GIL-like bottlenecks.
72
+ - **Hackable for JS Devs**: If you're in the Node ecosystem (e.g., integrating with web apps or CLIs), this fits like a glove—zero language switching required.
73
+ - **Benchmark Example**: On an RTX 3090 with 1531 sessions (~3565 chunks, 5000–9000 chars each), a full index rebuild takes just 1m48s (33 chunks/s). Comparable Python tools often take 8–40 minutes for similar workloads due to wrapper latencies and inefficient batching.
74
+
75
+ If speed and developer joy matter in your RAG/distillation pipeline, this toolkit shines.
76
  ---
77
 
78
  ## 🔧 How It Works (At a Glance)