Spaces:
Running
Running
| title: LFM2.5 Embedding API | |
| emoji: 🧠 | |
| colorFrom: indigo | |
| colorTo: red | |
| sdk: docker | |
| app_port: 7860 | |
| pinned: true | |
| license: mit | |
| tags: | |
| - embeddings | |
| - openai-compatible | |
| - llama-cpp | |
| - cpu-only | |
| - rag | |
| - vector-search | |
| - semantic-search | |
| - retrieval | |
| - text-embedding | |
| - gguf | |
| - quantization | |
| - liquid-ai | |
| - lfm2.5 | |
| - fastapi | |
| - docker | |
| - inference | |
| - ai | |
| - machine-learning | |
| - nlp | |
| - search | |
| - similarity | |
| - cosine-similarity | |
| - knowledge-base | |
| - document-indexing | |
| - multilingual | |
| short_description: OpenAI-compatible embeddings API running LFM2.5 on CPU | |
| # 🧠 LFM2.5 Embedding API (CPU-Optimized) | |
|  | |
|  | |
|  | |
|  | |
|  | |
| An **ultra-lightweight** and **100% OpenAI-compatible** REST API for generating embeddings using the [LiquidAI/LFM2.5-Embedding-350M](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M) model, optimized to run on **pure CPU** (no GPU required) on Hugging Face Spaces Free tier. | |
| ## ⚡ Key Features | |
| - 🚀 **Fast cold start** (~3-5s) thanks to pre-loaded Q8_0 GGUF model | |
| - 💾 **Minimal RAM usage** (~500MB) — runs comfortably within HF Free's 16GB limit | |
| - 🎯 **Asymmetric embeddings** — supports `query:` and `document:` prefixes for maximum RAG precision | |
| - 🔐 **Bearer Token authentication** via HF Secrets | |
| - 🤖 **Strict OpenAI standard** — works with any OpenAI-compatible client | |
| - 🌍 **Multilingual support** — LFM2.5 handles 100+ languages | |
| - 📦 **Easy deployment** — clone and run in 5 minutes | |
| ## ⚠️ Performance & Limitations (IMPORTANT) | |
| ### Current Performance on HF Free Tier | |
| | Metric | Value | Notes | | |
| |--------|-------|-------| | |
| | **Cold Start** | 3-5 seconds | First request after inactivity | | |
| | **Inference Time** | 10-15 seconds | Per embedding request (50-500 words) | | |
| | **Throughput** | ~6-8 requests/minute | Sustained rate | | |
| | **Max Context** | 512 tokens | Optimal for embeddings | | |
| | **Dimensions** | 1024 floats | Per embedding vector | | |
| ### Why 10-15 Seconds? | |
| This API runs on **Hugging Face Spaces Free tier**, which uses **shared CPU resources**: | |
| - **CPU Throttling**: The HF hypervisor dynamically limits CPU cycles when multiple containers compete for resources | |
| - **No AVX2 Optimization**: While the code is compiled with AVX2 support, the free tier's virtualization layer doesn't fully expose these CPU instructions | |
| - **Shared Infrastructure**: Your container shares physical CPU cores with other users' Spaces | |
| ### When to Use This API | |
| ✅ **Perfect for:** | |
| - Prototyping and development | |
| - Small-scale RAG applications (<1000 documents) | |
| - Personal projects and experimentation | |
| - Batch processing with async queues | |
| - Learning and education | |
| - Backup/fallback embedding service | |
| ❌ **Not ideal for:** | |
| - Real-time user-facing search (latency too high) | |
| - High-throughput production systems (>100 req/min) | |
| - Applications requiring <1s response times | |
| - Critical infrastructure with SLA requirements | |
| ### 🚀 Need Better Performance? | |
| If you need faster inference, consider these alternatives: | |
| | Option | Latency | Cost | Setup Complexity | | |
| |--------|---------|------|------------------| | |
| | **This Space (HF Free)** | 10-15s | $0 | ⭐ Minimal | | |
| | **HF Space Paid (Basic)** | 2-5s | ~$0.60/h | ⭐ Minimal | | |
| | **Cloudflare Workers AI** | 50-200ms | $0 (10k neurons/day) | ⭐⭐ Low | | |
| | **OpenAI Embeddings** | 100-300ms | $0.0001/1K tokens | ⭐⭐ Low | | |
| | **Self-hosted (GPU)** | 10-50ms | Hardware cost | ⭐⭐⭐⭐ High | | |
| **Recommended Alternative**: [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) with `@cf/qwen/qwen3-embedding-0.6b` offers 50-200ms latency on their global edge network, with 10,000 free neurons/day (~18k requests/day for 500-token embeddings). | |
| ## 🔌 Universal Compatibility | |
| This API can be used as an embedding backend for **any tool** that supports OpenAI-compatible endpoints: | |
| | Tool | Works? | Notes | | |
| |------|--------|-------| | |
| | **OpenClaw** | ✅ Full | Use `queryInputType: "query"` and `documentInputType: "document"` | | |
| | **Open WebUI** | ✅ Full | Configure as "OpenAI API" in RAG Settings | | |
| | **LangChain** | ✅ Full | Use `OpenAIEmbeddings(openai_api_base=...)` | | |
| | **LlamaIndex** | ✅ Full | Use `OpenAIEmbedding(api_base=...)` | | |
| | **Cursor** | ✅ Full | Point `api_base` to this Space | | |
| | **Continue** | ✅ Full | Configure in `config.json` | | |
| | **Dify** | ✅ Full | Use "OpenAI Embeddings" node | | |
| | **Flowise** | ✅ Full | Use OpenAI Embeddings component | | |
| | **n8n** | ✅ Full | Use OpenAI node with custom base URL | | |
| | **Haystack** | ✅ Full | Use `OpenAIEmbedder` with `api_base_url` | | |
| | **Semantic Kernel** | ✅ Full | Configure OpenAI connector | | |
| | **AutoGen** | ✅ Full | Use OpenAI-compatible embedding model | | |
| | **CrewAI** | ✅ Full | Configure embedding provider | | |
| | **OpenAI SDK (Python)** | ✅ Full | Override `base_url` and `api_key` | | |
| | **OpenAI SDK (Node.js)** | ✅ Full | Override `baseURL` and `apiKey` | | |
| ## 🛠️ Quick Start (How to Clone and Use) | |
| ### 1. Duplicate this Space | |
| Click the **three dots** (⋮) in the top-right corner → **"Duplicate this Space"** → choose visibility (Public/Private). | |
| ### 2. Configure the Secret | |
| In the duplicated Space, go to **Settings → Variables and Secrets** and add: | |
| | Name | Value | | |
| |------|-------| | |
| | `API_KEY` | `your-secret-key-here` | | |
| *(Use a strong string with 32+ characters. Tip: Use [1Password's Password Generator](https://1password.com/password-generator) for cryptographically secure random passwords)* | |
| ### 3. Wait for Build | |
| The Dockerfile will compile `llama.cpp` and download the Q8_0 model automatically (~5-8 minutes on first build). | |
| ### 4. Test with cURL | |
| ```bash | |
| curl -X POST "https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/embeddings" \ | |
| -H "Authorization: Bearer YOUR_SECRET_KEY" \ | |
| -H "Content-Type: application/json" \ | |
| -d '{ | |
| "input": "Text to generate embedding for", | |
| "model": "LiquidAI/LFM2.5-Embedding-350M", | |
| "input_type": "document" | |
| }' | |
| ``` | |
| ## 📝 Usage Examples | |
| ### Python with OpenAI SDK | |
| ```python | |
| from openai import OpenAI | |
| client = OpenAI( | |
| api_key="YOUR_SECRET_KEY", | |
| base_url="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/" | |
| ) | |
| # For indexing documents (store in vector DB) | |
| doc_response = client.embeddings.create( | |
| model="LiquidAI/LFM2.5-Embedding-350M", | |
| input="OpenClaw is a multilingual RAG tool.", | |
| extra_body={"input_type": "document"} | |
| ) | |
| # For searching (user query) | |
| query_response = client.embeddings.create( | |
| model="LiquidAI/LFM2.5-Embedding-350M", | |
| input="How does vector search work?", | |
| extra_body={"input_type": "query"} | |
| ) | |
| print(query_response.data[0].embedding[:5]) # First 5 floats | |
| ``` | |
| ### LangChain Integration | |
| ```python | |
| from langchain_openai import OpenAIEmbeddings | |
| embeddings = OpenAIEmbeddings( | |
| model="LiquidAI/LFM2.5-Embedding-350M", | |
| openai_api_key="YOUR_SECRET_KEY", | |
| openai_api_base="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/" | |
| ) | |
| # Generate embeddings | |
| vectors = embeddings.embed_documents(["Text 1", "Text 2"]) | |
| query_vector = embeddings.embed_query("Search query") | |
| ``` | |
| ### LlamaIndex Integration | |
| ```python | |
| from llama_index.embeddings.openai import OpenAIEmbedding | |
| embed_model = OpenAIEmbedding( | |
| model="LiquidAI/LFM2.5-Embedding-350M", | |
| api_key="YOUR_SECRET_KEY", | |
| api_base="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/" | |
| ) | |
| embeddings = embed_model.get_text_embedding("Your text here") | |
| ``` | |
| ### JavaScript/Node.js | |
| ```javascript | |
| import OpenAI from 'openai'; | |
| const client = new OpenAI({ | |
| apiKey: 'YOUR_SECRET_KEY', | |
| baseURL: 'https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/' | |
| }); | |
| const response = await client.embeddings.create({ | |
| model: 'LiquidAI/LFM2.5-Embedding-350M', | |
| input: 'Your text here' | |
| }); | |
| console.log(response.data[0].embedding.slice(0, 5)); | |
| ``` | |
| ## 🏗️ Architecture | |
| ``` | |
| +-------------------------------------------------------+ | |
| | Hugging Face Space (2 vCPU / 16GB RAM) | | |
| | | | |
| | +------------------+ +------------------+ | | |
| | | | | llama.cpp (C++) | | | |
| | | Docker +-------->| + Q8_0 GGUF | | | |
| | | Container | | (~380MB RAM) | | | |
| | | | +------------------+ | | |
| | +--------+---------+ | | |
| | | | | |
| | v | | |
| | +------------------+ +------------------+ | | |
| | | FastAPI | | /v1/embeddings | | | |
| | | + Uvicorn +-------->| (OpenAI-compat.) | | | |
| | +------------------+ +------------------+ | | |
| +-------------------------------------------------------+ | |
| ``` | |
| - **Engine**: `llama-cpp-python` (Python wrapper for llama.cpp) | |
| - **Model**: `LFM2.5-Embedding-350M-Q8_0.gguf` (8-bit quantization for maximum precision) | |
| - **Framework**: FastAPI + Uvicorn (1 worker) | |
| - **Pooling**: Native CLS Token (LFM2.5 standard) | |
| - **Dimensions**: 1024 floats per embedding | |
| - **Context**: 512 tokens (optimal for embeddings) | |
| ## 🎯 Best Practices | |
| ### 1. Use Asymmetric Embeddings | |
| Always specify `input_type` for better RAG performance: | |
| - `"document"` when indexing/storing text | |
| - `"query"` when searching | |
| ### 2. Implement Chunking | |
| Break long documents into ~400-token chunks with 50-token overlap for optimal retrieval. | |
| ### 3. Batch Requests | |
| Send multiple texts in a single request when possible: | |
| ```python | |
| response = client.embeddings.create( | |
| input=["Text 1", "Text 2", "Text 3"], | |
| model="LiquidAI/LFM2.5-Embedding-350M" | |
| ) | |
| ``` | |
| ### 4. Set Appropriate Timeouts | |
| Configure your HTTP client with 30-second timeouts to handle cold starts. | |
| ### 5. Use Async Queues for Indexing | |
| For large document collections, implement background processing to avoid blocking user interactions. | |
| ## ⚖️ Licensing | |
| This repository contains **two distinct layers** with different licenses: | |
| ### 1. API Code (Infrastructure) — MIT License | |
| All Python code (FastAPI, Dockerfile, scripts) is licensed under the **MIT License**. You are free to clone, modify, use commercially, and distribute the API infrastructure. | |
| ### 2. AI Model (Weights and GGUF) — LFM Open License v1.0 | |
| The model weights are owned by **Liquid AI, Inc.** and licensed under the **[LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M/blob/main/LICENSE)**. | |
| #### ⚠️ CRITICAL: Commercial Use Threshold | |
| **Commercial use is PERMITTED only if your Legal Entity's total annual revenue does NOT exceed $10,000,000 USD (ten million US dollars).** | |
| If your entity exceeds this threshold, you **must obtain a separate commercial license** from Liquid AI, Inc. | |
| **What this means for you:** | |
| ✅ **PERMITTED:** | |
| - Using this API to generate embeddings for RAG applications | |
| - Integration in commercial products (under the $10M threshold) | |
| - Academic research and non-commercial projects | |
| - Personal projects and experimentation | |
| ⚠️ **REQUIRES COMMERCIAL LICENSE:** | |
| - Entities with >$10M annual revenue must contact [Liquid AI](https://www.liquid.ai) | |
| - Exceeding the threshold without a license results in **automatic termination** (Section 11) | |
| 🚫 **PROHIBITED:** | |
| - Using Liquid AI trademarks ("Liquid AI", "LFM", etc.) to promote your product | |
| - Patent litigation against Liquid AI (triggers license termination) | |
| - Use for training competing foundation models | |
| 📄 **Full License Text**: See the [`LICENSE`](./LICENSE) file in this repository or the [official license](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M/blob/main/LICENSE) on Hugging Face. | |
| ## 🙏 Credits | |
| This project was architected in collaboration with: | |
| - 🤖 **Qwen3.7 Max** — DevOps architecture, Docker/CPU optimization, llama.cpp integration, and build troubleshooting | |
| - 🤖 **Gemini 3.1 Pro** — Code review, asymmetric embedding validation (query/document), and OpenClaw compliance analysis | |
| Base model by [Liquid AI](https://huggingface.co/LiquidAI). | |
| ## 📋 Compliance Checklist | |
| Before deploying this project in production, verify: | |
| - [ ] Your entity's annual revenue is under $10M USD, OR you have obtained a commercial license from Liquid AI | |
| - [ ] You have read and understood the full LFM Open License v1.0 | |
| - [ ] If redistributing, you have included the LICENSE file and preserved all copyright notices | |
| - [ ] You are not using Liquid AI trademarks to promote your product | |
| - [ ] You understand that violations result in automatic license termination | |
| - [ ] You have configured appropriate timeouts (30s+) in your client | |
| - [ ] You have implemented chunking for long documents | |
| - [ ] You understand the performance characteristics (10-15s per request) | |
| ## 🐛 Troubleshooting | |
| ### Issue: Cold Start Takes Too Long | |
| **Solution**: This is normal for HF Free tier. The first request after inactivity takes 3-5 seconds to load the model. Subsequent requests are faster. | |
| ### Issue: Requests Timeout | |
| **Solution**: Increase your HTTP client timeout to 30 seconds. The HF Free tier can be slow under load. | |
| ### Issue: Build Fails with OOMKilled | |
| **Solution**: This is a known HF builder limitation. The current Dockerfile uses `CMAKE_BUILD_PARALLEL_LEVEL=1` to avoid this. If it still fails, try a Factory Reboot. | |
| ### Issue: API Returns 401 Unauthorized | |
| **Solution**: Verify your `API_KEY` secret is correctly configured in Settings → Variables and Secrets. The name must be exactly `API_KEY` (uppercase). | |
| ### Issue: Poor Search Results | |
| **Solution**: Ensure you're using `input_type: "query"` for searches and `input_type: "document"` for indexing. This asymmetric approach significantly improves RAG quality. | |
| ## 🤝 Contributing | |
| Contributions are welcome! Please open an issue or pull request. | |
| ## 📄 License Summary | |
| - **API Code**: MIT License (see [`LICENSE`](./LICENSE)) | |
| - **Model Weights**: LFM Open License v1.0 by Liquid AI, Inc. | |
| --- | |
| **Built with ❤️ for the RAG and vector search open-source community.** | |
| **Need faster performance?** Consider [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) with `@cf/qwen/qwen3-embedding-0.6b` for 50-200ms latency on their global edge network. |