--- title: LFM2.5 Embedding API emoji: 🧠 colorFrom: indigo colorTo: red sdk: docker app_port: 7860 pinned: true license: mit tags: - embeddings - openai-compatible - llama-cpp - cpu-only - rag - vector-search - semantic-search - retrieval - text-embedding - gguf - quantization - liquid-ai - lfm2.5 - fastapi - docker - inference - ai - machine-learning - nlp - search - similarity - cosine-similarity - knowledge-base - document-indexing - multilingual short_description: OpenAI-compatible embeddings API running LFM2.5 on CPU --- # 🧠 LFM2.5 Embedding API (CPU-Optimized) ![Docker](https://img.shields.io/badge/Docker-Ready-blue) ![FastAPI](https://img.shields.io/badge/FastAPI-green) ![llama.cpp](https://img.shields.io/badge/llama.cpp-CPU%20Optimized-orange) ![OpenAI Compatible](https://img.shields.io/badge/OpenAI-Compatible-success) ![License](https://img.shields.io/badge/License-MIT%20%2B%20LFM--1.0-lightgrey) An **ultra-lightweight** and **100% OpenAI-compatible** REST API for generating embeddings using the [LiquidAI/LFM2.5-Embedding-350M](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M) model, optimized to run on **pure CPU** (no GPU required) on Hugging Face Spaces Free tier. ## ⚡ Key Features - 🚀 **Fast cold start** (~3-5s) thanks to pre-loaded Q8_0 GGUF model - 💾 **Minimal RAM usage** (~500MB) — runs comfortably within HF Free's 16GB limit - 🎯 **Asymmetric embeddings** — supports `query:` and `document:` prefixes for maximum RAG precision - 🔐 **Bearer Token authentication** via HF Secrets - 🤖 **Strict OpenAI standard** — works with any OpenAI-compatible client - 🌍 **Multilingual support** — LFM2.5 handles 100+ languages - 📦 **Easy deployment** — clone and run in 5 minutes ## ⚠️ Performance & Limitations (IMPORTANT) ### Current Performance on HF Free Tier | Metric | Value | Notes | |--------|-------|-------| | **Cold Start** | 3-5 seconds | First request after inactivity | | **Inference Time** | 10-15 seconds | Per embedding request (50-500 words) | | **Throughput** | ~6-8 requests/minute | Sustained rate | | **Max Context** | 512 tokens | Optimal for embeddings | | **Dimensions** | 1024 floats | Per embedding vector | ### Why 10-15 Seconds? This API runs on **Hugging Face Spaces Free tier**, which uses **shared CPU resources**: - **CPU Throttling**: The HF hypervisor dynamically limits CPU cycles when multiple containers compete for resources - **No AVX2 Optimization**: While the code is compiled with AVX2 support, the free tier's virtualization layer doesn't fully expose these CPU instructions - **Shared Infrastructure**: Your container shares physical CPU cores with other users' Spaces ### When to Use This API ✅ **Perfect for:** - Prototyping and development - Small-scale RAG applications (<1000 documents) - Personal projects and experimentation - Batch processing with async queues - Learning and education - Backup/fallback embedding service ❌ **Not ideal for:** - Real-time user-facing search (latency too high) - High-throughput production systems (>100 req/min) - Applications requiring <1s response times - Critical infrastructure with SLA requirements ### 🚀 Need Better Performance? If you need faster inference, consider these alternatives: | Option | Latency | Cost | Setup Complexity | |--------|---------|------|------------------| | **This Space (HF Free)** | 10-15s | $0 | ⭐ Minimal | | **HF Space Paid (Basic)** | 2-5s | ~$0.60/h | ⭐ Minimal | | **Cloudflare Workers AI** | 50-200ms | $0 (10k neurons/day) | ⭐⭐ Low | | **OpenAI Embeddings** | 100-300ms | $0.0001/1K tokens | ⭐⭐ Low | | **Self-hosted (GPU)** | 10-50ms | Hardware cost | ⭐⭐⭐⭐ High | **Recommended Alternative**: [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) with `@cf/qwen/qwen3-embedding-0.6b` offers 50-200ms latency on their global edge network, with 10,000 free neurons/day (~18k requests/day for 500-token embeddings). ## 🔌 Universal Compatibility This API can be used as an embedding backend for **any tool** that supports OpenAI-compatible endpoints: | Tool | Works? | Notes | |------|--------|-------| | **OpenClaw** | ✅ Full | Use `queryInputType: "query"` and `documentInputType: "document"` | | **Open WebUI** | ✅ Full | Configure as "OpenAI API" in RAG Settings | | **LangChain** | ✅ Full | Use `OpenAIEmbeddings(openai_api_base=...)` | | **LlamaIndex** | ✅ Full | Use `OpenAIEmbedding(api_base=...)` | | **Cursor** | ✅ Full | Point `api_base` to this Space | | **Continue** | ✅ Full | Configure in `config.json` | | **Dify** | ✅ Full | Use "OpenAI Embeddings" node | | **Flowise** | ✅ Full | Use OpenAI Embeddings component | | **n8n** | ✅ Full | Use OpenAI node with custom base URL | | **Haystack** | ✅ Full | Use `OpenAIEmbedder` with `api_base_url` | | **Semantic Kernel** | ✅ Full | Configure OpenAI connector | | **AutoGen** | ✅ Full | Use OpenAI-compatible embedding model | | **CrewAI** | ✅ Full | Configure embedding provider | | **OpenAI SDK (Python)** | ✅ Full | Override `base_url` and `api_key` | | **OpenAI SDK (Node.js)** | ✅ Full | Override `baseURL` and `apiKey` | ## 🛠️ Quick Start (How to Clone and Use) ### 1. Duplicate this Space Click the **three dots** (⋮) in the top-right corner → **"Duplicate this Space"** → choose visibility (Public/Private). ### 2. Configure the Secret In the duplicated Space, go to **Settings → Variables and Secrets** and add: | Name | Value | |------|-------| | `API_KEY` | `your-secret-key-here` | *(Use a strong string with 32+ characters. Tip: Use [1Password's Password Generator](https://1password.com/password-generator) for cryptographically secure random passwords)* ### 3. Wait for Build The Dockerfile will compile `llama.cpp` and download the Q8_0 model automatically (~5-8 minutes on first build). ### 4. Test with cURL ```bash curl -X POST "https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/embeddings" \ -H "Authorization: Bearer YOUR_SECRET_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "Text to generate embedding for", "model": "LiquidAI/LFM2.5-Embedding-350M", "input_type": "document" }' ``` ## 📝 Usage Examples ### Python with OpenAI SDK ```python from openai import OpenAI client = OpenAI( api_key="YOUR_SECRET_KEY", base_url="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/" ) # For indexing documents (store in vector DB) doc_response = client.embeddings.create( model="LiquidAI/LFM2.5-Embedding-350M", input="OpenClaw is a multilingual RAG tool.", extra_body={"input_type": "document"} ) # For searching (user query) query_response = client.embeddings.create( model="LiquidAI/LFM2.5-Embedding-350M", input="How does vector search work?", extra_body={"input_type": "query"} ) print(query_response.data[0].embedding[:5]) # First 5 floats ``` ### LangChain Integration ```python from langchain_openai import OpenAIEmbeddings embeddings = OpenAIEmbeddings( model="LiquidAI/LFM2.5-Embedding-350M", openai_api_key="YOUR_SECRET_KEY", openai_api_base="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/" ) # Generate embeddings vectors = embeddings.embed_documents(["Text 1", "Text 2"]) query_vector = embeddings.embed_query("Search query") ``` ### LlamaIndex Integration ```python from llama_index.embeddings.openai import OpenAIEmbedding embed_model = OpenAIEmbedding( model="LiquidAI/LFM2.5-Embedding-350M", api_key="YOUR_SECRET_KEY", api_base="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/" ) embeddings = embed_model.get_text_embedding("Your text here") ``` ### JavaScript/Node.js ```javascript import OpenAI from 'openai'; const client = new OpenAI({ apiKey: 'YOUR_SECRET_KEY', baseURL: 'https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/' }); const response = await client.embeddings.create({ model: 'LiquidAI/LFM2.5-Embedding-350M', input: 'Your text here' }); console.log(response.data[0].embedding.slice(0, 5)); ``` ## 🏗️ Architecture ``` +-------------------------------------------------------+ | Hugging Face Space (2 vCPU / 16GB RAM) | | | | +------------------+ +------------------+ | | | | | llama.cpp (C++) | | | | Docker +-------->| + Q8_0 GGUF | | | | Container | | (~380MB RAM) | | | | | +------------------+ | | +--------+---------+ | | | | | v | | +------------------+ +------------------+ | | | FastAPI | | /v1/embeddings | | | | + Uvicorn +-------->| (OpenAI-compat.) | | | +------------------+ +------------------+ | +-------------------------------------------------------+ ``` - **Engine**: `llama-cpp-python` (Python wrapper for llama.cpp) - **Model**: `LFM2.5-Embedding-350M-Q8_0.gguf` (8-bit quantization for maximum precision) - **Framework**: FastAPI + Uvicorn (1 worker) - **Pooling**: Native CLS Token (LFM2.5 standard) - **Dimensions**: 1024 floats per embedding - **Context**: 512 tokens (optimal for embeddings) ## 🎯 Best Practices ### 1. Use Asymmetric Embeddings Always specify `input_type` for better RAG performance: - `"document"` when indexing/storing text - `"query"` when searching ### 2. Implement Chunking Break long documents into ~400-token chunks with 50-token overlap for optimal retrieval. ### 3. Batch Requests Send multiple texts in a single request when possible: ```python response = client.embeddings.create( input=["Text 1", "Text 2", "Text 3"], model="LiquidAI/LFM2.5-Embedding-350M" ) ``` ### 4. Set Appropriate Timeouts Configure your HTTP client with 30-second timeouts to handle cold starts. ### 5. Use Async Queues for Indexing For large document collections, implement background processing to avoid blocking user interactions. ## ⚖️ Licensing This repository contains **two distinct layers** with different licenses: ### 1. API Code (Infrastructure) — MIT License All Python code (FastAPI, Dockerfile, scripts) is licensed under the **MIT License**. You are free to clone, modify, use commercially, and distribute the API infrastructure. ### 2. AI Model (Weights and GGUF) — LFM Open License v1.0 The model weights are owned by **Liquid AI, Inc.** and licensed under the **[LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M/blob/main/LICENSE)**. #### ⚠️ CRITICAL: Commercial Use Threshold **Commercial use is PERMITTED only if your Legal Entity's total annual revenue does NOT exceed $10,000,000 USD (ten million US dollars).** If your entity exceeds this threshold, you **must obtain a separate commercial license** from Liquid AI, Inc. **What this means for you:** ✅ **PERMITTED:** - Using this API to generate embeddings for RAG applications - Integration in commercial products (under the $10M threshold) - Academic research and non-commercial projects - Personal projects and experimentation ⚠️ **REQUIRES COMMERCIAL LICENSE:** - Entities with >$10M annual revenue must contact [Liquid AI](https://www.liquid.ai) - Exceeding the threshold without a license results in **automatic termination** (Section 11) 🚫 **PROHIBITED:** - Using Liquid AI trademarks ("Liquid AI", "LFM", etc.) to promote your product - Patent litigation against Liquid AI (triggers license termination) - Use for training competing foundation models 📄 **Full License Text**: See the [`LICENSE`](./LICENSE) file in this repository or the [official license](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M/blob/main/LICENSE) on Hugging Face. ## 🙏 Credits This project was architected in collaboration with: - 🤖 **Qwen3.7 Max** — DevOps architecture, Docker/CPU optimization, llama.cpp integration, and build troubleshooting - 🤖 **Gemini 3.1 Pro** — Code review, asymmetric embedding validation (query/document), and OpenClaw compliance analysis Base model by [Liquid AI](https://huggingface.co/LiquidAI). ## 📋 Compliance Checklist Before deploying this project in production, verify: - [ ] Your entity's annual revenue is under $10M USD, OR you have obtained a commercial license from Liquid AI - [ ] You have read and understood the full LFM Open License v1.0 - [ ] If redistributing, you have included the LICENSE file and preserved all copyright notices - [ ] You are not using Liquid AI trademarks to promote your product - [ ] You understand that violations result in automatic license termination - [ ] You have configured appropriate timeouts (30s+) in your client - [ ] You have implemented chunking for long documents - [ ] You understand the performance characteristics (10-15s per request) ## 🐛 Troubleshooting ### Issue: Cold Start Takes Too Long **Solution**: This is normal for HF Free tier. The first request after inactivity takes 3-5 seconds to load the model. Subsequent requests are faster. ### Issue: Requests Timeout **Solution**: Increase your HTTP client timeout to 30 seconds. The HF Free tier can be slow under load. ### Issue: Build Fails with OOMKilled **Solution**: This is a known HF builder limitation. The current Dockerfile uses `CMAKE_BUILD_PARALLEL_LEVEL=1` to avoid this. If it still fails, try a Factory Reboot. ### Issue: API Returns 401 Unauthorized **Solution**: Verify your `API_KEY` secret is correctly configured in Settings → Variables and Secrets. The name must be exactly `API_KEY` (uppercase). ### Issue: Poor Search Results **Solution**: Ensure you're using `input_type: "query"` for searches and `input_type: "document"` for indexing. This asymmetric approach significantly improves RAG quality. ## 🤝 Contributing Contributions are welcome! Please open an issue or pull request. ## 📄 License Summary - **API Code**: MIT License (see [`LICENSE`](./LICENSE)) - **Model Weights**: LFM Open License v1.0 by Liquid AI, Inc. --- **Built with ❤️ for the RAG and vector search open-source community.** **Need faster performance?** Consider [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) with `@cf/qwen/qwen3-embedding-0.6b` for 50-200ms latency on their global edge network.