README.md

---
title: LFM2.5 Embedding API
emoji: 🧠
colorFrom: indigo
colorTo: red
sdk: docker
app_port: 7860
pinned: true
license: mit
tags:
  - embeddings
  - openai-compatible
  - llama-cpp
  - cpu-only
  - rag
  - vector-search
  - semantic-search
  - retrieval
  - text-embedding
  - gguf
  - quantization
  - liquid-ai
  - lfm2.5
  - fastapi
  - docker
  - inference
  - ai
  - machine-learning
  - nlp
  - search
  - similarity
  - cosine-similarity
  - knowledge-base
  - document-indexing
  - multilingual
short_description: OpenAI-compatible embeddings API running LFM2.5 on CPU
---

# 🧠 LFM2.5 Embedding API (CPU-Optimized)

![Docker](https://img.shields.io/badge/Docker-Ready-blue)
![FastAPI](https://img.shields.io/badge/FastAPI-green)
![llama.cpp](https://img.shields.io/badge/llama.cpp-CPU%20Optimized-orange)
![OpenAI Compatible](https://img.shields.io/badge/OpenAI-Compatible-success)
![License](https://img.shields.io/badge/License-MIT%20%2B%20LFM--1.0-lightgrey)

An **ultra-lightweight** and **100% OpenAI-compatible** REST API for generating embeddings using the [LiquidAI/LFM2.5-Embedding-350M](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M) model, optimized to run on **pure CPU** (no GPU required) on Hugging Face Spaces Free tier.

## ⚡ Key Features

- 🚀 **Fast cold start** (~3-5s) thanks to pre-loaded Q8_0 GGUF model
- 💾 **Minimal RAM usage** (~500MB) — runs comfortably within HF Free's 16GB limit
- 🎯 **Asymmetric embeddings** — supports `query:` and `document:` prefixes for maximum RAG precision
- 🔐 **Bearer Token authentication** via HF Secrets
- 🤖 **Strict OpenAI standard** — works with any OpenAI-compatible client
- 🌍 **Multilingual support** — LFM2.5 handles 100+ languages
- 📦 **Easy deployment** — clone and run in 5 minutes

## ⚠️ Performance & Limitations (IMPORTANT)

### Current Performance on HF Free Tier

| Metric | Value | Notes |
|--------|-------|-------|
| **Cold Start** | 3-5 seconds | First request after inactivity |
| **Inference Time** | 10-15 seconds | Per embedding request (50-500 words) |
| **Throughput** | ~6-8 requests/minute | Sustained rate |
| **Max Context** | 512 tokens | Optimal for embeddings |
| **Dimensions** | 1024 floats | Per embedding vector |

### Why 10-15 Seconds?

This API runs on **Hugging Face Spaces Free tier**, which uses **shared CPU resources**:

- **CPU Throttling**: The HF hypervisor dynamically limits CPU cycles when multiple containers compete for resources
- **No AVX2 Optimization**: While the code is compiled with AVX2 support, the free tier's virtualization layer doesn't fully expose these CPU instructions
- **Shared Infrastructure**: Your container shares physical CPU cores with other users' Spaces

### When to Use This API

✅ **Perfect for:**
- Prototyping and development
- Small-scale RAG applications (<1000 documents)
- Personal projects and experimentation
- Batch processing with async queues
- Learning and education
- Backup/fallback embedding service

❌ **Not ideal for:**
- Real-time user-facing search (latency too high)
- High-throughput production systems (>100 req/min)
- Applications requiring <1s response times
- Critical infrastructure with SLA requirements

### 🚀 Need Better Performance?

If you need faster inference, consider these alternatives:

| Option | Latency | Cost | Setup Complexity |
|--------|---------|------|------------------|
| **This Space (HF Free)** | 10-15s | $0 | ⭐ Minimal |
| **HF Space Paid (Basic)** | 2-5s | ~$0.60/h | ⭐ Minimal |
| **Cloudflare Workers AI** | 50-200ms | $0 (10k neurons/day) | ⭐⭐ Low |
| **OpenAI Embeddings** | 100-300ms | $0.0001/1K tokens | ⭐⭐ Low |
| **Self-hosted (GPU)** | 10-50ms | Hardware cost | ⭐⭐⭐⭐ High |

**Recommended Alternative**: [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) with `@cf/qwen/qwen3-embedding-0.6b` offers 50-200ms latency on their global edge network, with 10,000 free neurons/day (~18k requests/day for 500-token embeddings).

## 🔌 Universal Compatibility

This API can be used as an embedding backend for **any tool** that supports OpenAI-compatible endpoints:

| Tool | Works? | Notes |
|------|--------|-------|
| **OpenClaw** | ✅ Full | Use `queryInputType: "query"` and `documentInputType: "document"` |
| **Open WebUI** | ✅ Full | Configure as "OpenAI API" in RAG Settings |
| **LangChain** | ✅ Full | Use `OpenAIEmbeddings(openai_api_base=...)` |
| **LlamaIndex** | ✅ Full | Use `OpenAIEmbedding(api_base=...)` |
| **Cursor** | ✅ Full | Point `api_base` to this Space |
| **Continue** | ✅ Full | Configure in `config.json` |
| **Dify** | ✅ Full | Use "OpenAI Embeddings" node |
| **Flowise** | ✅ Full | Use OpenAI Embeddings component |
| **n8n** | ✅ Full | Use OpenAI node with custom base URL |
| **Haystack** | ✅ Full | Use `OpenAIEmbedder` with `api_base_url` |
| **Semantic Kernel** | ✅ Full | Configure OpenAI connector |
| **AutoGen** | ✅ Full | Use OpenAI-compatible embedding model |
| **CrewAI** | ✅ Full | Configure embedding provider |
| **OpenAI SDK (Python)** | ✅ Full | Override `base_url` and `api_key` |
| **OpenAI SDK (Node.js)** | ✅ Full | Override `baseURL` and `apiKey` |

## 🛠️ Quick Start (How to Clone and Use)

### 1. Duplicate this Space
Click the **three dots** (⋮) in the top-right corner → **"Duplicate this Space"** → choose visibility (Public/Private).

### 2. Configure the Secret
In the duplicated Space, go to **Settings → Variables and Secrets** and add:

| Name | Value |
|------|-------|
| `API_KEY` | `your-secret-key-here` |

*(Use a strong string with 32+ characters. Tip: Use [1Password's Password Generator](https://1password.com/password-generator) for cryptographically secure random passwords)*

### 3. Wait for Build
The Dockerfile will compile `llama.cpp` and download the Q8_0 model automatically (~5-8 minutes on first build).

### 4. Test with cURL
```bash
curl -X POST "https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/embeddings" \
     -H "Authorization: Bearer YOUR_SECRET_KEY" \
     -H "Content-Type: application/json" \
     -d '{
           "input": "Text to generate embedding for",
           "model": "LiquidAI/LFM2.5-Embedding-350M",
           "input_type": "document"
         }'
```

## 📝 Usage Examples

### Python with OpenAI SDK

```python
from openai import OpenAI

client = OpenAI(
    api_key="YOUR_SECRET_KEY",
    base_url="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/"
)

# For indexing documents (store in vector DB)
doc_response = client.embeddings.create(
    model="LiquidAI/LFM2.5-Embedding-350M",
    input="OpenClaw is a multilingual RAG tool.",
    extra_body={"input_type": "document"}
)

# For searching (user query)
query_response = client.embeddings.create(
    model="LiquidAI/LFM2.5-Embedding-350M",
    input="How does vector search work?",
    extra_body={"input_type": "query"}
)

print(query_response.data[0].embedding[:5])  # First 5 floats
```

### LangChain Integration

```python
from langchain_openai import OpenAIEmbeddings

embeddings = OpenAIEmbeddings(
    model="LiquidAI/LFM2.5-Embedding-350M",
    openai_api_key="YOUR_SECRET_KEY",
    openai_api_base="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/"
)

# Generate embeddings
vectors = embeddings.embed_documents(["Text 1", "Text 2"])
query_vector = embeddings.embed_query("Search query")
```

### LlamaIndex Integration

```python
from llama_index.embeddings.openai import OpenAIEmbedding

embed_model = OpenAIEmbedding(
    model="LiquidAI/LFM2.5-Embedding-350M",
    api_key="YOUR_SECRET_KEY",
    api_base="https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/"
)

embeddings = embed_model.get_text_embedding("Your text here")
```

### JavaScript/Node.js

```javascript
import OpenAI from 'openai';

const client = new OpenAI({
  apiKey: 'YOUR_SECRET_KEY',
  baseURL: 'https://YOUR-USERNAME-YOUR-SPACE.hf.space/v1/'
});

const response = await client.embeddings.create({
  model: 'LiquidAI/LFM2.5-Embedding-350M',
  input: 'Your text here'
});

console.log(response.data[0].embedding.slice(0, 5));
```

## 🏗️ Architecture

```
+-------------------------------------------------------+
|  Hugging Face Space (2 vCPU / 16GB RAM)               |
|                                                       |
|   +------------------+         +------------------+   |
|   |                  |         | llama.cpp (C++)  |   |
|   |      Docker      +-------->|   + Q8_0 GGUF    |   |
|   |    Container     |         |  (~380MB RAM)    |   |
|   |                  |         +------------------+   |
|   +--------+---------+                                |
|            |                                          |
|            v                                          |
|   +------------------+         +------------------+   |
|   |     FastAPI      |         |  /v1/embeddings  |   |
|   |    + Uvicorn     +-------->| (OpenAI-compat.) |   |
|   +------------------+         +------------------+   |
+-------------------------------------------------------+
```

- **Engine**: `llama-cpp-python` (Python wrapper for llama.cpp)
- **Model**: `LFM2.5-Embedding-350M-Q8_0.gguf` (8-bit quantization for maximum precision)
- **Framework**: FastAPI + Uvicorn (1 worker)
- **Pooling**: Native CLS Token (LFM2.5 standard)
- **Dimensions**: 1024 floats per embedding
- **Context**: 512 tokens (optimal for embeddings)

## 🎯 Best Practices

### 1. Use Asymmetric Embeddings
Always specify `input_type` for better RAG performance:
- `"document"` when indexing/storing text
- `"query"` when searching

### 2. Implement Chunking
Break long documents into ~400-token chunks with 50-token overlap for optimal retrieval.

### 3. Batch Requests
Send multiple texts in a single request when possible:
```python
response = client.embeddings.create(
    input=["Text 1", "Text 2", "Text 3"],
    model="LiquidAI/LFM2.5-Embedding-350M"
)
```

### 4. Set Appropriate Timeouts
Configure your HTTP client with 30-second timeouts to handle cold starts.

### 5. Use Async Queues for Indexing
For large document collections, implement background processing to avoid blocking user interactions.

## ⚖️ Licensing

This repository contains **two distinct layers** with different licenses:

### 1. API Code (Infrastructure) — MIT License
All Python code (FastAPI, Dockerfile, scripts) is licensed under the **MIT License**. You are free to clone, modify, use commercially, and distribute the API infrastructure.

### 2. AI Model (Weights and GGUF) — LFM Open License v1.0
The model weights are owned by **Liquid AI, Inc.** and licensed under the **[LFM Open License v1.0](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M/blob/main/LICENSE)**.

#### ⚠️ CRITICAL: Commercial Use Threshold

**Commercial use is PERMITTED only if your Legal Entity's total annual revenue does NOT exceed $10,000,000 USD (ten million US dollars).**

If your entity exceeds this threshold, you **must obtain a separate commercial license** from Liquid AI, Inc.

**What this means for you:**

✅ **PERMITTED:**
- Using this API to generate embeddings for RAG applications
- Integration in commercial products (under the $10M threshold)
- Academic research and non-commercial projects
- Personal projects and experimentation

⚠️ **REQUIRES COMMERCIAL LICENSE:**
- Entities with >$10M annual revenue must contact [Liquid AI](https://www.liquid.ai)
- Exceeding the threshold without a license results in **automatic termination** (Section 11)

🚫 **PROHIBITED:**
- Using Liquid AI trademarks ("Liquid AI", "LFM", etc.) to promote your product
- Patent litigation against Liquid AI (triggers license termination)
- Use for training competing foundation models

📄 **Full License Text**: See the [`LICENSE`](./LICENSE) file in this repository or the [official license](https://huggingface.co/LiquidAI/LFM2.5-Embedding-350M/blob/main/LICENSE) on Hugging Face.

## 🙏 Credits

This project was architected in collaboration with:

- 🤖 **Qwen3.7 Max** — DevOps architecture, Docker/CPU optimization, llama.cpp integration, and build troubleshooting
- 🤖 **Gemini 3.1 Pro** — Code review, asymmetric embedding validation (query/document), and OpenClaw compliance analysis

Base model by [Liquid AI](https://huggingface.co/LiquidAI).

## 📋 Compliance Checklist

Before deploying this project in production, verify:

- [ ] Your entity's annual revenue is under $10M USD, OR you have obtained a commercial license from Liquid AI
- [ ] You have read and understood the full LFM Open License v1.0
- [ ] If redistributing, you have included the LICENSE file and preserved all copyright notices
- [ ] You are not using Liquid AI trademarks to promote your product
- [ ] You understand that violations result in automatic license termination
- [ ] You have configured appropriate timeouts (30s+) in your client
- [ ] You have implemented chunking for long documents
- [ ] You understand the performance characteristics (10-15s per request)

## 🐛 Troubleshooting

### Issue: Cold Start Takes Too Long
**Solution**: This is normal for HF Free tier. The first request after inactivity takes 3-5 seconds to load the model. Subsequent requests are faster.

### Issue: Requests Timeout
**Solution**: Increase your HTTP client timeout to 30 seconds. The HF Free tier can be slow under load.

### Issue: Build Fails with OOMKilled
**Solution**: This is a known HF builder limitation. The current Dockerfile uses `CMAKE_BUILD_PARALLEL_LEVEL=1` to avoid this. If it still fails, try a Factory Reboot.

### Issue: API Returns 401 Unauthorized
**Solution**: Verify your `API_KEY` secret is correctly configured in Settings → Variables and Secrets. The name must be exactly `API_KEY` (uppercase).

### Issue: Poor Search Results
**Solution**: Ensure you're using `input_type: "query"` for searches and `input_type: "document"` for indexing. This asymmetric approach significantly improves RAG quality.

## 🤝 Contributing

Contributions are welcome! Please open an issue or pull request.

## 📄 License Summary

- **API Code**: MIT License (see [`LICENSE`](./LICENSE))
- **Model Weights**: LFM Open License v1.0 by Liquid AI, Inc.

---

**Built with ❤️ for the RAG and vector search open-source community.**

**Need faster performance?** Consider [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/) with `@cf/qwen/qwen3-embedding-0.6b` for 50-200ms latency on their global edge network.