Spaces:
Runtime error
Embedding Cost Analysis — Gemini Embedding 2 vs BAAI/bge-small-en-v1.5
Created: 2026-05-16
Context: Decision document for embedding model selection and KB entry billing design.
Status: Gemini Embedding 2 migration recommended. See Section 6 for implementation plan.
Table of Contents
- Current Embedding Setup
- Gemini Embedding 2 Overview
- OpenAI Embedding Lineup (2026)
- Cost Per Entry by Modality
- Scale Analysis: 1,000 Tenants × 500 Entries
- Billing Model Recommendation
- Video & Audio — Transcript Strategy
- remork.org Specific Analysis
- Competitor Multimodal KB Support
- Qdrant Timestamp Filtering — False Claim Correction
- Credit System Note
1. Current Embedding Setup
- Model:
BAAI/bge-small-en-v1.5 - Inference: Local (sentence-transformers), runs in Docker container
- Dimensions: 384
- Cost: $0 per embedding (local inference, no API call)
- Context window: 512 tokens
- MTEB score: ~62
Why this was chosen: Zero cost, good enough for V1, no external API dependency.
Why to migrate: 512-token context limit truncates long KB entries. Quality gap is real (+6 MTEB points). Multimodal capability unavailable.
2. Gemini Embedding 2 Overview
| Attribute | Value |
|---|---|
| Model ID | gemini-embedding-2 (GA as of March 10, 2026) |
| Output Dimensions | 3,072 (Matryoshka flexible: 128–3,072; recommended: 768, 1,536, or 3,072) |
| Context Window | 8,192 tokens (16× larger than current bge-small) |
| Modalities | Text + Images (6×) + Video (120s) + Audio (80s) + PDF (6 pages) — all in same vector space |
| MTEB Score | 68.32 — top of leaderboard by 5+ points (2026) |
| Languages | 100+ languages |
| Deployment | Server-side only (never expose API key in browser) |
| Batch discount | 50% off for async batch jobs |
Pricing:
| Modality | Standard | Batch (50% off) |
|---|---|---|
| Text | $0.20 / 1M tokens | $0.10 / 1M tokens |
| Image | $0.45 / 1M tokens | $0.225 / 1M tokens |
| Video | $12.00 / 1M tokens | $6.00 / 1M tokens |
| Audio | $6.50 / 1M tokens | $3.25 / 1M tokens |
3. OpenAI Embedding Lineup (2026)
All OpenAI embedding models are text-only as of 2026. No multimodal support.
| Model | Dimensions | MTEB | Cost | Notes |
|---|---|---|---|---|
text-embedding-ada-002 |
1,536 | ~61 | $0.10/1M | Legacy, not deprecated |
text-embedding-3-small |
1,536 | 62.3 | $0.02/1M | Cheapest OpenAI option |
text-embedding-3-large |
3,072 | 64.6 | $0.13/1M | Best OpenAI text quality |
gemini-embedding-2 |
3,072 | 68.32 | $0.20/1M | Only model with native multimodal |
Conclusion: OpenAI has no path to multimodal embeddings. Gemini Embedding 2 is the only commercial model that handles video/audio natively (2026).
4. Cost Per Entry by Modality
Token counting methodology:
| Modality | Token counting | Assumed avg per entry |
|---|---|---|
| Text (typical SMB KB article) | 1 token ≈ 4 chars | 750 tokens |
| Text (remork.org — title + tags) | 1 token ≈ 4 chars | 65 tokens |
| Image (medium-res, 1 per entry) | ~768 tokens/image | 768 tokens |
| Video (10-min educational) | 1 frame/sec × 258 tokens/frame | 154,800 tokens |
| Audio (10-min recording) | ~32 tokens/sec | 19,200 tokens |
| PDF (5 pages, image-based) | ~768 tokens/page | 3,840 tokens |
Cost per single entry:
| Entry type | Tokens | Price/1M tokens | Cost per entry |
|---|---|---|---|
| Text (typical SMB) | 750 | $0.20 | $0.00015 |
| Text (remork.org style) | 65 | $0.20 | $0.000013 |
| Image only (1 image) | 768 | $0.45 | $0.00035 |
| Text + 1 image | 1,518 | blended | $0.00050 |
| Audio (10 min) | 19,200 | $6.50 | $0.125 |
| Video (10 min) | 154,800 | $12.00 | $1.86 |
| PDF (5 pages) | 3,840 | $0.20 | $0.00077 |
| Transcript from video (text only) | ~750 | $0.20 | $0.00015 |
The critical cliff: Text, images, and PDFs are all under $0.001 per entry. Video jumps to $1.86 per 10-minute entry — 12,400× more expensive than a text entry.
5. Scale Analysis: 1,000 Tenants × 500 Entries
| Scenario | Total entries | Total one-time cost | Cost per tenant |
|---|---|---|---|
| All text (typical SMB) | 500,000 | $75 | $0.15 |
| remork.org style (title + tags) | 500,000 | $6.50 | $0.0065 |
| Mixed text + images (e-commerce) | 500,000 | ~$160 | $0.16 |
| All 10-min videos (raw embedding) | 500,000 | $930,000 | $930 |
| All 10-min videos (transcript only) | 500,000 | $75 | $0.15 |
At batch pricing (50% off), halve all figures.
Key insight: Switching from raw video embedding to transcript-based embedding brings the cost from $930K down to $75 — a 12,400× cost reduction with better Q&A retrieval quality.
6. Billing Model Recommendation
⚠️ NOTE (2026-05-16): The credits abstraction is being removed entirely. All billing will use USD directly — tenants top up in USD, deductions are in USD, no conversion factor. See Section 11 and backend crucial_notes.md for the refactor plan. The table below uses USD throughout.
Rule: text and PDF entries are free. Media entries (image, audio, video) cost USD.
The billing system will deduct USD directly from tenants.balance_usd. Add a new event type: kb_embedding.
| Entry type | Platform cost | Charge tenant | Rationale |
|---|---|---|---|
| Text | $0.00015 | Free | Too small to meter; absorb as platform overhead |
| $0.00077 | Free | Same — absorb | |
| Image (per image in entry) | $0.00035 | ~$0.001 | Small fixed charge per image with markup |
| Audio (per minute) | ~$0.012/min | $0.02/min | Per-minute, transparent — or free if auto-transcribed |
| Video — raw embedding (per min) | ~$0.186/min | $0.20/min | Strongly discourage; show transcript path instead |
| Video — transcript only (Whisper) | $0 | Free | Strongly recommended default path |
| Audio — transcript only (Whisper) | $0 | Free | Strongly recommended default path |
Per-minute billing rationale
Tenants understand "longer video = more $" intuitively — mirrors how phone calls are billed. Before submission, show an estimate: "This video is 12 minutes. Direct embedding cost: ~$2.40. Transcript embedding: Free."
Billing implementation (post-credit-removal)
Add to billing_events.event_type enum: kb_embedding_image, kb_embedding_audio, kb_embedding_video
Add a kb_embedding_rates config table: USD per image / per audio-min / per video-min (mirrors the existing llm_models rate table pattern)
7. Video & Audio — Transcript Strategy
The core problem
Raw video/audio embedding is expensive and overkill for Q&A use cases. A transcript answers "what does this video cover?" just as well as video frame analysis — and does it faster, cheaper, and more accurately.
Recommended UX flow (applies to both video and audio)
Tenant uploads video/audio OR pastes YouTube URL
│
▼
Backend queues transcription job (Celery async)
│
├─ YouTube URL → YouTube Transcript API (free, instant)
│
└─ Uploaded file → Whisper (local, free) or Whisper API ($0.006/min)
│
▼
Frontend shows: "Transcript ready — review before embedding"
│
Editable transcript preview box (tenant can fix errors, add context)
│
├─ [Embed Transcript — Free] → stored as text KB entry, video URL in source_url
│
└─ [Embed Media Directly — X credits] → uses Gemini video/audio embedding
Transcription options by cost
| Method | Cost | Speed | Quality | Use case |
|---|---|---|---|---|
| YouTube Transcript API | Free | Instant | High (native captions) | YouTube URLs |
| Whisper local (Docker) | Free | ~1× realtime | High | Uploaded files, on-prem |
| OpenAI Whisper API | $0.006/min | Fast | High | Cloud alternative to local |
| Deepgram / AssemblyAI | ~$0.01–0.02/min | Fast | Very high | If better accuracy needed |
Recommendation: Run Whisper locally in Docker for uploaded files (zero cost, no API dependency). YouTube Transcript API for YouTube URLs.
What gets stored in the KB
KB entry (transcript-based video):
title: "Introduction to Khmer Literacy — Grade 3"
content: "[transcript text here — editable by tenant]"
source_url: "https://youtube.com/watch?v=abc123" ← video pointer
origin: "video_transcript"
source: "youtube" | "upload"
The chatbot can then:
- Retrieve the transcript for Q&A ("What topics are covered in the Grade 3 literacy video?")
- Include the video link in its response ("Watch the video here: [source_url]")
When raw video/audio embedding is still useful
Only one scenario justifies it: visual Q&A — "What is shown on the whiteboard at 3:20?" For educational platforms, this is rare. For product demos or training videos where visual elements matter, it's valid — but should be opt-in and credit-gated.
8. remork.org Specific Analysis
About remork.org: Non-profit educational platform serving thousands of Cambodian teachers. KB entries are video-based posts with title + tags + rare short descriptions.
Current state (videos stored on server)
- KB entries: title + tags + optional short description
- Average tokens per entry: ~65
- Cost per entry at Gemini Embedding 2: $0.000013
- Cost for 2,000 posts: $0.026 total one-time ingestion
- Verdict: essentially free. Absorb it without a second thought.
Future state (if videos move to YouTube)
Do NOT use raw video embedding. Use the transcript strategy:
- Paste YouTube video URL into KB entry form
- Backend calls YouTube Transcript API (free, instant)
- Admin reviews/edits the Khmer transcript
- Click "Embed Transcript"
- Cost: $0.000013 per entry (same as text) + free YouTube API call
Note on Khmer language: YouTube auto-captions support Khmer. Quality varies — human review before embedding is recommended. The editable transcript preview box solves this.
remork.org chatbot use case
A Cambodian teacher asks: "What videos are available about Grade 5 mathematics?"
The chatbot retrieves transcript-based entries, returns matched video titles + YouTube links.
Teacher clicks the link and watches the video.
This is the ideal flow: chatbot as a search/discovery layer over video content, not as a video analyzer.
9. Competitor Multimodal KB Support
| Competitor | Media KB | Notes |
|---|---|---|
| Intercom Fin | Images (chat only) | Analyzes user-uploaded screenshots in chat. Not KB upload. Enterprise/Pro. |
| Botpress | Images + video | Most advanced multimodal KB of all competitors. Mid-market. |
| Zendesk AI | Partial (roadmap) | Text-only currently; video KB planned. Enterprise. |
| Tidio Lyro | No | Text-only; uses Claude backend. SMB/mid-market. |
| Chatbase | Unclear | GPT-4o models available but no confirmed media KB upload. All tiers. |
| CustomGPT | No | Enterprise, text-only. |
| Voiceflow | No | Generic builder, text-only. |
Market position: If smart_chatbot ships image + transcript-based video KB, it matches Botpress and surpasses every other SMB-tier competitor.
Marketing one-liner: "Unlike Tidio or CustomGPT, your chatbot understands images and videos — not just PDFs and text."
10. Qdrant Timestamp Filtering — False Claim Correction
Claim heard: "Qdrant does not support timestamp/timeframe filtering."
Status: FALSE. Verified against Qdrant documentation (2026).
Qdrant fully supports datetime filtering via DatetimeRange payload filter conditions (available since v1.8.0+).
from qdrant_client import models
filter = models.Filter(
must=[
models.FieldCondition(
key="created_at",
range=models.DatetimeRange(
gte="2026-01-01T00:00:00Z",
lte="2026-05-16T23:59:59Z"
)
)
]
)
- RFC 3339 formatted dates
gt,gte,lt,lteoperators- UTC conversion handled automatically
- Create a payload index on the date field before high-volume ingestion
Known limitations (not blockers):
- Unindexed date filters degrade performance on large collections
- HNSW graph can degrade with very high-cardinality filters on 260M+ point datasets (Qdrant 1.16+ ACORN algorithm mitigates this)
Conclusion: No need to migrate away from Qdrant on this basis.
11. Credit System — Removal Decision (2026-05-16)
Decision: Remove the credits abstraction entirely. Use USD directly throughout the billing system.
Why credits were wrong
The system stored tenants.credits as a Decimal, where 1 credit = $0.001 USD (confirmed by migration math: LLM rates × 3× markup ÷ $0.001/credit = seeded credits_per_1k values). The billing router returned balance_usd = tenant.credits — a field named "USD" but containing raw credits. This created a silent 1000× mismatch: a tenant with credits = 100 was shown "100" under a USD label but actually had $0.10 in purchasing power.
Verified from the database (2026-05-16):
SELECT name, credits FROM tenants;
-- demo | 97.820590
-- Monireach | 100.000000
The demo tenant started with 100 credits ($0.10 real USD) and spent 2 credits ($0.002) across 9 chat interactions — each costing 0.135–0.518 credits.
What the refactor must do
See backend crucial_notes.md for the full logged task. Summary:
| Before | After |
|---|---|
tenants.credits |
tenants.balance_usd (true USD Decimal) |
credit_events table |
billing_events (or rename columns) |
credits_per_1k_input/output in llm_models |
usd_per_1k_input/output |
Deduction: amount in credits |
Deduction: amount in USD |
billing_mode = 'credits' |
billing_mode = 'pay_as_you_go' (or keep name) |
| 3× markup computed in credits | 3× markup computed in USD directly |
Seeded rates — what they become in USD
| Model | Old (credits/1K) | New (USD/1K) |
|---|---|---|
| gemini-2.5-flash input | 0.900000 credits | $0.000900 |
| gemini-2.5-flash output | 7.500000 credits | $0.007500 |
| qwen3-32b input | 0.870000 credits | $0.000870 |
| qwen3-32b output | 1.770000 credits | $0.001770 |
The numbers stay identical — only the unit label changes from "credits" to "USD". No business logic changes.