README.md

metadata

title: HyperLookup SaaS
emoji: ⚡
colorFrom: blue
colorTo: indigo
sdk: docker
app_port: 7860
pinned: false

⚡ HyperLookup API — Ultra-Fast 2.5 Billion Record SaaS Engine

An enterprise-grade, high-performance lookup API and search dashboard engineered to query across 227 GB of partitioned columnar Parquet data (~2.5+ Billion Records) in milliseconds — hosted 100% FREE with zero infrastructure bills.

---

🎯 Architecture Overview

                      +---------------------------------------+
                      |     Client (Web UI / REST API)        |
                      +-------------------+-------------------+
                                          |
                                          v
                      +---------------------------------------+
                      |     FastAPI Edge Server (Docker)      |
                      |  - In-Memory LRU Cache (0.05ms)       |
                      |  - O(1) Sparse Partition Router       |
                      +-------------------+-------------------+
                                          |
                                          v
                      +---------------------------------------+
                      |   DuckDB Vectorized Query Engine      |
                      |  - HTTP Range Requests (Byte Streaming)|
                      |  - Parquet Predicate Pushdown         |
                      +-------------------+-------------------+
                                          | (Co-located <5ms)
                                          v
+---------------------------------------------------------------------------------+
| Hugging Face Storage Bucket: ItsSecretz/icrm-hitek-full-db-mixed-bucket (227 GB)|
| - idx_phone.0.parquet   to   idx_phone.6.parquet   (~2.5 Billion Phone Records) |
| - idx_aadhar.0.parquet  to   idx_aadhar.4.parquet  (~1.4 Billion Aadhar Records)|
+---------------------------------------------------------------------------------+

Why It's Blazing Fast:

1. O(1) Sparse Router (partitions.json): Instead of scanning 227 GB, the router uses an in-memory boundary index to identify the exact single file partition in 0.05 milliseconds, skipping 99% of all database files.
2. Parquet Predicate Pushdown: DuckDB fetches only the Parquet metadata footer and streams only the single matching row group (~1MB) via HTTP Range requests.
3. In-Memory Query Cache: Popular or repeated queries return in 0.05ms from the in-memory TTL cache.
4. Warm Engine Speed: Once Parquet footers are cached in DuckDB memory, queries execute in 180ms – 300ms!

---

🚀 Benchmark Results (Live Tested)

Query Type	Database Scope	Latency	Result
Warm Point Lookup (Phone)	2.5 Billion Records (16.6 GB Partition)	200 ms - 219 ms	✅ Instant Exact Records
Warm Point Lookup (Aadhar)	1.4 Billion Records (15.7 GB Partition)	220 ms - 350 ms	✅ Instant Exact Records
In-Memory Cache Hit	Cached Query	0.05 ms	✅ Sub-millisecond
Cold Lookup (First Request)	Uncached remote fetch	3.8s - 10s	✅ Footers cached for warm queries

---

🌐 100% Free Hosting & Deployment (Professional SaaS Level)

🥇 Option 1: Hugging Face Spaces (RECOMMENDED — BEST PERFORMANCE)

Why it's the best:

• Specs: 2 vCPU, 16 GB RAM on the 100% Free Tier!
• Zero Latency: Hugging Face Spaces run inside the exact same AWS us-east-1 datacenter as the Storage Bucket (ItsSecretz/icrm-hitek-full-db-mixed-bucket). Network latency between your API and the bucket is < 5ms!
• Bandwidth: Unlimited, 100% Free egress.

How to Deploy on Hugging Face Spaces:

1. Go to Hugging Face Spaces and click "Create new Space".
2. Set Space Name: hyperlookup-api (or your choice).
3. Select Space SDK: Docker -> Blank.
4. Set Space Hardware: CPU Basic (2 vCPU, 16GB RAM) - Free.
5. Set Visibility: Public (or Private).
6. Clone the Space repo or push this codebase:

   git init
   git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/hyperlookup-api
   git add .
   git commit -m "Deploy HyperLookup SaaS"
   git push -u origin main
   

7. Hugging Face will automatically build the Dockerfile and launch your API on https://YOUR_USERNAME-hyperlookup-api.hf.space.

---

🥈 Option 2: Koyeb (Free Tier Alternative)

1. Sign up at Koyeb.com (Free Tier includes 512MB RAM micro instances).
2. Connect your GitHub repository.
3. Koyeb will detect the Dockerfile and auto-deploy with a global edge URL: https://your-app.koyeb.app.

---

🥉 Option 3: Render (Free Web Service)

1. Go to Render.com and click "New Web Service".
2. Connect your GitHub repository.
3. Select Environment: Docker.
4. Free plan: Click Deploy.

---

👑 Professional Custom Domain & Global CDN Setup (Cloudflare)

To give your project a professional SaaS brand (e.g., api.yourbrand.com or search.yourbrand.com):

1. Add Domain to Cloudflare (Free Plan).
2. In Cloudflare DNS settings:
   • Type: CNAME
   • Name: api (or @)
   • Target: YOUR_USERNAME-hyperlookup-api.hf.space
   • Proxy status: Proxied (Orange Cloud).
3. In Cloudflare SSL/TLS:
   • Set encryption mode to Full (strict).
4. In Cloudflare Caching:
   • Create a Cache Rule for /api/lookup* with Edge TTL 12 hours.
   • Now repeated queries across the globe are answered from Cloudflare's 300+ Edge Data Centers in under 20 milliseconds!

---

💻 Local Development & Testing

1. Install Requirements

pip install -r requirements.txt

2. Run the Server

uvicorn main:app --host 0.0.0.0 --port 8000 --reload

3. Open in Browser

• SaaS Dashboard: http://localhost:8000
• Interactive Swagger Docs: http://localhost:8000/docs
• System Health & Stats: http://localhost:8000/api/stats

---

📡 API Reference

1. Auto-Detect Lookup

Automatically identifies whether input is a 10-digit Phone or 12-digit Aadhar:

GET /api/lookup?q=9313336070

Sample JSON Response:

{
  "success": true,
  "query_type": "phone",
  "query": "9313336070",
  "formatted_query": "+91 9313336070",
  "count": 2,
  "records": [
    {
      "name": "Aakash Jagdishbhai Makhija",
      "fathersName": "Makhija Bhai",
      "phoneNumber": "9313336070",
      "aadharNumber": null,
      "otherNumber": "7567303287",
      "address": "705/5, Ambawadi Near Police Station, Ahmedabad, Gujarat, 382475",
      "district": "Ahmedabad",
      "pincode": "382475",
      "state": "Gujarat",
      "town": null,
      "source": "inddata"
    }
  ],
  "target_partitions": ["idx_phone.4.parquet"],
  "latency_ms": 218.4,
  "cached": false
}

2. Phone Lookup

GET /api/phone/9313336070

3. Aadhar Lookup

GET /api/aadhar/351104219878

4. Telegram User ID Lookup (Global Master DB - 65M+ Records)

Fast, direct partition-routed lookup for Telegram accounts by User ID:

GET /tg/lookup?user_id=74252719
# or direct path:
GET /tg/74252719

Sample Telegram JSON Response:

{
  "status": "success",
  "user_id": 74252719,
  "country": "Iran",
  "num": "+989192125069",
  "phone": "+989192125069",
  "formatted_phone": "+98 919 212 5069",
  "country_code": "IR",
  "query_type": "telegram",
  "target_partition": "part_0000_1_74252719.parquet",
  "latency_ms": 14.8,
  "cached": false
}

5. System Statistics

GET /api/stats

---

📦 Client Code Examples

Python:

import requests

url = "https://your-domain.com/api/lookup"
response = requests.get(url, params={"q": "9313336070"})
data = response.json()

print(f"Matches: {data['count']} in {data['latency_ms']} ms")
for record in data['records']:
    print(record['name'], record['phoneNumber'], record['address'])

JavaScript / Node.js:

const res = await fetch("https://your-domain.com/api/lookup?q=9313336070");
const data = await res.json();
console.log(`Latency: ${data.latency_ms}ms`, data.records);

cURL:

curl -X GET "https://your-domain.com/api/lookup?q=9313336070"

---

🛡️ License & Disclaimers

This repository is engineered for ultra-fast columnar analytical lookups using open-source tools (FastAPI, DuckDB, Hugging Face Hub).