--- title: HyperLookup SaaS emoji: ⚡ colorFrom: blue colorTo: indigo sdk: docker app_port: 7860 pinned: false --- # ⚡ HyperLookup API — Ultra-Fast 2.5 Billion Record SaaS Engine An enterprise-grade, high-performance lookup API and search dashboard engineered to query across **227 GB of partitioned columnar Parquet data (~2.5+ Billion Records)** in milliseconds — **hosted 100% FREE** with zero infrastructure bills. --- ## 🎯 Architecture Overview ``` +---------------------------------------+ | Client (Web UI / REST API) | +-------------------+-------------------+ | v +---------------------------------------+ | FastAPI Edge Server (Docker) | | - In-Memory LRU Cache (0.05ms) | | - O(1) Sparse Partition Router | +-------------------+-------------------+ | v +---------------------------------------+ | DuckDB Vectorized Query Engine | | - HTTP Range Requests (Byte Streaming)| | - Parquet Predicate Pushdown | +-------------------+-------------------+ | (Co-located <5ms) v +---------------------------------------------------------------------------------+ | Hugging Face Storage Bucket: ItsSecretz/icrm-hitek-full-db-mixed-bucket (227 GB)| | - idx_phone.0.parquet to idx_phone.6.parquet (~2.5 Billion Phone Records) | | - idx_aadhar.0.parquet to idx_aadhar.4.parquet (~1.4 Billion Aadhar Records)| +---------------------------------------------------------------------------------+ ``` ### Why It's Blazing Fast: 1. **O(1) Sparse Router (`partitions.json`)**: Instead of scanning 227 GB, the router uses an in-memory boundary index to identify the exact single file partition in **0.05 milliseconds**, skipping 99% of all database files. 2. **Parquet Predicate Pushdown**: DuckDB fetches only the Parquet metadata footer and streams only the single matching row group (~1MB) via HTTP Range requests. 3. **In-Memory Query Cache**: Popular or repeated queries return in **0.05ms** from the in-memory TTL cache. 4. **Warm Engine Speed**: Once Parquet footers are cached in DuckDB memory, queries execute in **180ms – 300ms**! --- ## 🚀 Benchmark Results (Live Tested) | Query Type | Database Scope | Latency | Result | | :--- | :--- | :--- | :--- | | **Warm Point Lookup (Phone)** | 2.5 Billion Records (16.6 GB Partition) | **200 ms - 219 ms** | ✅ Instant Exact Records | | **Warm Point Lookup (Aadhar)**| 1.4 Billion Records (15.7 GB Partition) | **220 ms - 350 ms** | ✅ Instant Exact Records | | **In-Memory Cache Hit** | Cached Query | **0.05 ms** | ✅ Sub-millisecond | | **Cold Lookup (First Request)**| Uncached remote fetch | **3.8s - 10s** | ✅ Footers cached for warm queries | --- ## 🌐 100% Free Hosting & Deployment (Professional SaaS Level) ### 🥇 Option 1: Hugging Face Spaces (RECOMMENDED — BEST PERFORMANCE) **Why it's the best:** - **Specs**: 2 vCPU, **16 GB RAM** on the 100% Free Tier! - **Zero Latency**: Hugging Face Spaces run inside the **exact same AWS us-east-1 datacenter** as the Storage Bucket (`ItsSecretz/icrm-hitek-full-db-mixed-bucket`). Network latency between your API and the bucket is **< 5ms**! - **Bandwidth**: Unlimited, 100% Free egress. #### How to Deploy on Hugging Face Spaces: 1. Go to [Hugging Face Spaces](https://huggingface.co/spaces) and click **"Create new Space"**. 2. Set Space Name: `hyperlookup-api` (or your choice). 3. Select **Space SDK**: **Docker** -> **Blank**. 4. Set Space Hardware: **CPU Basic (2 vCPU, 16GB RAM) - Free**. 5. Set Visibility: **Public** (or Private). 6. Clone the Space repo or push this codebase: ```bash git init git remote add origin https://huggingface.co/spaces/YOUR_USERNAME/hyperlookup-api git add . git commit -m "Deploy HyperLookup SaaS" git push -u origin main ``` 7. Hugging Face will automatically build the `Dockerfile` and launch your API on `https://YOUR_USERNAME-hyperlookup-api.hf.space`. --- ### 🥈 Option 2: Koyeb (Free Tier Alternative) 1. Sign up at [Koyeb.com](https://www.koyeb.com) (Free Tier includes 512MB RAM micro instances). 2. Connect your GitHub repository. 3. Koyeb will detect the `Dockerfile` and auto-deploy with a global edge URL: `https://your-app.koyeb.app`. --- ### 🥉 Option 3: Render (Free Web Service) 1. Go to [Render.com](https://render.com) and click **"New Web Service"**. 2. Connect your GitHub repository. 3. Select Environment: **Docker**. 4. Free plan: Click **Deploy**. --- ## 👑 Professional Custom Domain & Global CDN Setup (Cloudflare) To give your project a professional SaaS brand (e.g., `api.yourbrand.com` or `search.yourbrand.com`): 1. **Add Domain to Cloudflare** (Free Plan). 2. In Cloudflare DNS settings: - Type: `CNAME` - Name: `api` (or `@`) - Target: `YOUR_USERNAME-hyperlookup-api.hf.space` - Proxy status: **Proxied (Orange Cloud)**. 3. In Cloudflare **SSL/TLS**: - Set encryption mode to **Full (strict)**. 4. In Cloudflare **Caching**: - Create a Cache Rule for `/api/lookup*` with Edge TTL 12 hours. - Now repeated queries across the globe are answered from Cloudflare's 300+ Edge Data Centers in **under 20 milliseconds**! --- ## 💻 Local Development & Testing ### 1. Install Requirements ```bash pip install -r requirements.txt ``` ### 2. Run the Server ```bash uvicorn main:app --host 0.0.0.0 --port 8000 --reload ``` ### 3. Open in Browser - **SaaS Dashboard**: [http://localhost:8000](http://localhost:8000) - **Interactive Swagger Docs**: [http://localhost:8000/docs](http://localhost:8000/docs) - **System Health & Stats**: [http://localhost:8000/api/stats](http://localhost:8000/api/stats) --- ## 📡 API Reference ### 1. Auto-Detect Lookup Automatically identifies whether input is a 10-digit Phone or 12-digit Aadhar: ```bash GET /api/lookup?q=9313336070 ``` #### Sample JSON Response: ```json { "success": true, "query_type": "phone", "query": "9313336070", "formatted_query": "+91 9313336070", "count": 2, "records": [ { "name": "Aakash Jagdishbhai Makhija", "fathersName": "Makhija Bhai", "phoneNumber": "9313336070", "aadharNumber": null, "otherNumber": "7567303287", "address": "705/5, Ambawadi Near Police Station, Ahmedabad, Gujarat, 382475", "district": "Ahmedabad", "pincode": "382475", "state": "Gujarat", "town": null, "source": "inddata" } ], "target_partitions": ["idx_phone.4.parquet"], "latency_ms": 218.4, "cached": false } ``` ### 2. Phone Lookup ```bash GET /api/phone/9313336070 ``` ### 3. Aadhar Lookup ```bash GET /api/aadhar/351104219878 ``` ### 4. Telegram User ID Lookup (Global Master DB - 65M+ Records) Fast, direct partition-routed lookup for Telegram accounts by User ID: ```bash GET /tg/lookup?user_id=74252719 # or direct path: GET /tg/74252719 ``` #### Sample Telegram JSON Response: ```json { "status": "success", "user_id": 74252719, "country": "Iran", "num": "+989192125069", "phone": "+989192125069", "formatted_phone": "+98 919 212 5069", "country_code": "IR", "query_type": "telegram", "target_partition": "part_0000_1_74252719.parquet", "latency_ms": 14.8, "cached": false } ``` ### 5. System Statistics ```bash GET /api/stats ``` --- ## 📦 Client Code Examples ### Python: ```python import requests url = "https://your-domain.com/api/lookup" response = requests.get(url, params={"q": "9313336070"}) data = response.json() print(f"Matches: {data['count']} in {data['latency_ms']} ms") for record in data['records']: print(record['name'], record['phoneNumber'], record['address']) ``` ### JavaScript / Node.js: ```javascript const res = await fetch("https://your-domain.com/api/lookup?q=9313336070"); const data = await res.json(); console.log(`Latency: ${data.latency_ms}ms`, data.records); ``` ### cURL: ```bash curl -X GET "https://your-domain.com/api/lookup?q=9313336070" ``` --- ## 🛡️ License & Disclaimers This repository is engineered for ultra-fast columnar analytical lookups using open-source tools (FastAPI, DuckDB, Hugging Face Hub).