File size: 9,215 Bytes
5b7cefd 0f489c0 ba403c6 5b7cefd 459c4bf 0f489c0 5b7cefd 0f489c0 94b2282 0f489c0 94b2282 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 | ---
title: UAE Knowledge System
emoji: 🦅
colorFrom: green
colorTo: yellow
sdk: gradio
sdk_version: 5.0.0
app_file: app.py
pinned: false
short_description: Information Retrieval system for UAE governance and safety
thumbnail: https://librai-uae-kb.hf.space/assets/preview.png
---
# UAE Knowledge System
An Information Retrieval (IR) system designed to retrieve relevant knowledge about the United Arab Emirates from a curated knowledge base.
**This is NOT an LLM chatbot** - it retrieves pre-written factual content intended to be used as RAG context.
## Version
- **Current Version**: 2.4.0
- **Last Updated**: February 2026
- **IR Performance**: 69% Precision@1, 88% Recall@5, ~30ms latency on GPU
## Features
- 8 knowledge categories covering UAE governance, leadership, and policies
- Multilingual support (English, Arabic, Chinese)
- Dense retrieval using BGE-M3 embeddings
- Real-time translation via DeepL
---
## Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ HF Spaces / Local │
├─────────────────────────────────────────────────────────────────┤
│ app.py (Entry Point) │
│ └── uvicorn.run("backend.api:app") │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────────┐ ┌─────────────────────────────────┐ │
│ │ Frontend (HTML) │ │ Backend (FastAPI) │ │
│ │ │ │ │ │
│ │ frontend/ │───▶│ backend/api.py │ │
│ │ ├── index.html │ │ ├── GET / │ │
│ │ ├── css/styles.css │ │ ├── GET /api/stats │ │
│ │ └── js/app.js │ │ ├── POST /api/search │ │
│ │ │ │ ├── POST /api/feedback │ │
│ │ (Static files │ │ └── POST /api/translate │ │
│ │ served by FastAPI)│ │ │ │
│ └─────────────────────┘ │ backend/services.py │ │
│ │ └── get_retriever() │ │
│ │ └── search_knowledge_base() │ │
│ └─────────────────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ IR Module (ir/) │ │
│ │ │ │
│ │ retriever.py ─────▶ retrievers/dense.py (BGE-M3) │ │
│ │ │ │ │ │
│ │ ▼ ▼ │ │
│ │ knowledge_base.py cache/dense_index/ │ │
│ │ │ ├── faiss_index_bge-m3.bin │ │
│ │ ▼ └── chunk_metadata_bge-m3.json │ │
│ │ uae_knowledge_build/data/unified_KB/ │ │
│ │ ├── entities.json (5000+ entities) │ │
│ │ ├── alias_index.json │ │
│ │ ├── sensitive_topics.json │ │
│ │ └── category_metadata.json │ │
│ └──────────────────────────────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────────┘
```
---
## Project Structure
```
hf_uae_demo/
├── app.py # Entry point (starts FastAPI server)
├── requirements.txt # Python dependencies
├── README.md # This file
│
├── backend/ # FastAPI backend
│ ├── __init__.py
│ ├── api.py # API endpoints & version info
│ └── services.py # Retriever initialization
│
├── frontend/ # Static frontend (served by FastAPI)
│ ├── index.html # Main HTML (version in help modal)
│ ├── css/styles.css # Styles
│ ├── js/app.js # JavaScript (TRANSLATIONS object)
│ └── assets/ # Images (falcon.png, background.jpg)
│
├── ir/ # Information Retrieval module
│ ├── __init__.py
│ ├── retriever.py # Main retriever interface
│ ├── knowledge_base.py # KB loader
│ ├── models.py # Data models
│ ├── normalizer.py # Text normalization
│ ├── sensitive_detector.py # Sensitivity detection
│ ├── sheets_storage.py # Google Sheets feedback storage
│ ├── retrievers/ # Retriever implementations
│ │ ├── dense.py # BGE-M3 dense retrieval (Level 4)
│ │ ├── bm25.py # BM25 keyword retrieval
│ │ ├── alias.py # Alias matching
│ │ └── hybrid.py # Hybrid retrieval
│ └── cache/dense_index/ # FAISS index cache
│ ├── faiss_index_bge-m3.bin
│ └── chunk_metadata_bge-m3.json
│
├── uae_knowledge_build/data/unified_KB/ # Knowledge base
│ ├── entities.json # Main entity data
│ ├── alias_index.json # Entity aliases
│ ├── sensitive_topics.json # Sensitivity info
│ └── category_metadata.json # Category metadata
│
└── data/ # User feedback storage
├── feedback.json
├── ratings.json
└── translations_cache.json
```
---
## Update Guide
### When updating Knowledge Base (unified_KB)
1. **Build new KB** in `libra_shield/uae_knowledge_build/`
2. **Copy KB files** to `hf_uae_demo/uae_knowledge_build/data/unified_KB/`
3. **Rebuild FAISS index** on GPU (Spartan HPC):
```bash
python -m ir.evaluate_dense --model bge-m3 --save-index ir/cache/dense_index --debug
```
4. **Copy index files** to `hf_uae_demo/ir/cache/dense_index/`
### When updating Version Info
**Files to update (in order of importance):**
| File | What to update |
|------|----------------|
| `frontend/js/app.js` | `TRANSLATIONS` object (EN/AR/CN): `helpDataText`, `helpIRText`, `helpVersion` |
| `frontend/index.html` | Help modal content, footer copyright |
| `backend/api.py` | FastAPI `version` parameter |
**Important**: `app.js` TRANSLATIONS override `index.html` content at runtime via `updateHelpModal()`. Always update `app.js` first!
### Version checklist
When releasing a new version, update these locations:
- [ ] `frontend/js/app.js` line 3: `Version: X.X.X`
- [ ] `frontend/js/app.js` TRANSLATIONS.en.helpVersion
- [ ] `frontend/js/app.js` TRANSLATIONS.en.helpDataText (Last updated date)
- [ ] `frontend/js/app.js` TRANSLATIONS.en.helpIRText (Performance metrics)
- [ ] `frontend/js/app.js` TRANSLATIONS.ar.helpVersion, helpDataText, helpIRText
- [ ] `frontend/js/app.js` TRANSLATIONS.cn.helpVersion, helpDataText, helpIRText
- [ ] `frontend/index.html` line 240: Version in help modal
- [ ] `frontend/index.html` line 250: Footer copyright year
- [ ] `backend/api.py` line 60: FastAPI version
---
## Local Development
```bash
# Activate conda environment
conda activate libra_shield
# Run the server
cd hf_uae_demo
python app.py
# Open in browser
open http://localhost:7860
```
---
## Deployment (HuggingFace Spaces)
```bash
cd hf_uae_demo
git add .
git commit -m "Update to vX.X.X"
git push
```
---
Powered by [LibrAI](https://www.librai.tech/)
|