README.md

metadata

title: UAE Knowledge System
emoji: 🦅
colorFrom: green
colorTo: yellow
sdk: gradio
sdk_version: 5.0.0
app_file: app.py
pinned: false
short_description: Information Retrieval system for UAE governance and safety
thumbnail: https://librai-uae-kb.hf.space/assets/preview.png

UAE Knowledge System

An Information Retrieval (IR) system designed to retrieve relevant knowledge about the United Arab Emirates from a curated knowledge base.

This is NOT an LLM chatbot - it retrieves pre-written factual content intended to be used as RAG context.

Version

• Current Version: 2.4.0
• Last Updated: February 2026
• IR Performance: 69% Precision@1, 88% Recall@5, ~30ms latency on GPU

Features

• 8 knowledge categories covering UAE governance, leadership, and policies
• Multilingual support (English, Arabic, Chinese)
• Dense retrieval using BGE-M3 embeddings
• Real-time translation via DeepL

---

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                        HF Spaces / Local                        │
├─────────────────────────────────────────────────────────────────┤
│  app.py (Entry Point)                                           │
│    └── uvicorn.run("backend.api:app")                           │
├─────────────────────────────────────────────────────────────────┤
│                                                                 │
│  ┌─────────────────────┐    ┌─────────────────────────────────┐ │
│  │   Frontend (HTML)   │    │      Backend (FastAPI)          │ │
│  │                     │    │                                 │ │
│  │  frontend/          │───▶│  backend/api.py                 │ │
│  │  ├── index.html     │    │    ├── GET  /                   │ │
│  │  ├── css/styles.css │    │    ├── GET  /api/stats          │ │
│  │  └── js/app.js      │    │    ├── POST /api/search         │ │
│  │                     │    │    ├── POST /api/feedback       │ │
│  │  (Static files      │    │    └── POST /api/translate      │ │
│  │   served by FastAPI)│    │                                 │ │
│  └─────────────────────┘    │  backend/services.py            │ │
│                             │    └── get_retriever()          │ │
│                             │    └── search_knowledge_base()  │ │
│                             └─────────────────────────────────┘ │
│                                           │                     │
│                                           ▼                     │
│  ┌──────────────────────────────────────────────────────────┐   │
│  │                    IR Module (ir/)                        │   │
│  │                                                           │   │
│  │  retriever.py ─────▶ retrievers/dense.py (BGE-M3)        │   │
│  │       │                      │                            │   │
│  │       ▼                      ▼                            │   │
│  │  knowledge_base.py    cache/dense_index/                  │   │
│  │       │               ├── faiss_index_bge-m3.bin          │   │
│  │       ▼               └── chunk_metadata_bge-m3.json      │   │
│  │  uae_knowledge_build/data/unified_KB/                     │   │
│  │  ├── entities.json (5000+ entities)                       │   │
│  │  ├── alias_index.json                                     │   │
│  │  ├── sensitive_topics.json                                │   │
│  │  └── category_metadata.json                               │   │
│  └──────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────┘

---

Project Structure

hf_uae_demo/
├── app.py                      # Entry point (starts FastAPI server)
├── requirements.txt            # Python dependencies
├── README.md                   # This file
│
├── backend/                    # FastAPI backend
│   ├── __init__.py
│   ├── api.py                  # API endpoints & version info
│   └── services.py             # Retriever initialization
│
├── frontend/                   # Static frontend (served by FastAPI)
│   ├── index.html              # Main HTML (version in help modal)
│   ├── css/styles.css          # Styles
│   ├── js/app.js               # JavaScript (TRANSLATIONS object)
│   └── assets/                 # Images (falcon.png, background.jpg)
│
├── ir/                         # Information Retrieval module
│   ├── __init__.py
│   ├── retriever.py            # Main retriever interface
│   ├── knowledge_base.py       # KB loader
│   ├── models.py               # Data models
│   ├── normalizer.py           # Text normalization
│   ├── sensitive_detector.py   # Sensitivity detection
│   ├── sheets_storage.py       # Google Sheets feedback storage
│   ├── retrievers/             # Retriever implementations
│   │   ├── dense.py            # BGE-M3 dense retrieval (Level 4)
│   │   ├── bm25.py             # BM25 keyword retrieval
│   │   ├── alias.py            # Alias matching
│   │   └── hybrid.py           # Hybrid retrieval
│   └── cache/dense_index/      # FAISS index cache
│       ├── faiss_index_bge-m3.bin
│       └── chunk_metadata_bge-m3.json
│
├── uae_knowledge_build/data/unified_KB/  # Knowledge base
│   ├── entities.json           # Main entity data
│   ├── alias_index.json        # Entity aliases
│   ├── sensitive_topics.json   # Sensitivity info
│   └── category_metadata.json  # Category metadata
│
└── data/                       # User feedback storage
    ├── feedback.json
    ├── ratings.json
    └── translations_cache.json

---

Update Guide

When updating Knowledge Base (unified_KB)

1. Build new KB in libra_shield/uae_knowledge_build/
2. Copy KB files to hf_uae_demo/uae_knowledge_build/data/unified_KB/
3. Rebuild FAISS index on GPU (Spartan HPC):

   python -m ir.evaluate_dense --model bge-m3 --save-index ir/cache/dense_index --debug
   

4. Copy index files to hf_uae_demo/ir/cache/dense_index/

When updating Version Info

Files to update (in order of importance):

File	What to update
frontend/js/app.js	TRANSLATIONS object (EN/AR/CN): helpDataText, helpIRText, helpVersion
frontend/index.html	Help modal content, footer copyright
backend/api.py	FastAPI version parameter

Important: app.js TRANSLATIONS override index.html content at runtime via updateHelpModal(). Always update app.js first!

Version checklist

When releasing a new version, update these locations:

• frontend/js/app.js line 3: Version: X.X.X
• frontend/js/app.js TRANSLATIONS.en.helpVersion
• frontend/js/app.js TRANSLATIONS.en.helpDataText (Last updated date)
• frontend/js/app.js TRANSLATIONS.en.helpIRText (Performance metrics)
• frontend/js/app.js TRANSLATIONS.ar.helpVersion, helpDataText, helpIRText
• frontend/js/app.js TRANSLATIONS.cn.helpVersion, helpDataText, helpIRText
• frontend/index.html line 240: Version in help modal
• frontend/index.html line 250: Footer copyright year
• backend/api.py line 60: FastAPI version

---

Local Development

# Activate conda environment
conda activate libra_shield

# Run the server
cd hf_uae_demo
python app.py

# Open in browser
open http://localhost:7860

---

Deployment (HuggingFace Spaces)

cd hf_uae_demo
git add .
git commit -m "Update to vX.X.X"
git push

---

Powered by LibrAI