gnai-creator commited on
Commit
ad18bea
Β·
1 Parent(s): 8b2fcf8

1M training data

Browse files
DEPLOY_GUIDE.md DELETED
@@ -1,238 +0,0 @@
1
- # πŸš€ HuggingFace Space Deployment Guide
2
-
3
- This guide will help you deploy the AletheionGuard Space to HuggingFace.
4
-
5
- ## πŸ“‹ Prerequisites
6
-
7
- 1. **HuggingFace Account**: Sign up at https://huggingface.co/join
8
- 2. **Git LFS**: Install Git Large File Storage
9
- ```bash
10
- # Ubuntu/Debian
11
- sudo apt-get install git-lfs
12
-
13
- # macOS
14
- brew install git-lfs
15
-
16
- # Initialize
17
- git lfs install
18
- ```
19
- 3. **HuggingFace CLI** (optional but recommended):
20
- ```bash
21
- pip install huggingface_hub
22
- huggingface-cli login
23
- ```
24
-
25
- ## πŸ”‘ Step 1: Create Access Token
26
-
27
- 1. Go to https://huggingface.co/settings/tokens
28
- 2. Click **New token**
29
- 3. Name: `AletheionGuard Deploy`
30
- 4. Type: **Write** (needed to push to Space)
31
- 5. Copy the token (starts with `hf_...`)
32
-
33
- ## πŸ“¦ Step 2: Create New Space
34
-
35
- ### Option A: Via Web Interface (Easiest)
36
-
37
- 1. Go to https://huggingface.co/new-space
38
- 2. Fill in details:
39
- - **Owner**: Your username or organization
40
- - **Space name**: `aletheionguard` (or your preferred name)
41
- - **License**: AGPL-3.0
42
- - **SDK**: Docker
43
- - **Visibility**: Public or Private (choose Private for BYO-HF mode)
44
- 3. Click **Create Space**
45
- 4. You'll be redirected to your new Space
46
-
47
- ### Option B: Via CLI
48
-
49
- ```bash
50
- huggingface-cli repo create aletheionguard --type space --space_sdk docker
51
- ```
52
-
53
- ## πŸ“€ Step 3: Push Files to Space
54
-
55
- Navigate to the Space directory:
56
-
57
- ```bash
58
- cd /home/sapo/AletheionGuard/hf_space_example/AletheionGuard
59
- ```
60
-
61
- ### First-time Setup (if not already a git repo)
62
-
63
- ```bash
64
- # Check if already initialized
65
- git status
66
-
67
- # If not, initialize
68
- git init
69
- git lfs install
70
- git lfs track "*.pth"
71
- git lfs track "*.ckpt"
72
- ```
73
-
74
- ### Add HuggingFace Remote
75
-
76
- Replace `YOUR_USERNAME` with your HuggingFace username:
77
-
78
- ```bash
79
- git remote add hf https://huggingface.co/spaces/YOUR_USERNAME/aletheionguard
80
- ```
81
-
82
- ### Commit and Push
83
-
84
- ```bash
85
- # Stage all files
86
- git add .
87
-
88
- # Commit
89
- git commit -m "Initial commit: AletheionGuard Space with Trial 012 models"
90
-
91
- # Push to HuggingFace
92
- git push hf main
93
- ```
94
-
95
- If prompted for credentials:
96
- - **Username**: Your HuggingFace username
97
- - **Password**: Your HF token (starts with `hf_...`)
98
-
99
- ## πŸ”§ Step 4: Configure Space Settings
100
-
101
- 1. Go to your Space: `https://huggingface.co/spaces/YOUR_USERNAME/aletheionguard`
102
- 2. Click **Settings**
103
- 3. Configure:
104
- - **Hardware**: CPU Basic (free) or upgrade if needed
105
- - **Persistent Storage**: Optional (0 GB for this Space)
106
- - **Secrets**: Add `HUGGINGFACE_TOKEN` if needed for authentication
107
- 4. Click **Save**
108
-
109
- ## βœ… Step 5: Verify Deployment
110
-
111
- 1. Wait for build to complete (check **Build logs** tab)
112
- 2. Once "Running", test the endpoints:
113
-
114
- ```bash
115
- # Health check
116
- curl https://YOUR_USERNAME-aletheionguard.hf.space/health
117
-
118
- # Predict (requires auth if Space is private)
119
- curl -X POST https://YOUR_USERNAME-aletheionguard.hf.space/predict \
120
- -H "Authorization: Bearer hf_YOUR_TOKEN" \
121
- -H "Content-Type: application/json" \
122
- -d '{"text": "Paris is the capital of France"}'
123
- ```
124
-
125
- Expected response:
126
- ```json
127
- {
128
- "q1": 0.06,
129
- "q2": 0.08,
130
- "height": 0.90,
131
- "message": "Heuristic metrics computed successfully.",
132
- "verdict": "ACCEPT"
133
- }
134
- ```
135
-
136
- ## πŸ”„ Step 6: Update Models (After Fine-tuning)
137
-
138
- After fine-tuning with `scripts/finetune_real_data.py`:
139
-
140
- ```bash
141
- # Copy new models
142
- cp /home/sapo/AletheionGuard/models/real_finetuned/*.pth .
143
- cp /home/sapo/AletheionGuard/models/real_finetuned/q1q2-finetuned-*.ckpt q1q2_best.ckpt
144
-
145
- # Update metadata
146
- nano model_info.json # Update training info and metrics
147
-
148
- # Commit and push
149
- git add *.pth *.ckpt model_info.json
150
- git commit -m "Update to fine-tuned models (TruthfulQA + SQuAD)"
151
- git push hf main
152
- ```
153
-
154
- ## πŸ“Š Monitoring & Logs
155
-
156
- - **View logs**: Click **Logs** tab in your Space
157
- - **Restart Space**: Click **Factory reboot** if needed
158
- - **Monitor usage**: Check **Analytics** tab
159
-
160
- ## πŸ’° Costs & Limits
161
-
162
- ### Free Tier (CPU Basic)
163
- - βœ… Perfect for demos and low-traffic
164
- - βœ… Includes: 2 vCPU, 16 GB RAM
165
- - ❌ Limited to ~1000 requests/day
166
- - ❌ May sleep after inactivity
167
-
168
- ### Paid Tier (Upgrade Options)
169
- - **CPU Upgrade** ($0.03/hour): 4 vCPU, 32 GB RAM
170
- - **GPU T4** ($0.60/hour): For faster inference
171
- - **Persistent**: Space never sleeps
172
-
173
- To upgrade:
174
- 1. Go to Space β†’ Settings β†’ Hardware
175
- 2. Select tier β†’ Upgrade
176
-
177
- ## πŸ” Using with BYO-HF Mode
178
-
179
- After deployment, use your Space with AletheionGuard:
180
-
181
- ```python
182
- from aletheion_guard import EpistemicAuditor
183
-
184
- auditor = EpistemicAuditor(
185
- mode="byo-hf",
186
- hf_space_url="https://huggingface.co/spaces/YOUR_USERNAME/aletheionguard",
187
- hf_token="hf_YOUR_TOKEN" # Only if Space is private
188
- )
189
-
190
- result = auditor.evaluate("The Earth is flat")
191
- print(f"Verdict: {result.verdict}, Height: {result.height}")
192
- ```
193
-
194
- ## πŸ› Troubleshooting
195
-
196
- ### Build Failed
197
-
198
- Check **Build logs** for errors. Common issues:
199
-
200
- 1. **LFS bandwidth exceeded**: Use HuggingFace Pro ($9/month)
201
- 2. **File too large**: Compress models or use smaller batch size during training
202
- 3. **Docker build timeout**: Simplify Dockerfile or reduce dependencies
203
-
204
- ### Space Not Starting
205
-
206
- 1. Check **Logs** tab for runtime errors
207
- 2. Verify `app.py` has no syntax errors
208
- 3. Ensure all model files are present
209
- 4. Try **Factory reboot**
210
-
211
- ### Authentication Errors
212
-
213
- 1. Ensure HF token has **Write** permissions
214
- 2. For private Spaces, include token in requests
215
- 3. Check token expiration
216
-
217
- ### Slow Inference
218
-
219
- 1. Upgrade to GPU hardware
220
- 2. Optimize model (quantization, pruning)
221
- 3. Use caching for repeated requests
222
-
223
- ## πŸ“š Resources
224
-
225
- - **HuggingFace Spaces Docs**: https://huggingface.co/docs/hub/spaces
226
- - **Git LFS**: https://git-lfs.github.com/
227
- - **Docker for Spaces**: https://huggingface.co/docs/hub/spaces-sdks-docker
228
- - **AletheionGuard Docs**: https://docs.aletheion.com
229
-
230
- ## πŸ’¬ Support
231
-
232
- - **Email**: support@aletheion.com
233
- - **Discord**: https://discord.gg/aletheion
234
- - **GitHub Issues**: https://github.com/AletheionAGI/AletheionGuard/issues
235
-
236
- ---
237
-
238
- **Ready to deploy?** Follow the steps above and you'll have your Space running in minutes! πŸš€
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
FIX_GIT_LFS.md DELETED
@@ -1,125 +0,0 @@
1
- # πŸ”§ Fix Git LFS for HuggingFace Push
2
-
3
- O push falhou porque o Git LFS nΓ£o estΓ‘ configurado. Siga estes passos:
4
-
5
- ## Passo 1: Instalar Git LFS
6
-
7
- ```bash
8
- # Para Ubuntu/Debian/WSL
9
- curl -s https://packagecloud.io/install/repositories/github/git-lfs/script.deb.sh | sudo bash
10
- sudo apt-get install git-lfs
11
-
12
- # Verificar instalaΓ§Γ£o
13
- git lfs version
14
- ```
15
-
16
- ## Passo 2: Configurar Git LFS no RepositΓ³rio
17
-
18
- ```bash
19
- cd /home/sapo/AletheionGuard/hf_space_example/AletheionGuard
20
-
21
- # Inicializar Git LFS
22
- git lfs install
23
-
24
- # Verificar que .gitattributes existe e tem as configuraΓ§Γ΅es corretas
25
- cat .gitattributes
26
- ```
27
-
28
- ## Passo 3: Resetar o Commit Anterior e Re-adicionar com LFS
29
-
30
- ```bash
31
- # Voltar um commit (desfazer o commit que falhou no push)
32
- git reset --soft HEAD~1
33
-
34
- # Verificar arquivos que precisam de LFS
35
- ls -lh *.pth *.ckpt
36
-
37
- # Re-adicionar arquivos (agora com LFS)
38
- git add .gitattributes
39
- git add *.pth
40
- git add *.ckpt
41
- git add *.py
42
- git add *.md
43
- git add *.txt
44
- git add Dockerfile
45
-
46
- # Verificar que os arquivos grandes estΓ£o no LFS
47
- git lfs ls-files
48
-
49
- # Commit novamente
50
- git commit -m "Initial commit: AletheionGuard Space with Trial 012 models"
51
- ```
52
-
53
- ## Passo 4: Push para HuggingFace
54
-
55
- ```bash
56
- # Push
57
- git push
58
-
59
- # Se pedir credenciais:
60
- # Username: gnai-creator
61
- # Password: seu token HF (hf_...)
62
- ```
63
-
64
- ## Verificar Arquivos LFS
65
-
66
- ApΓ³s adicionar os arquivos, vocΓͺ deve ver algo assim:
67
-
68
- ```bash
69
- $ git lfs ls-files
70
- c9a5b2e1f3 * base_forces.pth
71
- d4f8c7a2b1 * height_gate.pth
72
- e3d9b4f8c2 * q1_gate.pth
73
- f2e8a3d7c1 * q2_gate.pth
74
- a1b2c3d4e5 * q1q2_best.ckpt
75
- ```
76
-
77
- ## SoluΓ§Γ£o Alternativa (Se Git LFS NΓ£o Funcionar)
78
-
79
- Se vocΓͺ nΓ£o conseguir instalar o Git LFS, pode:
80
-
81
- 1. **Remover os modelos grandes** e fazer upload manual depois:
82
- ```bash
83
- git rm --cached *.pth *.ckpt
84
- git commit -m "Remove large files temporarily"
85
- git push
86
- ```
87
-
88
- 2. **Upload manual pelo site**:
89
- - VΓ‘ para https://huggingface.co/spaces/gnai-creator/AletheionGuard
90
- - Clique em "Files" β†’ "Upload files"
91
- - FaΓ§a upload dos arquivos `.pth` e `.ckpt` manualmente
92
-
93
- 3. **Ou use modelos menores/mock** para demonstraΓ§Γ£o:
94
- - Edite `app.py` para usar apenas heurΓ­sticas (jΓ‘ implementado como fallback)
95
- - Remove a necessidade dos arquivos `.pth`
96
-
97
- ## Comandos Úteis
98
-
99
- ```bash
100
- # Ver status do LFS
101
- git lfs status
102
-
103
- # Ver arquivos rastreados pelo LFS
104
- git lfs ls-files
105
-
106
- # Ver tamanho dos arquivos
107
- du -sh *.pth *.ckpt
108
-
109
- # Limpar cache LFS se necessΓ‘rio
110
- git lfs prune
111
- ```
112
-
113
- ## Problema Comum: "last.ckpt"
114
-
115
- O arquivo `last.ckpt` nΓ£o deveria estar no repositΓ³rio. Para removΓͺ-lo:
116
-
117
- ```bash
118
- # Se ele foi adicionado por engano
119
- git rm last.ckpt
120
- git commit -m "Remove last.ckpt"
121
- ```
122
-
123
- ---
124
-
125
- **Depois de seguir estes passos, tente o push novamente!** πŸš€
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app.py.backup DELETED
@@ -1,158 +0,0 @@
1
- # SPDX-License-Identifier: AGPL-3.0-or-later
2
- # Copyright (c) 2024-2025 Felipe Maya Muniz
3
-
4
- """
5
- Reference Hugging Face Space for AletheionGuard BYO-HF mode.
6
-
7
- This is a minimal FastAPI endpoint that clients can deploy on Hugging Face Spaces
8
- to use with AletheionGuard's BYO-HF mode.
9
-
10
- Deploy this Space as PRIVATE and use your HF token + Space URL with AletheionGuard.
11
- """
12
-
13
- from fastapi import FastAPI, HTTPException, Header
14
- from pydantic import BaseModel
15
- from typing import Optional
16
- import logging
17
- import math
18
-
19
- logging.basicConfig(level=logging.INFO)
20
- logger = logging.getLogger(__name__)
21
-
22
- app = FastAPI(
23
- title="AletheionGuard HF Space",
24
- description="Reference endpoint for BYO-HF mode",
25
- version="1.0.0"
26
- )
27
-
28
-
29
- class PredictRequest(BaseModel):
30
- """Request model for /predict endpoint."""
31
- text: str
32
- context: Optional[str] = None
33
-
34
-
35
- class PredictResponse(BaseModel):
36
- """Response model for /predict endpoint."""
37
- q1: float
38
- q2: float
39
- height: float
40
- message: str
41
- verdict: Optional[str] = None # Optional debug field - NOT used by API
42
-
43
-
44
- def get_verdict(q1: float, q2: float, height: float) -> str:
45
- """
46
- Calculate verdict for debug purposes only.
47
-
48
- NOTE: This is NOT the official verdict. The official verdict is always
49
- calculated by the AletheionGuard API using the same rule.
50
-
51
- Official epistemic rule:
52
- - u = 1.0 - height (total uncertainty)
53
- - If q2 >= 0.35 OR u >= 0.60 β†’ REFUSED
54
- - If q1 >= 0.35 OR (0.30 <= u < 0.60) β†’ MAYBE
55
- - Otherwise β†’ ACCEPT
56
- """
57
- u = 1.0 - height # Total uncertainty
58
-
59
- if q2 >= 0.35 or u >= 0.60:
60
- return "REFUSED"
61
- if q1 >= 0.35 or (0.30 <= u < 0.60):
62
- return "MAYBE"
63
- return "ACCEPT"
64
-
65
-
66
- @app.get("/")
67
- def root():
68
- """Root endpoint."""
69
- return {
70
- "name": "AletheionGuard HF Space",
71
- "version": "1.0.0",
72
- "status": "operational"
73
- }
74
-
75
-
76
- @app.post("/predict", response_model=PredictResponse)
77
- def predict(
78
- request: PredictRequest,
79
- authorization: str = Header(...)
80
- ):
81
- """
82
- Predict endpoint for text analysis.
83
-
84
- Returns heuristic uncertainty metrics (q1, q2, height) and optional verdict.
85
-
86
- NOTE: This is an MVP implementation using heuristics. For production:
87
- 1. Load a sentence-transformer model
88
- 2. Use trained Q1/Q2 gates to compute actual metrics
89
- 3. Return embeddings/logits for calibration
90
-
91
- Args:
92
- request: Text and optional context
93
- authorization: Bearer token (verified by HF automatically)
94
-
95
- Returns:
96
- Heuristic metrics with optional debug verdict
97
-
98
- Example:
99
- >>> POST /predict
100
- >>> Headers: Authorization: Bearer hf_...
101
- >>> Body: {"text": "Paris is the capital of France", "context": "geography"}
102
- >>> Response: {"q1": 0.06, "q2": 0.18, "height": 0.81, "verdict": "ACCEPT"}
103
- """
104
- try:
105
- logger.info(f"Received prediction request - text_length={len(request.text)}")
106
-
107
- # MVP: Compute heuristic metrics (replace with actual model in production)
108
- # Simple heuristics based on text characteristics:
109
- text_len = len(request.text)
110
- word_count = len(request.text.split())
111
- has_context = request.context is not None
112
-
113
- # Heuristic Q1 (aleatoric): based on text ambiguity indicators
114
- # Lower for factual statements, higher for opinion/uncertain language
115
- q1 = min(0.30, 0.05 + (word_count / 200)) # Increases with verbosity
116
- if any(word in request.text.lower() for word in ["maybe", "possibly", "might", "could"]):
117
- q1 += 0.15
118
-
119
- # Heuristic Q2 (epistemic): based on model confidence indicators
120
- # Lower for common topics, higher for rare/complex topics
121
- q2 = 0.10 if text_len > 20 else 0.20 # More text = more context
122
- if has_context:
123
- q2 -= 0.05 # Context helps reduce epistemic uncertainty
124
- if any(word in request.text.lower() for word in ["quantum", "theoretical", "hypothetical"]):
125
- q2 += 0.20
126
-
127
- # Ensure bounds [0, 1]
128
- q1 = max(0.0, min(1.0, q1))
129
- q2 = max(0.0, min(1.0, q2))
130
-
131
- # Compute height from pyramidal formula
132
- height = max(0.0, min(1.0, 1.0 - math.sqrt(q1**2 + q2**2)))
133
-
134
- # Compute verdict (optional debug field)
135
- verdict = get_verdict(q1, q2, height)
136
-
137
- return PredictResponse(
138
- q1=round(q1, 3),
139
- q2=round(q2, 3),
140
- height=round(height, 3),
141
- message="Heuristic metrics computed successfully.",
142
- verdict=verdict # Debug only - API ignores this
143
- )
144
-
145
- except Exception as e:
146
- logger.error(f"Prediction failed: {str(e)}")
147
- raise HTTPException(status_code=500, detail=str(e))
148
-
149
-
150
- @app.get("/health")
151
- def health():
152
- """Health check endpoint."""
153
- return {"status": "healthy"}
154
-
155
-
156
- if __name__ == "__main__":
157
- import uvicorn
158
- uvicorn.run(app, host="0.0.0.0", port=7860) # HF Spaces use port 7860
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
app_fixed.py DELETED
@@ -1,351 +0,0 @@
1
- # SPDX-License-Identifier: AGPL-3.0-or-later
2
- # Copyright (c) 2024-2025 Felipe Maya Muniz
3
-
4
- """
5
- Production Hugging Face Space for AletheionGuard.
6
-
7
- This endpoint loads the trained neural models and provides accurate
8
- epistemic uncertainty estimation using the full AletheionGuard architecture.
9
- """
10
-
11
- from fastapi import FastAPI, HTTPException, Header
12
- from pydantic import BaseModel
13
- from typing import Optional
14
- import logging
15
- import math
16
- import torch
17
- import torch.nn as nn
18
- from sentence_transformers import SentenceTransformer
19
- from pathlib import Path
20
-
21
- logging.basicConfig(level=logging.INFO)
22
- logger = logging.getLogger(__name__)
23
-
24
- app = FastAPI(
25
- title="AletheionGuard HF Space",
26
- description="Production epistemic uncertainty estimation",
27
- version="2.0.0"
28
- )
29
-
30
-
31
- # ============================================================================
32
- # Model Definitions (copied from q1q2_gates.py)
33
- # ============================================================================
34
-
35
- class UncertaintyNetwork(nn.Module):
36
- """Base neural network for uncertainty estimation."""
37
-
38
- def __init__(
39
- self,
40
- input_dim: int = 384,
41
- hidden_dim: int = 256,
42
- num_layers: int = 3,
43
- dropout: float = 0.1
44
- ):
45
- super().__init__()
46
-
47
- self.input_dim = input_dim
48
- self.hidden_dim = hidden_dim
49
- self.num_layers = num_layers
50
-
51
- # Build MLP layers
52
- layers = []
53
-
54
- # Input layer
55
- layers.append(nn.Linear(input_dim, hidden_dim))
56
- layers.append(nn.ReLU())
57
- layers.append(nn.Dropout(dropout))
58
-
59
- # Hidden layers
60
- for _ in range(num_layers - 1):
61
- layers.append(nn.Linear(hidden_dim, hidden_dim))
62
- layers.append(nn.ReLU())
63
- layers.append(nn.Dropout(dropout))
64
-
65
- # Output layer (single uncertainty value)
66
- layers.append(nn.Linear(hidden_dim, 1))
67
- layers.append(nn.Sigmoid()) # Clamp to [0, 1]
68
-
69
- self.network = nn.Sequential(*layers)
70
-
71
- def forward(self, x: torch.Tensor) -> torch.Tensor:
72
- if x.dim() == 1:
73
- x = x.unsqueeze(0)
74
- single_sample = True
75
- else:
76
- single_sample = False
77
-
78
- output = self.network(x)
79
-
80
- if single_sample:
81
- output = output.squeeze(0)
82
-
83
- return output
84
-
85
-
86
- class Q1Gate(nn.Module):
87
- """Aleatoric uncertainty gate (Q1)."""
88
-
89
- def __init__(self, input_dim: int = 384, hidden_dim: int = 256):
90
- super().__init__()
91
- self.network = UncertaintyNetwork(
92
- input_dim=input_dim,
93
- hidden_dim=hidden_dim,
94
- num_layers=3,
95
- dropout=0.1
96
- )
97
-
98
- def forward(self, embeddings: torch.Tensor) -> torch.Tensor:
99
- return self.network(embeddings)
100
-
101
-
102
- class Q2Gate(nn.Module):
103
- """Epistemic uncertainty gate (Q2) - conditioned on Q1."""
104
-
105
- def __init__(self, input_dim: int = 384, hidden_dim: int = 256):
106
- super().__init__()
107
- # Q2 is conditioned on Q1, so input is embeddings + Q1 value
108
- self.network = UncertaintyNetwork(
109
- input_dim=input_dim + 1, # +1 for Q1 conditioning
110
- hidden_dim=hidden_dim,
111
- num_layers=3,
112
- dropout=0.1
113
- )
114
-
115
- def forward(self, embeddings: torch.Tensor, q1: torch.Tensor) -> torch.Tensor:
116
- # Handle single sample
117
- if embeddings.dim() == 1:
118
- embeddings = embeddings.unsqueeze(0)
119
- single_sample = True
120
- else:
121
- single_sample = False
122
-
123
- # Convert Q1 to tensor if needed
124
- if isinstance(q1, float):
125
- q1 = torch.tensor([[q1]], dtype=embeddings.dtype, device=embeddings.device)
126
- elif q1.dim() == 0:
127
- q1 = q1.unsqueeze(0).unsqueeze(0)
128
- elif q1.dim() == 1:
129
- q1 = q1.unsqueeze(1)
130
-
131
- # Concatenate embeddings with Q1 for conditioning
132
- combined = torch.cat([embeddings, q1], dim=1)
133
- output = self.network(combined)
134
-
135
- if single_sample:
136
- output = output.squeeze(0)
137
-
138
- return output
139
-
140
-
141
- # ============================================================================
142
- # Global Model State
143
- # ============================================================================
144
-
145
- class ModelState:
146
- """Global state for loaded models."""
147
-
148
- def __init__(self):
149
- self.encoder = None
150
- self.q1_gate = None
151
- self.q2_gate = None
152
- self.device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
153
- self.loaded = False
154
-
155
- def load_models(self):
156
- """Load all models at startup."""
157
- if self.loaded:
158
- return
159
-
160
- try:
161
- logger.info("πŸ”§ Loading models...")
162
-
163
- # 1. Load sentence transformer for embeddings
164
- logger.info(" Loading sentence transformer...")
165
- self.encoder = SentenceTransformer('all-MiniLM-L6-v2')
166
- self.encoder.eval()
167
- logger.info(" βœ“ Sentence transformer loaded")
168
-
169
- # 2. Load Q1 gate
170
- logger.info(" Loading Q1 gate...")
171
- self.q1_gate = Q1Gate(input_dim=384, hidden_dim=256)
172
- if Path('q1_gate.pth').exists():
173
- self.q1_gate.load_state_dict(torch.load('q1_gate.pth', map_location=self.device))
174
- logger.info(" βœ“ Q1 gate loaded from q1_gate.pth")
175
- else:
176
- logger.warning(" ⚠️ q1_gate.pth not found, using random weights")
177
-
178
- self.q1_gate.to(self.device)
179
- self.q1_gate.eval()
180
-
181
- # 3. Load Q2 gate
182
- logger.info(" Loading Q2 gate...")
183
- self.q2_gate = Q2Gate(input_dim=384, hidden_dim=256)
184
- if Path('q2_gate.pth').exists():
185
- self.q2_gate.load_state_dict(torch.load('q2_gate.pth', map_location=self.device))
186
- logger.info(" βœ“ Q2 gate loaded from q2_gate.pth")
187
- else:
188
- logger.warning(" ⚠️ q2_gate.pth not found, using random weights")
189
-
190
- self.q2_gate.to(self.device)
191
- self.q2_gate.eval()
192
-
193
- self.loaded = True
194
- logger.info(f"βœ… All models loaded successfully (device: {self.device})")
195
-
196
- except Exception as e:
197
- logger.error(f"❌ Failed to load models: {e}")
198
- raise
199
-
200
-
201
- # Global model state
202
- models = ModelState()
203
-
204
-
205
- # ============================================================================
206
- # API Models
207
- # ============================================================================
208
-
209
- class PredictRequest(BaseModel):
210
- """Request model for /predict endpoint."""
211
- text: str
212
- context: Optional[str] = None
213
-
214
-
215
- class PredictResponse(BaseModel):
216
- """Response model for /predict endpoint."""
217
- q1: float
218
- q2: float
219
- height: float
220
- message: str
221
- verdict: Optional[str] = None
222
-
223
-
224
- def get_verdict(q1: float, q2: float, height: float) -> str:
225
- """
226
- Calculate verdict using official epistemic rule.
227
-
228
- Official epistemic rule:
229
- - u = 1.0 - height (total uncertainty)
230
- - If q2 >= 0.35 OR u >= 0.60 β†’ REFUSED
231
- - If q1 >= 0.35 OR (0.30 <= u < 0.60) β†’ MAYBE
232
- - Otherwise β†’ ACCEPT
233
- """
234
- u = 1.0 - height # Total uncertainty
235
-
236
- if q2 >= 0.35 or u >= 0.60:
237
- return "REFUSED"
238
- if q1 >= 0.35 or (0.30 <= u < 0.60):
239
- return "MAYBE"
240
- return "ACCEPT"
241
-
242
-
243
- # ============================================================================
244
- # API Endpoints
245
- # ============================================================================
246
-
247
- @app.on_event("startup")
248
- async def startup_event():
249
- """Load models on startup."""
250
- models.load_models()
251
-
252
-
253
- @app.get("/")
254
- def root():
255
- """Root endpoint."""
256
- return {
257
- "name": "AletheionGuard HF Space",
258
- "version": "2.0.0",
259
- "status": "operational",
260
- "models_loaded": models.loaded
261
- }
262
-
263
-
264
- @app.post("/predict", response_model=PredictResponse)
265
- def predict(
266
- request: PredictRequest,
267
- authorization: str = Header(...)
268
- ):
269
- """
270
- Predict endpoint using trained neural models.
271
-
272
- Returns epistemic uncertainty metrics (q1, q2, height) computed by
273
- the trained AletheionGuard neural networks.
274
-
275
- Args:
276
- request: Text and optional context
277
- authorization: Bearer token (verified by HF automatically)
278
-
279
- Returns:
280
- Neural-computed metrics with verdict
281
-
282
- Example:
283
- >>> POST /predict
284
- >>> Headers: Authorization: Bearer hf_...
285
- >>> Body: {"text": "Paris is the capital of France", "context": "geography"}
286
- >>> Response: {"q1": 0.08, "q2": 0.12, "height": 0.86, "verdict": "ACCEPT"}
287
- """
288
- try:
289
- if not models.loaded:
290
- raise HTTPException(status_code=503, detail="Models not loaded")
291
-
292
- logger.info(f"Received prediction request - text_length={len(request.text)}")
293
-
294
- # Combine text and context for embedding
295
- full_text = request.text
296
- if request.context:
297
- full_text = f"{request.context}: {request.text}"
298
-
299
- # 1. Get embeddings from sentence transformer
300
- with torch.no_grad():
301
- embeddings = models.encoder.encode(
302
- full_text,
303
- convert_to_tensor=True,
304
- device=models.device
305
- )
306
-
307
- # 2. Compute Q1 (aleatoric uncertainty)
308
- q1_tensor = models.q1_gate(embeddings)
309
- q1 = float(q1_tensor.item())
310
-
311
- # 3. Compute Q2 (epistemic uncertainty) - conditioned on Q1
312
- q2_tensor = models.q2_gate(embeddings, q1_tensor)
313
- q2 = float(q2_tensor.item())
314
-
315
- # 4. Compute height from pyramidal formula
316
- # height = 1 - sqrt(q1^2 + q2^2)
317
- height = max(0.0, min(1.0, 1.0 - math.sqrt(q1**2 + q2**2)))
318
-
319
- # 5. Calculate verdict
320
- verdict = get_verdict(q1, q2, height)
321
-
322
- logger.info(f"Prediction: q1={q1:.3f}, q2={q2:.3f}, height={height:.3f}, verdict={verdict}")
323
-
324
- return PredictResponse(
325
- q1=round(q1, 3),
326
- q2=round(q2, 3),
327
- height=round(height, 3),
328
- message="Neural metrics computed successfully.",
329
- verdict=verdict
330
- )
331
-
332
- except HTTPException:
333
- raise
334
- except Exception as e:
335
- logger.error(f"Prediction failed: {str(e)}")
336
- raise HTTPException(status_code=500, detail=str(e))
337
-
338
-
339
- @app.get("/health")
340
- def health():
341
- """Health check endpoint."""
342
- return {
343
- "status": "healthy",
344
- "models_loaded": models.loaded,
345
- "device": str(models.device)
346
- }
347
-
348
-
349
- if __name__ == "__main__":
350
- import uvicorn
351
- uvicorn.run(app, host="0.0.0.0", port=7860)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
base_forces.pth CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:add0bf0c721dc6fb30fbaabcf5618d3ff53d571515df4a1e322d5e50dd13a87d
3
  size 201797
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:71808dab230b8abcb6e13e5fdd48179a2eaa170ddede73958cb04e87ce8427b1
3
  size 201797
height_gate.pth CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:1fb33b134d1cf6c2c1ffd457b6e76afc7daac1edc1f91991490d04e294f2bf1d
3
  size 234693
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d6b67f29aa7e17de5c454c9154d6346a197f03a7609f71f63c6f7f6f97978881
3
  size 234693
last.ckpt:Zone.Identifier DELETED
Binary file (25 Bytes)
 
q1_gate.pth CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:70e352e31f20203257c54e3b4fe7e582fe0e8f6aaa50fbe9552a8d685e08cd35
3
  size 925133
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:bb2529a7caf744365b77db5cc5077b1cd924c2792eb5ccfd2cd4a16347fa6d87
3
  size 925133
q1q2_best.ckpt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c2b608721183b9f4faf75091dd8d3d609a2d65a1b23b5ebc33f5af4ae92f30c3
3
- size 6861543
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6207b861536f987d67776fcac25bd661837ae2e2a4911091bd094c75dd1e11ca
3
+ size 6861479
q2_gate.pth CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2efa25c4ac9291ce4bbecbd768e7e9850b0977119a614bab6921fd3b32aebd7a
3
  size 926157
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f7d6bea4db289cbd56a344fb08e4de74fe28d5c3fbc2c08c07678c70180cef65
3
  size 926157
test_local.py DELETED
@@ -1,217 +0,0 @@
1
- #!/usr/bin/env python3
2
- """
3
- Test script to verify the HuggingFace Space locally before deployment.
4
-
5
- Usage:
6
- python test_local.py
7
- """
8
-
9
- import requests
10
- import time
11
- import sys
12
- from pathlib import Path
13
-
14
- # Colors for terminal output
15
- GREEN = '\033[92m'
16
- RED = '\033[91m'
17
- YELLOW = '\033[93m'
18
- BLUE = '\033[94m'
19
- RESET = '\033[0m'
20
-
21
- def print_success(msg):
22
- print(f"{GREEN}βœ“{RESET} {msg}")
23
-
24
- def print_error(msg):
25
- print(f"{RED}βœ—{RESET} {msg}")
26
-
27
- def print_info(msg):
28
- print(f"{BLUE}β„Ή{RESET} {msg}")
29
-
30
- def print_warning(msg):
31
- print(f"{YELLOW}⚠{RESET} {msg}")
32
-
33
- def check_files():
34
- """Check if all required files exist."""
35
- print_info("Checking required files...")
36
-
37
- required_files = [
38
- "app.py",
39
- "requirements.txt",
40
- "Dockerfile",
41
- "README.md",
42
- "model_info.json",
43
- "q1_gate.pth",
44
- "q2_gate.pth",
45
- "height_gate.pth",
46
- "base_forces.pth",
47
- "q1q2_best.ckpt",
48
- ]
49
-
50
- all_exist = True
51
- for file in required_files:
52
- path = Path(file)
53
- if path.exists():
54
- size = path.stat().st_size / 1024 # KB
55
- print_success(f"{file} ({size:.1f} KB)")
56
- else:
57
- print_error(f"{file} NOT FOUND")
58
- all_exist = False
59
-
60
- return all_exist
61
-
62
- def test_api():
63
- """Test the FastAPI endpoints."""
64
- print_info("\nStarting API tests...")
65
- print_warning("Make sure the API is running: python -m uvicorn app:app --port 7860\n")
66
-
67
- base_url = "http://localhost:7860"
68
-
69
- # Test 1: Root endpoint
70
- print_info("Test 1: Root endpoint (GET /)")
71
- try:
72
- response = requests.get(f"{base_url}/", timeout=5)
73
- if response.status_code == 200:
74
- data = response.json()
75
- print_success(f"Status: {response.status_code}")
76
- print_success(f"Response: {data}")
77
- else:
78
- print_error(f"Status: {response.status_code}")
79
- return False
80
- except requests.exceptions.ConnectionError:
81
- print_error("Cannot connect to API. Is it running?")
82
- print_info("Start it with: python -m uvicorn app:app --port 7860")
83
- return False
84
- except Exception as e:
85
- print_error(f"Error: {str(e)}")
86
- return False
87
-
88
- # Test 2: Health endpoint
89
- print_info("\nTest 2: Health check (GET /health)")
90
- try:
91
- response = requests.get(f"{base_url}/health", timeout=5)
92
- if response.status_code == 200:
93
- data = response.json()
94
- print_success(f"Status: {response.status_code}")
95
- print_success(f"Response: {data}")
96
- else:
97
- print_error(f"Status: {response.status_code}")
98
- return False
99
- except Exception as e:
100
- print_error(f"Error: {str(e)}")
101
- return False
102
-
103
- # Test 3: Predict endpoint - factual statement
104
- print_info("\nTest 3: Predict endpoint - Factual statement")
105
- try:
106
- response = requests.post(
107
- f"{base_url}/predict",
108
- headers={"Authorization": "Bearer test_token"},
109
- json={"text": "Paris is the capital of France"},
110
- timeout=5
111
- )
112
- if response.status_code == 200:
113
- data = response.json()
114
- print_success(f"Status: {response.status_code}")
115
- print_success(f"Q1: {data['q1']}, Q2: {data['q2']}, Height: {data['height']}")
116
- print_success(f"Verdict: {data['verdict']}")
117
-
118
- # Validate values
119
- if data['q1'] < 0 or data['q1'] > 1:
120
- print_error(f"Q1 out of range: {data['q1']}")
121
- return False
122
- if data['q2'] < 0 or data['q2'] > 1:
123
- print_error(f"Q2 out of range: {data['q2']}")
124
- return False
125
- if data['height'] < 0 or data['height'] > 1:
126
- print_error(f"Height out of range: {data['height']}")
127
- return False
128
- else:
129
- print_error(f"Status: {response.status_code}")
130
- print_error(f"Response: {response.text}")
131
- return False
132
- except Exception as e:
133
- print_error(f"Error: {str(e)}")
134
- return False
135
-
136
- # Test 4: Predict endpoint - uncertain statement
137
- print_info("\nTest 4: Predict endpoint - Uncertain statement")
138
- try:
139
- response = requests.post(
140
- f"{base_url}/predict",
141
- headers={"Authorization": "Bearer test_token"},
142
- json={"text": "Maybe quantum computing will solve all problems"},
143
- timeout=5
144
- )
145
- if response.status_code == 200:
146
- data = response.json()
147
- print_success(f"Status: {response.status_code}")
148
- print_success(f"Q1: {data['q1']}, Q2: {data['q2']}, Height: {data['height']}")
149
- print_success(f"Verdict: {data['verdict']}")
150
-
151
- # This should have higher uncertainty
152
- if data['q1'] < 0.15 and data['q2'] < 0.15:
153
- print_warning("Uncertainties seem low for an uncertain statement")
154
- else:
155
- print_error(f"Status: {response.status_code}")
156
- return False
157
- except Exception as e:
158
- print_error(f"Error: {str(e)}")
159
- return False
160
-
161
- # Test 5: Predict endpoint - with context
162
- print_info("\nTest 5: Predict endpoint - With context")
163
- try:
164
- response = requests.post(
165
- f"{base_url}/predict",
166
- headers={"Authorization": "Bearer test_token"},
167
- json={
168
- "text": "The Eiffel Tower is 324 meters tall",
169
- "context": "geography"
170
- },
171
- timeout=5
172
- )
173
- if response.status_code == 200:
174
- data = response.json()
175
- print_success(f"Status: {response.status_code}")
176
- print_success(f"Q1: {data['q1']}, Q2: {data['q2']}, Height: {data['height']}")
177
- print_success(f"Verdict: {data['verdict']}")
178
- else:
179
- print_error(f"Status: {response.status_code}")
180
- return False
181
- except Exception as e:
182
- print_error(f"Error: {str(e)}")
183
- return False
184
-
185
- return True
186
-
187
- def main():
188
- print(f"\n{BLUE}{'='*60}{RESET}")
189
- print(f"{BLUE} AletheionGuard HuggingFace Space - Local Test Suite{RESET}")
190
- print(f"{BLUE}{'='*60}{RESET}\n")
191
-
192
- # Check files
193
- if not check_files():
194
- print_error("\nSome required files are missing!")
195
- print_info("Make sure all model files are copied to this directory")
196
- sys.exit(1)
197
-
198
- print_success("\nAll required files present βœ“\n")
199
-
200
- # Test API
201
- print_info("="*60)
202
- if test_api():
203
- print(f"\n{GREEN}{'='*60}{RESET}")
204
- print(f"{GREEN} All tests passed! βœ“{RESET}")
205
- print(f"{GREEN}{'='*60}{RESET}\n")
206
- print_info("Your Space is ready for deployment!")
207
- print_info("Follow DEPLOY_GUIDE.md to push to HuggingFace\n")
208
- return 0
209
- else:
210
- print(f"\n{RED}{'='*60}{RESET}")
211
- print(f"{RED} Some tests failed βœ—{RESET}")
212
- print(f"{RED}{'='*60}{RESET}\n")
213
- print_info("Fix the errors above before deploying\n")
214
- return 1
215
-
216
- if __name__ == "__main__":
217
- sys.exit(main())