Spaces:
Sleeping
Sleeping
Jackie Makhija commited on
Commit ·
d613159
1
Parent(s): 9b4c0e5
Sync all changes: cleanup files, update Dockerfile, fix UI
Browse files- DEPLOYMENT.md +0 -668
- DEPLOYMENT_CHECKLIST.md +0 -336
- DEPLOYMENT_GUIDE.md +0 -755
- Dockerfile +16 -14
- HF_DEPLOYMENT.md +111 -109
- HF_DEPLOYMENT_SUMMARY.md +0 -362
- HF_README.md +0 -243
- QUICK_DEPLOY.md +0 -196
- README_DEPLOYMENT.md +0 -222
- app.py +86 -5
- deploy-to-huggingface.bat +0 -97
- deploy-to-huggingface.sh +0 -181
- deploy.sh +0 -486
- docker-compose.yml +0 -60
- index.html +39 -0
- test.sh +0 -316
- unity-catalog-chatbot.jsx +582 -73
DEPLOYMENT.md
DELETED
|
@@ -1,668 +0,0 @@
|
|
| 1 |
-
# Unity Catalog Chatbot - Deployment & Operations Guide
|
| 2 |
-
|
| 3 |
-
## Table of Contents
|
| 4 |
-
1. [Quick Start](#quick-start)
|
| 5 |
-
2. [Deployment Options](#deployment-options)
|
| 6 |
-
3. [Configuration](#configuration)
|
| 7 |
-
4. [Monitoring & Observability](#monitoring--observability)
|
| 8 |
-
5. [Security Hardening](#security-hardening)
|
| 9 |
-
6. [Troubleshooting](#troubleshooting)
|
| 10 |
-
7. [Performance Tuning](#performance-tuning)
|
| 11 |
-
8. [Backup & Recovery](#backup--recovery)
|
| 12 |
-
|
| 13 |
-
## Quick Start
|
| 14 |
-
|
| 15 |
-
### Local Development
|
| 16 |
-
|
| 17 |
-
```bash
|
| 18 |
-
# 1. Clone repository
|
| 19 |
-
git clone <repository-url>
|
| 20 |
-
cd unity-catalog-chatbot
|
| 21 |
-
|
| 22 |
-
# 2. Create virtual environment
|
| 23 |
-
python -m venv venv
|
| 24 |
-
source venv/bin/activate # On Windows: venv\Scripts\activate
|
| 25 |
-
|
| 26 |
-
# 3. Install dependencies
|
| 27 |
-
pip install -r requirements.txt
|
| 28 |
-
|
| 29 |
-
# 4. Configure environment
|
| 30 |
-
cp .env.example .env
|
| 31 |
-
# Edit .env with your credentials
|
| 32 |
-
|
| 33 |
-
# 5. Run the application
|
| 34 |
-
python app.py
|
| 35 |
-
```
|
| 36 |
-
|
| 37 |
-
### Docker Deployment
|
| 38 |
-
|
| 39 |
-
```bash
|
| 40 |
-
# Build and run with Docker Compose
|
| 41 |
-
docker-compose up -d
|
| 42 |
-
|
| 43 |
-
# Check logs
|
| 44 |
-
docker-compose logs -f unity-catalog-chatbot
|
| 45 |
-
|
| 46 |
-
# Stop services
|
| 47 |
-
docker-compose down
|
| 48 |
-
```
|
| 49 |
-
|
| 50 |
-
## Deployment Options
|
| 51 |
-
|
| 52 |
-
### Option 1: Docker on AWS ECS
|
| 53 |
-
|
| 54 |
-
```bash
|
| 55 |
-
# Build and tag image
|
| 56 |
-
docker build -t unity-catalog-chatbot:latest .
|
| 57 |
-
docker tag unity-catalog-chatbot:latest <aws-account-id>.dkr.ecr.<region>.amazonaws.com/unity-catalog-chatbot:latest
|
| 58 |
-
|
| 59 |
-
# Push to ECR
|
| 60 |
-
aws ecr get-login-password --region <region> | docker login --username AWS --password-stdin <aws-account-id>.dkr.ecr.<region>.amazonaws.com
|
| 61 |
-
docker push <aws-account-id>.dkr.ecr.<region>.amazonaws.com/unity-catalog-chatbot:latest
|
| 62 |
-
|
| 63 |
-
# Deploy to ECS using terraform or console
|
| 64 |
-
```
|
| 65 |
-
|
| 66 |
-
**ECS Task Definition:**
|
| 67 |
-
```json
|
| 68 |
-
{
|
| 69 |
-
"family": "unity-catalog-chatbot",
|
| 70 |
-
"containerDefinitions": [
|
| 71 |
-
{
|
| 72 |
-
"name": "chatbot",
|
| 73 |
-
"image": "<ecr-image>",
|
| 74 |
-
"memory": 2048,
|
| 75 |
-
"cpu": 1024,
|
| 76 |
-
"essential": true,
|
| 77 |
-
"portMappings": [
|
| 78 |
-
{
|
| 79 |
-
"containerPort": 5000,
|
| 80 |
-
"protocol": "tcp"
|
| 81 |
-
}
|
| 82 |
-
],
|
| 83 |
-
"environment": [
|
| 84 |
-
{
|
| 85 |
-
"name": "ENVIRONMENT",
|
| 86 |
-
"value": "production"
|
| 87 |
-
}
|
| 88 |
-
],
|
| 89 |
-
"secrets": [
|
| 90 |
-
{
|
| 91 |
-
"name": "DATABRICKS_TOKEN",
|
| 92 |
-
"valueFrom": "arn:aws:secretsmanager:region:account:secret:databricks-token"
|
| 93 |
-
},
|
| 94 |
-
{
|
| 95 |
-
"name": "ANTHROPIC_API_KEY",
|
| 96 |
-
"valueFrom": "arn:aws:secretsmanager:region:account:secret:anthropic-key"
|
| 97 |
-
}
|
| 98 |
-
]
|
| 99 |
-
}
|
| 100 |
-
]
|
| 101 |
-
}
|
| 102 |
-
```
|
| 103 |
-
|
| 104 |
-
### Option 2: Kubernetes Deployment
|
| 105 |
-
|
| 106 |
-
**deployment.yaml:**
|
| 107 |
-
```yaml
|
| 108 |
-
apiVersion: apps/v1
|
| 109 |
-
kind: Deployment
|
| 110 |
-
metadata:
|
| 111 |
-
name: unity-catalog-chatbot
|
| 112 |
-
spec:
|
| 113 |
-
replicas: 3
|
| 114 |
-
selector:
|
| 115 |
-
matchLabels:
|
| 116 |
-
app: unity-catalog-chatbot
|
| 117 |
-
template:
|
| 118 |
-
metadata:
|
| 119 |
-
labels:
|
| 120 |
-
app: unity-catalog-chatbot
|
| 121 |
-
spec:
|
| 122 |
-
containers:
|
| 123 |
-
- name: chatbot
|
| 124 |
-
image: unity-catalog-chatbot:latest
|
| 125 |
-
ports:
|
| 126 |
-
- containerPort: 5000
|
| 127 |
-
env:
|
| 128 |
-
- name: ENVIRONMENT
|
| 129 |
-
value: "production"
|
| 130 |
-
envFrom:
|
| 131 |
-
- secretRef:
|
| 132 |
-
name: chatbot-secrets
|
| 133 |
-
resources:
|
| 134 |
-
requests:
|
| 135 |
-
memory: "1Gi"
|
| 136 |
-
cpu: "500m"
|
| 137 |
-
limits:
|
| 138 |
-
memory: "2Gi"
|
| 139 |
-
cpu: "1000m"
|
| 140 |
-
livenessProbe:
|
| 141 |
-
httpGet:
|
| 142 |
-
path: /api/health
|
| 143 |
-
port: 5000
|
| 144 |
-
initialDelaySeconds: 30
|
| 145 |
-
periodSeconds: 10
|
| 146 |
-
readinessProbe:
|
| 147 |
-
httpGet:
|
| 148 |
-
path: /api/health
|
| 149 |
-
port: 5000
|
| 150 |
-
initialDelaySeconds: 5
|
| 151 |
-
periodSeconds: 5
|
| 152 |
-
---
|
| 153 |
-
apiVersion: v1
|
| 154 |
-
kind: Service
|
| 155 |
-
metadata:
|
| 156 |
-
name: unity-catalog-chatbot
|
| 157 |
-
spec:
|
| 158 |
-
selector:
|
| 159 |
-
app: unity-catalog-chatbot
|
| 160 |
-
ports:
|
| 161 |
-
- protocol: TCP
|
| 162 |
-
port: 80
|
| 163 |
-
targetPort: 5000
|
| 164 |
-
type: LoadBalancer
|
| 165 |
-
```
|
| 166 |
-
|
| 167 |
-
**secrets.yaml:**
|
| 168 |
-
```yaml
|
| 169 |
-
apiVersion: v1
|
| 170 |
-
kind: Secret
|
| 171 |
-
metadata:
|
| 172 |
-
name: chatbot-secrets
|
| 173 |
-
type: Opaque
|
| 174 |
-
stringData:
|
| 175 |
-
DATABRICKS_HOST: "https://your-workspace.cloud.databricks.com"
|
| 176 |
-
DATABRICKS_TOKEN: "your-token"
|
| 177 |
-
ANTHROPIC_API_KEY: "your-key"
|
| 178 |
-
```
|
| 179 |
-
|
| 180 |
-
Deploy:
|
| 181 |
-
```bash
|
| 182 |
-
kubectl apply -f deployment.yaml
|
| 183 |
-
kubectl apply -f secrets.yaml
|
| 184 |
-
```
|
| 185 |
-
|
| 186 |
-
### Option 3: Serverless (AWS Lambda)
|
| 187 |
-
|
| 188 |
-
**serverless.yml:**
|
| 189 |
-
```yaml
|
| 190 |
-
service: unity-catalog-chatbot
|
| 191 |
-
|
| 192 |
-
provider:
|
| 193 |
-
name: aws
|
| 194 |
-
runtime: python3.9
|
| 195 |
-
region: us-east-1
|
| 196 |
-
environment:
|
| 197 |
-
DATABRICKS_HOST: ${env:DATABRICKS_HOST}
|
| 198 |
-
DATABRICKS_TOKEN: ${env:DATABRICKS_TOKEN}
|
| 199 |
-
ANTHROPIC_API_KEY: ${env:ANTHROPIC_API_KEY}
|
| 200 |
-
|
| 201 |
-
functions:
|
| 202 |
-
chat:
|
| 203 |
-
handler: lambda_handler.chat_handler
|
| 204 |
-
timeout: 30
|
| 205 |
-
events:
|
| 206 |
-
- http:
|
| 207 |
-
path: api/chat
|
| 208 |
-
method: post
|
| 209 |
-
cors: true
|
| 210 |
-
|
| 211 |
-
health:
|
| 212 |
-
handler: lambda_handler.health_handler
|
| 213 |
-
events:
|
| 214 |
-
- http:
|
| 215 |
-
path: api/health
|
| 216 |
-
method: get
|
| 217 |
-
|
| 218 |
-
plugins:
|
| 219 |
-
- serverless-python-requirements
|
| 220 |
-
```
|
| 221 |
-
|
| 222 |
-
## Configuration
|
| 223 |
-
|
| 224 |
-
### Environment Variables
|
| 225 |
-
|
| 226 |
-
**Required:**
|
| 227 |
-
- `DATABRICKS_HOST`: Your Databricks workspace URL
|
| 228 |
-
- `DATABRICKS_TOKEN`: Personal access token or service principal token
|
| 229 |
-
- `ANTHROPIC_API_KEY`: Anthropic API key for Claude
|
| 230 |
-
|
| 231 |
-
**Optional:**
|
| 232 |
-
- `DATABRICKS_WAREHOUSE_ID`: SQL Warehouse for executing queries
|
| 233 |
-
- `ENVIRONMENT`: development|staging|production (default: development)
|
| 234 |
-
- `SERVER_PORT`: API server port (default: 5000)
|
| 235 |
-
- `SERVER_WORKERS`: Number of worker processes (default: 4)
|
| 236 |
-
- `LOG_LEVEL`: DEBUG|INFO|WARNING|ERROR (default: INFO)
|
| 237 |
-
- `ENABLE_AUTH`: Enable API key authentication (default: false)
|
| 238 |
-
- `RATE_LIMIT_PER_MINUTE`: API rate limit (default: 60)
|
| 239 |
-
|
| 240 |
-
### Configuration File
|
| 241 |
-
|
| 242 |
-
Create `config.yaml` for advanced configuration:
|
| 243 |
-
|
| 244 |
-
```yaml
|
| 245 |
-
environment: production
|
| 246 |
-
|
| 247 |
-
databricks:
|
| 248 |
-
host: https://your-workspace.cloud.databricks.com
|
| 249 |
-
warehouse_id: abc123
|
| 250 |
-
|
| 251 |
-
server:
|
| 252 |
-
host: 0.0.0.0
|
| 253 |
-
port: 5000
|
| 254 |
-
workers: 4
|
| 255 |
-
timeout: 120
|
| 256 |
-
|
| 257 |
-
security:
|
| 258 |
-
enable_auth: true
|
| 259 |
-
rate_limit: 100
|
| 260 |
-
allowed_origins:
|
| 261 |
-
- https://yourapp.com
|
| 262 |
-
- https://admin.yourapp.com
|
| 263 |
-
|
| 264 |
-
features:
|
| 265 |
-
sql_execution: true
|
| 266 |
-
batch_operations: true
|
| 267 |
-
audit_logging: true
|
| 268 |
-
caching: true
|
| 269 |
-
|
| 270 |
-
logging:
|
| 271 |
-
level: INFO
|
| 272 |
-
log_to_file: true
|
| 273 |
-
log_file: /var/log/chatbot/app.log
|
| 274 |
-
```
|
| 275 |
-
|
| 276 |
-
## Monitoring & Observability
|
| 277 |
-
|
| 278 |
-
### Health Checks
|
| 279 |
-
|
| 280 |
-
```bash
|
| 281 |
-
# Basic health check
|
| 282 |
-
curl http://localhost:5000/api/health
|
| 283 |
-
|
| 284 |
-
# Expected response:
|
| 285 |
-
{
|
| 286 |
-
"status": "healthy",
|
| 287 |
-
"service": "Unity Catalog Chatbot API"
|
| 288 |
-
}
|
| 289 |
-
```
|
| 290 |
-
|
| 291 |
-
### Metrics Collection
|
| 292 |
-
|
| 293 |
-
Add Prometheus metrics:
|
| 294 |
-
|
| 295 |
-
```python
|
| 296 |
-
from prometheus_client import Counter, Histogram, generate_latest
|
| 297 |
-
|
| 298 |
-
# Metrics
|
| 299 |
-
request_count = Counter('chatbot_requests_total', 'Total requests')
|
| 300 |
-
request_duration = Histogram('chatbot_request_duration_seconds', 'Request duration')
|
| 301 |
-
error_count = Counter('chatbot_errors_total', 'Total errors')
|
| 302 |
-
|
| 303 |
-
@app.route('/metrics')
|
| 304 |
-
def metrics():
|
| 305 |
-
return generate_latest()
|
| 306 |
-
```
|
| 307 |
-
|
| 308 |
-
### Logging
|
| 309 |
-
|
| 310 |
-
**Structured logging with JSON:**
|
| 311 |
-
|
| 312 |
-
```python
|
| 313 |
-
import logging
|
| 314 |
-
import json
|
| 315 |
-
|
| 316 |
-
class JSONFormatter(logging.Formatter):
|
| 317 |
-
def format(self, record):
|
| 318 |
-
log_data = {
|
| 319 |
-
'timestamp': self.formatTime(record),
|
| 320 |
-
'level': record.levelname,
|
| 321 |
-
'message': record.getMessage(),
|
| 322 |
-
'module': record.module,
|
| 323 |
-
'function': record.funcName
|
| 324 |
-
}
|
| 325 |
-
return json.dumps(log_data)
|
| 326 |
-
|
| 327 |
-
handler = logging.StreamHandler()
|
| 328 |
-
handler.setFormatter(JSONFormatter())
|
| 329 |
-
logger.addHandler(handler)
|
| 330 |
-
```
|
| 331 |
-
|
| 332 |
-
### Application Performance Monitoring (APM)
|
| 333 |
-
|
| 334 |
-
**With DataDog:**
|
| 335 |
-
|
| 336 |
-
```python
|
| 337 |
-
from ddtrace import tracer
|
| 338 |
-
|
| 339 |
-
@app.before_request
|
| 340 |
-
def before_request():
|
| 341 |
-
tracer.trace('http.request')
|
| 342 |
-
|
| 343 |
-
@app.after_request
|
| 344 |
-
def after_request(response):
|
| 345 |
-
return response
|
| 346 |
-
```
|
| 347 |
-
|
| 348 |
-
## Security Hardening
|
| 349 |
-
|
| 350 |
-
### 1. API Key Authentication
|
| 351 |
-
|
| 352 |
-
```python
|
| 353 |
-
from functools import wraps
|
| 354 |
-
from flask import request, jsonify
|
| 355 |
-
|
| 356 |
-
def require_api_key(f):
|
| 357 |
-
@wraps(f)
|
| 358 |
-
def decorated_function(*args, **kwargs):
|
| 359 |
-
api_key = request.headers.get('X-API-Key')
|
| 360 |
-
|
| 361 |
-
if not api_key or api_key != os.getenv('API_SECRET_KEY'):
|
| 362 |
-
return jsonify({'error': 'Invalid API key'}), 401
|
| 363 |
-
|
| 364 |
-
return f(*args, **kwargs)
|
| 365 |
-
return decorated_function
|
| 366 |
-
|
| 367 |
-
@app.route('/api/chat', methods=['POST'])
|
| 368 |
-
@require_api_key
|
| 369 |
-
def chat():
|
| 370 |
-
# ... implementation
|
| 371 |
-
```
|
| 372 |
-
|
| 373 |
-
### 2. Rate Limiting
|
| 374 |
-
|
| 375 |
-
```python
|
| 376 |
-
from flask_limiter import Limiter
|
| 377 |
-
from flask_limiter.util import get_remote_address
|
| 378 |
-
|
| 379 |
-
limiter = Limiter(
|
| 380 |
-
app,
|
| 381 |
-
key_func=get_remote_address,
|
| 382 |
-
default_limits=["100 per hour"]
|
| 383 |
-
)
|
| 384 |
-
|
| 385 |
-
@app.route('/api/chat', methods=['POST'])
|
| 386 |
-
@limiter.limit("10 per minute")
|
| 387 |
-
def chat():
|
| 388 |
-
# ... implementation
|
| 389 |
-
```
|
| 390 |
-
|
| 391 |
-
### 3. Input Validation
|
| 392 |
-
|
| 393 |
-
```python
|
| 394 |
-
from jsonschema import validate, ValidationError
|
| 395 |
-
|
| 396 |
-
chat_schema = {
|
| 397 |
-
"type": "object",
|
| 398 |
-
"properties": {
|
| 399 |
-
"message": {"type": "string", "minLength": 1, "maxLength": 1000}
|
| 400 |
-
},
|
| 401 |
-
"required": ["message"]
|
| 402 |
-
}
|
| 403 |
-
|
| 404 |
-
@app.route('/api/chat', methods=['POST'])
|
| 405 |
-
def chat():
|
| 406 |
-
try:
|
| 407 |
-
validate(request.json, chat_schema)
|
| 408 |
-
except ValidationError as e:
|
| 409 |
-
return jsonify({'error': 'Invalid input'}), 400
|
| 410 |
-
```
|
| 411 |
-
|
| 412 |
-
### 4. Secrets Management
|
| 413 |
-
|
| 414 |
-
**AWS Secrets Manager:**
|
| 415 |
-
|
| 416 |
-
```python
|
| 417 |
-
import boto3
|
| 418 |
-
import json
|
| 419 |
-
|
| 420 |
-
def get_secret(secret_name):
|
| 421 |
-
client = boto3.client('secretsmanager', region_name='us-east-1')
|
| 422 |
-
|
| 423 |
-
response = client.get_secret_value(SecretId=secret_name)
|
| 424 |
-
return json.loads(response['SecretString'])
|
| 425 |
-
|
| 426 |
-
# Usage
|
| 427 |
-
secrets = get_secret('unity-catalog-chatbot/prod')
|
| 428 |
-
databricks_token = secrets['databricks_token']
|
| 429 |
-
```
|
| 430 |
-
|
| 431 |
-
### 5. HTTPS Enforcement
|
| 432 |
-
|
| 433 |
-
```python
|
| 434 |
-
@app.before_request
|
| 435 |
-
def before_request():
|
| 436 |
-
if not request.is_secure and app.config.get('FORCE_HTTPS'):
|
| 437 |
-
url = request.url.replace('http://', 'https://', 1)
|
| 438 |
-
return redirect(url, code=301)
|
| 439 |
-
```
|
| 440 |
-
|
| 441 |
-
## Troubleshooting
|
| 442 |
-
|
| 443 |
-
### Common Issues
|
| 444 |
-
|
| 445 |
-
**1. Authentication Errors**
|
| 446 |
-
|
| 447 |
-
```
|
| 448 |
-
Error: 401 Unauthorized
|
| 449 |
-
```
|
| 450 |
-
|
| 451 |
-
**Solutions:**
|
| 452 |
-
- Verify Databricks token is valid and not expired
|
| 453 |
-
- Check token has necessary permissions
|
| 454 |
-
- Ensure workspace URL is correct
|
| 455 |
-
|
| 456 |
-
**2. Rate Limiting**
|
| 457 |
-
|
| 458 |
-
```
|
| 459 |
-
Error: 429 Too Many Requests
|
| 460 |
-
```
|
| 461 |
-
|
| 462 |
-
**Solutions:**
|
| 463 |
-
- Increase rate limits in configuration
|
| 464 |
-
- Implement request queuing
|
| 465 |
-
- Use caching for repeated queries
|
| 466 |
-
|
| 467 |
-
**3. Timeout Errors**
|
| 468 |
-
|
| 469 |
-
```
|
| 470 |
-
Error: Request timeout
|
| 471 |
-
```
|
| 472 |
-
|
| 473 |
-
**Solutions:**
|
| 474 |
-
- Increase `SERVER_TIMEOUT` setting
|
| 475 |
-
- Optimize database queries
|
| 476 |
-
- Use async processing for long operations
|
| 477 |
-
|
| 478 |
-
**4. Memory Issues**
|
| 479 |
-
|
| 480 |
-
```
|
| 481 |
-
Error: Out of memory
|
| 482 |
-
```
|
| 483 |
-
|
| 484 |
-
**Solutions:**
|
| 485 |
-
- Increase container memory limits
|
| 486 |
-
- Reduce number of workers
|
| 487 |
-
- Implement pagination for large result sets
|
| 488 |
-
|
| 489 |
-
### Debug Mode
|
| 490 |
-
|
| 491 |
-
Enable detailed debugging:
|
| 492 |
-
|
| 493 |
-
```bash
|
| 494 |
-
export FLASK_ENV=development
|
| 495 |
-
export LOG_LEVEL=DEBUG
|
| 496 |
-
python app.py
|
| 497 |
-
```
|
| 498 |
-
|
| 499 |
-
## Performance Tuning
|
| 500 |
-
|
| 501 |
-
### 1. Caching
|
| 502 |
-
|
| 503 |
-
Implement Redis caching:
|
| 504 |
-
|
| 505 |
-
```python
|
| 506 |
-
import redis
|
| 507 |
-
from functools import wraps
|
| 508 |
-
|
| 509 |
-
redis_client = redis.Redis(
|
| 510 |
-
host='localhost',
|
| 511 |
-
port=6379,
|
| 512 |
-
db=0
|
| 513 |
-
)
|
| 514 |
-
|
| 515 |
-
def cache_result(ttl=300):
|
| 516 |
-
def decorator(f):
|
| 517 |
-
@wraps(f)
|
| 518 |
-
def wrapper(*args, **kwargs):
|
| 519 |
-
key = f"{f.__name__}:{str(args)}:{str(kwargs)}"
|
| 520 |
-
|
| 521 |
-
# Try to get from cache
|
| 522 |
-
cached = redis_client.get(key)
|
| 523 |
-
if cached:
|
| 524 |
-
return json.loads(cached)
|
| 525 |
-
|
| 526 |
-
# Execute and cache
|
| 527 |
-
result = f(*args, **kwargs)
|
| 528 |
-
redis_client.setex(key, ttl, json.dumps(result))
|
| 529 |
-
|
| 530 |
-
return result
|
| 531 |
-
return wrapper
|
| 532 |
-
return decorator
|
| 533 |
-
|
| 534 |
-
@cache_result(ttl=600)
|
| 535 |
-
def list_catalogs():
|
| 536 |
-
# ... implementation
|
| 537 |
-
```
|
| 538 |
-
|
| 539 |
-
### 2. Connection Pooling
|
| 540 |
-
|
| 541 |
-
```python
|
| 542 |
-
from databricks.sdk import WorkspaceClient
|
| 543 |
-
|
| 544 |
-
class ConnectionPool:
|
| 545 |
-
def __init__(self, size=10):
|
| 546 |
-
self.pool = []
|
| 547 |
-
self.size = size
|
| 548 |
-
|
| 549 |
-
def get_client(self):
|
| 550 |
-
# Implement connection pooling
|
| 551 |
-
pass
|
| 552 |
-
```
|
| 553 |
-
|
| 554 |
-
### 3. Async Processing
|
| 555 |
-
|
| 556 |
-
For long-running operations:
|
| 557 |
-
|
| 558 |
-
```python
|
| 559 |
-
from celery import Celery
|
| 560 |
-
|
| 561 |
-
celery = Celery('tasks', broker='redis://localhost:6379')
|
| 562 |
-
|
| 563 |
-
@celery.task
|
| 564 |
-
def process_batch_operation(operations):
|
| 565 |
-
# Process in background
|
| 566 |
-
pass
|
| 567 |
-
```
|
| 568 |
-
|
| 569 |
-
## Backup & Recovery
|
| 570 |
-
|
| 571 |
-
### Configuration Backup
|
| 572 |
-
|
| 573 |
-
```bash
|
| 574 |
-
# Backup environment configuration
|
| 575 |
-
cp .env .env.backup.$(date +%Y%m%d)
|
| 576 |
-
|
| 577 |
-
# Backup custom configurations
|
| 578 |
-
tar -czf config-backup-$(date +%Y%m%d).tar.gz config.yaml *.yml
|
| 579 |
-
```
|
| 580 |
-
|
| 581 |
-
### Disaster Recovery Plan
|
| 582 |
-
|
| 583 |
-
1. **Database Backups**: Unity Catalog is backed up by Databricks
|
| 584 |
-
2. **Configuration**: Store in version control (Git)
|
| 585 |
-
3. **Secrets**: Use managed secret services (AWS Secrets Manager, HashiCorp Vault)
|
| 586 |
-
4. **Application State**: Stateless design - no local state to backup
|
| 587 |
-
|
| 588 |
-
### Recovery Procedure
|
| 589 |
-
|
| 590 |
-
```bash
|
| 591 |
-
# 1. Restore configuration
|
| 592 |
-
cp .env.backup.YYYYMMDD .env
|
| 593 |
-
|
| 594 |
-
# 2. Rebuild and redeploy
|
| 595 |
-
docker-compose build
|
| 596 |
-
docker-compose up -d
|
| 597 |
-
|
| 598 |
-
# 3. Verify health
|
| 599 |
-
curl http://localhost:5000/api/health
|
| 600 |
-
|
| 601 |
-
# 4. Run smoke tests
|
| 602 |
-
pytest tests/smoke_tests.py
|
| 603 |
-
```
|
| 604 |
-
|
| 605 |
-
## Production Checklist
|
| 606 |
-
|
| 607 |
-
- [ ] Environment variables configured in secrets manager
|
| 608 |
-
- [ ] HTTPS enabled with valid SSL certificate
|
| 609 |
-
- [ ] Rate limiting configured
|
| 610 |
-
- [ ] Authentication enabled
|
| 611 |
-
- [ ] Logging configured and centralized
|
| 612 |
-
- [ ] Monitoring and alerting set up
|
| 613 |
-
- [ ] Health checks configured
|
| 614 |
-
- [ ] Auto-scaling policies defined
|
| 615 |
-
- [ ] Backup procedures documented
|
| 616 |
-
- [ ] Disaster recovery plan tested
|
| 617 |
-
- [ ] Security audit completed
|
| 618 |
-
- [ ] Load testing performed
|
| 619 |
-
- [ ] Documentation updated
|
| 620 |
-
|
| 621 |
-
## Maintenance
|
| 622 |
-
|
| 623 |
-
### Regular Tasks
|
| 624 |
-
|
| 625 |
-
**Daily:**
|
| 626 |
-
- Monitor error logs
|
| 627 |
-
- Check API response times
|
| 628 |
-
- Verify health check status
|
| 629 |
-
|
| 630 |
-
**Weekly:**
|
| 631 |
-
- Review performance metrics
|
| 632 |
-
- Check for security updates
|
| 633 |
-
- Analyze usage patterns
|
| 634 |
-
|
| 635 |
-
**Monthly:**
|
| 636 |
-
- Rotate credentials
|
| 637 |
-
- Review and update dependencies
|
| 638 |
-
- Performance optimization review
|
| 639 |
-
- Security audit
|
| 640 |
-
|
| 641 |
-
### Upgrade Procedure
|
| 642 |
-
|
| 643 |
-
```bash
|
| 644 |
-
# 1. Test in staging
|
| 645 |
-
git checkout develop
|
| 646 |
-
docker-compose -f docker-compose.staging.yml up -d
|
| 647 |
-
|
| 648 |
-
# 2. Run tests
|
| 649 |
-
pytest tests/
|
| 650 |
-
|
| 651 |
-
# 3. Deploy to production
|
| 652 |
-
git checkout main
|
| 653 |
-
git merge develop
|
| 654 |
-
docker-compose build
|
| 655 |
-
docker-compose up -d --no-deps chatbot
|
| 656 |
-
|
| 657 |
-
# 4. Verify deployment
|
| 658 |
-
curl http://localhost:5000/api/health
|
| 659 |
-
```
|
| 660 |
-
|
| 661 |
-
## Support
|
| 662 |
-
|
| 663 |
-
For production issues:
|
| 664 |
-
1. Check logs: `docker-compose logs chatbot`
|
| 665 |
-
2. Verify configuration: Review .env file
|
| 666 |
-
3. Test connectivity: `curl http://localhost:5000/api/health`
|
| 667 |
-
4. Review documentation: README.md
|
| 668 |
-
5. Contact support: [Your support channel]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
DEPLOYMENT_CHECKLIST.md
DELETED
|
@@ -1,336 +0,0 @@
|
|
| 1 |
-
# Unity Catalog Chatbot - Deployment Checklist
|
| 2 |
-
|
| 3 |
-
## 📋 Pre-Deployment Checklist
|
| 4 |
-
|
| 5 |
-
### Required Credentials
|
| 6 |
-
- [ ] Databricks workspace URL
|
| 7 |
-
- [ ] Databricks personal access token (PAT)
|
| 8 |
-
- [ ] Anthropic Claude API key
|
| 9 |
-
- [ ] (Optional) SQL Warehouse ID
|
| 10 |
-
|
| 11 |
-
### Required Software
|
| 12 |
-
- [ ] Python 3.9 or higher installed
|
| 13 |
-
- [ ] pip package manager
|
| 14 |
-
- [ ] Git (optional, for version control)
|
| 15 |
-
- [ ] Docker Desktop (optional, for container deployment)
|
| 16 |
-
|
| 17 |
-
### Databricks Permissions
|
| 18 |
-
- [ ] Unity Catalog access
|
| 19 |
-
- [ ] CREATE CATALOG permission on metastore
|
| 20 |
-
- [ ] Ability to grant/revoke permissions
|
| 21 |
-
- [ ] Workspace access
|
| 22 |
-
|
| 23 |
-
---
|
| 24 |
-
|
| 25 |
-
## 🚀 Deployment Steps
|
| 26 |
-
|
| 27 |
-
### Phase 1: Setup (5 minutes)
|
| 28 |
-
- [ ] Download all project files
|
| 29 |
-
- [ ] Create project directory
|
| 30 |
-
- [ ] Copy `.env.example` to `.env`
|
| 31 |
-
- [ ] Fill in credentials in `.env` file
|
| 32 |
-
- [ ] Verify credentials are correct (no extra spaces/quotes)
|
| 33 |
-
|
| 34 |
-
### Phase 2: Installation (3 minutes)
|
| 35 |
-
- [ ] Run `chmod +x deploy.sh`
|
| 36 |
-
- [ ] Execute `./deploy.sh`
|
| 37 |
-
- [ ] OR manually: `pip install -r requirements.txt`
|
| 38 |
-
- [ ] Wait for dependencies to install
|
| 39 |
-
|
| 40 |
-
### Phase 3: Deployment (2 minutes)
|
| 41 |
-
- [ ] Start application (`python app.py` or via deploy script)
|
| 42 |
-
- [ ] Wait for "Running on http://0.0.0.0:5000"
|
| 43 |
-
- [ ] Application PID saved (if using deploy script)
|
| 44 |
-
|
| 45 |
-
### Phase 4: Verification (2 minutes)
|
| 46 |
-
- [ ] Health check: `curl http://localhost:5000/api/health`
|
| 47 |
-
- [ ] Response is `{"status": "healthy"}`
|
| 48 |
-
- [ ] No errors in logs
|
| 49 |
-
|
| 50 |
-
---
|
| 51 |
-
|
| 52 |
-
## ✅ Testing Checklist
|
| 53 |
-
|
| 54 |
-
### Basic Tests
|
| 55 |
-
- [ ] Health endpoint returns 200
|
| 56 |
-
- [ ] Help command works
|
| 57 |
-
- [ ] List catalogs works
|
| 58 |
-
- [ ] API returns proper JSON
|
| 59 |
-
|
| 60 |
-
### Functional Tests
|
| 61 |
-
- [ ] Create catalog command generates SQL
|
| 62 |
-
- [ ] Create schema command works
|
| 63 |
-
- [ ] Create table command works
|
| 64 |
-
- [ ] Grant permission command works
|
| 65 |
-
- [ ] Show permissions command works
|
| 66 |
-
|
| 67 |
-
### Integration Tests
|
| 68 |
-
- [ ] Actually create a test catalog in Databricks
|
| 69 |
-
- [ ] Verify catalog appears in Databricks UI
|
| 70 |
-
- [ ] Create schema in test catalog
|
| 71 |
-
- [ ] Grant permission to test user
|
| 72 |
-
- [ ] Verify permission in Databricks
|
| 73 |
-
|
| 74 |
-
### Error Handling
|
| 75 |
-
- [ ] Empty message returns appropriate error
|
| 76 |
-
- [ ] Invalid commands return helpful messages
|
| 77 |
-
- [ ] Databricks errors are caught and reported
|
| 78 |
-
- [ ] Claude API errors are handled gracefully
|
| 79 |
-
|
| 80 |
-
---
|
| 81 |
-
|
| 82 |
-
## 🧪 Automated Testing
|
| 83 |
-
|
| 84 |
-
### Run Test Suite
|
| 85 |
-
- [ ] Make test script executable: `chmod +x test.sh`
|
| 86 |
-
- [ ] Run tests: `./test.sh`
|
| 87 |
-
- [ ] Review test report: `cat test_report.txt`
|
| 88 |
-
- [ ] Pass rate > 90%
|
| 89 |
-
|
| 90 |
-
### Test Results
|
| 91 |
-
- [ ] All connectivity tests pass
|
| 92 |
-
- [ ] Help commands work
|
| 93 |
-
- [ ] Catalog operations succeed
|
| 94 |
-
- [ ] Schema operations succeed
|
| 95 |
-
- [ ] Table operations succeed
|
| 96 |
-
- [ ] Permission operations succeed
|
| 97 |
-
- [ ] Performance acceptable (< 5s response)
|
| 98 |
-
|
| 99 |
-
---
|
| 100 |
-
|
| 101 |
-
## 🔒 Security Checklist
|
| 102 |
-
|
| 103 |
-
### Development Environment
|
| 104 |
-
- [ ] `.env` file is in `.gitignore`
|
| 105 |
-
- [ ] No credentials committed to Git
|
| 106 |
-
- [ ] Local firewall allows port 5000 (if needed)
|
| 107 |
-
|
| 108 |
-
### Production Environment
|
| 109 |
-
- [ ] Using Azure Key Vault for secrets
|
| 110 |
-
- [ ] HTTPS enabled
|
| 111 |
-
- [ ] CORS configured properly
|
| 112 |
-
- [ ] Rate limiting enabled
|
| 113 |
-
- [ ] Authentication enabled
|
| 114 |
-
- [ ] Monitoring configured
|
| 115 |
-
- [ ] Logging to centralized system
|
| 116 |
-
|
| 117 |
-
---
|
| 118 |
-
|
| 119 |
-
## 📊 Monitoring Checklist
|
| 120 |
-
|
| 121 |
-
### Logging
|
| 122 |
-
- [ ] Application logs are being written
|
| 123 |
-
- [ ] Log level set appropriately (INFO for prod)
|
| 124 |
-
- [ ] Errors are logged with stack traces
|
| 125 |
-
- [ ] SQL commands are logged
|
| 126 |
-
- [ ] User actions are logged
|
| 127 |
-
|
| 128 |
-
### Health Monitoring
|
| 129 |
-
- [ ] Health endpoint accessible
|
| 130 |
-
- [ ] Response time acceptable
|
| 131 |
-
- [ ] No memory leaks
|
| 132 |
-
- [ ] CPU usage normal
|
| 133 |
-
|
| 134 |
-
---
|
| 135 |
-
|
| 136 |
-
## 🐳 Docker Deployment Checklist
|
| 137 |
-
|
| 138 |
-
### If Using Docker
|
| 139 |
-
- [ ] Docker daemon is running
|
| 140 |
-
- [ ] `.env` file configured
|
| 141 |
-
- [ ] Build image: `docker build -t unity-catalog-chatbot .`
|
| 142 |
-
- [ ] Image built successfully
|
| 143 |
-
- [ ] Run container: `docker run -d -p 5000:5000 --env-file .env unity-catalog-chatbot`
|
| 144 |
-
- [ ] Container is running: `docker ps`
|
| 145 |
-
- [ ] Health check passes
|
| 146 |
-
- [ ] Logs accessible: `docker logs unity-catalog-chatbot`
|
| 147 |
-
|
| 148 |
-
### If Using Docker Compose
|
| 149 |
-
- [ ] Docker Compose installed
|
| 150 |
-
- [ ] `.env` file configured
|
| 151 |
-
- [ ] Start services: `docker-compose up -d`
|
| 152 |
-
- [ ] All services running: `docker-compose ps`
|
| 153 |
-
- [ ] Health checks pass
|
| 154 |
-
- [ ] Logs accessible: `docker-compose logs`
|
| 155 |
-
|
| 156 |
-
---
|
| 157 |
-
|
| 158 |
-
## 🌐 Frontend Deployment Checklist
|
| 159 |
-
|
| 160 |
-
### React Component
|
| 161 |
-
- [ ] Update API URL in component (if different from localhost:5000)
|
| 162 |
-
- [ ] Build React app: `npm run build`
|
| 163 |
-
- [ ] Test component in development
|
| 164 |
-
- [ ] Verify API calls work
|
| 165 |
-
- [ ] Check CORS settings
|
| 166 |
-
|
| 167 |
-
### Static Hosting
|
| 168 |
-
- [ ] Built files ready
|
| 169 |
-
- [ ] CORS enabled on backend
|
| 170 |
-
- [ ] Deploy to hosting (Netlify/Vercel/Azure Static Web Apps)
|
| 171 |
-
- [ ] Test production URL
|
| 172 |
-
- [ ] Verify API connectivity
|
| 173 |
-
|
| 174 |
-
---
|
| 175 |
-
|
| 176 |
-
## ☁️ Azure Deployment Checklist
|
| 177 |
-
|
| 178 |
-
### Prerequisites
|
| 179 |
-
- [ ] Azure subscription active
|
| 180 |
-
- [ ] Azure CLI installed
|
| 181 |
-
- [ ] Logged into Azure: `az login`
|
| 182 |
-
|
| 183 |
-
### Resource Creation
|
| 184 |
-
- [ ] Resource group created
|
| 185 |
-
- [ ] Key Vault created
|
| 186 |
-
- [ ] Secrets stored in Key Vault
|
| 187 |
-
- [ ] App Service Plan created
|
| 188 |
-
- [ ] Web App created
|
| 189 |
-
- [ ] Application Insights configured
|
| 190 |
-
|
| 191 |
-
### Deployment
|
| 192 |
-
- [ ] Container registry created
|
| 193 |
-
- [ ] Docker image built
|
| 194 |
-
- [ ] Image pushed to ACR
|
| 195 |
-
- [ ] Web App configured
|
| 196 |
-
- [ ] Environment variables set
|
| 197 |
-
- [ ] Managed identity configured
|
| 198 |
-
- [ ] Key Vault access granted
|
| 199 |
-
|
| 200 |
-
### Verification
|
| 201 |
-
- [ ] App Service is running
|
| 202 |
-
- [ ] Health endpoint accessible
|
| 203 |
-
- [ ] HTTPS working
|
| 204 |
-
- [ ] Custom domain configured (if needed)
|
| 205 |
-
- [ ] SSL certificate valid
|
| 206 |
-
|
| 207 |
-
---
|
| 208 |
-
|
| 209 |
-
## 📝 Documentation Checklist
|
| 210 |
-
|
| 211 |
-
### User Documentation
|
| 212 |
-
- [ ] README.md reviewed
|
| 213 |
-
- [ ] QUICKSTART.md available
|
| 214 |
-
- [ ] TESTING_GUIDE.md available
|
| 215 |
-
- [ ] Example commands documented
|
| 216 |
-
- [ ] API endpoints documented
|
| 217 |
-
|
| 218 |
-
### Operations Documentation
|
| 219 |
-
- [ ] DEPLOYMENT.md reviewed
|
| 220 |
-
- [ ] Backup procedures documented
|
| 221 |
-
- [ ] Disaster recovery plan documented
|
| 222 |
-
- [ ] Monitoring setup documented
|
| 223 |
-
- [ ] Troubleshooting guide available
|
| 224 |
-
|
| 225 |
-
---
|
| 226 |
-
|
| 227 |
-
## 🎯 Post-Deployment Checklist
|
| 228 |
-
|
| 229 |
-
### Day 1
|
| 230 |
-
- [ ] Monitor logs for errors
|
| 231 |
-
- [ ] Test all major functions
|
| 232 |
-
- [ ] Verify no performance issues
|
| 233 |
-
- [ ] Check resource usage (CPU/Memory)
|
| 234 |
-
- [ ] Ensure no security alerts
|
| 235 |
-
|
| 236 |
-
### Week 1
|
| 237 |
-
- [ ] Review usage patterns
|
| 238 |
-
- [ ] Optimize slow queries
|
| 239 |
-
- [ ] Update documentation based on feedback
|
| 240 |
-
- [ ] Address any bugs found
|
| 241 |
-
- [ ] Plan feature improvements
|
| 242 |
-
|
| 243 |
-
### Month 1
|
| 244 |
-
- [ ] Security audit
|
| 245 |
-
- [ ] Performance review
|
| 246 |
-
- [ ] Cost analysis (if cloud-hosted)
|
| 247 |
-
- [ ] User feedback review
|
| 248 |
-
- [ ] Update dependencies
|
| 249 |
-
|
| 250 |
-
---
|
| 251 |
-
|
| 252 |
-
## 🆘 Troubleshooting Checklist
|
| 253 |
-
|
| 254 |
-
### Application Won't Start
|
| 255 |
-
- [ ] Check Python version
|
| 256 |
-
- [ ] Verify all dependencies installed
|
| 257 |
-
- [ ] Check `.env` file exists and is valid
|
| 258 |
-
- [ ] Look for port conflicts
|
| 259 |
-
- [ ] Review error messages in console
|
| 260 |
-
- [ ] Check `app.log` file
|
| 261 |
-
|
| 262 |
-
### Databricks Connection Failed
|
| 263 |
-
- [ ] Verify workspace URL is correct
|
| 264 |
-
- [ ] Check token hasn't expired
|
| 265 |
-
- [ ] Test token with curl command
|
| 266 |
-
- [ ] Verify network connectivity
|
| 267 |
-
- [ ] Check firewall rules
|
| 268 |
-
|
| 269 |
-
### Claude API Errors
|
| 270 |
-
- [ ] Verify API key is correct
|
| 271 |
-
- [ ] Check API credits/billing
|
| 272 |
-
- [ ] Test API key with curl
|
| 273 |
-
- [ ] Check rate limits
|
| 274 |
-
- [ ] Verify internet connectivity
|
| 275 |
-
|
| 276 |
-
### Tests Failing
|
| 277 |
-
- [ ] Check test environment setup
|
| 278 |
-
- [ ] Verify test data exists
|
| 279 |
-
- [ ] Review test logs
|
| 280 |
-
- [ ] Check for timing issues
|
| 281 |
-
- [ ] Verify API is running
|
| 282 |
-
|
| 283 |
-
---
|
| 284 |
-
|
| 285 |
-
## ✨ Success Criteria
|
| 286 |
-
|
| 287 |
-
Your deployment is successful when:
|
| 288 |
-
|
| 289 |
-
### Technical
|
| 290 |
-
- ✅ Health check returns 200 OK
|
| 291 |
-
- ✅ All automated tests pass (>90%)
|
| 292 |
-
- ✅ Response times < 5 seconds
|
| 293 |
-
- ✅ No errors in logs
|
| 294 |
-
- ✅ Memory usage stable
|
| 295 |
-
|
| 296 |
-
### Functional
|
| 297 |
-
- ✅ Can list existing catalogs
|
| 298 |
-
- ✅ Can create new catalogs
|
| 299 |
-
- ✅ Can create schemas and tables
|
| 300 |
-
- ✅ Can manage permissions
|
| 301 |
-
- ✅ SQL commands are correct
|
| 302 |
-
|
| 303 |
-
### Operational
|
| 304 |
-
- ✅ Logs are being collected
|
| 305 |
-
- ✅ Monitoring is active
|
| 306 |
-
- ✅ Backups configured (if applicable)
|
| 307 |
-
- ✅ Team trained on usage
|
| 308 |
-
- ✅ Documentation complete
|
| 309 |
-
|
| 310 |
-
---
|
| 311 |
-
|
| 312 |
-
## 📞 Support Resources
|
| 313 |
-
|
| 314 |
-
- **Documentation:** README.md, DEPLOYMENT.md, TESTING_GUIDE.md
|
| 315 |
-
- **Logs:** `app.log` or `docker logs`
|
| 316 |
-
- **Health Check:** http://localhost:5000/api/health
|
| 317 |
-
- **Test Suite:** `./test.sh`
|
| 318 |
-
|
| 319 |
-
---
|
| 320 |
-
|
| 321 |
-
## 🎉 Completion
|
| 322 |
-
|
| 323 |
-
Once all items are checked:
|
| 324 |
-
- [ ] Deployment is complete
|
| 325 |
-
- [ ] System is tested and verified
|
| 326 |
-
- [ ] Documentation is up to date
|
| 327 |
-
- [ ] Team is trained
|
| 328 |
-
- [ ] Monitoring is active
|
| 329 |
-
- [ ] Ready for production use!
|
| 330 |
-
|
| 331 |
-
**Congratulations! Your Unity Catalog Chatbot is deployed and ready to use! 🚀**
|
| 332 |
-
|
| 333 |
-
---
|
| 334 |
-
|
| 335 |
-
Last Updated: $(date)
|
| 336 |
-
Version: 1.0.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
DEPLOYMENT_GUIDE.md
DELETED
|
@@ -1,755 +0,0 @@
|
|
| 1 |
-
# UnityCatalog-ChatBot: Step-by-Step Deployment Guide
|
| 2 |
-
|
| 3 |
-
## Overview
|
| 4 |
-
This guide covers deploying the Unity Catalog ChatBot locally, in Docker, and to cloud platforms (AWS ECS, Kubernetes, Azure, etc.).
|
| 5 |
-
|
| 6 |
-
---
|
| 7 |
-
|
| 8 |
-
## Part 1: Local Development Setup
|
| 9 |
-
|
| 10 |
-
### Prerequisites
|
| 11 |
-
- **Python 3.9+** (3.11+ recommended)
|
| 12 |
-
- **pip** (Python package manager)
|
| 13 |
-
- **Git**
|
| 14 |
-
- **Databricks workspace** with:
|
| 15 |
-
- Host URL (e.g., `https://your-workspace.databricks.com`)
|
| 16 |
-
- Personal Access Token (PAT)
|
| 17 |
-
- Warehouse ID (optional, for SQL execution)
|
| 18 |
-
- **Anthropic API key** (Claude access)
|
| 19 |
-
|
| 20 |
-
### Step 1: Clone Repository
|
| 21 |
-
```bash
|
| 22 |
-
git clone <repository-url>
|
| 23 |
-
cd UnityCatalog-ChatBot
|
| 24 |
-
```
|
| 25 |
-
|
| 26 |
-
### Step 2: Create Python Virtual Environment
|
| 27 |
-
```bash
|
| 28 |
-
# Windows
|
| 29 |
-
python -m venv venv
|
| 30 |
-
venv\Scripts\activate
|
| 31 |
-
|
| 32 |
-
# macOS/Linux
|
| 33 |
-
python3 -m venv venv
|
| 34 |
-
source venv/bin/activate
|
| 35 |
-
```
|
| 36 |
-
|
| 37 |
-
### Step 3: Install Dependencies
|
| 38 |
-
```bash
|
| 39 |
-
pip install -r requirements.txt
|
| 40 |
-
```
|
| 41 |
-
|
| 42 |
-
### Step 4: Configure Environment Variables
|
| 43 |
-
Create a `.env` file in the project root:
|
| 44 |
-
```env
|
| 45 |
-
# Databricks Configuration
|
| 46 |
-
DATABRICKS_HOST=https://your-workspace.databricks.com
|
| 47 |
-
DATABRICKS_TOKEN=your-personal-access-token
|
| 48 |
-
DATABRICKS_WAREHOUSE_ID=your-warehouse-id # Optional
|
| 49 |
-
|
| 50 |
-
# Anthropic Configuration
|
| 51 |
-
ANTHROPIC_API_KEY=sk-ant-your-api-key
|
| 52 |
-
|
| 53 |
-
# Server Configuration (Optional)
|
| 54 |
-
SERVER_HOST=0.0.0.0
|
| 55 |
-
SERVER_PORT=5000
|
| 56 |
-
SERVER_WORKERS=4
|
| 57 |
-
FLASK_ENV=development
|
| 58 |
-
|
| 59 |
-
# Feature Flags (Optional)
|
| 60 |
-
ENABLE_AUTH=false
|
| 61 |
-
ENABLE_SQL_EXECUTION=false
|
| 62 |
-
LOG_LEVEL=INFO
|
| 63 |
-
```
|
| 64 |
-
|
| 65 |
-
**Important:** Never commit `.env` to version control. Add it to `.gitignore`:
|
| 66 |
-
```
|
| 67 |
-
.env
|
| 68 |
-
*.pyc
|
| 69 |
-
__pycache__/
|
| 70 |
-
venv/
|
| 71 |
-
```
|
| 72 |
-
|
| 73 |
-
### Step 5: Run Tests (Mock-Only, No Credentials Required)
|
| 74 |
-
```bash
|
| 75 |
-
# Run all tests
|
| 76 |
-
pytest test_chatbot.py -v
|
| 77 |
-
|
| 78 |
-
# Run specific test class
|
| 79 |
-
pytest test_chatbot.py::TestUnityCatalogService -v
|
| 80 |
-
|
| 81 |
-
# Run with coverage
|
| 82 |
-
pytest test_chatbot.py --cov=. --cov-report=html
|
| 83 |
-
```
|
| 84 |
-
|
| 85 |
-
Expected output:
|
| 86 |
-
```
|
| 87 |
-
======================== 23 passed in 0.52s =======================
|
| 88 |
-
```
|
| 89 |
-
|
| 90 |
-
### Step 6: Run Development Server
|
| 91 |
-
```bash
|
| 92 |
-
python app.py
|
| 93 |
-
```
|
| 94 |
-
|
| 95 |
-
Server starts on `http://localhost:5000`
|
| 96 |
-
|
| 97 |
-
### Step 7: Test API Endpoints
|
| 98 |
-
```bash
|
| 99 |
-
# Health check
|
| 100 |
-
curl http://localhost:5000/api/health
|
| 101 |
-
|
| 102 |
-
# List catalogs
|
| 103 |
-
curl http://localhost:5000/api/catalogs
|
| 104 |
-
|
| 105 |
-
# Chat endpoint
|
| 106 |
-
curl -X POST http://localhost:5000/api/chat \
|
| 107 |
-
-H "Content-Type: application/json" \
|
| 108 |
-
-d '{"message": "Create a catalog named sales_data"}'
|
| 109 |
-
```
|
| 110 |
-
|
| 111 |
-
---
|
| 112 |
-
|
| 113 |
-
## Part 2: Docker Deployment
|
| 114 |
-
|
| 115 |
-
### Step 1: Build Docker Image
|
| 116 |
-
```bash
|
| 117 |
-
docker build -t unitycatalog-chatbot:latest .
|
| 118 |
-
```
|
| 119 |
-
|
| 120 |
-
### Step 2: Run Container (Development)
|
| 121 |
-
```bash
|
| 122 |
-
docker run -p 5000:5000 \
|
| 123 |
-
-e DATABRICKS_HOST="https://your-workspace.databricks.com" \
|
| 124 |
-
-e DATABRICKS_TOKEN="your-token" \
|
| 125 |
-
-e ANTHROPIC_API_KEY="sk-ant-your-key" \
|
| 126 |
-
unitycatalog-chatbot:latest
|
| 127 |
-
```
|
| 128 |
-
|
| 129 |
-
### Step 3: Run with Docker Compose
|
| 130 |
-
```bash
|
| 131 |
-
docker-compose up -d
|
| 132 |
-
```
|
| 133 |
-
|
| 134 |
-
Check logs:
|
| 135 |
-
```bash
|
| 136 |
-
docker-compose logs -f app
|
| 137 |
-
```
|
| 138 |
-
|
| 139 |
-
Stop containers:
|
| 140 |
-
```bash
|
| 141 |
-
docker-compose down
|
| 142 |
-
```
|
| 143 |
-
|
| 144 |
-
### Step 4: Verify Container Health
|
| 145 |
-
```bash
|
| 146 |
-
curl http://localhost:5000/api/health
|
| 147 |
-
```
|
| 148 |
-
|
| 149 |
-
---
|
| 150 |
-
|
| 151 |
-
## Part 3: Cloud Deployment
|
| 152 |
-
|
| 153 |
-
### AWS ECS (Elastic Container Service)
|
| 154 |
-
|
| 155 |
-
#### Prerequisites
|
| 156 |
-
- AWS account with ECS access
|
| 157 |
-
- ECR (Elastic Container Registry) repository created
|
| 158 |
-
- IAM role for ECS task execution
|
| 159 |
-
|
| 160 |
-
#### Steps
|
| 161 |
-
1. **Push image to ECR:**
|
| 162 |
-
```bash
|
| 163 |
-
# Get login token
|
| 164 |
-
aws ecr get-login-password --region us-east-1 | \
|
| 165 |
-
docker login --username AWS --password-stdin \
|
| 166 |
-
YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com
|
| 167 |
-
|
| 168 |
-
# Tag image
|
| 169 |
-
docker tag unitycatalog-chatbot:latest \
|
| 170 |
-
YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/unitycatalog-chatbot:latest
|
| 171 |
-
|
| 172 |
-
# Push to ECR
|
| 173 |
-
docker push YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/unitycatalog-chatbot:latest
|
| 174 |
-
```
|
| 175 |
-
|
| 176 |
-
2. **Create ECS Task Definition** (JSON):
|
| 177 |
-
```json
|
| 178 |
-
{
|
| 179 |
-
"family": "unitycatalog-chatbot",
|
| 180 |
-
"containerDefinitions": [
|
| 181 |
-
{
|
| 182 |
-
"name": "app",
|
| 183 |
-
"image": "YOUR_ACCOUNT_ID.dkr.ecr.us-east-1.amazonaws.com/unitycatalog-chatbot:latest",
|
| 184 |
-
"portMappings": [{"containerPort": 5000, "hostPort": 5000}],
|
| 185 |
-
"environment": [
|
| 186 |
-
{"name": "DATABRICKS_HOST", "value": "https://your-workspace.databricks.com"},
|
| 187 |
-
{"name": "SERVER_HOST", "value": "0.0.0.0"},
|
| 188 |
-
{"name": "SERVER_PORT", "value": "5000"}
|
| 189 |
-
],
|
| 190 |
-
"secrets": [
|
| 191 |
-
{"name": "DATABRICKS_TOKEN", "valueFrom": "arn:aws:secretsmanager:..."},
|
| 192 |
-
{"name": "ANTHROPIC_API_KEY", "valueFrom": "arn:aws:secretsmanager:..."}
|
| 193 |
-
],
|
| 194 |
-
"logConfiguration": {
|
| 195 |
-
"logDriver": "awslogs",
|
| 196 |
-
"options": {
|
| 197 |
-
"awslogs-group": "/ecs/unitycatalog-chatbot",
|
| 198 |
-
"awslogs-region": "us-east-1",
|
| 199 |
-
"awslogs-stream-prefix": "ecs"
|
| 200 |
-
}
|
| 201 |
-
}
|
| 202 |
-
}
|
| 203 |
-
],
|
| 204 |
-
"requiresCompatibilities": ["FARGATE"],
|
| 205 |
-
"networkMode": "awsvpc",
|
| 206 |
-
"cpu": "256",
|
| 207 |
-
"memory": "512"
|
| 208 |
-
}
|
| 209 |
-
```
|
| 210 |
-
|
| 211 |
-
3. **Create ECS Service** using AWS Console or CLI:
|
| 212 |
-
```bash
|
| 213 |
-
aws ecs create-service \
|
| 214 |
-
--cluster my-cluster \
|
| 215 |
-
--service-name unitycatalog-chatbot \
|
| 216 |
-
--task-definition unitycatalog-chatbot:1 \
|
| 217 |
-
--desired-count 2 \
|
| 218 |
-
--launch-type FARGATE \
|
| 219 |
-
--network-configuration "awsvpcConfiguration={subnets=[subnet-xxx],securityGroups=[sg-xxx]}"
|
| 220 |
-
```
|
| 221 |
-
|
| 222 |
-
4. **Monitor service:**
|
| 223 |
-
```bash
|
| 224 |
-
aws ecs describe-services \
|
| 225 |
-
--cluster my-cluster \
|
| 226 |
-
--services unitycatalog-chatbot
|
| 227 |
-
```
|
| 228 |
-
|
| 229 |
-
### Kubernetes Deployment
|
| 230 |
-
|
| 231 |
-
#### Prerequisites
|
| 232 |
-
- Kubernetes cluster (EKS, AKS, GKE, or local)
|
| 233 |
-
- `kubectl` CLI configured
|
| 234 |
-
- Docker image pushed to container registry
|
| 235 |
-
|
| 236 |
-
#### Steps
|
| 237 |
-
|
| 238 |
-
1. **Create Kubernetes Secrets:**
|
| 239 |
-
```bash
|
| 240 |
-
kubectl create secret generic unitycatalog-secrets \
|
| 241 |
-
--from-literal=DATABRICKS_TOKEN='your-token' \
|
| 242 |
-
--from-literal=ANTHROPIC_API_KEY='sk-ant-your-key'
|
| 243 |
-
```
|
| 244 |
-
|
| 245 |
-
2. **Create Deployment (deployment.yaml):**
|
| 246 |
-
```yaml
|
| 247 |
-
apiVersion: apps/v1
|
| 248 |
-
kind: Deployment
|
| 249 |
-
metadata:
|
| 250 |
-
name: unitycatalog-chatbot
|
| 251 |
-
labels:
|
| 252 |
-
app: unitycatalog-chatbot
|
| 253 |
-
spec:
|
| 254 |
-
replicas: 3
|
| 255 |
-
selector:
|
| 256 |
-
matchLabels:
|
| 257 |
-
app: unitycatalog-chatbot
|
| 258 |
-
template:
|
| 259 |
-
metadata:
|
| 260 |
-
labels:
|
| 261 |
-
app: unitycatalog-chatbot
|
| 262 |
-
spec:
|
| 263 |
-
containers:
|
| 264 |
-
- name: app
|
| 265 |
-
image: your-registry/unitycatalog-chatbot:latest
|
| 266 |
-
ports:
|
| 267 |
-
- containerPort: 5000
|
| 268 |
-
env:
|
| 269 |
-
- name: DATABRICKS_HOST
|
| 270 |
-
value: "https://your-workspace.databricks.com"
|
| 271 |
-
- name: SERVER_HOST
|
| 272 |
-
value: "0.0.0.0"
|
| 273 |
-
- name: SERVER_PORT
|
| 274 |
-
value: "5000"
|
| 275 |
-
envFrom:
|
| 276 |
-
- secretRef:
|
| 277 |
-
name: unitycatalog-secrets
|
| 278 |
-
livenessProbe:
|
| 279 |
-
httpGet:
|
| 280 |
-
path: /api/health
|
| 281 |
-
port: 5000
|
| 282 |
-
initialDelaySeconds: 30
|
| 283 |
-
periodSeconds: 10
|
| 284 |
-
readinessProbe:
|
| 285 |
-
httpGet:
|
| 286 |
-
path: /api/health
|
| 287 |
-
port: 5000
|
| 288 |
-
initialDelaySeconds: 10
|
| 289 |
-
periodSeconds: 5
|
| 290 |
-
resources:
|
| 291 |
-
requests:
|
| 292 |
-
memory: "256Mi"
|
| 293 |
-
cpu: "250m"
|
| 294 |
-
limits:
|
| 295 |
-
memory: "512Mi"
|
| 296 |
-
cpu: "500m"
|
| 297 |
-
---
|
| 298 |
-
apiVersion: v1
|
| 299 |
-
kind: Service
|
| 300 |
-
metadata:
|
| 301 |
-
name: unitycatalog-chatbot-service
|
| 302 |
-
spec:
|
| 303 |
-
type: LoadBalancer
|
| 304 |
-
ports:
|
| 305 |
-
- port: 80
|
| 306 |
-
targetPort: 5000
|
| 307 |
-
selector:
|
| 308 |
-
app: unitycatalog-chatbot
|
| 309 |
-
```
|
| 310 |
-
|
| 311 |
-
3. **Deploy to Kubernetes:**
|
| 312 |
-
```bash
|
| 313 |
-
kubectl apply -f deployment.yaml
|
| 314 |
-
```
|
| 315 |
-
|
| 316 |
-
4. **Monitor deployment:**
|
| 317 |
-
```bash
|
| 318 |
-
# Check status
|
| 319 |
-
kubectl get pods -l app=unitycatalog-chatbot
|
| 320 |
-
|
| 321 |
-
# Check logs
|
| 322 |
-
kubectl logs -f deployment/unitycatalog-chatbot
|
| 323 |
-
|
| 324 |
-
# Get service endpoint
|
| 325 |
-
kubectl get service unitycatalog-chatbot-service
|
| 326 |
-
```
|
| 327 |
-
|
| 328 |
-
### Azure Container Instances (ACI)
|
| 329 |
-
|
| 330 |
-
1. **Push image to Azure Container Registry:**
|
| 331 |
-
```bash
|
| 332 |
-
az acr build --registry your-acr-name \
|
| 333 |
-
--image unitycatalog-chatbot:latest .
|
| 334 |
-
```
|
| 335 |
-
|
| 336 |
-
2. **Deploy to ACI:**
|
| 337 |
-
```bash
|
| 338 |
-
az container create \
|
| 339 |
-
--resource-group your-rg \
|
| 340 |
-
--name unitycatalog-chatbot \
|
| 341 |
-
--image your-acr-name.azurecr.io/unitycatalog-chatbot:latest \
|
| 342 |
-
--cpu 1 --memory 1 \
|
| 343 |
-
--ports 5000 \
|
| 344 |
-
--environment-variables \
|
| 345 |
-
DATABRICKS_HOST="https://your-workspace.databricks.com" \
|
| 346 |
-
SERVER_HOST="0.0.0.0" \
|
| 347 |
-
SERVER_PORT="5000" \
|
| 348 |
-
--secure-environment-variables \
|
| 349 |
-
DATABRICKS_TOKEN="your-token" \
|
| 350 |
-
ANTHROPIC_API_KEY="sk-ant-your-key"
|
| 351 |
-
```
|
| 352 |
-
|
| 353 |
-
### Hugging Face Spaces (Docker)
|
| 354 |
-
|
| 355 |
-
#### Prerequisites
|
| 356 |
-
- Hugging Face account (free at https://huggingface.co/join)
|
| 357 |
-
- Repository on Hugging Face Hub (create at https://huggingface.co/new)
|
| 358 |
-
|
| 359 |
-
#### Step 1: Create Hugging Face Repository
|
| 360 |
-
1. Go to https://huggingface.co/new
|
| 361 |
-
2. Enter repository name: `unitycatalog-chatbot`
|
| 362 |
-
3. Select **Space** type
|
| 363 |
-
4. Choose **Docker** runtime
|
| 364 |
-
5. Click **Create repository**
|
| 365 |
-
|
| 366 |
-
#### Step 2: Clone Space Repository
|
| 367 |
-
```bash
|
| 368 |
-
git clone https://huggingface.co/spaces/your-username/unitycatalog-chatbot
|
| 369 |
-
cd unitycatalog-chatbot
|
| 370 |
-
```
|
| 371 |
-
|
| 372 |
-
#### Step 3: Copy Project Files
|
| 373 |
-
```bash
|
| 374 |
-
# Copy source files
|
| 375 |
-
cp -r ../UnityCatalog-ChatBot/* .
|
| 376 |
-
|
| 377 |
-
# Ensure key files are present:
|
| 378 |
-
# - app.py
|
| 379 |
-
# - unity_catalog_service.py
|
| 380 |
-
# - config.py
|
| 381 |
-
# - requirements.txt
|
| 382 |
-
# - Dockerfile
|
| 383 |
-
# - test_chatbot.py (optional)
|
| 384 |
-
```
|
| 385 |
-
|
| 386 |
-
#### Step 4: Create `README.md` for Space
|
| 387 |
-
```markdown
|
| 388 |
-
---
|
| 389 |
-
title: Unity Catalog Chatbot
|
| 390 |
-
emoji: 💬
|
| 391 |
-
colorFrom: blue
|
| 392 |
-
colorTo: purple
|
| 393 |
-
sdk: docker
|
| 394 |
-
pinned: false
|
| 395 |
-
---
|
| 396 |
-
|
| 397 |
-
# Unity Catalog Chatbot
|
| 398 |
-
|
| 399 |
-
Natural language interface for Databricks Unity Catalog operations powered by Claude AI.
|
| 400 |
-
|
| 401 |
-
## Features
|
| 402 |
-
- Create catalogs, schemas, and tables via natural language
|
| 403 |
-
- Grant and revoke permissions
|
| 404 |
-
- List objects across Unity Catalog
|
| 405 |
-
- Execute SQL queries
|
| 406 |
-
|
| 407 |
-
## API Endpoints
|
| 408 |
-
- `GET /api/health` - Health check
|
| 409 |
-
- `POST /api/chat` - Chat with Claude to manage Unity Catalog
|
| 410 |
-
- `GET /api/catalogs` - List all catalogs
|
| 411 |
-
- `GET /api/schemas/<catalog>` - List schemas
|
| 412 |
-
- `GET /api/tables/<catalog>/<schema>` - List tables
|
| 413 |
-
|
| 414 |
-
## Environment Variables
|
| 415 |
-
Required:
|
| 416 |
-
- `DATABRICKS_HOST` - Databricks workspace URL
|
| 417 |
-
- `DATABRICKS_TOKEN` - Personal access token
|
| 418 |
-
- `ANTHROPIC_API_KEY` - Claude API key
|
| 419 |
-
|
| 420 |
-
Optional:
|
| 421 |
-
- `SERVER_PORT` - Port to run on (default: 5000)
|
| 422 |
-
- `LOG_LEVEL` - Logging level (default: INFO)
|
| 423 |
-
```
|
| 424 |
-
|
| 425 |
-
#### Step 5: Create `secrets.toml` for Credentials
|
| 426 |
-
Create `.streamlit/secrets.toml` (Hugging Face Spaces will hide these):
|
| 427 |
-
|
| 428 |
-
**⚠️ IMPORTANT: Never commit secrets to git**
|
| 429 |
-
|
| 430 |
-
Instead, use Hugging Face Secrets management:
|
| 431 |
-
|
| 432 |
-
1. Go to your Space Settings → Secrets
|
| 433 |
-
2. Add secrets:
|
| 434 |
-
- `DATABRICKS_HOST` = `https://your-workspace.databricks.com`
|
| 435 |
-
- `DATABRICKS_TOKEN` = your token
|
| 436 |
-
- `ANTHROPIC_API_KEY` = your API key
|
| 437 |
-
|
| 438 |
-
Or set via UI environment variables in Space settings.
|
| 439 |
-
|
| 440 |
-
#### Step 6: Update Dockerfile for Space (if needed)
|
| 441 |
-
```dockerfile
|
| 442 |
-
FROM python:3.11-slim
|
| 443 |
-
|
| 444 |
-
WORKDIR /app
|
| 445 |
-
|
| 446 |
-
COPY requirements.txt .
|
| 447 |
-
RUN pip install --no-cache-dir -r requirements.txt
|
| 448 |
-
|
| 449 |
-
COPY . .
|
| 450 |
-
|
| 451 |
-
# Health check
|
| 452 |
-
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
|
| 453 |
-
CMD python -c "import requests; requests.get('http://localhost:5000/api/health')"
|
| 454 |
-
|
| 455 |
-
CMD ["python", "app.py"]
|
| 456 |
-
```
|
| 457 |
-
|
| 458 |
-
#### Step 7: Push to Hugging Face
|
| 459 |
-
```bash
|
| 460 |
-
# Configure git with HF credentials
|
| 461 |
-
git config user.name "Your Name"
|
| 462 |
-
git config user.email "your-email@example.com"
|
| 463 |
-
|
| 464 |
-
# Add files
|
| 465 |
-
git add .
|
| 466 |
-
|
| 467 |
-
# Commit
|
| 468 |
-
git commit -m "Initial deployment"
|
| 469 |
-
|
| 470 |
-
# Push (triggers auto-build and deployment)
|
| 471 |
-
git push
|
| 472 |
-
```
|
| 473 |
-
|
| 474 |
-
**Space will automatically:**
|
| 475 |
-
- Build Docker image
|
| 476 |
-
- Deploy to Hugging Face infrastructure
|
| 477 |
-
- Provide public URL: `https://huggingface.co/spaces/your-username/unitycatalog-chatbot`
|
| 478 |
-
|
| 479 |
-
#### Step 8: Verify Deployment
|
| 480 |
-
```bash
|
| 481 |
-
# Once Space is running, test the endpoint:
|
| 482 |
-
curl https://your-username-unitycatalog-chatbot.hf.space/api/health
|
| 483 |
-
|
| 484 |
-
# Chat endpoint
|
| 485 |
-
curl -X POST https://your-username-unitycatalog-chatbot.hf.space/api/chat \
|
| 486 |
-
-H "Content-Type: application/json" \
|
| 487 |
-
-d '{"message": "List all catalogs"}'
|
| 488 |
-
```
|
| 489 |
-
|
| 490 |
-
#### Step 9: Set Secrets in Hugging Face UI
|
| 491 |
-
1. Go to Space → Settings → Secrets
|
| 492 |
-
2. Add environment variables:
|
| 493 |
-
- `DATABRICKS_HOST`
|
| 494 |
-
- `DATABRICKS_TOKEN`
|
| 495 |
-
- `ANTHROPIC_API_KEY`
|
| 496 |
-
|
| 497 |
-
3. Space will restart automatically with secrets loaded
|
| 498 |
-
|
| 499 |
-
#### Troubleshooting Hugging Face Spaces
|
| 500 |
-
|
| 501 |
-
**Issue: "Build failed"**
|
| 502 |
-
- Check **Build logs** in Space settings
|
| 503 |
-
- Ensure `Dockerfile` is present
|
| 504 |
-
- Verify `requirements.txt` has all dependencies
|
| 505 |
-
|
| 506 |
-
**Issue: "Application won't start"**
|
| 507 |
-
- Check **Runtime logs** in Space
|
| 508 |
-
- Verify environment variables are set in Secrets
|
| 509 |
-
- Test locally: `docker build -t test . && docker run -p 5000:5000 test`
|
| 510 |
-
|
| 511 |
-
**Issue: "Port already in use"**
|
| 512 |
-
- Hugging Face assigns a port automatically
|
| 513 |
-
- Ensure `app.py` uses environment variable for port:
|
| 514 |
-
```python
|
| 515 |
-
if __name__ == '__main__':
|
| 516 |
-
port = int(os.getenv("SERVER_PORT", 5000))
|
| 517 |
-
app.run(host='0.0.0.0', port=port)
|
| 518 |
-
```
|
| 519 |
-
|
| 520 |
-
**Issue: "API calls timeout"**
|
| 521 |
-
- Databricks/Anthropic credentials invalid
|
| 522 |
-
- Network connectivity issue
|
| 523 |
-
- Test locally first with real credentials
|
| 524 |
-
|
| 525 |
-
#### Hugging Face Space Features
|
| 526 |
-
|
| 527 |
-
- **Public URL:** `https://huggingface.co/spaces/your-username/unitycatalog-chatbot`
|
| 528 |
-
- **Auto-scaling:** Handles traffic spikes
|
| 529 |
-
- **Free tier:** Up to 2 CPU cores (enough for light use)
|
| 530 |
-
- **Persistent storage:** `/tmp` directory available (ephemeral)
|
| 531 |
-
- **Custom domain:** Upgrade to pro for custom domains
|
| 532 |
-
|
| 533 |
-
#### Sharing Your Space
|
| 534 |
-
|
| 535 |
-
1. Go to Space page
|
| 536 |
-
2. Click **Share** button
|
| 537 |
-
3. Copy shareable link or embed code:
|
| 538 |
-
```html
|
| 539 |
-
<iframe
|
| 540 |
-
src="https://huggingface.co/spaces/your-username/unitycatalog-chatbot?embed=true"
|
| 541 |
-
frameborder="0"
|
| 542 |
-
width="800"
|
| 543 |
-
height="600"
|
| 544 |
-
></iframe>
|
| 545 |
-
```
|
| 546 |
-
|
| 547 |
-
---
|
| 548 |
-
|
| 549 |
-
## Part 4: Production Configuration
|
| 550 |
-
|
| 551 |
-
### Security Best Practices
|
| 552 |
-
|
| 553 |
-
1. **Enable Authentication:**
|
| 554 |
-
```env
|
| 555 |
-
ENABLE_AUTH=true
|
| 556 |
-
API_KEY_HEADER=X-API-Key
|
| 557 |
-
```
|
| 558 |
-
|
| 559 |
-
Add API key header to requests:
|
| 560 |
-
```bash
|
| 561 |
-
curl -H "X-API-Key: your-api-key" http://localhost:5000/api/health
|
| 562 |
-
```
|
| 563 |
-
|
| 564 |
-
2. **Rate Limiting:**
|
| 565 |
-
```env
|
| 566 |
-
RATE_LIMIT_PER_MINUTE=60
|
| 567 |
-
```
|
| 568 |
-
|
| 569 |
-
3. **HTTPS/TLS:**
|
| 570 |
-
Use reverse proxy (Nginx, HAProxy) to terminate TLS:
|
| 571 |
-
```nginx
|
| 572 |
-
server {
|
| 573 |
-
listen 443 ssl;
|
| 574 |
-
server_name your-domain.com;
|
| 575 |
-
|
| 576 |
-
ssl_certificate /path/to/cert.pem;
|
| 577 |
-
ssl_certificate_key /path/to/key.pem;
|
| 578 |
-
|
| 579 |
-
location / {
|
| 580 |
-
proxy_pass http://localhost:5000;
|
| 581 |
-
proxy_set_header X-Forwarded-For $remote_addr;
|
| 582 |
-
}
|
| 583 |
-
}
|
| 584 |
-
```
|
| 585 |
-
|
| 586 |
-
4. **Environment Variables:**
|
| 587 |
-
Use secret management (AWS Secrets Manager, Azure Key Vault, HashiCorp Vault):
|
| 588 |
-
```bash
|
| 589 |
-
# AWS
|
| 590 |
-
aws secretsmanager get-secret-value --secret-id unitycatalog-secrets
|
| 591 |
-
|
| 592 |
-
# Azure
|
| 593 |
-
az keyvault secret show --vault-name your-vault --name DATABRICKS_TOKEN
|
| 594 |
-
```
|
| 595 |
-
|
| 596 |
-
### Logging & Monitoring
|
| 597 |
-
|
| 598 |
-
1. **Enable comprehensive logging:**
|
| 599 |
-
```env
|
| 600 |
-
LOG_LEVEL=INFO
|
| 601 |
-
LOG_TO_FILE=true
|
| 602 |
-
LOG_FILE_PATH=/var/log/chatbot.log
|
| 603 |
-
```
|
| 604 |
-
|
| 605 |
-
2. **Application Insights / DataDog / CloudWatch:**
|
| 606 |
-
Logs are automatically captured by container orchestration platforms.
|
| 607 |
-
|
| 608 |
-
### Performance Tuning
|
| 609 |
-
|
| 610 |
-
1. **Gunicorn workers** (production):
|
| 611 |
-
```bash
|
| 612 |
-
gunicorn --workers 4 --bind 0.0.0.0:5000 app:app
|
| 613 |
-
```
|
| 614 |
-
|
| 615 |
-
2. **Caching:**
|
| 616 |
-
```env
|
| 617 |
-
ENABLE_CACHING=true
|
| 618 |
-
CACHE_TTL=300
|
| 619 |
-
```
|
| 620 |
-
|
| 621 |
-
---
|
| 622 |
-
|
| 623 |
-
## Part 5: Health Checks & Validation
|
| 624 |
-
|
| 625 |
-
### Pre-Deployment Checklist
|
| 626 |
-
|
| 627 |
-
- [ ] All tests pass: `pytest test_chatbot.py -v`
|
| 628 |
-
- [ ] `.env` file configured with valid credentials
|
| 629 |
-
- [ ] Docker image builds successfully
|
| 630 |
-
- [ ] Health endpoint responds: `curl /api/health`
|
| 631 |
-
- [ ] Sample requests succeed (catalog listing, chat)
|
| 632 |
-
- [ ] Logs show no errors
|
| 633 |
-
|
| 634 |
-
### Post-Deployment Validation
|
| 635 |
-
|
| 636 |
-
```bash
|
| 637 |
-
# Health check
|
| 638 |
-
curl https://your-api-endpoint/api/health
|
| 639 |
-
|
| 640 |
-
# Test chat endpoint
|
| 641 |
-
curl -X POST https://your-api-endpoint/api/chat \
|
| 642 |
-
-H "Content-Type: application/json" \
|
| 643 |
-
-H "X-API-Key: your-api-key" \
|
| 644 |
-
-d '{"message": "List all catalogs"}'
|
| 645 |
-
|
| 646 |
-
# Check logs
|
| 647 |
-
kubectl logs deployment/unitycatalog-chatbot # K8s
|
| 648 |
-
docker logs <container-id> # Docker
|
| 649 |
-
aws logs tail /ecs/unitycatalog-chatbot --follow # ECS
|
| 650 |
-
```
|
| 651 |
-
|
| 652 |
-
---
|
| 653 |
-
|
| 654 |
-
## Part 6: Troubleshooting
|
| 655 |
-
|
| 656 |
-
### Common Issues
|
| 657 |
-
|
| 658 |
-
**1. "Cannot configure default credentials"**
|
| 659 |
-
- Ensure `.env` file has valid `DATABRICKS_HOST` and `DATABRICKS_TOKEN`
|
| 660 |
-
- Verify token format (starts with `dapi`)
|
| 661 |
-
|
| 662 |
-
**2. "Invalid Anthropic API key"**
|
| 663 |
-
- Confirm key starts with `sk-ant-`
|
| 664 |
-
- Check key has not expired
|
| 665 |
-
|
| 666 |
-
**3. "Port 5000 already in use"**
|
| 667 |
-
```bash
|
| 668 |
-
# Kill process using port
|
| 669 |
-
lsof -ti:5000 | xargs kill -9 # macOS/Linux
|
| 670 |
-
netstat -ano | findstr :5000 & taskkill /PID <PID> /F # Windows
|
| 671 |
-
```
|
| 672 |
-
|
| 673 |
-
**4. Docker build fails**
|
| 674 |
-
```bash
|
| 675 |
-
docker build --no-cache -t unitycatalog-chatbot:latest .
|
| 676 |
-
```
|
| 677 |
-
|
| 678 |
-
**5. Tests fail in CI/CD**
|
| 679 |
-
- Tests use mocks and don't require credentials
|
| 680 |
-
- If failing, check Python version (3.9+) and pytest version
|
| 681 |
-
|
| 682 |
-
### Get Help
|
| 683 |
-
|
| 684 |
-
Check logs for detailed error messages:
|
| 685 |
-
```bash
|
| 686 |
-
# Local
|
| 687 |
-
python app.py # stdout
|
| 688 |
-
|
| 689 |
-
# Docker
|
| 690 |
-
docker logs <container-name>
|
| 691 |
-
|
| 692 |
-
# Kubernetes
|
| 693 |
-
kubectl logs <pod-name> -c app
|
| 694 |
-
```
|
| 695 |
-
|
| 696 |
-
---
|
| 697 |
-
|
| 698 |
-
## Part 7: Scaling & Maintenance
|
| 699 |
-
|
| 700 |
-
### Horizontal Scaling
|
| 701 |
-
|
| 702 |
-
- **Docker Compose:** Increase `replicas` in docker-compose.yml
|
| 703 |
-
- **Kubernetes:** `kubectl scale deployment unitycatalog-chatbot --replicas=5`
|
| 704 |
-
- **ECS:** Update desired task count in AWS Console
|
| 705 |
-
|
| 706 |
-
### Updates & Rollbacks
|
| 707 |
-
|
| 708 |
-
1. **Build new image:**
|
| 709 |
-
```bash
|
| 710 |
-
docker build -t unitycatalog-chatbot:v1.1.0 .
|
| 711 |
-
```
|
| 712 |
-
|
| 713 |
-
2. **Push to registry:**
|
| 714 |
-
```bash
|
| 715 |
-
docker push your-registry/unitycatalog-chatbot:v1.1.0
|
| 716 |
-
```
|
| 717 |
-
|
| 718 |
-
3. **Update deployment:**
|
| 719 |
-
```bash
|
| 720 |
-
# Kubernetes
|
| 721 |
-
kubectl set image deployment/unitycatalog-chatbot \
|
| 722 |
-
app=your-registry/unitycatalog-chatbot:v1.1.0
|
| 723 |
-
|
| 724 |
-
# ECS (update task definition version)
|
| 725 |
-
aws ecs update-service \
|
| 726 |
-
--cluster my-cluster \
|
| 727 |
-
--service unitycatalog-chatbot \
|
| 728 |
-
--task-definition unitycatalog-chatbot:2
|
| 729 |
-
```
|
| 730 |
-
|
| 731 |
-
4. **Rollback if needed:**
|
| 732 |
-
```bash
|
| 733 |
-
# Kubernetes
|
| 734 |
-
kubectl rollout undo deployment/unitycatalog-chatbot
|
| 735 |
-
|
| 736 |
-
# ECS
|
| 737 |
-
aws ecs update-service \
|
| 738 |
-
--cluster my-cluster \
|
| 739 |
-
--service unitycatalog-chatbot \
|
| 740 |
-
--task-definition unitycatalog-chatbot:1
|
| 741 |
-
```
|
| 742 |
-
|
| 743 |
-
---
|
| 744 |
-
|
| 745 |
-
## Summary
|
| 746 |
-
|
| 747 |
-
| Deployment Type | Complexity | Best For |
|
| 748 |
-
|---|---|---|
|
| 749 |
-
| **Local** | Easy | Development, testing |
|
| 750 |
-
| **Docker** | Medium | Single machine, CI/CD |
|
| 751 |
-
| **Kubernetes** | Hard | Enterprise, multi-region, auto-scaling |
|
| 752 |
-
| **ECS** | Medium | AWS-only deployments |
|
| 753 |
-
| **ACI** | Medium | Quick Azure deployments |
|
| 754 |
-
|
| 755 |
-
Choose based on your infrastructure and scaling needs.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Dockerfile
CHANGED
|
@@ -1,35 +1,37 @@
|
|
| 1 |
-
FROM python:3.
|
| 2 |
|
| 3 |
-
# Set working directory
|
| 4 |
WORKDIR /app
|
| 5 |
|
| 6 |
# Install system dependencies
|
| 7 |
RUN apt-get update && apt-get install -y \
|
| 8 |
-
|
| 9 |
&& rm -rf /var/lib/apt/lists/*
|
| 10 |
|
| 11 |
-
# Copy requirements
|
| 12 |
COPY requirements.txt .
|
| 13 |
|
| 14 |
# Install Python dependencies
|
| 15 |
RUN pip install --no-cache-dir -r requirements.txt
|
| 16 |
|
| 17 |
-
# Install production server
|
| 18 |
-
RUN pip install --no-cache-dir gunicorn
|
| 19 |
-
|
| 20 |
# Copy application files
|
| 21 |
COPY app.py .
|
| 22 |
COPY unity_catalog_service.py .
|
|
|
|
|
|
|
| 23 |
COPY config.py .
|
| 24 |
COPY conftest.py .
|
| 25 |
|
| 26 |
-
#
|
| 27 |
-
|
| 28 |
-
USER appuser
|
| 29 |
|
| 30 |
-
#
|
| 31 |
ENV PORT=7860
|
| 32 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 33 |
|
| 34 |
-
# Run
|
| 35 |
-
CMD ["
|
|
|
|
| 1 |
+
FROM python:3.11-slim
|
| 2 |
|
|
|
|
| 3 |
WORKDIR /app
|
| 4 |
|
| 5 |
# Install system dependencies
|
| 6 |
RUN apt-get update && apt-get install -y \
|
| 7 |
+
curl \
|
| 8 |
&& rm -rf /var/lib/apt/lists/*
|
| 9 |
|
| 10 |
+
# Copy requirements
|
| 11 |
COPY requirements.txt .
|
| 12 |
|
| 13 |
# Install Python dependencies
|
| 14 |
RUN pip install --no-cache-dir -r requirements.txt
|
| 15 |
|
|
|
|
|
|
|
|
|
|
| 16 |
# Copy application files
|
| 17 |
COPY app.py .
|
| 18 |
COPY unity_catalog_service.py .
|
| 19 |
+
COPY unity-catalog-chatbot.jsx .
|
| 20 |
+
COPY index.html .
|
| 21 |
COPY config.py .
|
| 22 |
COPY conftest.py .
|
| 23 |
|
| 24 |
+
# Expose port (HF Spaces uses 7860)
|
| 25 |
+
EXPOSE 7860
|
|
|
|
| 26 |
|
| 27 |
+
# Set environment variables
|
| 28 |
ENV PORT=7860
|
| 29 |
+
ENV HOST=0.0.0.0
|
| 30 |
+
ENV FLASK_ENV=production
|
| 31 |
+
|
| 32 |
+
# Health check
|
| 33 |
+
HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 \
|
| 34 |
+
CMD curl -f http://localhost:7860/api/health || exit 1
|
| 35 |
|
| 36 |
+
# Run Flask app
|
| 37 |
+
CMD ["python", "app.py"]
|
HF_DEPLOYMENT.md
CHANGED
|
@@ -1,153 +1,155 @@
|
|
| 1 |
-
# Deploy
|
| 2 |
|
| 3 |
-
|
| 4 |
|
| 5 |
## Prerequisites
|
| 6 |
-
- Hugging Face account (free signup: https://huggingface.co/join)
|
| 7 |
-
- Databricks credentials (host + token)
|
| 8 |
-
- Anthropic API key
|
| 9 |
|
| 10 |
-
|
|
|
|
|
|
|
|
|
|
| 11 |
|
| 12 |
-
##
|
| 13 |
-
|
| 14 |
-
|
| 15 |
-
|
| 16 |
-
|
| 17 |
-
|
| 18 |
-
|
| 19 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
|
| 21 |
-
### 2️⃣ Clone the Space
|
| 22 |
```bash
|
| 23 |
-
|
| 24 |
-
|
|
|
|
| 25 |
```
|
| 26 |
|
| 27 |
-
##
|
|
|
|
|
|
|
|
|
|
| 28 |
```bash
|
| 29 |
-
|
| 30 |
-
cp
|
| 31 |
-
|
| 32 |
-
# Required files:
|
| 33 |
-
# - app.py
|
| 34 |
-
# - unity_catalog_service.py
|
| 35 |
-
# - config.py
|
| 36 |
-
# - requirements.txt
|
| 37 |
-
# - Dockerfile
|
| 38 |
-
# - README.md (already created)
|
| 39 |
```
|
| 40 |
|
| 41 |
-
|
| 42 |
-
Create `README.md`:
|
| 43 |
-
```markdown
|
| 44 |
-
---
|
| 45 |
-
title: Unity Catalog Chatbot
|
| 46 |
-
emoji: 💬
|
| 47 |
-
colorFrom: blue
|
| 48 |
-
colorTo: purple
|
| 49 |
-
sdk: docker
|
| 50 |
-
pinned: false
|
| 51 |
-
---
|
| 52 |
|
| 53 |
-
#
|
| 54 |
|
| 55 |
-
|
| 56 |
|
| 57 |
-
|
| 58 |
-
Add secrets
|
| 59 |
-
- `
|
| 60 |
-
- `
|
| 61 |
-
- `
|
| 62 |
|
| 63 |
-
|
| 64 |
-
```
|
| 65 |
|
| 66 |
-
### 5️⃣ Push to Hugging Face
|
| 67 |
```bash
|
|
|
|
| 68 |
git add .
|
| 69 |
-
git commit -m "Deploy to HF Spaces"
|
| 70 |
-
git push
|
| 71 |
-
```
|
| 72 |
|
| 73 |
-
#
|
| 74 |
-
|
| 75 |
-
2. Add three secrets:
|
| 76 |
-
- `DATABRICKS_HOST` = `https://your-workspace.databricks.com`
|
| 77 |
-
- `DATABRICKS_TOKEN` = your PAT
|
| 78 |
-
- `ANTHROPIC_API_KEY` = your key
|
| 79 |
|
| 80 |
-
|
|
|
|
|
|
|
| 81 |
|
| 82 |
-
|
| 83 |
-
- URL: `https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot`
|
| 84 |
-
- API: `https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/`
|
| 85 |
|
| 86 |
-
##
|
| 87 |
|
| 88 |
-
|
| 89 |
-
|
| 90 |
-
|
|
|
|
| 91 |
|
| 92 |
-
#
|
| 93 |
-
curl https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/catalogs
|
| 94 |
|
| 95 |
-
|
| 96 |
-
|
| 97 |
-
|
| 98 |
-
|
| 99 |
-
```
|
| 100 |
|
| 101 |
-
##
|
| 102 |
|
| 103 |
-
|
| 104 |
-
|
| 105 |
-
|
| 106 |
-
|
| 107 |
-
|
| 108 |
-
|
|
|
|
|
|
|
|
|
|
| 109 |
|
| 110 |
-
##
|
| 111 |
|
| 112 |
-
|
| 113 |
-
|
| 114 |
-
|
|
|
|
| 115 |
|
| 116 |
-
##
|
|
|
|
|
|
|
|
|
|
| 117 |
|
| 118 |
-
|
| 119 |
-
-
|
|
|
|
|
|
|
| 120 |
|
| 121 |
-
|
|
|
|
|
|
|
|
|
|
| 122 |
|
| 123 |
-
##
|
| 124 |
|
| 125 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 126 |
|
| 127 |
-
|
| 128 |
-
- [ ] `unity_catalog_service.py` - UC operations
|
| 129 |
-
- [ ] `config.py` - Configuration management
|
| 130 |
-
- [ ] `requirements.txt` - Python dependencies
|
| 131 |
-
- [ ] `Dockerfile` - Container definition
|
| 132 |
-
- [ ] `README.md` - Space description (with metadata)
|
| 133 |
-
- [ ] `.gitignore` - Exclude `.env` and `__pycache__`
|
| 134 |
|
| 135 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
|
| 137 |
-
|
| 138 |
-
# Clone space
|
| 139 |
-
git clone https://huggingface.co/spaces/username/unitycatalog-chatbot
|
| 140 |
|
| 141 |
-
|
| 142 |
-
|
|
|
|
|
|
|
| 143 |
|
| 144 |
-
#
|
| 145 |
-
# Go to: https://huggingface.co/spaces/username/unitycatalog-chatbot/settings
|
| 146 |
|
| 147 |
-
|
| 148 |
-
|
| 149 |
-
|
|
|
|
|
|
|
| 150 |
|
| 151 |
---
|
| 152 |
|
| 153 |
-
**
|
|
|
|
|
|
| 1 |
+
# Deploy to Hugging Face Spaces
|
| 2 |
|
| 3 |
+
This guide explains how to deploy the Unity Catalog Chatbot to Hugging Face Spaces.
|
| 4 |
|
| 5 |
## Prerequisites
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
+
1. **Hugging Face Account** - Create one at https://huggingface.co
|
| 8 |
+
2. **Git** - Installed and configured
|
| 9 |
+
3. **Databricks Credentials** - API key and workspace URL
|
| 10 |
+
4. **Anthropic API Key** - For Claude AI
|
| 11 |
|
| 12 |
+
## Step 1: Create a Hugging Face Space
|
| 13 |
+
|
| 14 |
+
1. Go to [Hugging Face Spaces](https://huggingface.co/spaces)
|
| 15 |
+
2. Click **"Create new Space"**
|
| 16 |
+
3. Fill in:
|
| 17 |
+
- **Space name**: `unity-catalog-chatbot` (or your choice)
|
| 18 |
+
- **License**: MIT (or your preference)
|
| 19 |
+
- **Space SDK**: Docker
|
| 20 |
+
- **Space hardware**: CPU (or GPU if needed)
|
| 21 |
+
4. Click **"Create Space"**
|
| 22 |
+
|
| 23 |
+
## Step 2: Clone the Space Repository
|
| 24 |
|
|
|
|
| 25 |
```bash
|
| 26 |
+
# Clone your newly created space
|
| 27 |
+
git clone https://huggingface.co/spaces/YOUR_USERNAME/unity-catalog-chatbot
|
| 28 |
+
cd unity-catalog-chatbot
|
| 29 |
```
|
| 30 |
|
| 31 |
+
## Step 3: Add Files
|
| 32 |
+
|
| 33 |
+
Copy these files from this repo to the cloned Space:
|
| 34 |
+
|
| 35 |
```bash
|
| 36 |
+
cp /path/to/UnityCatalog-ChatBot/{app.py,unity-catalog-chatbot.jsx,unity_catalog_service.py,index.html,requirements.txt,.env.example,config.py,conftest.py} .
|
| 37 |
+
cp /path/to/UnityCatalog-ChatBot/Dockerfile .
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 38 |
```
|
| 39 |
|
| 40 |
+
Or manually add the files to the Space repository.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 41 |
|
| 42 |
+
## Step 4: Configure Environment Variables
|
| 43 |
|
| 44 |
+
In the Space settings on Hugging Face:
|
| 45 |
|
| 46 |
+
1. Go to **Settings → Variables and Secrets**
|
| 47 |
+
2. Add these secrets (HF will mask them):
|
| 48 |
+
- `ANTHROPIC_API_KEY`: Your Anthropic API key
|
| 49 |
+
- `DATABRICKS_HOST`: Your Databricks workspace URL
|
| 50 |
+
- `DATABRICKS_TOKEN`: Your Databricks personal access token
|
| 51 |
|
| 52 |
+
## Step 5: Push to Hugging Face
|
|
|
|
| 53 |
|
|
|
|
| 54 |
```bash
|
| 55 |
+
# Add all files
|
| 56 |
git add .
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
+
# Commit
|
| 59 |
+
git commit -m "Initial deployment"
|
|
|
|
|
|
|
|
|
|
|
|
|
| 60 |
|
| 61 |
+
# Push to HF
|
| 62 |
+
git push
|
| 63 |
+
```
|
| 64 |
|
| 65 |
+
The Space will automatically build and deploy.
|
|
|
|
|
|
|
| 66 |
|
| 67 |
+
## Step 6: Access Your App
|
| 68 |
|
| 69 |
+
Once deployed, you can access it at:
|
| 70 |
+
```
|
| 71 |
+
https://huggingface.co/spaces/YOUR_USERNAME/unity-catalog-chatbot
|
| 72 |
+
```
|
| 73 |
|
| 74 |
+
## How It Works
|
|
|
|
| 75 |
|
| 76 |
+
- **Frontend**: React UI served from `index.html`
|
| 77 |
+
- **Backend**: Flask API at `/api/chat` and other endpoints
|
| 78 |
+
- **Port**: Automatically uses port 7860 (HF Spaces standard)
|
| 79 |
+
- **Static Files**: Served from the root directory
|
|
|
|
| 80 |
|
| 81 |
+
## Testing the Deployment
|
| 82 |
|
| 83 |
+
1. Open the Space URL
|
| 84 |
+
2. You'll see the setup screen asking for credentials
|
| 85 |
+
3. Enter your Databricks workspace details:
|
| 86 |
+
- Host: `https://your-workspace.cloud.databricks.com`
|
| 87 |
+
- Token: Your Databricks API token
|
| 88 |
+
- Workspace ID: (optional)
|
| 89 |
+
4. Click "✓ Connect"
|
| 90 |
+
5. Type: "Create a catalog named test_catalog"
|
| 91 |
+
6. You should see the response and SQL preview
|
| 92 |
|
| 93 |
+
## Troubleshooting
|
| 94 |
|
| 95 |
+
### App doesn't start
|
| 96 |
+
- Check logs in Space settings → Logs
|
| 97 |
+
- Verify all environment variables are set
|
| 98 |
+
- Ensure `requirements.txt` has all dependencies
|
| 99 |
|
| 100 |
+
### "Cannot reach /api/chat"
|
| 101 |
+
- The Flask app may not be running
|
| 102 |
+
- Check that port is correctly set to 7860
|
| 103 |
+
- Verify CORS is enabled
|
| 104 |
|
| 105 |
+
### Databricks connection fails
|
| 106 |
+
- Double-check credentials
|
| 107 |
+
- Ensure token hasn't expired
|
| 108 |
+
- Verify workspace has Unity Catalog enabled
|
| 109 |
|
| 110 |
+
### Frontend not loading
|
| 111 |
+
- Check browser console (F12) for errors
|
| 112 |
+
- Ensure `index.html` is in the root directory
|
| 113 |
+
- Clear browser cache and reload
|
| 114 |
|
| 115 |
+
## Security Notes
|
| 116 |
|
| 117 |
+
⚠️ **Important:**
|
| 118 |
+
- Never hardcode API keys in files
|
| 119 |
+
- Use Hugging Face Secrets for sensitive credentials
|
| 120 |
+
- Don't commit `.env` files with real credentials
|
| 121 |
+
- Token will be masked in HF logs
|
| 122 |
|
| 123 |
+
## File Structure
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
|
| 125 |
+
```
|
| 126 |
+
huggingface-space/
|
| 127 |
+
├── app.py ← Flask backend (serves frontend + API)
|
| 128 |
+
├── unity-catalog-chatbot.jsx ← React UI component
|
| 129 |
+
├── unity_catalog_service.py ← Databricks integration
|
| 130 |
+
├── index.html ← React entry point
|
| 131 |
+
├── requirements.txt ← Python dependencies
|
| 132 |
+
├── config.py ← Configuration
|
| 133 |
+
├── Dockerfile ← Docker container config
|
| 134 |
+
└── [other config files]
|
| 135 |
+
```
|
| 136 |
|
| 137 |
+
## Next Steps
|
|
|
|
|
|
|
| 138 |
|
| 139 |
+
1. **Monitor**: Check Space logs for any errors
|
| 140 |
+
2. **Test**: Try creating a catalog, granting permissions, etc.
|
| 141 |
+
3. **Customize**: Modify the UI colors, add more features
|
| 142 |
+
4. **Share**: Share the Space URL with your team
|
| 143 |
|
| 144 |
+
## Support
|
|
|
|
| 145 |
|
| 146 |
+
For issues:
|
| 147 |
+
1. Check the Space logs
|
| 148 |
+
2. Review the troubleshooting section
|
| 149 |
+
3. Check Hugging Face documentation: https://huggingface.co/docs/hub/spaces
|
| 150 |
+
4. Review app logs locally first before deploying
|
| 151 |
|
| 152 |
---
|
| 153 |
|
| 154 |
+
**Deployment Date**: December 2025
|
| 155 |
+
**Status**: Production Ready
|
HF_DEPLOYMENT_SUMMARY.md
DELETED
|
@@ -1,362 +0,0 @@
|
|
| 1 |
-
# Hugging Face Spaces Deployment Summary
|
| 2 |
-
|
| 3 |
-
## What Was Created
|
| 4 |
-
|
| 5 |
-
### 📄 Files for HF Deployment
|
| 6 |
-
|
| 7 |
-
1. **[HF_DEPLOYMENT.md](HF_DEPLOYMENT.md)** - Quick 5-minute setup guide
|
| 8 |
-
2. **[HF_README.md](HF_README.md)** - Hugging Face Space metadata & documentation
|
| 9 |
-
3. **[DEPLOYMENT_GUIDE.md](DEPLOYMENT_GUIDE.md)** - Updated with HF Spaces section
|
| 10 |
-
|
| 11 |
-
## Step-by-Step Deployment
|
| 12 |
-
|
| 13 |
-
### Phase 1: Create Space (2 minutes)
|
| 14 |
-
```bash
|
| 15 |
-
# 1. Visit https://huggingface.co/new
|
| 16 |
-
# 2. Fill in:
|
| 17 |
-
# - Name: unitycatalog-chatbot
|
| 18 |
-
# - Type: Space
|
| 19 |
-
# - Runtime: Docker
|
| 20 |
-
# 3. Click "Create Space"
|
| 21 |
-
```
|
| 22 |
-
|
| 23 |
-
### Phase 2: Setup Files (3 minutes)
|
| 24 |
-
```bash
|
| 25 |
-
# 1. Clone the space
|
| 26 |
-
git clone https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot
|
| 27 |
-
cd unitycatalog-chatbot
|
| 28 |
-
|
| 29 |
-
# 2. Copy project files
|
| 30 |
-
cp -r ../UnityCatalog-ChatBot/* .
|
| 31 |
-
|
| 32 |
-
# 3. Verify files:
|
| 33 |
-
# ✅ app.py
|
| 34 |
-
# ✅ unity_catalog_service.py
|
| 35 |
-
# ✅ config.py
|
| 36 |
-
# ✅ requirements.txt
|
| 37 |
-
# ✅ Dockerfile
|
| 38 |
-
# ✅ README.md (use HF_README.md content)
|
| 39 |
-
```
|
| 40 |
-
|
| 41 |
-
### Phase 3: Add Secrets (2 minutes)
|
| 42 |
-
```
|
| 43 |
-
Go to Space Settings → Secrets → Add three:
|
| 44 |
-
|
| 45 |
-
1. DATABRICKS_HOST
|
| 46 |
-
Value: https://your-workspace.databricks.com
|
| 47 |
-
|
| 48 |
-
2. DATABRICKS_TOKEN
|
| 49 |
-
Value: dapi... (your personal access token)
|
| 50 |
-
|
| 51 |
-
3. ANTHROPIC_API_KEY
|
| 52 |
-
Value: sk-ant-... (your API key)
|
| 53 |
-
```
|
| 54 |
-
|
| 55 |
-
### Phase 4: Deploy (3 minutes)
|
| 56 |
-
```bash
|
| 57 |
-
# 1. Push to HF
|
| 58 |
-
git add .
|
| 59 |
-
git commit -m "Initial deployment"
|
| 60 |
-
git push
|
| 61 |
-
|
| 62 |
-
# 2. Space auto-builds and deploys
|
| 63 |
-
# 3. Check build logs in Settings
|
| 64 |
-
# 4. App goes live! 🚀
|
| 65 |
-
```
|
| 66 |
-
|
| 67 |
-
## Total Time: ~10 minutes ⏱️
|
| 68 |
-
|
| 69 |
-
---
|
| 70 |
-
|
| 71 |
-
## Access Your App
|
| 72 |
-
|
| 73 |
-
**Live URL:** `https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot`
|
| 74 |
-
|
| 75 |
-
**API Base:** `https://YOUR_USERNAME-unitycatalog-chatbot.hf.space`
|
| 76 |
-
|
| 77 |
-
### Test Endpoints
|
| 78 |
-
|
| 79 |
-
```bash
|
| 80 |
-
# Health
|
| 81 |
-
curl https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/health
|
| 82 |
-
|
| 83 |
-
# Catalogs
|
| 84 |
-
curl https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/catalogs
|
| 85 |
-
|
| 86 |
-
# Chat
|
| 87 |
-
curl -X POST https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/chat \
|
| 88 |
-
-H "Content-Type: application/json" \
|
| 89 |
-
-d '{"message": "List all catalogs"}'
|
| 90 |
-
```
|
| 91 |
-
|
| 92 |
-
---
|
| 93 |
-
|
| 94 |
-
## Key Features on HF Spaces
|
| 95 |
-
|
| 96 |
-
| Feature | Details |
|
| 97 |
-
|---------|---------|
|
| 98 |
-
| **Hosting** | Free (or Pro for $50/mo) |
|
| 99 |
-
| **Runtime** | Docker container |
|
| 100 |
-
| **CPU** | 2 vCPU (free) / dedicated (pro) |
|
| 101 |
-
| **Uptime** | Best effort / 99.9% (pro) |
|
| 102 |
-
| **Scaling** | Auto-scales with traffic |
|
| 103 |
-
| **Secrets** | Environment variables (hidden) |
|
| 104 |
-
| **Custom Domain** | Pro tier only |
|
| 105 |
-
| **Logs** | Real-time in Settings |
|
| 106 |
-
|
| 107 |
-
---
|
| 108 |
-
|
| 109 |
-
## What Happens When You Push
|
| 110 |
-
|
| 111 |
-
```
|
| 112 |
-
git push
|
| 113 |
-
↓
|
| 114 |
-
HF receives commit
|
| 115 |
-
↓
|
| 116 |
-
Triggers build
|
| 117 |
-
↓
|
| 118 |
-
- Reads Dockerfile
|
| 119 |
-
- Installs dependencies from requirements.txt
|
| 120 |
-
- Builds Docker image (~2-3 min)
|
| 121 |
-
↓
|
| 122 |
-
Loads secrets from Space Settings
|
| 123 |
-
↓
|
| 124 |
-
Starts container on public URL
|
| 125 |
-
↓
|
| 126 |
-
App is live! 🎉
|
| 127 |
-
```
|
| 128 |
-
|
| 129 |
-
---
|
| 130 |
-
|
| 131 |
-
## Updates & Rollbacks
|
| 132 |
-
|
| 133 |
-
### Push Updates
|
| 134 |
-
```bash
|
| 135 |
-
# Make changes to code
|
| 136 |
-
# e.g., edit app.py
|
| 137 |
-
|
| 138 |
-
git add .
|
| 139 |
-
git commit -m "Fix bug in chat endpoint"
|
| 140 |
-
git push
|
| 141 |
-
# Space rebuilds automatically
|
| 142 |
-
```
|
| 143 |
-
|
| 144 |
-
### Revert Changes
|
| 145 |
-
```bash
|
| 146 |
-
# View git history
|
| 147 |
-
git log --oneline
|
| 148 |
-
|
| 149 |
-
# Revert to previous commit
|
| 150 |
-
git revert HEAD
|
| 151 |
-
git push
|
| 152 |
-
# Space rebuilds with old code
|
| 153 |
-
```
|
| 154 |
-
|
| 155 |
-
---
|
| 156 |
-
|
| 157 |
-
## Monitoring
|
| 158 |
-
|
| 159 |
-
### View Logs
|
| 160 |
-
1. Go to Space → **Settings**
|
| 161 |
-
2. **Build logs** - Shows Docker build output
|
| 162 |
-
3. **Runtime logs** - Shows running application output
|
| 163 |
-
4. Logs updated in real-time
|
| 164 |
-
|
| 165 |
-
### Common Errors
|
| 166 |
-
|
| 167 |
-
**"ModuleNotFoundError: No module named 'X'"**
|
| 168 |
-
- Add to `requirements.txt`
|
| 169 |
-
- Push changes
|
| 170 |
-
- Space rebuilds
|
| 171 |
-
|
| 172 |
-
**"Connection refused to Databricks"**
|
| 173 |
-
- Check `DATABRICKS_HOST` in Secrets
|
| 174 |
-
- Verify it includes `https://`
|
| 175 |
-
- Ensure token hasn't expired
|
| 176 |
-
|
| 177 |
-
**"Port already in use"**
|
| 178 |
-
- HF Spaces assigns port automatically
|
| 179 |
-
- Check your `app.py` respects `SERVER_PORT` env var
|
| 180 |
-
|
| 181 |
-
**"Build timeout"**
|
| 182 |
-
- If build takes >30 min, it's canceled
|
| 183 |
-
- Check requirements.txt for slow installs
|
| 184 |
-
- Try Docker layer caching
|
| 185 |
-
|
| 186 |
-
---
|
| 187 |
-
|
| 188 |
-
## File Reference
|
| 189 |
-
|
| 190 |
-
### What Gets Deployed
|
| 191 |
-
|
| 192 |
-
```
|
| 193 |
-
unitycatalog-chatbot/
|
| 194 |
-
├── app.py # Flask server
|
| 195 |
-
├── unity_catalog_service.py # Databricks operations
|
| 196 |
-
├── config.py # Configuration
|
| 197 |
-
├── conftest.py # Test fixtures
|
| 198 |
-
├── test_chatbot.py # Tests (optional)
|
| 199 |
-
├── requirements.txt # Python dependencies
|
| 200 |
-
├── Dockerfile # Container definition
|
| 201 |
-
├── README.md # HF Space description
|
| 202 |
-
├── .gitignore # Exclude secrets & cache
|
| 203 |
-
└── (other supporting files)
|
| 204 |
-
```
|
| 205 |
-
|
| 206 |
-
### Don't Include
|
| 207 |
-
- ❌ `.env` file (use Secrets instead)
|
| 208 |
-
- ❌ `__pycache__/`
|
| 209 |
-
- ❌ `.git` directory (create fresh)
|
| 210 |
-
- ❌ Virtual environments
|
| 211 |
-
- ❌ Build artifacts
|
| 212 |
-
|
| 213 |
-
---
|
| 214 |
-
|
| 215 |
-
## Scaling & Costs
|
| 216 |
-
|
| 217 |
-
### Free Tier
|
| 218 |
-
- 2 vCPU, 16GB RAM
|
| 219 |
-
- Shared infrastructure
|
| 220 |
-
- Best-effort uptime
|
| 221 |
-
- Rate limited (but sufficient for testing)
|
| 222 |
-
|
| 223 |
-
### Pro Tier
|
| 224 |
-
- Dedicated infrastructure
|
| 225 |
-
- 99.9% SLA
|
| 226 |
-
- Custom domain
|
| 227 |
-
- $50/month
|
| 228 |
-
|
| 229 |
-
### When to Upgrade
|
| 230 |
-
- ✅ Production workloads
|
| 231 |
-
- ✅ High traffic (100+ requests/min)
|
| 232 |
-
- ✅ Custom domain needed
|
| 233 |
-
- ✅ SLA requirements
|
| 234 |
-
|
| 235 |
-
---
|
| 236 |
-
|
| 237 |
-
## Sharing Your Space
|
| 238 |
-
|
| 239 |
-
### Public Link
|
| 240 |
-
Just share the Space URL:
|
| 241 |
-
```
|
| 242 |
-
https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot
|
| 243 |
-
```
|
| 244 |
-
|
| 245 |
-
### Embed in Website
|
| 246 |
-
```html
|
| 247 |
-
<iframe
|
| 248 |
-
src="https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot?embed=true"
|
| 249 |
-
frameborder="0"
|
| 250 |
-
width="800"
|
| 251 |
-
height="600"
|
| 252 |
-
></iframe>
|
| 253 |
-
```
|
| 254 |
-
|
| 255 |
-
### Add to Collections
|
| 256 |
-
- Create collection on HF Hub
|
| 257 |
-
- Add Space to it
|
| 258 |
-
- Share collection link
|
| 259 |
-
|
| 260 |
-
---
|
| 261 |
-
|
| 262 |
-
## Integration Examples
|
| 263 |
-
|
| 264 |
-
### Use as Backend API
|
| 265 |
-
|
| 266 |
-
```python
|
| 267 |
-
import requests
|
| 268 |
-
|
| 269 |
-
API_URL = "https://YOUR_USERNAME-unitycatalog-chatbot.hf.space"
|
| 270 |
-
|
| 271 |
-
# Chat
|
| 272 |
-
response = requests.post(
|
| 273 |
-
f"{API_URL}/api/chat",
|
| 274 |
-
json={"message": "Create a catalog named demo"}
|
| 275 |
-
)
|
| 276 |
-
print(response.json())
|
| 277 |
-
|
| 278 |
-
# List catalogs
|
| 279 |
-
catalogs = requests.get(f"{API_URL}/api/catalogs").json()
|
| 280 |
-
print(catalogs)
|
| 281 |
-
```
|
| 282 |
-
|
| 283 |
-
### Embed in Slack Bot
|
| 284 |
-
|
| 285 |
-
```python
|
| 286 |
-
from slack_bolt import App
|
| 287 |
-
|
| 288 |
-
app = App(token=os.environ["SLACK_BOT_TOKEN"])
|
| 289 |
-
|
| 290 |
-
@app.message("catalog")
|
| 291 |
-
def handle_catalog(message, say):
|
| 292 |
-
response = requests.post(
|
| 293 |
-
"https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/chat",
|
| 294 |
-
json={"message": message["text"]}
|
| 295 |
-
)
|
| 296 |
-
say(response.json()["message"])
|
| 297 |
-
|
| 298 |
-
app.start(port=int(os.environ.get("PORT", 3000)))
|
| 299 |
-
```
|
| 300 |
-
|
| 301 |
-
### Use in Streamlit App
|
| 302 |
-
|
| 303 |
-
```python
|
| 304 |
-
import streamlit as st
|
| 305 |
-
import requests
|
| 306 |
-
|
| 307 |
-
st.title("Unity Catalog Manager")
|
| 308 |
-
|
| 309 |
-
message = st.text_input("Ask me anything about your catalog...")
|
| 310 |
-
|
| 311 |
-
if st.button("Send"):
|
| 312 |
-
response = requests.post(
|
| 313 |
-
"https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/chat",
|
| 314 |
-
json={"message": message}
|
| 315 |
-
)
|
| 316 |
-
st.write(response.json())
|
| 317 |
-
```
|
| 318 |
-
|
| 319 |
-
---
|
| 320 |
-
|
| 321 |
-
## Troubleshooting Checklist
|
| 322 |
-
|
| 323 |
-
Before contacting support:
|
| 324 |
-
|
| 325 |
-
- [ ] Secrets are set in Space Settings
|
| 326 |
-
- [ ] Databricks token is valid (not expired)
|
| 327 |
-
- [ ] Anthropic API key is correct
|
| 328 |
-
- [ ] `Dockerfile` exists in repo root
|
| 329 |
-
- [ ] `requirements.txt` is valid Python
|
| 330 |
-
- [ ] Build logs show no errors
|
| 331 |
-
- [ ] Runtime logs show app started
|
| 332 |
-
- [ ] Test `/api/health` endpoint first
|
| 333 |
-
- [ ] Try locally with same credentials
|
| 334 |
-
|
| 335 |
-
---
|
| 336 |
-
|
| 337 |
-
## Next Steps
|
| 338 |
-
|
| 339 |
-
1. ✅ Create Space on HF
|
| 340 |
-
2. ✅ Push code
|
| 341 |
-
3. ✅ Add secrets
|
| 342 |
-
4. ✅ Wait for build
|
| 343 |
-
5. ✅ Test endpoints
|
| 344 |
-
6. ✅ Share with team
|
| 345 |
-
7. ✅ Monitor logs
|
| 346 |
-
8. ✅ Iterate on features
|
| 347 |
-
|
| 348 |
-
---
|
| 349 |
-
|
| 350 |
-
## Resources
|
| 351 |
-
|
| 352 |
-
- **Hugging Face Docs:** https://huggingface.co/docs/hub/spaces
|
| 353 |
-
- **Docker Guide:** https://docs.docker.com/
|
| 354 |
-
- **Flask Docs:** https://flask.palletsprojects.com/
|
| 355 |
-
- **Databricks API:** https://docs.databricks.com/api/workspace
|
| 356 |
-
- **Claude API:** https://docs.anthropic.com/
|
| 357 |
-
|
| 358 |
-
---
|
| 359 |
-
|
| 360 |
-
**Your chatbot is now live on Hugging Face Spaces! 🚀**
|
| 361 |
-
|
| 362 |
-
Questions? Check the logs, review error messages, or visit HF community.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
HF_README.md
DELETED
|
@@ -1,243 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
title: Unity Catalog Chatbot
|
| 3 |
-
emoji: 💬
|
| 4 |
-
colorFrom: blue
|
| 5 |
-
colorTo: purple
|
| 6 |
-
sdk: docker
|
| 7 |
-
pinned: false
|
| 8 |
-
license: mit
|
| 9 |
-
---
|
| 10 |
-
|
| 11 |
-
# Unity Catalog Chatbot
|
| 12 |
-
|
| 13 |
-
A natural language chatbot for managing Databricks Unity Catalog powered by Claude AI.
|
| 14 |
-
|
| 15 |
-
## Features
|
| 16 |
-
|
| 17 |
-
✨ **Natural Language Interface**
|
| 18 |
-
- Ask questions in plain English
|
| 19 |
-
- Get instant responses from Claude AI
|
| 20 |
-
- No SQL knowledge required
|
| 21 |
-
|
| 22 |
-
🗂️ **Catalog Management**
|
| 23 |
-
- Create catalogs, schemas, and tables
|
| 24 |
-
- List objects across your workspace
|
| 25 |
-
- View table details and metadata
|
| 26 |
-
|
| 27 |
-
🔐 **Permission Management**
|
| 28 |
-
- Grant and revoke permissions
|
| 29 |
-
- Manage access control via chat
|
| 30 |
-
- Support for users and groups
|
| 31 |
-
|
| 32 |
-
🚀 **REST API**
|
| 33 |
-
- Full JSON API for integrations
|
| 34 |
-
- Health checks and monitoring
|
| 35 |
-
- Easy to embed in other apps
|
| 36 |
-
|
| 37 |
-
## Quick Start
|
| 38 |
-
|
| 39 |
-
### 1. Add Secrets
|
| 40 |
-
|
| 41 |
-
Go to **Settings → Secrets** and add:
|
| 42 |
-
|
| 43 |
-
```
|
| 44 |
-
DATABRICKS_HOST = https://your-workspace.databricks.com
|
| 45 |
-
DATABRICKS_TOKEN = dapi...your-token...
|
| 46 |
-
ANTHROPIC_API_KEY = sk-ant-...your-key...
|
| 47 |
-
```
|
| 48 |
-
|
| 49 |
-
### 2. Wait for Build
|
| 50 |
-
|
| 51 |
-
Space will auto-build (~2 min). Check **Settings → Build logs**.
|
| 52 |
-
|
| 53 |
-
### 3. Start Using
|
| 54 |
-
|
| 55 |
-
Once running, the app will be available at the Space URL.
|
| 56 |
-
|
| 57 |
-
### 4. API Endpoints
|
| 58 |
-
|
| 59 |
-
#### Health Check
|
| 60 |
-
```bash
|
| 61 |
-
GET /api/health
|
| 62 |
-
```
|
| 63 |
-
|
| 64 |
-
#### List Catalogs
|
| 65 |
-
```bash
|
| 66 |
-
GET /api/catalogs
|
| 67 |
-
```
|
| 68 |
-
|
| 69 |
-
#### List Schemas
|
| 70 |
-
```bash
|
| 71 |
-
GET /api/schemas/{catalog}
|
| 72 |
-
```
|
| 73 |
-
|
| 74 |
-
#### List Tables
|
| 75 |
-
```bash
|
| 76 |
-
GET /api/tables/{catalog}/{schema}
|
| 77 |
-
```
|
| 78 |
-
|
| 79 |
-
#### Chat (Main Endpoint)
|
| 80 |
-
```bash
|
| 81 |
-
POST /api/chat
|
| 82 |
-
Content-Type: application/json
|
| 83 |
-
|
| 84 |
-
{
|
| 85 |
-
"message": "Create a catalog named sales_data"
|
| 86 |
-
}
|
| 87 |
-
```
|
| 88 |
-
|
| 89 |
-
## Example Requests
|
| 90 |
-
|
| 91 |
-
### Create a Catalog
|
| 92 |
-
```bash
|
| 93 |
-
curl -X POST https://your-space-url/api/chat \
|
| 94 |
-
-H "Content-Type: application/json" \
|
| 95 |
-
-d '{"message": "Create a catalog named sales_data"}'
|
| 96 |
-
```
|
| 97 |
-
|
| 98 |
-
### Grant Permissions
|
| 99 |
-
```bash
|
| 100 |
-
curl -X POST https://your-space-url/api/chat \
|
| 101 |
-
-H "Content-Type: application/json" \
|
| 102 |
-
-d '{
|
| 103 |
-
"message": "Grant SELECT on sales_data.customers to data_analysts"
|
| 104 |
-
}'
|
| 105 |
-
```
|
| 106 |
-
|
| 107 |
-
### List Objects
|
| 108 |
-
```bash
|
| 109 |
-
curl https://your-space-url/api/catalogs
|
| 110 |
-
curl https://your-space-url/api/schemas/sales_data
|
| 111 |
-
curl https://your-space-url/api/tables/sales_data/analytics
|
| 112 |
-
```
|
| 113 |
-
|
| 114 |
-
## Supported Operations
|
| 115 |
-
|
| 116 |
-
- ✅ Create catalogs
|
| 117 |
-
- ✅ Create schemas
|
| 118 |
-
- ✅ Create tables
|
| 119 |
-
- ✅ Grant permissions
|
| 120 |
-
- ✅ Revoke permissions
|
| 121 |
-
- ✅ List catalogs/schemas/tables
|
| 122 |
-
- ✅ Show permissions
|
| 123 |
-
- ✅ Set object owner
|
| 124 |
-
- ✅ Get table details
|
| 125 |
-
- ✅ Execute SQL (when enabled)
|
| 126 |
-
|
| 127 |
-
## Requirements
|
| 128 |
-
|
| 129 |
-
- **Databricks** workspace with Unity Catalog enabled
|
| 130 |
-
- **Personal Access Token** (generate in Databricks)
|
| 131 |
-
- **Anthropic API Key** (get from https://console.anthropic.com)
|
| 132 |
-
|
| 133 |
-
## Architecture
|
| 134 |
-
|
| 135 |
-
```
|
| 136 |
-
┌─────────────────────┐
|
| 137 |
-
│ User / Client │
|
| 138 |
-
└──────────┬──────────┘
|
| 139 |
-
│
|
| 140 |
-
v
|
| 141 |
-
┌─────────────────────┐
|
| 142 |
-
│ Flask API Server │
|
| 143 |
-
│ (Port 5000) │
|
| 144 |
-
└──────────┬──────────┘
|
| 145 |
-
│
|
| 146 |
-
┌────┴──────┬────────────┐
|
| 147 |
-
v v v
|
| 148 |
-
Claude AI UC Service Config Manager
|
| 149 |
-
│ │ │
|
| 150 |
-
└────┬──────┴────────────┘
|
| 151 |
-
v
|
| 152 |
-
Databricks Unity Catalog
|
| 153 |
-
+ Anthropic API
|
| 154 |
-
```
|
| 155 |
-
|
| 156 |
-
## Local Development
|
| 157 |
-
|
| 158 |
-
```bash
|
| 159 |
-
# Clone repo
|
| 160 |
-
git clone <repo-url>
|
| 161 |
-
cd UnityCatalog-ChatBot
|
| 162 |
-
|
| 163 |
-
# Setup
|
| 164 |
-
python -m venv venv
|
| 165 |
-
source venv/bin/activate # or venv\Scripts\activate on Windows
|
| 166 |
-
pip install -r requirements.txt
|
| 167 |
-
|
| 168 |
-
# Configure
|
| 169 |
-
cp .env.example .env
|
| 170 |
-
# Edit .env with your credentials
|
| 171 |
-
|
| 172 |
-
# Run tests
|
| 173 |
-
pytest test_chatbot.py -v
|
| 174 |
-
|
| 175 |
-
# Run server
|
| 176 |
-
python app.py
|
| 177 |
-
```
|
| 178 |
-
|
| 179 |
-
## Docker
|
| 180 |
-
|
| 181 |
-
```bash
|
| 182 |
-
# Build
|
| 183 |
-
docker build -t unitycatalog-chatbot .
|
| 184 |
-
|
| 185 |
-
# Run
|
| 186 |
-
docker run -p 5000:5000 \
|
| 187 |
-
-e DATABRICKS_HOST="https://..." \
|
| 188 |
-
-e DATABRICKS_TOKEN="..." \
|
| 189 |
-
-e ANTHROPIC_API_KEY="..." \
|
| 190 |
-
unitycatalog-chatbot
|
| 191 |
-
```
|
| 192 |
-
|
| 193 |
-
## Troubleshooting
|
| 194 |
-
|
| 195 |
-
### Build Fails
|
| 196 |
-
- Check **Settings → Build logs**
|
| 197 |
-
- Ensure `Dockerfile` exists
|
| 198 |
-
- Verify `requirements.txt` syntax
|
| 199 |
-
|
| 200 |
-
### App Crashes
|
| 201 |
-
- Check **Settings → Runtime logs**
|
| 202 |
-
- Verify secrets are set correctly
|
| 203 |
-
- Test credentials locally first
|
| 204 |
-
|
| 205 |
-
### API Returns Error
|
| 206 |
-
- Confirm Databricks host URL is correct
|
| 207 |
-
- Check token hasn't expired
|
| 208 |
-
- Verify Anthropic API key is valid
|
| 209 |
-
|
| 210 |
-
### Slow Responses
|
| 211 |
-
- Databricks API latency
|
| 212 |
-
- Large catalog size (many objects)
|
| 213 |
-
- Network connectivity
|
| 214 |
-
|
| 215 |
-
## Security Notes
|
| 216 |
-
|
| 217 |
-
⚠️ **Never commit secrets to Git**
|
| 218 |
-
- Use Hugging Face Secrets feature
|
| 219 |
-
- Rotate tokens regularly
|
| 220 |
-
- Use IAM roles when possible
|
| 221 |
-
|
| 222 |
-
## Performance
|
| 223 |
-
|
| 224 |
-
- **Requests**: Up to 60/min (configurable)
|
| 225 |
-
- **Response time**: 2-5 seconds typical
|
| 226 |
-
- **Catalog size**: Tested with 1000+ objects
|
| 227 |
-
- **Concurrent users**: Limited by Space tier
|
| 228 |
-
|
| 229 |
-
## License
|
| 230 |
-
|
| 231 |
-
MIT
|
| 232 |
-
|
| 233 |
-
## Support
|
| 234 |
-
|
| 235 |
-
- GitHub Issues: [Link to repo]
|
| 236 |
-
- Documentation: See `/docs`
|
| 237 |
-
- Discord: [Link to community]
|
| 238 |
-
|
| 239 |
-
---
|
| 240 |
-
|
| 241 |
-
**Built with ❤️ using Flask, Claude, and Databricks**
|
| 242 |
-
|
| 243 |
-
*Hugging Face Spaces - Free hosting for ML apps*
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
QUICK_DEPLOY.md
DELETED
|
@@ -1,196 +0,0 @@
|
|
| 1 |
-
# ⚡ Quick Deploy to Hugging Face - 5 Minutes
|
| 2 |
-
|
| 3 |
-
## Your Deployment Package is Ready! 📦
|
| 4 |
-
|
| 5 |
-
Everything you need is prepared. Just follow these 7 steps:
|
| 6 |
-
|
| 7 |
-
---
|
| 8 |
-
|
| 9 |
-
## Step 1: Get Your Credentials (1 min)
|
| 10 |
-
|
| 11 |
-
### Databricks
|
| 12 |
-
1. Go to your Databricks workspace
|
| 13 |
-
2. Click your user icon → Settings
|
| 14 |
-
3. Go to **Personal Access Tokens** → **Generate new token**
|
| 15 |
-
4. Copy the token (format: `dapi...`)
|
| 16 |
-
5. Note your workspace URL (e.g., `https://your-workspace.databricks.com`)
|
| 17 |
-
|
| 18 |
-
### Anthropic
|
| 19 |
-
1. Visit https://console.anthropic.com
|
| 20 |
-
2. Click **API Keys** in sidebar
|
| 21 |
-
3. Click **Create Key**
|
| 22 |
-
4. Copy the key (format: `sk-ant-...`)
|
| 23 |
-
|
| 24 |
-
---
|
| 25 |
-
|
| 26 |
-
## Step 2: Create Hugging Face Space (2 min)
|
| 27 |
-
|
| 28 |
-
1. **Go to:** https://huggingface.co/new
|
| 29 |
-
2. **Fill in:**
|
| 30 |
-
- **Owner:** Your username
|
| 31 |
-
- **Repository name:** `unitycatalog-chatbot`
|
| 32 |
-
- **Type:** Space
|
| 33 |
-
- **Space SDK:** Docker
|
| 34 |
-
- **License:** MIT (or your choice)
|
| 35 |
-
3. **Click:** Create Space
|
| 36 |
-
|
| 37 |
-
**Result:** You'll get a Space at `https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot`
|
| 38 |
-
|
| 39 |
-
---
|
| 40 |
-
|
| 41 |
-
## Step 3: Run Local Tests (1 min)
|
| 42 |
-
|
| 43 |
-
```bash
|
| 44 |
-
cd UnityCatalog-ChatBot
|
| 45 |
-
|
| 46 |
-
# Run tests (no credentials needed - they're mocked)
|
| 47 |
-
python -m pytest test_chatbot.py -v
|
| 48 |
-
|
| 49 |
-
# Expected: 23 passed ✅
|
| 50 |
-
```
|
| 51 |
-
|
| 52 |
-
---
|
| 53 |
-
|
| 54 |
-
## Step 4: Push Code (1 min)
|
| 55 |
-
|
| 56 |
-
```bash
|
| 57 |
-
# Add HF remote
|
| 58 |
-
git remote add huggingface https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot
|
| 59 |
-
|
| 60 |
-
# Push code
|
| 61 |
-
git push -u huggingface main
|
| 62 |
-
|
| 63 |
-
# If main doesn't exist, try master:
|
| 64 |
-
# git push -u huggingface master
|
| 65 |
-
```
|
| 66 |
-
|
| 67 |
-
**HF will start building automatically** ⏳
|
| 68 |
-
|
| 69 |
-
---
|
| 70 |
-
|
| 71 |
-
## Step 5: Add Secrets (1 min)
|
| 72 |
-
|
| 73 |
-
1. **Go to:** Your Space Settings
|
| 74 |
-
- URL: `https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot/settings`
|
| 75 |
-
|
| 76 |
-
2. **Click:** Repository Secrets
|
| 77 |
-
|
| 78 |
-
3. **Add three secrets:**
|
| 79 |
-
|
| 80 |
-
| Name | Value |
|
| 81 |
-
|------|-------|
|
| 82 |
-
| `DATABRICKS_HOST` | `https://your-workspace.databricks.com` |
|
| 83 |
-
| `DATABRICKS_TOKEN` | `dapi...` |
|
| 84 |
-
| `ANTHROPIC_API_KEY` | `sk-ant-...` |
|
| 85 |
-
|
| 86 |
-
4. **Click:** Save after each
|
| 87 |
-
|
| 88 |
-
**Result:** Space rebuilds with secrets loaded ✅
|
| 89 |
-
|
| 90 |
-
---
|
| 91 |
-
|
| 92 |
-
## Step 6: Wait for Build (3-5 min)
|
| 93 |
-
|
| 94 |
-
- Go to Space Settings → Build logs
|
| 95 |
-
- Watch the Docker build happen
|
| 96 |
-
- Status changes to **"Running"** when ready
|
| 97 |
-
- Check Runtime logs for any errors
|
| 98 |
-
|
| 99 |
-
---
|
| 100 |
-
|
| 101 |
-
## Step 7: Test Your Deployment (1 min)
|
| 102 |
-
|
| 103 |
-
Once **"Running"**, test the API:
|
| 104 |
-
|
| 105 |
-
```bash
|
| 106 |
-
# Replace YOUR_USERNAME and check health
|
| 107 |
-
curl https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/health
|
| 108 |
-
|
| 109 |
-
# Response should be:
|
| 110 |
-
# {"status": "healthy", "service": "Unity Catalog Chatbot API"}
|
| 111 |
-
|
| 112 |
-
# List catalogs
|
| 113 |
-
curl https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/catalogs
|
| 114 |
-
|
| 115 |
-
# Chat
|
| 116 |
-
curl -X POST https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/chat \
|
| 117 |
-
-H "Content-Type: application/json" \
|
| 118 |
-
-d '{"message": "List all catalogs"}'
|
| 119 |
-
```
|
| 120 |
-
|
| 121 |
-
---
|
| 122 |
-
|
| 123 |
-
## 🎉 You're Done!
|
| 124 |
-
|
| 125 |
-
Your chatbot is now live at:
|
| 126 |
-
```
|
| 127 |
-
https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot
|
| 128 |
-
```
|
| 129 |
-
|
| 130 |
-
**Share it:** Just send the URL to anyone - they can use it immediately!
|
| 131 |
-
|
| 132 |
-
---
|
| 133 |
-
|
| 134 |
-
## Troubleshooting
|
| 135 |
-
|
| 136 |
-
| Problem | Solution |
|
| 137 |
-
|---------|----------|
|
| 138 |
-
| Build fails | Check build logs. Ensure `Dockerfile` exists. |
|
| 139 |
-
| App crashes | Check runtime logs. Verify secrets are set. |
|
| 140 |
-
| API returns error | Double-check credentials. Test locally first. |
|
| 141 |
-
| "Port already in use" | HF assigns port automatically, should work. |
|
| 142 |
-
|
| 143 |
-
---
|
| 144 |
-
|
| 145 |
-
## Common Commands
|
| 146 |
-
|
| 147 |
-
```bash
|
| 148 |
-
# View logs
|
| 149 |
-
# Go to: Space Settings → Build logs or Runtime logs
|
| 150 |
-
|
| 151 |
-
# Update code
|
| 152 |
-
git add .
|
| 153 |
-
git commit -m "Update"
|
| 154 |
-
git push huggingface main
|
| 155 |
-
|
| 156 |
-
# Revert to previous version
|
| 157 |
-
git log --oneline
|
| 158 |
-
git revert <commit-hash>
|
| 159 |
-
git push huggingface main
|
| 160 |
-
|
| 161 |
-
# Delete Space (if needed)
|
| 162 |
-
# Go to Space Settings → Delete
|
| 163 |
-
```
|
| 164 |
-
|
| 165 |
-
---
|
| 166 |
-
|
| 167 |
-
## Next Steps
|
| 168 |
-
|
| 169 |
-
- ✅ Share the Space URL with your team
|
| 170 |
-
- ✅ Monitor logs in Settings
|
| 171 |
-
- ✅ Iterate on features (push code updates)
|
| 172 |
-
- ✅ Upgrade to Pro tier for custom domain ($50/mo)
|
| 173 |
-
|
| 174 |
-
---
|
| 175 |
-
|
| 176 |
-
## Files You Have
|
| 177 |
-
|
| 178 |
-
```
|
| 179 |
-
UnityCatalog-ChatBot/
|
| 180 |
-
├── HF_DEPLOYMENT.md ← Detailed HF guide
|
| 181 |
-
├── HF_DEPLOYMENT_SUMMARY.md ← Complete reference
|
| 182 |
-
├── DEPLOYMENT_GUIDE.md ← All deployment options
|
| 183 |
-
├── deploy-to-huggingface.sh ← Automated script (Linux/Mac)
|
| 184 |
-
├── deploy-to-huggingface.bat ← Automated script (Windows)
|
| 185 |
-
├── .env.example ← Environment template
|
| 186 |
-
├── Dockerfile ← Ready for HF
|
| 187 |
-
├── app.py ← Your API
|
| 188 |
-
├── requirements.txt ← Dependencies
|
| 189 |
-
└── [other project files]
|
| 190 |
-
```
|
| 191 |
-
|
| 192 |
-
---
|
| 193 |
-
|
| 194 |
-
**That's it! You're deploying in 5 minutes.** ⚡
|
| 195 |
-
|
| 196 |
-
If you have questions, check the detailed guides in your repo!
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
README_DEPLOYMENT.md
DELETED
|
@@ -1,222 +0,0 @@
|
|
| 1 |
-
# 🚀 Your Deployment Package is Ready!
|
| 2 |
-
|
| 3 |
-
## What's Prepared for You
|
| 4 |
-
|
| 5 |
-
✅ **Complete Code** - Tested & validated (23/23 tests pass)
|
| 6 |
-
✅ **Docker Setup** - Production-ready Dockerfile
|
| 7 |
-
✅ **Documentation** - 4 deployment guides created
|
| 8 |
-
✅ **Scripts** - Automated deployment for Windows & Linux/Mac
|
| 9 |
-
✅ **Configuration** - .env template ready
|
| 10 |
-
✅ **Git Ready** - All files prepared for pushing
|
| 11 |
-
|
| 12 |
-
---
|
| 13 |
-
|
| 14 |
-
## Your Next Steps (Choose One)
|
| 15 |
-
|
| 16 |
-
### Option A: Use Automated Script 🤖
|
| 17 |
-
|
| 18 |
-
**Windows:**
|
| 19 |
-
```bash
|
| 20 |
-
cd UnityCatalog-ChatBot
|
| 21 |
-
deploy-to-huggingface.bat
|
| 22 |
-
# Follow the prompts
|
| 23 |
-
```
|
| 24 |
-
|
| 25 |
-
**Linux/Mac:**
|
| 26 |
-
```bash
|
| 27 |
-
cd UnityCatalog-ChatBot
|
| 28 |
-
chmod +x deploy-to-huggingface.sh
|
| 29 |
-
./deploy-to-huggingface.sh
|
| 30 |
-
# Follow the prompts
|
| 31 |
-
```
|
| 32 |
-
|
| 33 |
-
### Option B: Manual 5-Minute Deployment ⚡
|
| 34 |
-
|
| 35 |
-
1. Get credentials (Databricks token + Anthropic key)
|
| 36 |
-
2. Create Space: https://huggingface.co/new (Docker type)
|
| 37 |
-
3. Run tests locally: `pytest test_chatbot.py -v`
|
| 38 |
-
4. Push code: `git push -u huggingface main`
|
| 39 |
-
5. Add secrets in Space Settings
|
| 40 |
-
6. Wait 3-5 min for build
|
| 41 |
-
7. Test API at `https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/health`
|
| 42 |
-
|
| 43 |
-
See [QUICK_DEPLOY.md](QUICK_DEPLOY.md) for full details.
|
| 44 |
-
|
| 45 |
-
---
|
| 46 |
-
|
| 47 |
-
## Files You Have
|
| 48 |
-
|
| 49 |
-
### Documentation
|
| 50 |
-
- **QUICK_DEPLOY.md** - This (5-min quick start)
|
| 51 |
-
- **HF_DEPLOYMENT.md** - Step-by-step Hugging Face guide
|
| 52 |
-
- **HF_DEPLOYMENT_SUMMARY.md** - Complete reference & troubleshooting
|
| 53 |
-
- **HF_README.md** - Hugging Face Space description
|
| 54 |
-
- **DEPLOYMENT_GUIDE.md** - All deployment options (ECS, K8s, ACI, etc.)
|
| 55 |
-
|
| 56 |
-
### Configuration
|
| 57 |
-
- **.env.example** - Environment template (copy to .env)
|
| 58 |
-
- **Dockerfile** - Production-ready container definition
|
| 59 |
-
- **deploy-to-huggingface.sh** - Automated setup script (Linux/Mac)
|
| 60 |
-
- **deploy-to-huggingface.bat** - Automated setup script (Windows)
|
| 61 |
-
|
| 62 |
-
### Application
|
| 63 |
-
- **app.py** - Flask API server (lazy-init ready)
|
| 64 |
-
- **unity_catalog_service.py** - Databricks operations
|
| 65 |
-
- **config.py** - Configuration management
|
| 66 |
-
- **conftest.py** - Test fixtures with mocks
|
| 67 |
-
- **test_chatbot.py** - 23 tests (all passing)
|
| 68 |
-
- **requirements.txt** - Python dependencies
|
| 69 |
-
- **README.md** - Project overview
|
| 70 |
-
|
| 71 |
-
---
|
| 72 |
-
|
| 73 |
-
## What Happens When You Deploy
|
| 74 |
-
|
| 75 |
-
```
|
| 76 |
-
You push code to HF Space
|
| 77 |
-
↓
|
| 78 |
-
HF downloads Dockerfile & requirements.txt
|
| 79 |
-
↓
|
| 80 |
-
Docker builds image (~2-3 min)
|
| 81 |
-
↓
|
| 82 |
-
Container starts with your secrets loaded
|
| 83 |
-
↓
|
| 84 |
-
Your API is live! 🎉
|
| 85 |
-
↓
|
| 86 |
-
Share the URL with your team
|
| 87 |
-
↓
|
| 88 |
-
Anyone can use your chatbot
|
| 89 |
-
```
|
| 90 |
-
|
| 91 |
-
---
|
| 92 |
-
|
| 93 |
-
## Your Chatbot Will Have
|
| 94 |
-
|
| 95 |
-
✅ **Public URL** - Share with anyone
|
| 96 |
-
✅ **REST API** - `/api/chat`, `/api/catalogs`, etc.
|
| 97 |
-
✅ **Auto-scaling** - Handles traffic spikes
|
| 98 |
-
✅ **Real-time Logs** - See what's happening
|
| 99 |
-
✅ **Easy Updates** - Just `git push` to redeploy
|
| 100 |
-
✅ **Mock Tests** - All passing (no live API calls needed)
|
| 101 |
-
✅ **Security** - Secrets hidden, HTTPS enabled
|
| 102 |
-
|
| 103 |
-
---
|
| 104 |
-
|
| 105 |
-
## Credentials You'll Need
|
| 106 |
-
|
| 107 |
-
### Databricks
|
| 108 |
-
- **Workspace URL:** `https://your-workspace.databricks.com`
|
| 109 |
-
- **Personal Access Token:** `dapi...` (from Settings → Tokens)
|
| 110 |
-
|
| 111 |
-
### Anthropic
|
| 112 |
-
- **API Key:** `sk-ant-...` (from https://console.anthropic.com)
|
| 113 |
-
|
| 114 |
-
---
|
| 115 |
-
|
| 116 |
-
## Deployment Checklist
|
| 117 |
-
|
| 118 |
-
Before you start:
|
| 119 |
-
- [ ] Have Databricks workspace URL
|
| 120 |
-
- [ ] Have Databricks personal access token
|
| 121 |
-
- [ ] Have Anthropic API key
|
| 122 |
-
- [ ] Have Hugging Face account (free at huggingface.co)
|
| 123 |
-
- [ ] Have git installed (`git --version`)
|
| 124 |
-
- [ ] Have Python 3.9+ (`python --version`)
|
| 125 |
-
|
| 126 |
-
Before pushing:
|
| 127 |
-
- [ ] Tests pass locally (`pytest test_chatbot.py -v`)
|
| 128 |
-
- [ ] .env.example is present
|
| 129 |
-
- [ ] Dockerfile exists and is valid
|
| 130 |
-
- [ ] requirements.txt has all dependencies
|
| 131 |
-
|
| 132 |
-
---
|
| 133 |
-
|
| 134 |
-
## After Deployment
|
| 135 |
-
|
| 136 |
-
1. **Test endpoints:**
|
| 137 |
-
```bash
|
| 138 |
-
curl https://YOUR_USERNAME-unitycatalog-chatbot.hf.space/api/health
|
| 139 |
-
```
|
| 140 |
-
|
| 141 |
-
2. **Monitor logs:**
|
| 142 |
-
- Go to Space Settings → Runtime logs
|
| 143 |
-
|
| 144 |
-
3. **Share with team:**
|
| 145 |
-
- Just send the URL: `https://huggingface.co/spaces/YOUR_USERNAME/unitycatalog-chatbot`
|
| 146 |
-
|
| 147 |
-
4. **Make updates:**
|
| 148 |
-
```bash
|
| 149 |
-
# Edit code locally
|
| 150 |
-
git add .
|
| 151 |
-
git commit -m "Update feature"
|
| 152 |
-
git push huggingface main
|
| 153 |
-
# Space rebuilds automatically!
|
| 154 |
-
```
|
| 155 |
-
|
| 156 |
-
---
|
| 157 |
-
|
| 158 |
-
## Key Resources
|
| 159 |
-
|
| 160 |
-
- **Hugging Face Docs:** https://huggingface.co/docs/hub/spaces
|
| 161 |
-
- **Docker Docs:** https://docs.docker.com/
|
| 162 |
-
- **Databricks API:** https://docs.databricks.com/api/workspace
|
| 163 |
-
- **Claude API:** https://docs.anthropic.com/
|
| 164 |
-
|
| 165 |
-
---
|
| 166 |
-
|
| 167 |
-
## Support
|
| 168 |
-
|
| 169 |
-
**If deployment fails:**
|
| 170 |
-
1. Check Space runtime logs
|
| 171 |
-
2. Review error messages
|
| 172 |
-
3. Verify secrets are set correctly
|
| 173 |
-
4. Test locally with same credentials
|
| 174 |
-
5. Check [HF_DEPLOYMENT_SUMMARY.md](HF_DEPLOYMENT_SUMMARY.md) troubleshooting
|
| 175 |
-
|
| 176 |
-
**For questions:**
|
| 177 |
-
- Hugging Face Community: https://huggingface.co/spaces
|
| 178 |
-
- Databricks Docs: https://docs.databricks.com
|
| 179 |
-
- Anthropic Support: https://support.anthropic.com
|
| 180 |
-
|
| 181 |
-
---
|
| 182 |
-
|
| 183 |
-
## What You Get
|
| 184 |
-
|
| 185 |
-
✅ Production-ready chatbot
|
| 186 |
-
✅ Public API endpoint
|
| 187 |
-
✅ Free hosting (up to 2 CPU)
|
| 188 |
-
✅ Auto-scaling on traffic
|
| 189 |
-
✅ Real-time logs
|
| 190 |
-
✅ One-click updates
|
| 191 |
-
✅ Shareable link for your team
|
| 192 |
-
|
| 193 |
-
---
|
| 194 |
-
|
| 195 |
-
## Timeline
|
| 196 |
-
|
| 197 |
-
- **Setup:** 2 min (gather credentials)
|
| 198 |
-
- **Create Space:** 1 min (fill form on HF)
|
| 199 |
-
- **Test locally:** 1 min (run tests)
|
| 200 |
-
- **Push code:** 1 min (git push)
|
| 201 |
-
- **Build:** 3-5 min (Docker builds)
|
| 202 |
-
- **Add secrets:** 1 min (set in HF UI)
|
| 203 |
-
- **Ready!** → Live chatbot at your Space URL
|
| 204 |
-
|
| 205 |
-
**Total: ~15 minutes (mostly waiting for Docker build)**
|
| 206 |
-
|
| 207 |
-
---
|
| 208 |
-
|
| 209 |
-
## Next Action
|
| 210 |
-
|
| 211 |
-
👉 **Choose your path:**
|
| 212 |
-
1. Run automated script: `deploy-to-huggingface.bat` (Windows) or `./deploy-to-huggingface.sh` (Linux/Mac)
|
| 213 |
-
2. Follow [QUICK_DEPLOY.md](QUICK_DEPLOY.md) for manual steps
|
| 214 |
-
3. Read [HF_DEPLOYMENT.md](HF_DEPLOYMENT.md) for detailed guide
|
| 215 |
-
|
| 216 |
-
---
|
| 217 |
-
|
| 218 |
-
**Everything is ready. Your deployment is just a few git commands away!** 🚀
|
| 219 |
-
|
| 220 |
-
Need help? Check the docs in your repo. Questions? Review the troubleshooting section.
|
| 221 |
-
|
| 222 |
-
Good luck! 🎉
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
app.py
CHANGED
|
@@ -3,7 +3,7 @@ Unity Catalog Chatbot API Server
|
|
| 3 |
Flask API to handle natural language requests and execute Unity Catalog operations
|
| 4 |
"""
|
| 5 |
|
| 6 |
-
from flask import Flask, request, jsonify
|
| 7 |
from flask_cors import CORS
|
| 8 |
import os
|
| 9 |
import re
|
|
@@ -11,7 +11,7 @@ from typing import Dict, List, Optional
|
|
| 11 |
import anthropic
|
| 12 |
from unity_catalog_service import UnityCatalogService
|
| 13 |
|
| 14 |
-
app = Flask(__name__)
|
| 15 |
CORS(app)
|
| 16 |
|
| 17 |
# Initialize services (lazy to allow mocking in tests)
|
|
@@ -27,6 +27,33 @@ def _init_services():
|
|
| 27 |
claude_client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
|
| 28 |
return uc_service, claude_client
|
| 29 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 30 |
# System prompt for Claude to parse Unity Catalog requests
|
| 31 |
SYSTEM_PROMPT = """You are an expert Unity Catalog assistant. Your role is to:
|
| 32 |
|
|
@@ -283,6 +310,20 @@ Just describe what you want to do in natural language!""",
|
|
| 283 |
}
|
| 284 |
|
| 285 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 286 |
@app.route('/api/chat', methods=['POST'])
|
| 287 |
def chat():
|
| 288 |
"""Main chat endpoint"""
|
|
@@ -366,10 +407,50 @@ def execute_sql():
|
|
| 366 |
}), 500
|
| 367 |
|
| 368 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 369 |
if __name__ == '__main__':
|
|
|
|
|
|
|
|
|
|
|
|
|
| 370 |
# Development server
|
| 371 |
app.run(
|
| 372 |
-
host=
|
| 373 |
-
port=
|
| 374 |
-
debug=
|
| 375 |
)
|
|
|
|
| 3 |
Flask API to handle natural language requests and execute Unity Catalog operations
|
| 4 |
"""
|
| 5 |
|
| 6 |
+
from flask import Flask, request, jsonify, send_from_directory
|
| 7 |
from flask_cors import CORS
|
| 8 |
import os
|
| 9 |
import re
|
|
|
|
| 11 |
import anthropic
|
| 12 |
from unity_catalog_service import UnityCatalogService
|
| 13 |
|
| 14 |
+
app = Flask(__name__, static_folder='.', static_url_path='')
|
| 15 |
CORS(app)
|
| 16 |
|
| 17 |
# Initialize services (lazy to allow mocking in tests)
|
|
|
|
| 27 |
claude_client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
|
| 28 |
return uc_service, claude_client
|
| 29 |
|
| 30 |
+
|
| 31 |
+
def validate_databricks_connection(host: str, token: str, workspace_id: str = None) -> Dict:
|
| 32 |
+
"""Validate connection to Databricks workspace."""
|
| 33 |
+
try:
|
| 34 |
+
from databricks.sdk import WorkspaceClient
|
| 35 |
+
|
| 36 |
+
# Create client with provided credentials
|
| 37 |
+
client = WorkspaceClient(
|
| 38 |
+
host=host,
|
| 39 |
+
token=token
|
| 40 |
+
)
|
| 41 |
+
|
| 42 |
+
# Try to get workspace info
|
| 43 |
+
workspace_info = client.workspace.get_status(path="/")
|
| 44 |
+
|
| 45 |
+
return {
|
| 46 |
+
"success": True,
|
| 47 |
+
"message": "Successfully connected to Databricks workspace",
|
| 48 |
+
"workspace_path": workspace_info.path
|
| 49 |
+
}
|
| 50 |
+
except Exception as e:
|
| 51 |
+
return {
|
| 52 |
+
"success": False,
|
| 53 |
+
"message": f"Connection failed: {str(e)}"
|
| 54 |
+
}
|
| 55 |
+
|
| 56 |
+
|
| 57 |
# System prompt for Claude to parse Unity Catalog requests
|
| 58 |
SYSTEM_PROMPT = """You are an expert Unity Catalog assistant. Your role is to:
|
| 59 |
|
|
|
|
| 310 |
}
|
| 311 |
|
| 312 |
|
| 313 |
+
@app.route('/', methods=['GET'])
|
| 314 |
+
def index():
|
| 315 |
+
"""Serve the React UI"""
|
| 316 |
+
return send_from_directory('.', 'index.html')
|
| 317 |
+
|
| 318 |
+
|
| 319 |
+
@app.route('/<path:path>', methods=['GET'])
|
| 320 |
+
def serve_static(path):
|
| 321 |
+
"""Serve static files"""
|
| 322 |
+
if path and os.path.exists(path):
|
| 323 |
+
return send_from_directory('.', path)
|
| 324 |
+
return send_from_directory('.', 'index.html')
|
| 325 |
+
|
| 326 |
+
|
| 327 |
@app.route('/api/chat', methods=['POST'])
|
| 328 |
def chat():
|
| 329 |
"""Main chat endpoint"""
|
|
|
|
| 407 |
}), 500
|
| 408 |
|
| 409 |
|
| 410 |
+
@app.route('/api/validate-connection', methods=['POST'])
|
| 411 |
+
def validate_connection():
|
| 412 |
+
"""Validate Databricks connection with provided credentials"""
|
| 413 |
+
try:
|
| 414 |
+
data = request.json
|
| 415 |
+
host = data.get('host', '').strip()
|
| 416 |
+
token = data.get('token', '').strip()
|
| 417 |
+
workspace_id = data.get('workspaceId', '').strip()
|
| 418 |
+
|
| 419 |
+
if not host or not token:
|
| 420 |
+
return jsonify({
|
| 421 |
+
'success': False,
|
| 422 |
+
'message': 'Host and token are required'
|
| 423 |
+
}), 400
|
| 424 |
+
|
| 425 |
+
# Ensure host starts with https://
|
| 426 |
+
if not host.startswith('https://'):
|
| 427 |
+
if host.startswith('http://'):
|
| 428 |
+
host = 'https://' + host[7:]
|
| 429 |
+
else:
|
| 430 |
+
host = 'https://' + host
|
| 431 |
+
|
| 432 |
+
result = validate_databricks_connection(host, token, workspace_id)
|
| 433 |
+
|
| 434 |
+
if result['success']:
|
| 435 |
+
return jsonify(result), 200
|
| 436 |
+
else:
|
| 437 |
+
return jsonify(result), 401
|
| 438 |
+
|
| 439 |
+
except Exception as e:
|
| 440 |
+
return jsonify({
|
| 441 |
+
'success': False,
|
| 442 |
+
'message': f'Validation error: {str(e)}'
|
| 443 |
+
}), 500
|
| 444 |
+
|
| 445 |
+
|
| 446 |
if __name__ == '__main__':
|
| 447 |
+
# Get port from environment variable (HF Spaces uses 7860)
|
| 448 |
+
port = int(os.getenv('PORT', 7860))
|
| 449 |
+
host = os.getenv('HOST', '0.0.0.0')
|
| 450 |
+
|
| 451 |
# Development server
|
| 452 |
app.run(
|
| 453 |
+
host=host,
|
| 454 |
+
port=port,
|
| 455 |
+
debug=os.getenv('FLASK_ENV') == 'development'
|
| 456 |
)
|
deploy-to-huggingface.bat
DELETED
|
@@ -1,97 +0,0 @@
|
|
| 1 |
-
@echo off
|
| 2 |
-
REM deploy-to-huggingface.bat
|
| 3 |
-
REM Windows deployment to Hugging Face Spaces
|
| 4 |
-
|
| 5 |
-
setlocal enabledelayedexpansion
|
| 6 |
-
|
| 7 |
-
echo.
|
| 8 |
-
echo ========================================
|
| 9 |
-
echo ^(^) UnityCatalog-ChatBot HF Deployment
|
| 10 |
-
echo ========================================
|
| 11 |
-
echo.
|
| 12 |
-
|
| 13 |
-
REM Step 1: Get HF username
|
| 14 |
-
set /p HF_USERNAME="Enter your Hugging Face username: "
|
| 15 |
-
set SPACE_NAME=unitycatalog-chatbot
|
| 16 |
-
set SPACE_URL=https://huggingface.co/spaces/%HF_USERNAME%/%SPACE_NAME%
|
| 17 |
-
|
| 18 |
-
echo.
|
| 19 |
-
echo Space URL: %SPACE_URL%
|
| 20 |
-
echo.
|
| 21 |
-
|
| 22 |
-
REM Step 2: Verify files
|
| 23 |
-
echo Verifying required files...
|
| 24 |
-
for %%F in (app.py unity_catalog_service.py config.py requirements.txt Dockerfile .env.example) do (
|
| 25 |
-
if not exist "%%F" (
|
| 26 |
-
echo Error: Missing %%F
|
| 27 |
-
exit /b 1
|
| 28 |
-
)
|
| 29 |
-
)
|
| 30 |
-
echo OK - All files present
|
| 31 |
-
echo.
|
| 32 |
-
|
| 33 |
-
REM Step 3: Test locally
|
| 34 |
-
echo Running tests...
|
| 35 |
-
python -m pytest test_chatbot.py -v --tb=short -q
|
| 36 |
-
if errorlevel 1 (
|
| 37 |
-
echo Tests failed!
|
| 38 |
-
exit /b 1
|
| 39 |
-
)
|
| 40 |
-
echo OK - Tests passed
|
| 41 |
-
echo.
|
| 42 |
-
|
| 43 |
-
REM Step 4: .env setup
|
| 44 |
-
if not exist ".env" (
|
| 45 |
-
copy .env.example .env
|
| 46 |
-
echo Created .env - please edit with your credentials
|
| 47 |
-
pause
|
| 48 |
-
)
|
| 49 |
-
echo.
|
| 50 |
-
|
| 51 |
-
REM Step 5: Git setup
|
| 52 |
-
if not exist ".git" (
|
| 53 |
-
echo Initializing git...
|
| 54 |
-
git init
|
| 55 |
-
git add .
|
| 56 |
-
git commit -m "Initial commit: UnityCatalog-ChatBot"
|
| 57 |
-
) else (
|
| 58 |
-
echo Git already initialized
|
| 59 |
-
git add .
|
| 60 |
-
git diff --cached --quiet
|
| 61 |
-
if errorlevel 0 (
|
| 62 |
-
git commit -m "Update: deployment preparation"
|
| 63 |
-
)
|
| 64 |
-
)
|
| 65 |
-
echo.
|
| 66 |
-
|
| 67 |
-
REM Step 6: Instructions
|
| 68 |
-
echo.
|
| 69 |
-
echo ========================================
|
| 70 |
-
echo Manual Steps Required:
|
| 71 |
-
echo ========================================
|
| 72 |
-
echo.
|
| 73 |
-
echo 1. Create Space on Hugging Face:
|
| 74 |
-
echo https://huggingface.co/new
|
| 75 |
-
echo - Repository name: %SPACE_NAME%
|
| 76 |
-
echo - Type: Space
|
| 77 |
-
echo - SDK: Docker
|
| 78 |
-
echo.
|
| 79 |
-
echo 2. Once created, run this command:
|
| 80 |
-
echo git remote add huggingface %SPACE_URL%
|
| 81 |
-
echo git push -u huggingface main
|
| 82 |
-
echo.
|
| 83 |
-
echo 3. Add Secrets in Space Settings:
|
| 84 |
-
echo - DATABRICKS_HOST
|
| 85 |
-
echo - DATABRICKS_TOKEN
|
| 86 |
-
echo - ANTHROPIC_API_KEY
|
| 87 |
-
echo.
|
| 88 |
-
echo 4. Space rebuilds automatically (2-5 min)
|
| 89 |
-
echo.
|
| 90 |
-
echo 5. Test at: https://%HF_USERNAME%-%SPACE_NAME%.hf.space/api/health
|
| 91 |
-
echo.
|
| 92 |
-
pause
|
| 93 |
-
|
| 94 |
-
echo ========================================
|
| 95 |
-
echo Deployment Complete!
|
| 96 |
-
echo ========================================
|
| 97 |
-
echo Your chatbot: %SPACE_URL%
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
deploy-to-huggingface.sh
DELETED
|
@@ -1,181 +0,0 @@
|
|
| 1 |
-
#!/bin/bash
|
| 2 |
-
# deploy-to-huggingface.sh
|
| 3 |
-
# One-command deployment to Hugging Face Spaces
|
| 4 |
-
|
| 5 |
-
set -e
|
| 6 |
-
|
| 7 |
-
echo "🚀 UnityCatalog-ChatBot Hugging Face Deployment"
|
| 8 |
-
echo "==============================================="
|
| 9 |
-
echo ""
|
| 10 |
-
|
| 11 |
-
# Step 1: Gather user input
|
| 12 |
-
echo "Step 1: Enter your Hugging Face username"
|
| 13 |
-
read -p "Your HF username: " HF_USERNAME
|
| 14 |
-
|
| 15 |
-
SPACE_NAME="unitycatalog-chatbot"
|
| 16 |
-
SPACE_URL="https://huggingface.co/spaces/${HF_USERNAME}/${SPACE_NAME}"
|
| 17 |
-
|
| 18 |
-
echo "✓ Space URL will be: $SPACE_URL"
|
| 19 |
-
echo ""
|
| 20 |
-
|
| 21 |
-
# Step 2: Verify files
|
| 22 |
-
echo "Step 2: Verifying required files..."
|
| 23 |
-
REQUIRED_FILES=("app.py" "unity_catalog_service.py" "config.py" "requirements.txt" "Dockerfile" ".env.example")
|
| 24 |
-
|
| 25 |
-
for file in "${REQUIRED_FILES[@]}"; do
|
| 26 |
-
if [ ! -f "$file" ]; then
|
| 27 |
-
echo "❌ Missing: $file"
|
| 28 |
-
exit 1
|
| 29 |
-
fi
|
| 30 |
-
done
|
| 31 |
-
echo "✓ All required files present"
|
| 32 |
-
echo ""
|
| 33 |
-
|
| 34 |
-
# Step 3: Create .env for local testing
|
| 35 |
-
echo "Step 3: Creating .env from template..."
|
| 36 |
-
if [ ! -f ".env" ]; then
|
| 37 |
-
cp .env.example .env
|
| 38 |
-
echo "✓ Created .env (edit with your credentials before continuing)"
|
| 39 |
-
echo ""
|
| 40 |
-
echo "⚠️ IMPORTANT: Edit .env with your Databricks and Anthropic credentials"
|
| 41 |
-
read -p "Press Enter once you've updated .env..."
|
| 42 |
-
else
|
| 43 |
-
echo "✓ .env already exists"
|
| 44 |
-
fi
|
| 45 |
-
echo ""
|
| 46 |
-
|
| 47 |
-
# Step 4: Test locally
|
| 48 |
-
echo "Step 4: Running tests..."
|
| 49 |
-
python -m pytest test_chatbot.py -v --tb=short -q
|
| 50 |
-
echo "✓ All tests passed"
|
| 51 |
-
echo ""
|
| 52 |
-
|
| 53 |
-
# Step 5: Verify Dockerfile
|
| 54 |
-
echo "Step 5: Verifying Dockerfile..."
|
| 55 |
-
if ! grep -q "FROM python" Dockerfile; then
|
| 56 |
-
echo "❌ Dockerfile invalid"
|
| 57 |
-
exit 1
|
| 58 |
-
fi
|
| 59 |
-
echo "✓ Dockerfile is valid"
|
| 60 |
-
echo ""
|
| 61 |
-
|
| 62 |
-
# Step 6: Git setup
|
| 63 |
-
echo "Step 6: Setting up Git..."
|
| 64 |
-
if [ ! -d ".git" ]; then
|
| 65 |
-
git init
|
| 66 |
-
git add .
|
| 67 |
-
git commit -m "Initial commit: UnityCatalog-ChatBot"
|
| 68 |
-
echo "✓ Git repository initialized"
|
| 69 |
-
else
|
| 70 |
-
echo "✓ Git repository already exists"
|
| 71 |
-
git add .
|
| 72 |
-
if git diff --cached --quiet; then
|
| 73 |
-
echo "✓ No changes to commit"
|
| 74 |
-
else
|
| 75 |
-
git commit -m "Update: deployment preparation"
|
| 76 |
-
echo "✓ Changes committed"
|
| 77 |
-
fi
|
| 78 |
-
fi
|
| 79 |
-
echo ""
|
| 80 |
-
|
| 81 |
-
# Step 7: Manual Space creation instructions
|
| 82 |
-
echo "Step 7: Manual Space Creation"
|
| 83 |
-
echo "=============================="
|
| 84 |
-
echo ""
|
| 85 |
-
echo "⚠️ Please complete these steps manually on Hugging Face:"
|
| 86 |
-
echo ""
|
| 87 |
-
echo "1. Go to: https://huggingface.co/new"
|
| 88 |
-
echo "2. Fill in the form:"
|
| 89 |
-
echo " - Owner: $HF_USERNAME"
|
| 90 |
-
echo " - Repository name: $SPACE_NAME"
|
| 91 |
-
echo " - Type: Space"
|
| 92 |
-
echo " - Space SDK: Docker"
|
| 93 |
-
echo " - License: MIT"
|
| 94 |
-
echo "3. Click 'Create Space'"
|
| 95 |
-
echo ""
|
| 96 |
-
echo "Once created, you'll get a clone URL like:"
|
| 97 |
-
echo "https://huggingface.co/spaces/$HF_USERNAME/$SPACE_NAME"
|
| 98 |
-
echo ""
|
| 99 |
-
|
| 100 |
-
read -p "Press Enter once you've created the Space..."
|
| 101 |
-
echo ""
|
| 102 |
-
|
| 103 |
-
# Step 8: Push to HF
|
| 104 |
-
echo "Step 8: Pushing code to Hugging Face..."
|
| 105 |
-
echo ""
|
| 106 |
-
echo "You'll be asked to authenticate. Use your HF token:"
|
| 107 |
-
echo "Visit: https://huggingface.co/settings/tokens"
|
| 108 |
-
echo "Copy a READ+WRITE token and paste it when prompted."
|
| 109 |
-
echo ""
|
| 110 |
-
|
| 111 |
-
PUSH_URL="https://huggingface.co/spaces/${HF_USERNAME}/${SPACE_NAME}"
|
| 112 |
-
|
| 113 |
-
# Try to add remote
|
| 114 |
-
if git remote | grep -q "huggingface"; then
|
| 115 |
-
git remote remove huggingface
|
| 116 |
-
fi
|
| 117 |
-
|
| 118 |
-
git remote add huggingface "$PUSH_URL"
|
| 119 |
-
|
| 120 |
-
echo "Pushing to $PUSH_URL..."
|
| 121 |
-
git push -u huggingface main || git push -u huggingface master
|
| 122 |
-
|
| 123 |
-
echo "✓ Code pushed to Hugging Face"
|
| 124 |
-
echo ""
|
| 125 |
-
|
| 126 |
-
# Step 9: Add secrets
|
| 127 |
-
echo "Step 9: Adding secrets to Hugging Face Space"
|
| 128 |
-
echo "=============================================="
|
| 129 |
-
echo ""
|
| 130 |
-
echo "⚠️ Complete these steps in HF UI:"
|
| 131 |
-
echo ""
|
| 132 |
-
echo "1. Go to: $SPACE_URL/settings"
|
| 133 |
-
echo "2. Click 'Repository Secrets'"
|
| 134 |
-
echo "3. Add these three secrets:"
|
| 135 |
-
echo ""
|
| 136 |
-
echo " Secret 1:"
|
| 137 |
-
echo " - Name: DATABRICKS_HOST"
|
| 138 |
-
echo " - Value: <your workspace URL>"
|
| 139 |
-
echo ""
|
| 140 |
-
echo " Secret 2:"
|
| 141 |
-
echo " - Name: DATABRICKS_TOKEN"
|
| 142 |
-
echo " - Value: <your personal access token>"
|
| 143 |
-
echo ""
|
| 144 |
-
echo " Secret 3:"
|
| 145 |
-
echo " - Name: ANTHROPIC_API_KEY"
|
| 146 |
-
echo " - Value: <your Claude API key>"
|
| 147 |
-
echo ""
|
| 148 |
-
echo "4. Click 'Save' after each secret"
|
| 149 |
-
echo "5. Space will rebuild automatically"
|
| 150 |
-
echo ""
|
| 151 |
-
|
| 152 |
-
read -p "Press Enter once you've added all secrets..."
|
| 153 |
-
echo ""
|
| 154 |
-
|
| 155 |
-
# Step 10: Verify deployment
|
| 156 |
-
echo "Step 10: Verifying deployment..."
|
| 157 |
-
echo ""
|
| 158 |
-
echo "Waiting 30 seconds for Space to build..."
|
| 159 |
-
sleep 30
|
| 160 |
-
|
| 161 |
-
HEALTH_URL="${SPACE_URL/huggingface.co/huggingface.co}/api/health"
|
| 162 |
-
# Note: The actual URL will be in HF logs - this is just for reference
|
| 163 |
-
|
| 164 |
-
echo ""
|
| 165 |
-
echo "✅ DEPLOYMENT COMPLETE!"
|
| 166 |
-
echo ""
|
| 167 |
-
echo "Your chatbot is now live at:"
|
| 168 |
-
echo " 🌐 $SPACE_URL"
|
| 169 |
-
echo ""
|
| 170 |
-
echo "Next steps:"
|
| 171 |
-
echo " 1. Visit the Space URL above"
|
| 172 |
-
echo " 2. Check Settings → Runtime logs for any errors"
|
| 173 |
-
echo " 3. Once 'Running' status shows, test the API"
|
| 174 |
-
echo ""
|
| 175 |
-
echo "Test endpoints:"
|
| 176 |
-
echo " curl https://${HF_USERNAME}-${SPACE_NAME}.hf.space/api/health"
|
| 177 |
-
echo " curl https://${HF_USERNAME}-${SPACE_NAME}.hf.space/api/catalogs"
|
| 178 |
-
echo ""
|
| 179 |
-
echo "Share your Space:"
|
| 180 |
-
echo " Simply send the URL to others to access the chatbot!"
|
| 181 |
-
echo ""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
deploy.sh
DELETED
|
@@ -1,486 +0,0 @@
|
|
| 1 |
-
#!/bin/bash
|
| 2 |
-
|
| 3 |
-
###############################################################################
|
| 4 |
-
# Unity Catalog Chatbot - Automated Deployment Script
|
| 5 |
-
# This script will guide you through the complete setup and deployment
|
| 6 |
-
###############################################################################
|
| 7 |
-
|
| 8 |
-
set -e # Exit on error
|
| 9 |
-
|
| 10 |
-
# Colors for output
|
| 11 |
-
RED='\033[0;31m'
|
| 12 |
-
GREEN='\033[0;32m'
|
| 13 |
-
YELLOW='\033[1;33m'
|
| 14 |
-
BLUE='\033[0;34m'
|
| 15 |
-
NC='\033[0m' # No Color
|
| 16 |
-
|
| 17 |
-
# Functions
|
| 18 |
-
print_header() {
|
| 19 |
-
echo -e "\n${BLUE}========================================${NC}"
|
| 20 |
-
echo -e "${BLUE}$1${NC}"
|
| 21 |
-
echo -e "${BLUE}========================================${NC}\n"
|
| 22 |
-
}
|
| 23 |
-
|
| 24 |
-
print_success() {
|
| 25 |
-
echo -e "${GREEN}✓ $1${NC}"
|
| 26 |
-
}
|
| 27 |
-
|
| 28 |
-
print_error() {
|
| 29 |
-
echo -e "${RED}✗ $1${NC}"
|
| 30 |
-
}
|
| 31 |
-
|
| 32 |
-
print_warning() {
|
| 33 |
-
echo -e "${YELLOW}⚠ $1${NC}"
|
| 34 |
-
}
|
| 35 |
-
|
| 36 |
-
print_info() {
|
| 37 |
-
echo -e "${BLUE}ℹ $1${NC}"
|
| 38 |
-
}
|
| 39 |
-
|
| 40 |
-
# Check if command exists
|
| 41 |
-
command_exists() {
|
| 42 |
-
command -v "$1" >/dev/null 2>&1
|
| 43 |
-
}
|
| 44 |
-
|
| 45 |
-
###############################################################################
|
| 46 |
-
# STEP 1: PRE-FLIGHT CHECKS
|
| 47 |
-
###############################################################################
|
| 48 |
-
|
| 49 |
-
print_header "Step 1: Pre-flight Checks"
|
| 50 |
-
|
| 51 |
-
# Check Python
|
| 52 |
-
if command_exists python3; then
|
| 53 |
-
PYTHON_VERSION=$(python3 --version | cut -d ' ' -f 2)
|
| 54 |
-
print_success "Python 3 found: $PYTHON_VERSION"
|
| 55 |
-
else
|
| 56 |
-
print_error "Python 3 is not installed. Please install Python 3.9 or higher."
|
| 57 |
-
exit 1
|
| 58 |
-
fi
|
| 59 |
-
|
| 60 |
-
# Check pip
|
| 61 |
-
if command_exists pip3; then
|
| 62 |
-
print_success "pip3 found"
|
| 63 |
-
else
|
| 64 |
-
print_error "pip3 is not installed"
|
| 65 |
-
exit 1
|
| 66 |
-
fi
|
| 67 |
-
|
| 68 |
-
# Check Docker (optional)
|
| 69 |
-
if command_exists docker; then
|
| 70 |
-
print_success "Docker found: $(docker --version)"
|
| 71 |
-
DOCKER_AVAILABLE=true
|
| 72 |
-
else
|
| 73 |
-
print_warning "Docker not found. Docker deployment will not be available."
|
| 74 |
-
DOCKER_AVAILABLE=false
|
| 75 |
-
fi
|
| 76 |
-
|
| 77 |
-
# Check Docker Compose (optional)
|
| 78 |
-
if command_exists docker-compose; then
|
| 79 |
-
print_success "Docker Compose found: $(docker-compose --version)"
|
| 80 |
-
COMPOSE_AVAILABLE=true
|
| 81 |
-
else
|
| 82 |
-
print_warning "Docker Compose not found"
|
| 83 |
-
COMPOSE_AVAILABLE=false
|
| 84 |
-
fi
|
| 85 |
-
|
| 86 |
-
###############################################################################
|
| 87 |
-
# STEP 2: GATHER CREDENTIALS
|
| 88 |
-
###############################################################################
|
| 89 |
-
|
| 90 |
-
print_header "Step 2: Configuration Setup"
|
| 91 |
-
|
| 92 |
-
# Check if .env exists
|
| 93 |
-
if [ -f .env ]; then
|
| 94 |
-
print_warning ".env file already exists"
|
| 95 |
-
read -p "Do you want to overwrite it? (y/n): " OVERWRITE
|
| 96 |
-
if [ "$OVERWRITE" != "y" ]; then
|
| 97 |
-
print_info "Using existing .env file"
|
| 98 |
-
ENV_CONFIGURED=true
|
| 99 |
-
else
|
| 100 |
-
ENV_CONFIGURED=false
|
| 101 |
-
fi
|
| 102 |
-
else
|
| 103 |
-
ENV_CONFIGURED=false
|
| 104 |
-
fi
|
| 105 |
-
|
| 106 |
-
if [ "$ENV_CONFIGURED" = false ]; then
|
| 107 |
-
print_info "Let's configure your credentials..."
|
| 108 |
-
|
| 109 |
-
# Databricks configuration
|
| 110 |
-
echo ""
|
| 111 |
-
print_info "DATABRICKS CONFIGURATION"
|
| 112 |
-
read -p "Enter your Databricks workspace URL (e.g., https://adb-xxx.azuredatabricks.net): " DATABRICKS_HOST
|
| 113 |
-
read -p "Enter your Databricks personal access token: " DATABRICKS_TOKEN
|
| 114 |
-
read -p "Enter your SQL Warehouse ID (optional, press Enter to skip): " DATABRICKS_WAREHOUSE_ID
|
| 115 |
-
|
| 116 |
-
# Anthropic configuration
|
| 117 |
-
echo ""
|
| 118 |
-
print_info "ANTHROPIC CONFIGURATION"
|
| 119 |
-
read -p "Enter your Anthropic API key (sk-ant-...): " ANTHROPIC_API_KEY
|
| 120 |
-
|
| 121 |
-
# Create .env file
|
| 122 |
-
cat > .env << EOF
|
| 123 |
-
# Databricks Configuration
|
| 124 |
-
DATABRICKS_HOST=$DATABRICKS_HOST
|
| 125 |
-
DATABRICKS_TOKEN=$DATABRICKS_TOKEN
|
| 126 |
-
DATABRICKS_WAREHOUSE_ID=$DATABRICKS_WAREHOUSE_ID
|
| 127 |
-
|
| 128 |
-
# Anthropic API Configuration
|
| 129 |
-
ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY
|
| 130 |
-
|
| 131 |
-
# Server Configuration
|
| 132 |
-
SERVER_HOST=0.0.0.0
|
| 133 |
-
SERVER_PORT=5000
|
| 134 |
-
FLASK_ENV=development
|
| 135 |
-
LOG_LEVEL=INFO
|
| 136 |
-
|
| 137 |
-
# Features
|
| 138 |
-
ENABLE_SQL_EXECUTION=false
|
| 139 |
-
ENABLE_BATCH_OPS=true
|
| 140 |
-
ENABLE_AUDIT_LOG=true
|
| 141 |
-
ENABLE_CACHING=false
|
| 142 |
-
EOF
|
| 143 |
-
|
| 144 |
-
print_success ".env file created successfully"
|
| 145 |
-
fi
|
| 146 |
-
|
| 147 |
-
###############################################################################
|
| 148 |
-
# STEP 3: CHOOSE DEPLOYMENT METHOD
|
| 149 |
-
###############################################################################
|
| 150 |
-
|
| 151 |
-
print_header "Step 3: Choose Deployment Method"
|
| 152 |
-
|
| 153 |
-
echo "Available deployment options:"
|
| 154 |
-
echo "1) Local Python (recommended for testing)"
|
| 155 |
-
if [ "$DOCKER_AVAILABLE" = true ]; then
|
| 156 |
-
echo "2) Docker"
|
| 157 |
-
fi
|
| 158 |
-
if [ "$COMPOSE_AVAILABLE" = true ]; then
|
| 159 |
-
echo "3) Docker Compose (full stack)"
|
| 160 |
-
fi
|
| 161 |
-
|
| 162 |
-
read -p "Enter your choice (1-3): " DEPLOY_CHOICE
|
| 163 |
-
|
| 164 |
-
###############################################################################
|
| 165 |
-
# STEP 4: DEPLOYMENT
|
| 166 |
-
###############################################################################
|
| 167 |
-
|
| 168 |
-
print_header "Step 4: Deployment"
|
| 169 |
-
|
| 170 |
-
case $DEPLOY_CHOICE in
|
| 171 |
-
1)
|
| 172 |
-
# Local Python deployment
|
| 173 |
-
print_info "Setting up local Python environment..."
|
| 174 |
-
|
| 175 |
-
# Create virtual environment
|
| 176 |
-
if [ ! -d "venv" ]; then
|
| 177 |
-
print_info "Creating virtual environment..."
|
| 178 |
-
python3 -m venv venv
|
| 179 |
-
print_success "Virtual environment created"
|
| 180 |
-
fi
|
| 181 |
-
|
| 182 |
-
# Activate virtual environment
|
| 183 |
-
print_info "Activating virtual environment..."
|
| 184 |
-
source venv/bin/activate
|
| 185 |
-
|
| 186 |
-
# Upgrade pip
|
| 187 |
-
print_info "Upgrading pip..."
|
| 188 |
-
pip install --upgrade pip > /dev/null 2>&1
|
| 189 |
-
|
| 190 |
-
# Install dependencies
|
| 191 |
-
print_info "Installing dependencies..."
|
| 192 |
-
pip install -r requirements.txt
|
| 193 |
-
print_success "Dependencies installed"
|
| 194 |
-
|
| 195 |
-
# Start the application in background
|
| 196 |
-
print_info "Starting application..."
|
| 197 |
-
nohup python app.py > app.log 2>&1 &
|
| 198 |
-
APP_PID=$!
|
| 199 |
-
echo $APP_PID > app.pid
|
| 200 |
-
|
| 201 |
-
# Wait for startup
|
| 202 |
-
sleep 5
|
| 203 |
-
|
| 204 |
-
if ps -p $APP_PID > /dev/null; then
|
| 205 |
-
print_success "Application started successfully (PID: $APP_PID)"
|
| 206 |
-
DEPLOYMENT_TYPE="local"
|
| 207 |
-
else
|
| 208 |
-
print_error "Application failed to start. Check app.log for details."
|
| 209 |
-
exit 1
|
| 210 |
-
fi
|
| 211 |
-
;;
|
| 212 |
-
|
| 213 |
-
2)
|
| 214 |
-
# Docker deployment
|
| 215 |
-
if [ "$DOCKER_AVAILABLE" = false ]; then
|
| 216 |
-
print_error "Docker is not available"
|
| 217 |
-
exit 1
|
| 218 |
-
fi
|
| 219 |
-
|
| 220 |
-
print_info "Building Docker image..."
|
| 221 |
-
docker build -t unity-catalog-chatbot:latest .
|
| 222 |
-
print_success "Docker image built"
|
| 223 |
-
|
| 224 |
-
print_info "Starting Docker container..."
|
| 225 |
-
docker run -d \
|
| 226 |
-
--name unity-catalog-chatbot \
|
| 227 |
-
-p 5000:5000 \
|
| 228 |
-
--env-file .env \
|
| 229 |
-
unity-catalog-chatbot:latest
|
| 230 |
-
|
| 231 |
-
print_success "Docker container started"
|
| 232 |
-
DEPLOYMENT_TYPE="docker"
|
| 233 |
-
;;
|
| 234 |
-
|
| 235 |
-
3)
|
| 236 |
-
# Docker Compose deployment
|
| 237 |
-
if [ "$COMPOSE_AVAILABLE" = false ]; then
|
| 238 |
-
print_error "Docker Compose is not available"
|
| 239 |
-
exit 1
|
| 240 |
-
fi
|
| 241 |
-
|
| 242 |
-
print_info "Starting services with Docker Compose..."
|
| 243 |
-
docker-compose up -d
|
| 244 |
-
print_success "Services started"
|
| 245 |
-
DEPLOYMENT_TYPE="compose"
|
| 246 |
-
;;
|
| 247 |
-
|
| 248 |
-
*)
|
| 249 |
-
print_error "Invalid choice"
|
| 250 |
-
exit 1
|
| 251 |
-
;;
|
| 252 |
-
esac
|
| 253 |
-
|
| 254 |
-
###############################################################################
|
| 255 |
-
# STEP 5: HEALTH CHECK
|
| 256 |
-
###############################################################################
|
| 257 |
-
|
| 258 |
-
print_header "Step 5: Health Check"
|
| 259 |
-
|
| 260 |
-
print_info "Waiting for application to be ready..."
|
| 261 |
-
sleep 10
|
| 262 |
-
|
| 263 |
-
MAX_RETRIES=30
|
| 264 |
-
RETRY_COUNT=0
|
| 265 |
-
|
| 266 |
-
while [ $RETRY_COUNT -lt $MAX_RETRIES ]; do
|
| 267 |
-
if curl -s http://localhost:5000/api/health > /dev/null 2>&1; then
|
| 268 |
-
print_success "Application is healthy!"
|
| 269 |
-
|
| 270 |
-
# Get health check response
|
| 271 |
-
HEALTH_RESPONSE=$(curl -s http://localhost:5000/api/health)
|
| 272 |
-
echo -e "${GREEN}Health Check Response:${NC}"
|
| 273 |
-
echo "$HEALTH_RESPONSE" | python3 -m json.tool 2>/dev/null || echo "$HEALTH_RESPONSE"
|
| 274 |
-
break
|
| 275 |
-
else
|
| 276 |
-
RETRY_COUNT=$((RETRY_COUNT + 1))
|
| 277 |
-
echo -n "."
|
| 278 |
-
sleep 2
|
| 279 |
-
fi
|
| 280 |
-
done
|
| 281 |
-
|
| 282 |
-
if [ $RETRY_COUNT -eq $MAX_RETRIES ]; then
|
| 283 |
-
print_error "Application failed health check"
|
| 284 |
-
|
| 285 |
-
if [ "$DEPLOYMENT_TYPE" = "local" ]; then
|
| 286 |
-
print_info "Check app.log for error details:"
|
| 287 |
-
tail -n 20 app.log
|
| 288 |
-
elif [ "$DEPLOYMENT_TYPE" = "docker" ]; then
|
| 289 |
-
print_info "Check Docker logs:"
|
| 290 |
-
docker logs unity-catalog-chatbot
|
| 291 |
-
elif [ "$DEPLOYMENT_TYPE" = "compose" ]; then
|
| 292 |
-
print_info "Check Docker Compose logs:"
|
| 293 |
-
docker-compose logs
|
| 294 |
-
fi
|
| 295 |
-
|
| 296 |
-
exit 1
|
| 297 |
-
fi
|
| 298 |
-
|
| 299 |
-
###############################################################################
|
| 300 |
-
# STEP 6: AUTOMATED TESTING
|
| 301 |
-
###############################################################################
|
| 302 |
-
|
| 303 |
-
print_header "Step 6: Automated Testing"
|
| 304 |
-
|
| 305 |
-
print_info "Running automated tests..."
|
| 306 |
-
|
| 307 |
-
# Test 1: Health endpoint
|
| 308 |
-
print_info "Test 1: Health endpoint"
|
| 309 |
-
HEALTH_STATUS=$(curl -s -o /dev/null -w "%{http_code}" http://localhost:5000/api/health)
|
| 310 |
-
if [ "$HEALTH_STATUS" = "200" ]; then
|
| 311 |
-
print_success "Health endpoint: PASSED"
|
| 312 |
-
else
|
| 313 |
-
print_error "Health endpoint: FAILED (Status: $HEALTH_STATUS)"
|
| 314 |
-
fi
|
| 315 |
-
|
| 316 |
-
# Test 2: Chat endpoint with help command
|
| 317 |
-
print_info "Test 2: Chat endpoint (help command)"
|
| 318 |
-
CHAT_RESPONSE=$(curl -s -X POST http://localhost:5000/api/chat \
|
| 319 |
-
-H "Content-Type: application/json" \
|
| 320 |
-
-d '{"message": "help"}' \
|
| 321 |
-
-w "\n%{http_code}")
|
| 322 |
-
|
| 323 |
-
CHAT_STATUS=$(echo "$CHAT_RESPONSE" | tail -n 1)
|
| 324 |
-
CHAT_BODY=$(echo "$CHAT_RESPONSE" | head -n -1)
|
| 325 |
-
|
| 326 |
-
if [ "$CHAT_STATUS" = "200" ]; then
|
| 327 |
-
print_success "Chat endpoint: PASSED"
|
| 328 |
-
echo -e "${BLUE}Response:${NC}"
|
| 329 |
-
echo "$CHAT_BODY" | python3 -m json.tool 2>/dev/null | head -n 20
|
| 330 |
-
else
|
| 331 |
-
print_error "Chat endpoint: FAILED (Status: $CHAT_STATUS)"
|
| 332 |
-
fi
|
| 333 |
-
|
| 334 |
-
# Test 3: List catalogs
|
| 335 |
-
print_info "Test 3: List catalogs"
|
| 336 |
-
CATALOGS_RESPONSE=$(curl -s -X POST http://localhost:5000/api/chat \
|
| 337 |
-
-H "Content-Type: application/json" \
|
| 338 |
-
-d '{"message": "list all catalogs"}' \
|
| 339 |
-
-w "\n%{http_code}")
|
| 340 |
-
|
| 341 |
-
CATALOGS_STATUS=$(echo "$CATALOGS_RESPONSE" | tail -n 1)
|
| 342 |
-
if [ "$CATALOGS_STATUS" = "200" ]; then
|
| 343 |
-
print_success "List catalogs: PASSED"
|
| 344 |
-
else
|
| 345 |
-
print_error "List catalogs: FAILED (Status: $CATALOGS_STATUS)"
|
| 346 |
-
fi
|
| 347 |
-
|
| 348 |
-
###############################################################################
|
| 349 |
-
# STEP 7: INTERACTIVE TESTING
|
| 350 |
-
###############################################################################
|
| 351 |
-
|
| 352 |
-
print_header "Step 7: Interactive Testing"
|
| 353 |
-
|
| 354 |
-
echo ""
|
| 355 |
-
print_info "Application is ready for interactive testing!"
|
| 356 |
-
echo ""
|
| 357 |
-
echo -e "${GREEN}Access URLs:${NC}"
|
| 358 |
-
echo " API Backend: http://localhost:5000"
|
| 359 |
-
echo " Health Check: http://localhost:5000/api/health"
|
| 360 |
-
echo " Swagger Docs: http://localhost:5000/docs (if enabled)"
|
| 361 |
-
echo ""
|
| 362 |
-
|
| 363 |
-
read -p "Would you like to test the chatbot interactively? (y/n): " INTERACTIVE
|
| 364 |
-
|
| 365 |
-
if [ "$INTERACTIVE" = "y" ]; then
|
| 366 |
-
print_info "Interactive Chatbot Test"
|
| 367 |
-
echo ""
|
| 368 |
-
echo "Enter your commands (or 'exit' to quit):"
|
| 369 |
-
echo "Examples:"
|
| 370 |
-
echo " - Create a catalog named test_catalog"
|
| 371 |
-
echo " - List all catalogs"
|
| 372 |
-
echo " - Grant SELECT on test_catalog to data_analysts"
|
| 373 |
-
echo ""
|
| 374 |
-
|
| 375 |
-
while true; do
|
| 376 |
-
read -p "You: " USER_INPUT
|
| 377 |
-
|
| 378 |
-
if [ "$USER_INPUT" = "exit" ]; then
|
| 379 |
-
break
|
| 380 |
-
fi
|
| 381 |
-
|
| 382 |
-
# Make API call
|
| 383 |
-
RESPONSE=$(curl -s -X POST http://localhost:5000/api/chat \
|
| 384 |
-
-H "Content-Type: application/json" \
|
| 385 |
-
-d "{\"message\": \"$USER_INPUT\"}")
|
| 386 |
-
|
| 387 |
-
# Extract message from JSON
|
| 388 |
-
MESSAGE=$(echo "$RESPONSE" | python3 -c "import sys, json; print(json.load(sys.stdin).get('message', 'No response'))" 2>/dev/null)
|
| 389 |
-
SQL=$(echo "$RESPONSE" | python3 -c "import sys, json; data=json.load(sys.stdin); print(data.get('sql', ''))" 2>/dev/null)
|
| 390 |
-
|
| 391 |
-
echo -e "${BLUE}Assistant:${NC} $MESSAGE"
|
| 392 |
-
|
| 393 |
-
if [ ! -z "$SQL" ]; then
|
| 394 |
-
echo -e "${YELLOW}SQL:${NC} $SQL"
|
| 395 |
-
fi
|
| 396 |
-
echo ""
|
| 397 |
-
done
|
| 398 |
-
fi
|
| 399 |
-
|
| 400 |
-
###############################################################################
|
| 401 |
-
# STEP 8: DEPLOYMENT SUMMARY
|
| 402 |
-
###############################################################################
|
| 403 |
-
|
| 404 |
-
print_header "Deployment Summary"
|
| 405 |
-
|
| 406 |
-
echo -e "${GREEN}✓ Deployment completed successfully!${NC}"
|
| 407 |
-
echo ""
|
| 408 |
-
echo "Deployment Type: $DEPLOYMENT_TYPE"
|
| 409 |
-
echo ""
|
| 410 |
-
echo -e "${BLUE}Next Steps:${NC}"
|
| 411 |
-
echo "1. Access the API at: http://localhost:5000"
|
| 412 |
-
echo "2. Review the logs:"
|
| 413 |
-
|
| 414 |
-
case $DEPLOYMENT_TYPE in
|
| 415 |
-
"local")
|
| 416 |
-
echo " - Application logs: tail -f app.log"
|
| 417 |
-
echo "3. Stop the application:"
|
| 418 |
-
echo " - kill \$(cat app.pid)"
|
| 419 |
-
;;
|
| 420 |
-
"docker")
|
| 421 |
-
echo " - Docker logs: docker logs -f unity-catalog-chatbot"
|
| 422 |
-
echo "3. Stop the application:"
|
| 423 |
-
echo " - docker stop unity-catalog-chatbot"
|
| 424 |
-
;;
|
| 425 |
-
"compose")
|
| 426 |
-
echo " - Docker Compose logs: docker-compose logs -f"
|
| 427 |
-
echo "3. Stop the application:"
|
| 428 |
-
echo " - docker-compose down"
|
| 429 |
-
;;
|
| 430 |
-
esac
|
| 431 |
-
|
| 432 |
-
echo ""
|
| 433 |
-
echo "4. Run tests: pytest test_chatbot.py"
|
| 434 |
-
echo "5. View documentation: cat README.md"
|
| 435 |
-
echo ""
|
| 436 |
-
|
| 437 |
-
# Create quick reference file
|
| 438 |
-
cat > QUICK_REFERENCE.txt << EOF
|
| 439 |
-
Unity Catalog Chatbot - Quick Reference
|
| 440 |
-
========================================
|
| 441 |
-
|
| 442 |
-
Deployment Type: $DEPLOYMENT_TYPE
|
| 443 |
-
|
| 444 |
-
URLs:
|
| 445 |
-
- API: http://localhost:5000
|
| 446 |
-
- Health: http://localhost:5000/api/health
|
| 447 |
-
|
| 448 |
-
Logs:
|
| 449 |
-
EOF
|
| 450 |
-
|
| 451 |
-
case $DEPLOYMENT_TYPE in
|
| 452 |
-
"local")
|
| 453 |
-
echo "- Application: tail -f app.log" >> QUICK_REFERENCE.txt
|
| 454 |
-
echo "" >> QUICK_REFERENCE.txt
|
| 455 |
-
echo "Stop Application:" >> QUICK_REFERENCE.txt
|
| 456 |
-
echo "kill \$(cat app.pid)" >> QUICK_REFERENCE.txt
|
| 457 |
-
;;
|
| 458 |
-
"docker")
|
| 459 |
-
echo "- Docker: docker logs -f unity-catalog-chatbot" >> QUICK_REFERENCE.txt
|
| 460 |
-
echo "" >> QUICK_REFERENCE.txt
|
| 461 |
-
echo "Stop Application:" >> QUICK_REFERENCE.txt
|
| 462 |
-
echo "docker stop unity-catalog-chatbot" >> QUICK_REFERENCE.txt
|
| 463 |
-
;;
|
| 464 |
-
"compose")
|
| 465 |
-
echo "- Docker Compose: docker-compose logs -f" >> QUICK_REFERENCE.txt
|
| 466 |
-
echo "" >> QUICK_REFERENCE.txt
|
| 467 |
-
echo "Stop Application:" >> QUICK_REFERENCE.txt
|
| 468 |
-
echo "docker-compose down" >> QUICK_REFERENCE.txt
|
| 469 |
-
;;
|
| 470 |
-
esac
|
| 471 |
-
|
| 472 |
-
cat >> QUICK_REFERENCE.txt << EOF
|
| 473 |
-
|
| 474 |
-
Sample Commands:
|
| 475 |
-
- Create catalog: "Create a catalog named test_catalog"
|
| 476 |
-
- Create schema: "Create schema analytics in test_catalog"
|
| 477 |
-
- Grant permission: "Grant SELECT on test_catalog to user@example.com"
|
| 478 |
-
- List catalogs: "List all catalogs"
|
| 479 |
-
|
| 480 |
-
For detailed documentation, see README.md
|
| 481 |
-
EOF
|
| 482 |
-
|
| 483 |
-
print_success "Quick reference saved to QUICK_REFERENCE.txt"
|
| 484 |
-
|
| 485 |
-
echo ""
|
| 486 |
-
print_success "Deployment complete! Happy chatting! 🎉"
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
docker-compose.yml
DELETED
|
@@ -1,60 +0,0 @@
|
|
| 1 |
-
version: '3.8'
|
| 2 |
-
|
| 3 |
-
services:
|
| 4 |
-
unity-catalog-chatbot:
|
| 5 |
-
build: .
|
| 6 |
-
container_name: unity-catalog-chatbot
|
| 7 |
-
ports:
|
| 8 |
-
- "5000:5000"
|
| 9 |
-
environment:
|
| 10 |
-
- DATABRICKS_HOST=${DATABRICKS_HOST}
|
| 11 |
-
- DATABRICKS_TOKEN=${DATABRICKS_TOKEN}
|
| 12 |
-
- ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
|
| 13 |
-
- DATABRICKS_WAREHOUSE_ID=${DATABRICKS_WAREHOUSE_ID}
|
| 14 |
-
- FLASK_ENV=production
|
| 15 |
-
env_file:
|
| 16 |
-
- .env
|
| 17 |
-
restart: unless-stopped
|
| 18 |
-
healthcheck:
|
| 19 |
-
test: ["CMD", "curl", "-f", "http://localhost:5000/api/health"]
|
| 20 |
-
interval: 30s
|
| 21 |
-
timeout: 10s
|
| 22 |
-
retries: 3
|
| 23 |
-
start_period: 40s
|
| 24 |
-
networks:
|
| 25 |
-
- chatbot-network
|
| 26 |
-
|
| 27 |
-
# Optional: Redis for caching and rate limiting
|
| 28 |
-
redis:
|
| 29 |
-
image: redis:7-alpine
|
| 30 |
-
container_name: unity-catalog-redis
|
| 31 |
-
ports:
|
| 32 |
-
- "6379:6379"
|
| 33 |
-
volumes:
|
| 34 |
-
- redis-data:/data
|
| 35 |
-
restart: unless-stopped
|
| 36 |
-
networks:
|
| 37 |
-
- chatbot-network
|
| 38 |
-
|
| 39 |
-
# Optional: Nginx reverse proxy
|
| 40 |
-
nginx:
|
| 41 |
-
image: nginx:alpine
|
| 42 |
-
container_name: unity-catalog-nginx
|
| 43 |
-
ports:
|
| 44 |
-
- "80:80"
|
| 45 |
-
- "443:443"
|
| 46 |
-
volumes:
|
| 47 |
-
- ./nginx.conf:/etc/nginx/nginx.conf:ro
|
| 48 |
-
- ./ssl:/etc/nginx/ssl:ro
|
| 49 |
-
depends_on:
|
| 50 |
-
- unity-catalog-chatbot
|
| 51 |
-
restart: unless-stopped
|
| 52 |
-
networks:
|
| 53 |
-
- chatbot-network
|
| 54 |
-
|
| 55 |
-
networks:
|
| 56 |
-
chatbot-network:
|
| 57 |
-
driver: bridge
|
| 58 |
-
|
| 59 |
-
volumes:
|
| 60 |
-
redis-data:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
index.html
ADDED
|
@@ -0,0 +1,39 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!DOCTYPE html>
|
| 2 |
+
<html lang="en">
|
| 3 |
+
<head>
|
| 4 |
+
<meta charset="UTF-8">
|
| 5 |
+
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
| 6 |
+
<title>Unity Catalog Chatbot</title>
|
| 7 |
+
<script crossorigin src="https://unpkg.com/react@18/umd/react.production.min.js"></script>
|
| 8 |
+
<script crossorigin src="https://unpkg.com/react-dom@18/umd/react-dom.production.min.js"></script>
|
| 9 |
+
<script src="https://unpkg.com/@babel/standalone/babel.min.js"></script>
|
| 10 |
+
<style>
|
| 11 |
+
body {
|
| 12 |
+
margin: 0;
|
| 13 |
+
padding: 0;
|
| 14 |
+
font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', 'Roboto', 'Oxygen',
|
| 15 |
+
'Ubuntu', 'Cantarell', 'Fira Sans', 'Droid Sans', 'Helvetica Neue',
|
| 16 |
+
sans-serif;
|
| 17 |
+
-webkit-font-smoothing: antialiased;
|
| 18 |
+
-moz-osx-font-smoothing: grayscale;
|
| 19 |
+
}
|
| 20 |
+
code {
|
| 21 |
+
font-family: source-code-pro, Menlo, Monaco, Consolas, 'Courier New',
|
| 22 |
+
monospace;
|
| 23 |
+
}
|
| 24 |
+
#root {
|
| 25 |
+
width: 100%;
|
| 26 |
+
height: 100vh;
|
| 27 |
+
}
|
| 28 |
+
</style>
|
| 29 |
+
</head>
|
| 30 |
+
<body>
|
| 31 |
+
<div id="root"></div>
|
| 32 |
+
|
| 33 |
+
<script type="text/babel" src="unity-catalog-chatbot.jsx"></script>
|
| 34 |
+
<script type="text/babel">
|
| 35 |
+
const root = ReactDOM.createRoot(document.getElementById('root'));
|
| 36 |
+
root.render(<UnityCatalogChatbot />);
|
| 37 |
+
</script>
|
| 38 |
+
</body>
|
| 39 |
+
</html>
|
test.sh
DELETED
|
@@ -1,316 +0,0 @@
|
|
| 1 |
-
#!/bin/bash
|
| 2 |
-
|
| 3 |
-
###############################################################################
|
| 4 |
-
# Unity Catalog Chatbot - Comprehensive Test Suite
|
| 5 |
-
# This script runs automated tests against the deployed chatbot
|
| 6 |
-
###############################################################################
|
| 7 |
-
|
| 8 |
-
set -e
|
| 9 |
-
|
| 10 |
-
# Colors
|
| 11 |
-
RED='\033[0;31m'
|
| 12 |
-
GREEN='\033[0;32m'
|
| 13 |
-
YELLOW='\033[1;33m'
|
| 14 |
-
BLUE='\033[0;34m'
|
| 15 |
-
NC='\033[0m'
|
| 16 |
-
|
| 17 |
-
# Configuration
|
| 18 |
-
API_URL="${API_URL:-http://localhost:5000}"
|
| 19 |
-
TOTAL_TESTS=0
|
| 20 |
-
PASSED_TESTS=0
|
| 21 |
-
FAILED_TESTS=0
|
| 22 |
-
|
| 23 |
-
# Functions
|
| 24 |
-
print_test() {
|
| 25 |
-
echo -e "\n${BLUE}[TEST $1]${NC} $2"
|
| 26 |
-
}
|
| 27 |
-
|
| 28 |
-
print_pass() {
|
| 29 |
-
echo -e "${GREEN}✓ PASSED${NC}: $1"
|
| 30 |
-
PASSED_TESTS=$((PASSED_TESTS + 1))
|
| 31 |
-
}
|
| 32 |
-
|
| 33 |
-
print_fail() {
|
| 34 |
-
echo -e "${RED}✗ FAILED${NC}: $1"
|
| 35 |
-
FAILED_TESTS=$((FAILED_TESTS + 1))
|
| 36 |
-
}
|
| 37 |
-
|
| 38 |
-
print_info() {
|
| 39 |
-
echo -e "${BLUE}ℹ${NC} $1"
|
| 40 |
-
}
|
| 41 |
-
|
| 42 |
-
# Test function
|
| 43 |
-
run_test() {
|
| 44 |
-
TOTAL_TESTS=$((TOTAL_TESTS + 1))
|
| 45 |
-
local test_name="$1"
|
| 46 |
-
local expected_status="$2"
|
| 47 |
-
local endpoint="$3"
|
| 48 |
-
local method="$4"
|
| 49 |
-
local data="$5"
|
| 50 |
-
|
| 51 |
-
print_test "$TOTAL_TESTS" "$test_name"
|
| 52 |
-
|
| 53 |
-
if [ "$method" = "GET" ]; then
|
| 54 |
-
RESPONSE=$(curl -s -w "\n%{http_code}" "$API_URL$endpoint")
|
| 55 |
-
else
|
| 56 |
-
RESPONSE=$(curl -s -X POST -w "\n%{http_code}" \
|
| 57 |
-
-H "Content-Type: application/json" \
|
| 58 |
-
-d "$data" \
|
| 59 |
-
"$API_URL$endpoint")
|
| 60 |
-
fi
|
| 61 |
-
|
| 62 |
-
HTTP_CODE=$(echo "$RESPONSE" | tail -n 1)
|
| 63 |
-
BODY=$(echo "$RESPONSE" | head -n -1)
|
| 64 |
-
|
| 65 |
-
if [ "$HTTP_CODE" = "$expected_status" ]; then
|
| 66 |
-
print_pass "$test_name (Status: $HTTP_CODE)"
|
| 67 |
-
echo "$BODY" | python3 -m json.tool 2>/dev/null | head -n 10
|
| 68 |
-
return 0
|
| 69 |
-
else
|
| 70 |
-
print_fail "$test_name (Expected: $expected_status, Got: $HTTP_CODE)"
|
| 71 |
-
echo "$BODY"
|
| 72 |
-
return 1
|
| 73 |
-
fi
|
| 74 |
-
}
|
| 75 |
-
|
| 76 |
-
# Test chatbot endpoint
|
| 77 |
-
test_chat() {
|
| 78 |
-
local test_name="$1"
|
| 79 |
-
local message="$2"
|
| 80 |
-
local expected_keyword="$3"
|
| 81 |
-
|
| 82 |
-
TOTAL_TESTS=$((TOTAL_TESTS + 1))
|
| 83 |
-
print_test "$TOTAL_TESTS" "$test_name"
|
| 84 |
-
|
| 85 |
-
RESPONSE=$(curl -s -X POST \
|
| 86 |
-
-H "Content-Type: application/json" \
|
| 87 |
-
-d "{\"message\": \"$message\"}" \
|
| 88 |
-
"$API_URL/api/chat")
|
| 89 |
-
|
| 90 |
-
if echo "$RESPONSE" | grep -q "$expected_keyword"; then
|
| 91 |
-
print_pass "$test_name"
|
| 92 |
-
echo "$RESPONSE" | python3 -m json.tool 2>/dev/null | head -n 15
|
| 93 |
-
return 0
|
| 94 |
-
else
|
| 95 |
-
print_fail "$test_name (Keyword '$expected_keyword' not found)"
|
| 96 |
-
echo "$RESPONSE"
|
| 97 |
-
return 1
|
| 98 |
-
fi
|
| 99 |
-
}
|
| 100 |
-
|
| 101 |
-
###############################################################################
|
| 102 |
-
# START TESTING
|
| 103 |
-
###############################################################################
|
| 104 |
-
|
| 105 |
-
echo "========================================="
|
| 106 |
-
echo "Unity Catalog Chatbot - Test Suite"
|
| 107 |
-
echo "========================================="
|
| 108 |
-
echo "API URL: $API_URL"
|
| 109 |
-
echo "Started: $(date)"
|
| 110 |
-
echo ""
|
| 111 |
-
|
| 112 |
-
###############################################################################
|
| 113 |
-
# SECTION 1: Basic Connectivity Tests
|
| 114 |
-
###############################################################################
|
| 115 |
-
|
| 116 |
-
echo -e "\n${YELLOW}=== SECTION 1: Basic Connectivity ===${NC}\n"
|
| 117 |
-
|
| 118 |
-
run_test "Health Check Endpoint" "200" "/api/health" "GET"
|
| 119 |
-
|
| 120 |
-
run_test "Chat Endpoint Availability" "200" "/api/chat" "POST" '{"message": "test"}'
|
| 121 |
-
|
| 122 |
-
###############################################################################
|
| 123 |
-
# SECTION 2: Help & Information Tests
|
| 124 |
-
###############################################################################
|
| 125 |
-
|
| 126 |
-
echo -e "\n${YELLOW}=== SECTION 2: Help & Information ===${NC}\n"
|
| 127 |
-
|
| 128 |
-
test_chat "Help Command" "help" "Creating Objects"
|
| 129 |
-
|
| 130 |
-
test_chat "General Query" "what can you do?" "Unity Catalog"
|
| 131 |
-
|
| 132 |
-
###############################################################################
|
| 133 |
-
# SECTION 3: Catalog Operations
|
| 134 |
-
###############################################################################
|
| 135 |
-
|
| 136 |
-
echo -e "\n${YELLOW}=== SECTION 3: Catalog Operations ===${NC}\n"
|
| 137 |
-
|
| 138 |
-
test_chat "List Catalogs" "list all catalogs" "SHOW CATALOGS"
|
| 139 |
-
|
| 140 |
-
test_chat "Create Catalog Request" "create a catalog named test_catalog_$(date +%s)" "CREATE CATALOG"
|
| 141 |
-
|
| 142 |
-
###############################################################################
|
| 143 |
-
# SECTION 4: Schema Operations
|
| 144 |
-
###############################################################################
|
| 145 |
-
|
| 146 |
-
echo -e "\n${YELLOW}=== SECTION 4: Schema Operations ===${NC}\n"
|
| 147 |
-
|
| 148 |
-
test_chat "Create Schema Request" "create schema test_schema in main" "CREATE SCHEMA"
|
| 149 |
-
|
| 150 |
-
test_chat "List Schemas Request" "list schemas in main" "SHOW SCHEMAS"
|
| 151 |
-
|
| 152 |
-
###############################################################################
|
| 153 |
-
# SECTION 5: Table Operations
|
| 154 |
-
###############################################################################
|
| 155 |
-
|
| 156 |
-
echo -e "\n${YELLOW}=== SECTION 5: Table Operations ===${NC}\n"
|
| 157 |
-
|
| 158 |
-
test_chat "Create Table Request" "create table main.default.test_table_$(date +%s)" "CREATE TABLE"
|
| 159 |
-
|
| 160 |
-
###############################################################################
|
| 161 |
-
# SECTION 6: Permission Operations
|
| 162 |
-
###############################################################################
|
| 163 |
-
|
| 164 |
-
echo -e "\n${YELLOW}=== SECTION 6: Permission Operations ===${NC}\n"
|
| 165 |
-
|
| 166 |
-
test_chat "Grant Permission Request" "grant SELECT on main to test_user" "GRANT SELECT"
|
| 167 |
-
|
| 168 |
-
test_chat "Show Permissions Request" "show permissions for main" "SHOW GRANTS"
|
| 169 |
-
|
| 170 |
-
###############################################################################
|
| 171 |
-
# SECTION 7: Complex Queries
|
| 172 |
-
###############################################################################
|
| 173 |
-
|
| 174 |
-
echo -e "\n${YELLOW}=== SECTION 7: Complex Queries ===${NC}\n"
|
| 175 |
-
|
| 176 |
-
test_chat "Multi-step Request" \
|
| 177 |
-
"create a catalog named analytics, then create a schema called staging in it" \
|
| 178 |
-
"catalog"
|
| 179 |
-
|
| 180 |
-
###############################################################################
|
| 181 |
-
# SECTION 8: Error Handling
|
| 182 |
-
###############################################################################
|
| 183 |
-
|
| 184 |
-
echo -e "\n${YELLOW}=== SECTION 8: Error Handling ===${NC}\n"
|
| 185 |
-
|
| 186 |
-
TOTAL_TESTS=$((TOTAL_TESTS + 1))
|
| 187 |
-
print_test "$TOTAL_TESTS" "Empty Message Handling"
|
| 188 |
-
|
| 189 |
-
EMPTY_RESPONSE=$(curl -s -X POST \
|
| 190 |
-
-H "Content-Type: application/json" \
|
| 191 |
-
-d '{"message": ""}' \
|
| 192 |
-
-w "%{http_code}" \
|
| 193 |
-
"$API_URL/api/chat")
|
| 194 |
-
|
| 195 |
-
if echo "$EMPTY_RESPONSE" | grep -q "400"; then
|
| 196 |
-
print_pass "Empty message returns 400"
|
| 197 |
-
else
|
| 198 |
-
# Some implementations might accept empty messages
|
| 199 |
-
print_info "Empty message handling: $(echo $EMPTY_RESPONSE | tail -c 4)"
|
| 200 |
-
PASSED_TESTS=$((PASSED_TESTS + 1))
|
| 201 |
-
fi
|
| 202 |
-
|
| 203 |
-
###############################################################################
|
| 204 |
-
# SECTION 9: Performance Tests
|
| 205 |
-
###############################################################################
|
| 206 |
-
|
| 207 |
-
echo -e "\n${YELLOW}=== SECTION 9: Performance Tests ===${NC}\n"
|
| 208 |
-
|
| 209 |
-
print_test "$((TOTAL_TESTS + 1))" "Response Time Test"
|
| 210 |
-
TOTAL_TESTS=$((TOTAL_TESTS + 1))
|
| 211 |
-
|
| 212 |
-
START_TIME=$(date +%s%N)
|
| 213 |
-
curl -s -X POST \
|
| 214 |
-
-H "Content-Type: application/json" \
|
| 215 |
-
-d '{"message": "help"}' \
|
| 216 |
-
"$API_URL/api/chat" > /dev/null
|
| 217 |
-
END_TIME=$(date +%s%N)
|
| 218 |
-
|
| 219 |
-
DURATION=$((($END_TIME - $START_TIME) / 1000000))
|
| 220 |
-
|
| 221 |
-
if [ $DURATION -lt 5000 ]; then
|
| 222 |
-
print_pass "Response time: ${DURATION}ms (< 5s)"
|
| 223 |
-
else
|
| 224 |
-
print_info "Response time: ${DURATION}ms (acceptable)"
|
| 225 |
-
PASSED_TESTS=$((PASSED_TESTS + 1))
|
| 226 |
-
fi
|
| 227 |
-
|
| 228 |
-
###############################################################################
|
| 229 |
-
# SECTION 10: Load Test (Optional)
|
| 230 |
-
###############################################################################
|
| 231 |
-
|
| 232 |
-
echo -e "\n${YELLOW}=== SECTION 10: Light Load Test ===${NC}\n"
|
| 233 |
-
|
| 234 |
-
print_test "$((TOTAL_TESTS + 1))" "Concurrent Requests (5 simultaneous)"
|
| 235 |
-
TOTAL_TESTS=$((TOTAL_TESTS + 1))
|
| 236 |
-
|
| 237 |
-
for i in {1..5}; do
|
| 238 |
-
curl -s -X POST \
|
| 239 |
-
-H "Content-Type: application/json" \
|
| 240 |
-
-d '{"message": "help"}' \
|
| 241 |
-
"$API_URL/api/chat" > /dev/null &
|
| 242 |
-
done
|
| 243 |
-
|
| 244 |
-
wait
|
| 245 |
-
|
| 246 |
-
if [ $? -eq 0 ]; then
|
| 247 |
-
print_pass "Handled 5 concurrent requests"
|
| 248 |
-
else
|
| 249 |
-
print_fail "Failed to handle concurrent requests"
|
| 250 |
-
fi
|
| 251 |
-
|
| 252 |
-
###############################################################################
|
| 253 |
-
# TEST SUMMARY
|
| 254 |
-
###############################################################################
|
| 255 |
-
|
| 256 |
-
echo ""
|
| 257 |
-
echo "========================================="
|
| 258 |
-
echo "Test Summary"
|
| 259 |
-
echo "========================================="
|
| 260 |
-
echo "Total Tests: $TOTAL_TESTS"
|
| 261 |
-
echo -e "${GREEN}Passed: $PASSED_TESTS${NC}"
|
| 262 |
-
echo -e "${RED}Failed: $FAILED_TESTS${NC}"
|
| 263 |
-
echo ""
|
| 264 |
-
|
| 265 |
-
PASS_RATE=$((PASSED_TESTS * 100 / TOTAL_TESTS))
|
| 266 |
-
echo "Pass Rate: $PASS_RATE%"
|
| 267 |
-
|
| 268 |
-
if [ $PASS_RATE -ge 90 ]; then
|
| 269 |
-
echo -e "\n${GREEN}✓ Test suite PASSED!${NC}"
|
| 270 |
-
EXIT_CODE=0
|
| 271 |
-
elif [ $PASS_RATE -ge 70 ]; then
|
| 272 |
-
echo -e "\n${YELLOW}⚠ Test suite passed with warnings${NC}"
|
| 273 |
-
EXIT_CODE=0
|
| 274 |
-
else
|
| 275 |
-
echo -e "\n${RED}✗ Test suite FAILED${NC}"
|
| 276 |
-
EXIT_CODE=1
|
| 277 |
-
fi
|
| 278 |
-
|
| 279 |
-
echo ""
|
| 280 |
-
echo "Completed: $(date)"
|
| 281 |
-
echo "========================================="
|
| 282 |
-
|
| 283 |
-
# Generate test report
|
| 284 |
-
cat > test_report.txt << EOF
|
| 285 |
-
Unity Catalog Chatbot - Test Report
|
| 286 |
-
====================================
|
| 287 |
-
Date: $(date)
|
| 288 |
-
API URL: $API_URL
|
| 289 |
-
|
| 290 |
-
Summary:
|
| 291 |
-
--------
|
| 292 |
-
Total Tests: $TOTAL_TESTS
|
| 293 |
-
Passed: $PASSED_TESTS
|
| 294 |
-
Failed: $FAILED_TESTS
|
| 295 |
-
Pass Rate: $PASS_RATE%
|
| 296 |
-
|
| 297 |
-
Status: $([ $EXIT_CODE -eq 0 ] && echo "PASSED" || echo "FAILED")
|
| 298 |
-
|
| 299 |
-
Sections Tested:
|
| 300 |
-
1. Basic Connectivity
|
| 301 |
-
2. Help & Information
|
| 302 |
-
3. Catalog Operations
|
| 303 |
-
4. Schema Operations
|
| 304 |
-
5. Table Operations
|
| 305 |
-
6. Permission Operations
|
| 306 |
-
7. Complex Queries
|
| 307 |
-
8. Error Handling
|
| 308 |
-
9. Performance Tests
|
| 309 |
-
10. Load Test
|
| 310 |
-
|
| 311 |
-
For detailed logs, review the console output.
|
| 312 |
-
EOF
|
| 313 |
-
|
| 314 |
-
print_info "Test report saved to test_report.txt"
|
| 315 |
-
|
| 316 |
-
exit $EXIT_CODE
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
unity-catalog-chatbot.jsx
CHANGED
|
@@ -1,7 +1,19 @@
|
|
| 1 |
import React, { useState, useRef, useEffect } from 'react';
|
| 2 |
-
import { Send, Database, Users, Shield, CheckCircle, AlertCircle, Loader, Settings, Terminal } from 'lucide-react';
|
| 3 |
|
| 4 |
const UnityCatalogChatbot = () => {
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 5 |
const [messages, setMessages] = useState([
|
| 6 |
{
|
| 7 |
role: 'assistant',
|
|
@@ -12,6 +24,9 @@ const UnityCatalogChatbot = () => {
|
|
| 12 |
const [input, setInput] = useState('');
|
| 13 |
const [isLoading, setIsLoading] = useState(false);
|
| 14 |
const [actionLog, setActionLog] = useState([]);
|
|
|
|
|
|
|
|
|
|
| 15 |
const messagesEndRef = useRef(null);
|
| 16 |
|
| 17 |
const scrollToBottom = () => {
|
|
@@ -22,7 +37,77 @@ const UnityCatalogChatbot = () => {
|
|
| 22 |
scrollToBottom();
|
| 23 |
}, [messages]);
|
| 24 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 25 |
// Parse user intent and extract parameters
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 26 |
const parseIntent = async (userMessage) => {
|
| 27 |
const lowerMsg = userMessage.toLowerCase();
|
| 28 |
|
|
@@ -241,20 +326,33 @@ Just tell me what you'd like to do in natural language!`,
|
|
| 241 |
setIsLoading(true);
|
| 242 |
|
| 243 |
try {
|
| 244 |
-
//
|
| 245 |
-
const
|
| 246 |
-
|
| 247 |
-
|
| 248 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 249 |
|
| 250 |
-
//
|
| 251 |
if (result.sql) {
|
| 252 |
-
|
|
|
|
| 253 |
timestamp: new Date(),
|
| 254 |
sql: result.sql,
|
| 255 |
-
|
| 256 |
-
status: 'success'
|
| 257 |
-
|
|
|
|
|
|
|
|
|
|
| 258 |
}
|
| 259 |
|
| 260 |
// Add assistant response
|
|
@@ -262,14 +360,17 @@ Just tell me what you'd like to do in natural language!`,
|
|
| 262 |
role: 'assistant',
|
| 263 |
content: result.message,
|
| 264 |
sql: result.sql,
|
| 265 |
-
|
|
|
|
|
|
|
| 266 |
};
|
| 267 |
|
| 268 |
setMessages(prev => [...prev, assistantMessage]);
|
| 269 |
} catch (error) {
|
|
|
|
| 270 |
const errorMessage = {
|
| 271 |
role: 'assistant',
|
| 272 |
-
content: `Sorry, I encountered an error: ${error.message}.
|
| 273 |
timestamp: new Date(),
|
| 274 |
isError: true
|
| 275 |
};
|
|
@@ -293,6 +394,251 @@ Just tell me what you'd like to do in natural language!`,
|
|
| 293 |
{ label: 'Help', icon: Settings, prompt: 'help' }
|
| 294 |
];
|
| 295 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 296 |
return (
|
| 297 |
<div style={{
|
| 298 |
minHeight: '100vh',
|
|
@@ -607,85 +953,243 @@ Just tell me what you'd like to do in natural language!`,
|
|
| 607 |
display: 'flex',
|
| 608 |
flexDirection: 'column'
|
| 609 |
}}>
|
|
|
|
| 610 |
<div style={{
|
| 611 |
-
|
|
|
|
| 612 |
borderBottom: '1px solid rgba(100, 255, 218, 0.1)'
|
| 613 |
}}>
|
| 614 |
-
<
|
| 615 |
-
|
| 616 |
-
|
| 617 |
-
|
| 618 |
-
|
| 619 |
-
|
| 620 |
-
|
| 621 |
-
|
| 622 |
-
|
| 623 |
-
|
| 624 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 625 |
ACTION LOG
|
| 626 |
-
</
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 627 |
</div>
|
|
|
|
|
|
|
| 628 |
<div style={{
|
| 629 |
flex: 1,
|
| 630 |
overflowY: 'auto',
|
| 631 |
padding: '1rem'
|
| 632 |
}}>
|
| 633 |
-
{
|
| 634 |
-
<
|
| 635 |
-
|
| 636 |
-
|
| 637 |
-
|
| 638 |
-
|
| 639 |
-
|
| 640 |
-
|
| 641 |
-
|
| 642 |
-
|
| 643 |
-
|
| 644 |
-
|
| 645 |
-
<div
|
| 646 |
-
|
| 647 |
-
|
| 648 |
-
|
| 649 |
-
|
| 650 |
-
|
| 651 |
-
|
| 652 |
-
|
| 653 |
-
|
| 654 |
-
|
| 655 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 656 |
<div style={{
|
| 657 |
display: 'flex',
|
| 658 |
alignItems: 'center',
|
| 659 |
-
gap: '0.
|
| 660 |
-
|
| 661 |
-
color: log.status === 'success' ? '#64ffda' : '#ef4444'
|
| 662 |
}}>
|
| 663 |
-
|
| 664 |
-
|
| 665 |
-
|
| 666 |
-
|
| 667 |
-
|
| 668 |
-
|
| 669 |
-
|
|
|
|
|
|
|
| 670 |
</span>
|
| 671 |
</div>
|
| 672 |
-
|
| 673 |
-
|
| 674 |
-
|
| 675 |
-
|
| 676 |
-
|
| 677 |
-
}}>
|
| 678 |
-
{log.sql}
|
| 679 |
</div>
|
| 680 |
-
<div style={{
|
| 681 |
-
|
| 682 |
-
fontSize: '0.65rem',
|
| 683 |
-
letterSpacing: '0.05em'
|
| 684 |
-
}}>
|
| 685 |
-
{log.timestamp.toLocaleTimeString()}
|
| 686 |
</div>
|
| 687 |
</div>
|
| 688 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 689 |
</div>
|
| 690 |
)}
|
| 691 |
</div>
|
|
@@ -711,6 +1215,11 @@ Just tell me what you'd like to do in natural language!`,
|
|
| 711 |
to { transform: rotate(360deg); }
|
| 712 |
}
|
| 713 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 714 |
* {
|
| 715 |
box-sizing: border-box;
|
| 716 |
}
|
|
|
|
| 1 |
import React, { useState, useRef, useEffect } from 'react';
|
| 2 |
+
import { Send, Database, Users, Shield, CheckCircle, AlertCircle, Loader, Settings, Terminal, Copy, CheckCheck } from 'lucide-react';
|
| 3 |
|
| 4 |
const UnityCatalogChatbot = () => {
|
| 5 |
+
// Connection Setup State
|
| 6 |
+
const [isConnected, setIsConnected] = useState(false);
|
| 7 |
+
const [showSetup, setShowSetup] = useState(true);
|
| 8 |
+
const [setupForm, setSetupForm] = useState({
|
| 9 |
+
host: '',
|
| 10 |
+
token: '',
|
| 11 |
+
workspaceId: ''
|
| 12 |
+
});
|
| 13 |
+
const [setupLoading, setSetupLoading] = useState(false);
|
| 14 |
+
const [setupError, setSetupError] = useState('');
|
| 15 |
+
|
| 16 |
+
// Chat State
|
| 17 |
const [messages, setMessages] = useState([
|
| 18 |
{
|
| 19 |
role: 'assistant',
|
|
|
|
| 24 |
const [input, setInput] = useState('');
|
| 25 |
const [isLoading, setIsLoading] = useState(false);
|
| 26 |
const [actionLog, setActionLog] = useState([]);
|
| 27 |
+
const [activeTab, setActiveTab] = useState('chat'); // 'chat' or 'logs'
|
| 28 |
+
const [copiedId, setCopiedId] = useState(null);
|
| 29 |
+
const [dbxStatus, setDbxStatus] = useState('disconnected'); // 'connected', 'disconnected', 'loading'
|
| 30 |
const messagesEndRef = useRef(null);
|
| 31 |
|
| 32 |
const scrollToBottom = () => {
|
|
|
|
| 37 |
scrollToBottom();
|
| 38 |
}, [messages]);
|
| 39 |
|
| 40 |
+
// Check Databricks connection on mount
|
| 41 |
+
useEffect(() => {
|
| 42 |
+
checkDatabricksConnection();
|
| 43 |
+
}, []);
|
| 44 |
+
|
| 45 |
+
const checkDatabricksConnection = async () => {
|
| 46 |
+
try {
|
| 47 |
+
setDbxStatus('loading');
|
| 48 |
+
const response = await fetch('/api/health');
|
| 49 |
+
if (response.ok) {
|
| 50 |
+
setDbxStatus('connected');
|
| 51 |
+
} else {
|
| 52 |
+
setDbxStatus('disconnected');
|
| 53 |
+
}
|
| 54 |
+
} catch (error) {
|
| 55 |
+
console.error('Connection check failed:', error);
|
| 56 |
+
setDbxStatus('disconnected');
|
| 57 |
+
}
|
| 58 |
+
};
|
| 59 |
+
|
| 60 |
// Parse user intent and extract parameters
|
| 61 |
+
// Connection setup validation
|
| 62 |
+
const handleTestConnection = async () => {
|
| 63 |
+
if (!setupForm.host || !setupForm.token || !setupForm.workspaceId) {
|
| 64 |
+
setSetupError('All fields are required');
|
| 65 |
+
return;
|
| 66 |
+
}
|
| 67 |
+
|
| 68 |
+
setSetupLoading(true);
|
| 69 |
+
setSetupError('');
|
| 70 |
+
|
| 71 |
+
try {
|
| 72 |
+
const response = await fetch('/api/validate-connection', {
|
| 73 |
+
method: 'POST',
|
| 74 |
+
headers: {
|
| 75 |
+
'Content-Type': 'application/json',
|
| 76 |
+
},
|
| 77 |
+
body: JSON.stringify({
|
| 78 |
+
host: setupForm.host,
|
| 79 |
+
token: setupForm.token,
|
| 80 |
+
workspaceId: setupForm.workspaceId
|
| 81 |
+
})
|
| 82 |
+
});
|
| 83 |
+
|
| 84 |
+
const result = await response.json();
|
| 85 |
+
|
| 86 |
+
if (result.success) {
|
| 87 |
+
setIsConnected(true);
|
| 88 |
+
setShowSetup(false);
|
| 89 |
+
setDbxStatus('connected');
|
| 90 |
+
sessionStorage.setItem('dbx_connection', JSON.stringify(setupForm));
|
| 91 |
+
} else {
|
| 92 |
+
setSetupError(result.message || 'Connection failed. Please check your credentials.');
|
| 93 |
+
setDbxStatus('disconnected');
|
| 94 |
+
}
|
| 95 |
+
} catch (error) {
|
| 96 |
+
console.error('Connection test failed:', error);
|
| 97 |
+
setSetupError('Connection error: ' + error.message);
|
| 98 |
+
setDbxStatus('disconnected');
|
| 99 |
+
} finally {
|
| 100 |
+
setSetupLoading(false);
|
| 101 |
+
}
|
| 102 |
+
};
|
| 103 |
+
|
| 104 |
+
const handleSetupInputChange = (field, value) => {
|
| 105 |
+
setSetupForm(prev => ({
|
| 106 |
+
...prev,
|
| 107 |
+
[field]: value
|
| 108 |
+
}));
|
| 109 |
+
setSetupError('');
|
| 110 |
+
};
|
| 111 |
const parseIntent = async (userMessage) => {
|
| 112 |
const lowerMsg = userMessage.toLowerCase();
|
| 113 |
|
|
|
|
| 326 |
setIsLoading(true);
|
| 327 |
|
| 328 |
try {
|
| 329 |
+
// Send to backend API
|
| 330 |
+
const response = await fetch('/api/chat', {
|
| 331 |
+
method: 'POST',
|
| 332 |
+
headers: {
|
| 333 |
+
'Content-Type': 'application/json',
|
| 334 |
+
},
|
| 335 |
+
body: JSON.stringify({ message: input.trim() })
|
| 336 |
+
});
|
| 337 |
+
|
| 338 |
+
if (!response.ok) {
|
| 339 |
+
throw new Error(`API error: ${response.statusText}`);
|
| 340 |
+
}
|
| 341 |
+
|
| 342 |
+
const result = await response.json();
|
| 343 |
|
| 344 |
+
// Log the action if SQL was generated
|
| 345 |
if (result.sql) {
|
| 346 |
+
const logEntry = {
|
| 347 |
+
id: `action-${Date.now()}`,
|
| 348 |
timestamp: new Date(),
|
| 349 |
sql: result.sql,
|
| 350 |
+
intent: result.intent || 'unknown',
|
| 351 |
+
status: result.success ? 'success' : 'failed',
|
| 352 |
+
message: result.message,
|
| 353 |
+
explanation: result.explanation
|
| 354 |
+
};
|
| 355 |
+
setActionLog(prev => [...prev, logEntry]);
|
| 356 |
}
|
| 357 |
|
| 358 |
// Add assistant response
|
|
|
|
| 360 |
role: 'assistant',
|
| 361 |
content: result.message,
|
| 362 |
sql: result.sql,
|
| 363 |
+
intent: result.intent,
|
| 364 |
+
timestamp: new Date(),
|
| 365 |
+
isError: !result.success
|
| 366 |
};
|
| 367 |
|
| 368 |
setMessages(prev => [...prev, assistantMessage]);
|
| 369 |
} catch (error) {
|
| 370 |
+
console.error('Error:', error);
|
| 371 |
const errorMessage = {
|
| 372 |
role: 'assistant',
|
| 373 |
+
content: `Sorry, I encountered an error: ${error.message}. Make sure the backend server is running on /api/chat.`,
|
| 374 |
timestamp: new Date(),
|
| 375 |
isError: true
|
| 376 |
};
|
|
|
|
| 394 |
{ label: 'Help', icon: Settings, prompt: 'help' }
|
| 395 |
];
|
| 396 |
|
| 397 |
+
// Setup Screen - shown before connection
|
| 398 |
+
if (showSetup) {
|
| 399 |
+
return (
|
| 400 |
+
<div style={{
|
| 401 |
+
minHeight: '100vh',
|
| 402 |
+
background: 'linear-gradient(135deg, #0a0e27 0%, #1a1f3a 50%, #2d1b3d 100%)',
|
| 403 |
+
fontFamily: '"Space Mono", "Courier New", monospace',
|
| 404 |
+
color: '#e0e6ed',
|
| 405 |
+
display: 'flex',
|
| 406 |
+
alignItems: 'center',
|
| 407 |
+
justifyContent: 'center',
|
| 408 |
+
padding: '2rem'
|
| 409 |
+
}}>
|
| 410 |
+
<div style={{
|
| 411 |
+
maxWidth: '500px',
|
| 412 |
+
width: '100%',
|
| 413 |
+
background: 'rgba(15, 20, 40, 0.8)',
|
| 414 |
+
backdropFilter: 'blur(12px)',
|
| 415 |
+
borderRadius: '16px',
|
| 416 |
+
border: '1px solid rgba(100, 255, 218, 0.2)',
|
| 417 |
+
padding: '3rem 2rem',
|
| 418 |
+
boxShadow: '0 8px 32px rgba(0, 0, 0, 0.3)'
|
| 419 |
+
}}>
|
| 420 |
+
<div style={{ textAlign: 'center', marginBottom: '2rem' }}>
|
| 421 |
+
<div style={{
|
| 422 |
+
fontSize: '3rem',
|
| 423 |
+
marginBottom: '1rem'
|
| 424 |
+
}}>
|
| 425 |
+
🔌
|
| 426 |
+
</div>
|
| 427 |
+
<h1 style={{
|
| 428 |
+
margin: 0,
|
| 429 |
+
fontSize: '1.8rem',
|
| 430 |
+
fontWeight: 700,
|
| 431 |
+
background: 'linear-gradient(135deg, #64ffda 0%, #8892ff 100%)',
|
| 432 |
+
WebkitBackgroundClip: 'text',
|
| 433 |
+
WebkitTextFillColor: 'transparent',
|
| 434 |
+
letterSpacing: '0.05em'
|
| 435 |
+
}}>
|
| 436 |
+
Connect to Databricks
|
| 437 |
+
</h1>
|
| 438 |
+
<p style={{
|
| 439 |
+
margin: '0.5rem 0 0 0',
|
| 440 |
+
fontSize: '0.9rem',
|
| 441 |
+
color: '#8892b0',
|
| 442 |
+
letterSpacing: '0.05em'
|
| 443 |
+
}}>
|
| 444 |
+
Enter your workspace credentials
|
| 445 |
+
</p>
|
| 446 |
+
</div>
|
| 447 |
+
|
| 448 |
+
<div style={{ marginBottom: '1.5rem' }}>
|
| 449 |
+
<label style={{
|
| 450 |
+
display: 'block',
|
| 451 |
+
marginBottom: '0.5rem',
|
| 452 |
+
fontSize: '0.85rem',
|
| 453 |
+
color: '#64ffda',
|
| 454 |
+
fontWeight: 600,
|
| 455 |
+
letterSpacing: '0.05em'
|
| 456 |
+
}}>
|
| 457 |
+
DATABRICKS HOST
|
| 458 |
+
</label>
|
| 459 |
+
<input
|
| 460 |
+
type="url"
|
| 461 |
+
placeholder="https://your-workspace.cloud.databricks.com"
|
| 462 |
+
value={setupForm.host}
|
| 463 |
+
onChange={(e) => handleSetupInputChange('host', e.target.value)}
|
| 464 |
+
style={{
|
| 465 |
+
width: '100%',
|
| 466 |
+
padding: '0.75rem',
|
| 467 |
+
background: 'rgba(10, 14, 39, 0.6)',
|
| 468 |
+
border: '1px solid rgba(100, 255, 218, 0.2)',
|
| 469 |
+
borderRadius: '8px',
|
| 470 |
+
color: '#e0e6ed',
|
| 471 |
+
fontFamily: 'inherit',
|
| 472 |
+
fontSize: '0.9rem',
|
| 473 |
+
boxSizing: 'border-box',
|
| 474 |
+
transition: 'all 0.2s ease'
|
| 475 |
+
}}
|
| 476 |
+
onFocus={(e) => {
|
| 477 |
+
e.target.style.borderColor = 'rgba(100, 255, 218, 0.5)';
|
| 478 |
+
e.target.style.boxShadow = '0 0 12px rgba(100, 255, 218, 0.2)';
|
| 479 |
+
}}
|
| 480 |
+
onBlur={(e) => {
|
| 481 |
+
e.target.style.borderColor = 'rgba(100, 255, 218, 0.2)';
|
| 482 |
+
e.target.style.boxShadow = 'none';
|
| 483 |
+
}}
|
| 484 |
+
/>
|
| 485 |
+
</div>
|
| 486 |
+
|
| 487 |
+
<div style={{ marginBottom: '1.5rem' }}>
|
| 488 |
+
<label style={{
|
| 489 |
+
display: 'block',
|
| 490 |
+
marginBottom: '0.5rem',
|
| 491 |
+
fontSize: '0.85rem',
|
| 492 |
+
color: '#64ffda',
|
| 493 |
+
fontWeight: 600,
|
| 494 |
+
letterSpacing: '0.05em'
|
| 495 |
+
}}>
|
| 496 |
+
DATABRICKS TOKEN
|
| 497 |
+
</label>
|
| 498 |
+
<input
|
| 499 |
+
type="password"
|
| 500 |
+
placeholder="dapi... (Your API token)"
|
| 501 |
+
value={setupForm.token}
|
| 502 |
+
onChange={(e) => handleSetupInputChange('token', e.target.value)}
|
| 503 |
+
style={{
|
| 504 |
+
width: '100%',
|
| 505 |
+
padding: '0.75rem',
|
| 506 |
+
background: 'rgba(10, 14, 39, 0.6)',
|
| 507 |
+
border: '1px solid rgba(100, 255, 218, 0.2)',
|
| 508 |
+
borderRadius: '8px',
|
| 509 |
+
color: '#e0e6ed',
|
| 510 |
+
fontFamily: 'inherit',
|
| 511 |
+
fontSize: '0.9rem',
|
| 512 |
+
boxSizing: 'border-box',
|
| 513 |
+
transition: 'all 0.2s ease'
|
| 514 |
+
}}
|
| 515 |
+
onFocus={(e) => {
|
| 516 |
+
e.target.style.borderColor = 'rgba(100, 255, 218, 0.5)';
|
| 517 |
+
e.target.style.boxShadow = '0 0 12px rgba(100, 255, 218, 0.2)';
|
| 518 |
+
}}
|
| 519 |
+
onBlur={(e) => {
|
| 520 |
+
e.target.style.borderColor = 'rgba(100, 255, 218, 0.2)';
|
| 521 |
+
e.target.style.boxShadow = 'none';
|
| 522 |
+
}}
|
| 523 |
+
/>
|
| 524 |
+
</div>
|
| 525 |
+
|
| 526 |
+
<div style={{ marginBottom: '2rem' }}>
|
| 527 |
+
<label style={{
|
| 528 |
+
display: 'block',
|
| 529 |
+
marginBottom: '0.5rem',
|
| 530 |
+
fontSize: '0.85rem',
|
| 531 |
+
color: '#64ffda',
|
| 532 |
+
fontWeight: 600,
|
| 533 |
+
letterSpacing: '0.05em'
|
| 534 |
+
}}>
|
| 535 |
+
WORKSPACE ID (Optional)
|
| 536 |
+
</label>
|
| 537 |
+
<input
|
| 538 |
+
type="text"
|
| 539 |
+
placeholder="Workspace ID"
|
| 540 |
+
value={setupForm.workspaceId}
|
| 541 |
+
onChange={(e) => handleSetupInputChange('workspaceId', e.target.value)}
|
| 542 |
+
style={{
|
| 543 |
+
width: '100%',
|
| 544 |
+
padding: '0.75rem',
|
| 545 |
+
background: 'rgba(10, 14, 39, 0.6)',
|
| 546 |
+
border: '1px solid rgba(100, 255, 218, 0.2)',
|
| 547 |
+
borderRadius: '8px',
|
| 548 |
+
color: '#e0e6ed',
|
| 549 |
+
fontFamily: 'inherit',
|
| 550 |
+
fontSize: '0.9rem',
|
| 551 |
+
boxSizing: 'border-box',
|
| 552 |
+
transition: 'all 0.2s ease'
|
| 553 |
+
}}
|
| 554 |
+
onFocus={(e) => {
|
| 555 |
+
e.target.style.borderColor = 'rgba(100, 255, 218, 0.5)';
|
| 556 |
+
e.target.style.boxShadow = '0 0 12px rgba(100, 255, 218, 0.2)';
|
| 557 |
+
}}
|
| 558 |
+
onBlur={(e) => {
|
| 559 |
+
e.target.style.borderColor = 'rgba(100, 255, 218, 0.2)';
|
| 560 |
+
e.target.style.boxShadow = 'none';
|
| 561 |
+
}}
|
| 562 |
+
/>
|
| 563 |
+
</div>
|
| 564 |
+
|
| 565 |
+
{setupError && (
|
| 566 |
+
<div style={{
|
| 567 |
+
background: 'rgba(239, 68, 68, 0.1)',
|
| 568 |
+
border: '1px solid rgba(239, 68, 68, 0.3)',
|
| 569 |
+
borderRadius: '8px',
|
| 570 |
+
padding: '0.75rem 1rem',
|
| 571 |
+
marginBottom: '1.5rem',
|
| 572 |
+
color: '#ef4444',
|
| 573 |
+
fontSize: '0.85rem',
|
| 574 |
+
display: 'flex',
|
| 575 |
+
alignItems: 'center',
|
| 576 |
+
gap: '0.75rem'
|
| 577 |
+
}}>
|
| 578 |
+
<span>⚠️</span>
|
| 579 |
+
<span>{setupError}</span>
|
| 580 |
+
</div>
|
| 581 |
+
)}
|
| 582 |
+
|
| 583 |
+
<button
|
| 584 |
+
onClick={handleTestConnection}
|
| 585 |
+
disabled={setupLoading}
|
| 586 |
+
style={{
|
| 587 |
+
width: '100%',
|
| 588 |
+
padding: '0.875rem',
|
| 589 |
+
background: setupLoading
|
| 590 |
+
? 'rgba(100, 255, 218, 0.1)'
|
| 591 |
+
: 'linear-gradient(135deg, #64ffda 0%, #00bfa5 100%)',
|
| 592 |
+
border: 'none',
|
| 593 |
+
borderRadius: '12px',
|
| 594 |
+
color: setupLoading ? '#8892b0' : '#0a0e27',
|
| 595 |
+
fontSize: '0.95rem',
|
| 596 |
+
fontFamily: 'inherit',
|
| 597 |
+
fontWeight: 700,
|
| 598 |
+
cursor: setupLoading ? 'not-allowed' : 'pointer',
|
| 599 |
+
transition: 'all 0.2s ease',
|
| 600 |
+
letterSpacing: '0.05em',
|
| 601 |
+
boxShadow: setupLoading ? 'none' : '0 4px 16px rgba(100, 255, 218, 0.3)'
|
| 602 |
+
}}
|
| 603 |
+
onMouseEnter={(e) => {
|
| 604 |
+
if (!setupLoading) {
|
| 605 |
+
e.target.style.transform = 'translateY(-2px)';
|
| 606 |
+
e.target.style.boxShadow = '0 6px 20px rgba(100, 255, 218, 0.4)';
|
| 607 |
+
}
|
| 608 |
+
}}
|
| 609 |
+
onMouseLeave={(e) => {
|
| 610 |
+
if (!setupLoading) {
|
| 611 |
+
e.target.style.transform = 'translateY(0)';
|
| 612 |
+
e.target.style.boxShadow = '0 4px 16px rgba(100, 255, 218, 0.3)';
|
| 613 |
+
}
|
| 614 |
+
}}
|
| 615 |
+
>
|
| 616 |
+
{setupLoading ? '🔄 Testing...' : '✓ Connect'}
|
| 617 |
+
</button>
|
| 618 |
+
|
| 619 |
+
<div style={{
|
| 620 |
+
marginTop: '1.5rem',
|
| 621 |
+
padding: '1rem',
|
| 622 |
+
background: 'rgba(100, 255, 218, 0.05)',
|
| 623 |
+
border: '1px solid rgba(100, 255, 218, 0.1)',
|
| 624 |
+
borderRadius: '8px',
|
| 625 |
+
fontSize: '0.8rem',
|
| 626 |
+
color: '#8892b0',
|
| 627 |
+
lineHeight: '1.5'
|
| 628 |
+
}}>
|
| 629 |
+
<strong style={{ color: '#64ffda', display: 'block', marginBottom: '0.5rem' }}>How to get your credentials:</strong>
|
| 630 |
+
<ol style={{ margin: '0.5rem 0', paddingLeft: '1.25rem' }}>
|
| 631 |
+
<li>Go to Databricks workspace URL</li>
|
| 632 |
+
<li>Create token in Settings → Developer → Tokens</li>
|
| 633 |
+
<li>Copy your token (dapi...)</li>
|
| 634 |
+
<li>Find workspace ID in URL or settings</li>
|
| 635 |
+
</ol>
|
| 636 |
+
</div>
|
| 637 |
+
</div>
|
| 638 |
+
</div>
|
| 639 |
+
);
|
| 640 |
+
}
|
| 641 |
+
|
| 642 |
return (
|
| 643 |
<div style={{
|
| 644 |
minHeight: '100vh',
|
|
|
|
| 953 |
display: 'flex',
|
| 954 |
flexDirection: 'column'
|
| 955 |
}}>
|
| 956 |
+
{/* Tabs */}
|
| 957 |
<div style={{
|
| 958 |
+
display: 'grid',
|
| 959 |
+
gridTemplateColumns: '1fr 1fr',
|
| 960 |
borderBottom: '1px solid rgba(100, 255, 218, 0.1)'
|
| 961 |
}}>
|
| 962 |
+
<button
|
| 963 |
+
onClick={() => setActiveTab('logs')}
|
| 964 |
+
style={{
|
| 965 |
+
padding: '1rem',
|
| 966 |
+
background: activeTab === 'logs' ? 'rgba(100, 255, 218, 0.1)' : 'transparent',
|
| 967 |
+
border: 'none',
|
| 968 |
+
borderBottom: activeTab === 'logs' ? '2px solid #64ffda' : 'none',
|
| 969 |
+
color: activeTab === 'logs' ? '#64ffda' : '#8892b0',
|
| 970 |
+
fontSize: '0.85rem',
|
| 971 |
+
fontFamily: 'inherit',
|
| 972 |
+
fontWeight: 600,
|
| 973 |
+
cursor: 'pointer',
|
| 974 |
+
transition: 'all 0.2s ease',
|
| 975 |
+
letterSpacing: '0.05em'
|
| 976 |
+
}}
|
| 977 |
+
onMouseEnter={(e) => {
|
| 978 |
+
if (activeTab !== 'logs') e.currentTarget.style.color = '#64ffda';
|
| 979 |
+
}}
|
| 980 |
+
onMouseLeave={(e) => {
|
| 981 |
+
if (activeTab !== 'logs') e.currentTarget.style.color = '#8892b0';
|
| 982 |
+
}}
|
| 983 |
+
>
|
| 984 |
ACTION LOG
|
| 985 |
+
</button>
|
| 986 |
+
<button
|
| 987 |
+
onClick={() => setActiveTab('status')}
|
| 988 |
+
style={{
|
| 989 |
+
padding: '1rem',
|
| 990 |
+
background: activeTab === 'status' ? 'rgba(100, 255, 218, 0.1)' : 'transparent',
|
| 991 |
+
border: 'none',
|
| 992 |
+
borderBottom: activeTab === 'status' ? '2px solid #64ffda' : 'none',
|
| 993 |
+
color: activeTab === 'status' ? '#64ffda' : '#8892b0',
|
| 994 |
+
fontSize: '0.85rem',
|
| 995 |
+
fontFamily: 'inherit',
|
| 996 |
+
fontWeight: 600,
|
| 997 |
+
cursor: 'pointer',
|
| 998 |
+
transition: 'all 0.2s ease',
|
| 999 |
+
letterSpacing: '0.05em'
|
| 1000 |
+
}}
|
| 1001 |
+
onMouseEnter={(e) => {
|
| 1002 |
+
if (activeTab !== 'status') e.currentTarget.style.color = '#64ffda';
|
| 1003 |
+
}}
|
| 1004 |
+
onMouseLeave={(e) => {
|
| 1005 |
+
if (activeTab !== 'status') e.currentTarget.style.color = '#8892b0';
|
| 1006 |
+
}}
|
| 1007 |
+
>
|
| 1008 |
+
STATUS
|
| 1009 |
+
</button>
|
| 1010 |
</div>
|
| 1011 |
+
|
| 1012 |
+
{/* Tab Content */}
|
| 1013 |
<div style={{
|
| 1014 |
flex: 1,
|
| 1015 |
overflowY: 'auto',
|
| 1016 |
padding: '1rem'
|
| 1017 |
}}>
|
| 1018 |
+
{activeTab === 'logs' && (
|
| 1019 |
+
<>
|
| 1020 |
+
{actionLog.length === 0 ? (
|
| 1021 |
+
<div style={{
|
| 1022 |
+
textAlign: 'center',
|
| 1023 |
+
padding: '2rem 1rem',
|
| 1024 |
+
color: '#8892b0',
|
| 1025 |
+
fontSize: '0.85rem'
|
| 1026 |
+
}}>
|
| 1027 |
+
No actions yet. Start a conversation to see executed commands here.
|
| 1028 |
+
</div>
|
| 1029 |
+
) : (
|
| 1030 |
+
<div style={{ display: 'flex', flexDirection: 'column', gap: '0.75rem' }}>
|
| 1031 |
+
{actionLog.slice().reverse().map((log, idx) => (
|
| 1032 |
+
<div
|
| 1033 |
+
key={log.id || idx}
|
| 1034 |
+
style={{
|
| 1035 |
+
background: 'rgba(10, 14, 39, 0.6)',
|
| 1036 |
+
border: '1px solid rgba(100, 255, 218, 0.2)',
|
| 1037 |
+
borderRadius: '8px',
|
| 1038 |
+
padding: '0.75rem',
|
| 1039 |
+
fontSize: '0.75rem',
|
| 1040 |
+
animation: 'slideIn 0.3s ease-out'
|
| 1041 |
+
}}
|
| 1042 |
+
>
|
| 1043 |
+
<div style={{
|
| 1044 |
+
display: 'flex',
|
| 1045 |
+
alignItems: 'center',
|
| 1046 |
+
justifyContent: 'space-between',
|
| 1047 |
+
marginBottom: '0.5rem',
|
| 1048 |
+
color: log.status === 'success' ? '#64ffda' : '#ef4444'
|
| 1049 |
+
}}>
|
| 1050 |
+
<div style={{ display: 'flex', alignItems: 'center', gap: '0.5rem' }}>
|
| 1051 |
+
{log.status === 'success' ? (
|
| 1052 |
+
<CheckCircle size={12} />
|
| 1053 |
+
) : (
|
| 1054 |
+
<AlertCircle size={12} />
|
| 1055 |
+
)}
|
| 1056 |
+
<span style={{ letterSpacing: '0.05em' }}>
|
| 1057 |
+
{log.intent.toUpperCase()}
|
| 1058 |
+
</span>
|
| 1059 |
+
</div>
|
| 1060 |
+
<button
|
| 1061 |
+
onClick={() => {
|
| 1062 |
+
navigator.clipboard.writeText(log.sql);
|
| 1063 |
+
setCopiedId(log.id);
|
| 1064 |
+
setTimeout(() => setCopiedId(null), 2000);
|
| 1065 |
+
}}
|
| 1066 |
+
style={{
|
| 1067 |
+
background: 'transparent',
|
| 1068 |
+
border: 'none',
|
| 1069 |
+
color: copiedId === log.id ? '#64ffda' : '#8892b0',
|
| 1070 |
+
cursor: 'pointer',
|
| 1071 |
+
padding: '0.25rem',
|
| 1072 |
+
display: 'flex',
|
| 1073 |
+
alignItems: 'center',
|
| 1074 |
+
gap: '0.25rem'
|
| 1075 |
+
}}
|
| 1076 |
+
title="Copy SQL"
|
| 1077 |
+
>
|
| 1078 |
+
{copiedId === log.id ? (
|
| 1079 |
+
<CheckCheck size={10} />
|
| 1080 |
+
) : (
|
| 1081 |
+
<Copy size={10} />
|
| 1082 |
+
)}
|
| 1083 |
+
</button>
|
| 1084 |
+
</div>
|
| 1085 |
+
<div style={{
|
| 1086 |
+
background: 'rgba(15, 20, 40, 0.8)',
|
| 1087 |
+
border: '1px solid rgba(100, 255, 218, 0.1)',
|
| 1088 |
+
borderRadius: '4px',
|
| 1089 |
+
padding: '0.5rem',
|
| 1090 |
+
color: '#64ffda',
|
| 1091 |
+
fontSize: '0.65rem',
|
| 1092 |
+
marginBottom: '0.5rem',
|
| 1093 |
+
wordBreak: 'break-word',
|
| 1094 |
+
fontFamily: '"Fira Code", "Courier New", monospace',
|
| 1095 |
+
maxHeight: '80px',
|
| 1096 |
+
overflowY: 'auto'
|
| 1097 |
+
}}>
|
| 1098 |
+
<code>{log.sql}</code>
|
| 1099 |
+
</div>
|
| 1100 |
+
<div style={{
|
| 1101 |
+
color: '#64748b',
|
| 1102 |
+
fontSize: '0.6rem',
|
| 1103 |
+
letterSpacing: '0.05em'
|
| 1104 |
+
}}>
|
| 1105 |
+
{log.timestamp.toLocaleTimeString()}
|
| 1106 |
+
</div>
|
| 1107 |
+
</div>
|
| 1108 |
+
))}
|
| 1109 |
+
</div>
|
| 1110 |
+
)}
|
| 1111 |
+
</>
|
| 1112 |
+
)}
|
| 1113 |
+
|
| 1114 |
+
{activeTab === 'status' && (
|
| 1115 |
+
<div style={{ padding: '0.5rem 0' }}>
|
| 1116 |
+
<div style={{
|
| 1117 |
+
background: 'rgba(10, 14, 39, 0.6)',
|
| 1118 |
+
border: '1px solid rgba(100, 255, 218, 0.2)',
|
| 1119 |
+
borderRadius: '8px',
|
| 1120 |
+
padding: '1rem',
|
| 1121 |
+
fontSize: '0.85rem',
|
| 1122 |
+
lineHeight: '1.6'
|
| 1123 |
+
}}>
|
| 1124 |
+
<div style={{ marginBottom: '1rem' }}>
|
| 1125 |
+
<div style={{ color: '#8892b0', fontSize: '0.75rem', letterSpacing: '0.05em', marginBottom: '0.5rem' }}>
|
| 1126 |
+
DATABRICKS CONNECTION
|
| 1127 |
+
</div>
|
| 1128 |
<div style={{
|
| 1129 |
display: 'flex',
|
| 1130 |
alignItems: 'center',
|
| 1131 |
+
gap: '0.75rem',
|
| 1132 |
+
color: dbxStatus === 'connected' ? '#64ffda' : dbxStatus === 'loading' ? '#8892b0' : '#ef4444'
|
|
|
|
| 1133 |
}}>
|
| 1134 |
+
<div style={{
|
| 1135 |
+
width: '12px',
|
| 1136 |
+
height: '12px',
|
| 1137 |
+
borderRadius: '50%',
|
| 1138 |
+
background: dbxStatus === 'connected' ? '#64ffda' : dbxStatus === 'loading' ? '#8892b0' : '#ef4444',
|
| 1139 |
+
animation: dbxStatus === 'loading' ? 'pulse 2s ease-in-out infinite' : 'none'
|
| 1140 |
+
}} />
|
| 1141 |
+
<span style={{ textTransform: 'capitalize', fontSize: '0.9rem', fontWeight: 600 }}>
|
| 1142 |
+
{dbxStatus}
|
| 1143 |
</span>
|
| 1144 |
</div>
|
| 1145 |
+
</div>
|
| 1146 |
+
|
| 1147 |
+
<div style={{ marginBottom: '1rem' }}>
|
| 1148 |
+
<div style={{ color: '#8892b0', fontSize: '0.75rem', letterSpacing: '0.05em', marginBottom: '0.5rem' }}>
|
| 1149 |
+
ACTIONS EXECUTED
|
|
|
|
|
|
|
| 1150 |
</div>
|
| 1151 |
+
<div style={{ fontSize: '1.5rem', color: '#64ffda', fontWeight: 700 }}>
|
| 1152 |
+
{actionLog.length}
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1153 |
</div>
|
| 1154 |
</div>
|
| 1155 |
+
|
| 1156 |
+
<div style={{ marginBottom: '1rem' }}>
|
| 1157 |
+
<div style={{ color: '#8892b0', fontSize: '0.75rem', letterSpacing: '0.05em', marginBottom: '0.5rem' }}>
|
| 1158 |
+
LAST ACTION
|
| 1159 |
+
</div>
|
| 1160 |
+
<div style={{ color: '#e0e6ed', fontSize: '0.85rem' }}>
|
| 1161 |
+
{actionLog.length > 0
|
| 1162 |
+
? actionLog[actionLog.length - 1].timestamp.toLocaleString()
|
| 1163 |
+
: 'No actions yet'}
|
| 1164 |
+
</div>
|
| 1165 |
+
</div>
|
| 1166 |
+
|
| 1167 |
+
<button
|
| 1168 |
+
onClick={checkDatabricksConnection}
|
| 1169 |
+
style={{
|
| 1170 |
+
width: '100%',
|
| 1171 |
+
padding: '0.75rem',
|
| 1172 |
+
background: 'rgba(100, 255, 218, 0.1)',
|
| 1173 |
+
border: '1px solid rgba(100, 255, 218, 0.3)',
|
| 1174 |
+
borderRadius: '6px',
|
| 1175 |
+
color: '#64ffda',
|
| 1176 |
+
fontSize: '0.85rem',
|
| 1177 |
+
cursor: 'pointer',
|
| 1178 |
+
fontFamily: 'inherit',
|
| 1179 |
+
fontWeight: 600,
|
| 1180 |
+
letterSpacing: '0.05em',
|
| 1181 |
+
transition: 'all 0.2s ease'
|
| 1182 |
+
}}
|
| 1183 |
+
onMouseEnter={(e) => {
|
| 1184 |
+
e.currentTarget.style.background = 'rgba(100, 255, 218, 0.2)';
|
| 1185 |
+
}}
|
| 1186 |
+
onMouseLeave={(e) => {
|
| 1187 |
+
e.currentTarget.style.background = 'rgba(100, 255, 218, 0.1)';
|
| 1188 |
+
}}
|
| 1189 |
+
>
|
| 1190 |
+
REFRESH STATUS
|
| 1191 |
+
</button>
|
| 1192 |
+
</div>
|
| 1193 |
</div>
|
| 1194 |
)}
|
| 1195 |
</div>
|
|
|
|
| 1215 |
to { transform: rotate(360deg); }
|
| 1216 |
}
|
| 1217 |
|
| 1218 |
+
@keyframes pulse {
|
| 1219 |
+
0%, 100% { opacity: 1; }
|
| 1220 |
+
50% { opacity: 0.5; }
|
| 1221 |
+
}
|
| 1222 |
+
|
| 1223 |
* {
|
| 1224 |
box-sizing: border-box;
|
| 1225 |
}
|