README_HF_SPACE.md

---
title: Job Failure Prediction & Anomaly Detection API
emoji: 🔮
colorFrom: blue
colorTo: purple
sdk: docker
sdk_version: "latest"
app_file: app.py
pinned: false
---

# Job Failure Prediction & Anomaly Detection API

Production-ready ML service for predicting job failures and detecting anomalies in job execution data.

## 🚀 Quick Start

The API is automatically deployed and available at:
```
https://[your-space-name].hf.space
```

## 📡 API Endpoints

### Health Check
```bash
GET /health
```

### Job Failure Prediction
```bash
POST /predict/job-fail
Content-Type: application/json

{
  "zone": "prod",
  "job_nm": "daily_export_customer",
  "job_start_time": "2026-01-21T01:00:00",
  "duration_sec": 5400,
  "status": "SUCCESS",
  "err_msg": "",
  "explain": true
}
```

**Response:**
```json
{
  "fail_probability": 0.79,
  "risk_level": "MEDIUM",
  "top_drivers": [
    {
      "feature": "failure_rate_7",
      "shap_value": 0.30,
      "effect": "increase"
    }
  ],
  "recommended_actions": [
    "Monitor upstream dependencies and recent job history"
  ]
}
```

### Anomaly Detection
```bash
POST /detect/anomaly
Content-Type: application/json

{
  "features": {
    "duration_sec": 5400,
    "duration_zscore": 1.6,
    "err_msg_len": 0
  },
  "threshold": 0.01
}
```

**Response:**
```json
{
  "reconstruction_error": 0.0235,
  "is_anomaly": true,
  "threshold": 0.01,
  "top_drivers": [
    {
      "feature": "duration_zscore",
      "error": 0.0142
    }
  ]
}
```

## 📚 API Documentation

Interactive API documentation is available at:
- Swagger UI: `/docs`
- ReDoc: `/redoc`

## 🎯 Features

- **Job Failure Prediction**: XGBoost-based classifier with SHAP explainability
- **Anomaly Detection**: Autoencoder-based unsupervised anomaly detection
- **FastAPI REST API**: Production-ready endpoints
- **Docker Deployment**: Containerized for Hugging Face Spaces

## 📊 Risk Levels

- **MINIMAL**: `fail_probability < 0.3`
- **LOW**: `0.3 <= fail_probability < 0.5`
- **MEDIUM**: `0.5 <= fail_probability < 0.8`
- **CRITICAL**: `fail_probability >= 0.8`

## 🔧 Model Details

### Job Failure Prediction
- **Algorithm**: XGBoost Classifier
- **Preprocessing**: StandardScaler for numeric, OneHotEncoder for categorical
- **Explainability**: SHAP values for feature importance

### Anomaly Detection
- **Algorithm**: Autoencoder (TensorFlow/Keras)
- **Architecture**: Input → 64 → 32 → 64 → Output
- **Threshold**: Per-job 97th percentile or global threshold

## 📝 Example Usage

### Python
```python
import requests

url = "https://[your-space-name].hf.space/predict/job-fail"
response = requests.post(url, json={
    "zone": "prod",
    "job_nm": "daily_export",
    "job_start_time": "2026-01-21T01:00:00",
    "duration_sec": 5400,
    "status": "SUCCESS",
    "explain": True
})
print(response.json())
```

### cURL
```bash
curl -X POST https://[your-space-name].hf.space/predict/job-fail \
  -H "Content-Type: application/json" \
  -d '{
    "zone": "prod",
    "job_nm": "daily_export",
    "job_start_time": "2026-01-21T01:00:00",
    "duration_sec": 5400,
    "status": "SUCCESS"
  }'
```

## 📖 Full Documentation

See the main [README.md](README.md) for complete documentation, training instructions, and local development setup.

## ⚙️ Configuration

Models are loaded from the `models/` directory. Ensure all required model files are committed to the repository:
- `job_fail_pipeline_cpu.joblib`
- `anomaly_autoencoder_cpu.keras`
- `anomaly_scaler.joblib`
- `feature_schema.json`
- `shap_background.npy`
- `anomaly_features.joblib`
- `anomaly_threshold.joblib`

## 🔒 Production Ready

- Health check endpoint for monitoring
- Error handling and validation
- CORS enabled for cross-origin requests
- Dockerized for consistent deployment
- Optimized for Hugging Face Spaces infrastructure