README_HF_SPACE.md

metadata

title: Job Failure Prediction & Anomaly Detection API
emoji: 🔮
colorFrom: blue
colorTo: purple
sdk: docker
sdk_version: latest
app_file: app.py
pinned: false

Job Failure Prediction & Anomaly Detection API

Production-ready ML service for predicting job failures and detecting anomalies in job execution data.

🚀 Quick Start

The API is automatically deployed and available at:

https://[your-space-name].hf.space

📡 API Endpoints

Health Check

GET /health

Job Failure Prediction

POST /predict/job-fail
Content-Type: application/json

{
  "zone": "prod",
  "job_nm": "daily_export_customer",
  "job_start_time": "2026-01-21T01:00:00",
  "duration_sec": 5400,
  "status": "SUCCESS",
  "err_msg": "",
  "explain": true
}

Response:

{
  "fail_probability": 0.79,
  "risk_level": "MEDIUM",
  "top_drivers": [
    {
      "feature": "failure_rate_7",
      "shap_value": 0.30,
      "effect": "increase"
    }
  ],
  "recommended_actions": [
    "Monitor upstream dependencies and recent job history"
  ]
}

Anomaly Detection

POST /detect/anomaly
Content-Type: application/json

{
  "features": {
    "duration_sec": 5400,
    "duration_zscore": 1.6,
    "err_msg_len": 0
  },
  "threshold": 0.01
}

Response:

{
  "reconstruction_error": 0.0235,
  "is_anomaly": true,
  "threshold": 0.01,
  "top_drivers": [
    {
      "feature": "duration_zscore",
      "error": 0.0142
    }
  ]
}

📚 API Documentation

Interactive API documentation is available at:

• Swagger UI: /docs
• ReDoc: /redoc

🎯 Features

• Job Failure Prediction: XGBoost-based classifier with SHAP explainability
• Anomaly Detection: Autoencoder-based unsupervised anomaly detection
• FastAPI REST API: Production-ready endpoints
• Docker Deployment: Containerized for Hugging Face Spaces

📊 Risk Levels

• MINIMAL: fail_probability < 0.3
• LOW: 0.3 <= fail_probability < 0.5
• MEDIUM: 0.5 <= fail_probability < 0.8
• CRITICAL: fail_probability >= 0.8

🔧 Model Details

Job Failure Prediction

• Algorithm: XGBoost Classifier
• Preprocessing: StandardScaler for numeric, OneHotEncoder for categorical
• Explainability: SHAP values for feature importance

Anomaly Detection

• Algorithm: Autoencoder (TensorFlow/Keras)
• Architecture: Input → 64 → 32 → 64 → Output
• Threshold: Per-job 97th percentile or global threshold

📝 Example Usage

Python

import requests

url = "https://[your-space-name].hf.space/predict/job-fail"
response = requests.post(url, json={
    "zone": "prod",
    "job_nm": "daily_export",
    "job_start_time": "2026-01-21T01:00:00",
    "duration_sec": 5400,
    "status": "SUCCESS",
    "explain": True
})
print(response.json())

cURL

curl -X POST https://[your-space-name].hf.space/predict/job-fail \
  -H "Content-Type: application/json" \
  -d '{
    "zone": "prod",
    "job_nm": "daily_export",
    "job_start_time": "2026-01-21T01:00:00",
    "duration_sec": 5400,
    "status": "SUCCESS"
  }'

📖 Full Documentation

See the main README.md for complete documentation, training instructions, and local development setup.

⚙️ Configuration

Models are loaded from the models/ directory. Ensure all required model files are committed to the repository:

• job_fail_pipeline_cpu.joblib
• anomaly_autoencoder_cpu.keras
• anomaly_scaler.joblib
• feature_schema.json
• shap_background.npy
• anomaly_features.joblib
• anomaly_threshold.joblib

🔒 Production Ready

• Health check endpoint for monitoring
• Error handling and validation
• CORS enabled for cross-origin requests
• Dockerized for consistent deployment
• Optimized for Hugging Face Spaces infrastructure