n8n_workflow_examples.md

n8n Workflow Examples for Job Failure Prediction

This document provides examples for integrating the ML service with n8n workflows.

Service Endpoints

• Base URL: http://job-ml:8000 (internal) or http://localhost:8000 (local)
• Health Check: GET /health
• Job Failure Prediction: POST /predict/job-fail
• Anomaly Detection: POST /detect/anomaly

Example 1: Job Failure Prediction Workflow

Workflow Structure

1. Trigger: Database query / Webhook / Schedule
2. HTTP Request: Call prediction endpoint
3. IF Node: Check risk level
4. Action: Send alert (Slack / PagerDuty / Email)

HTTP Request Node Configuration

Method: POST
URL: http://job-ml:8000/predict/job-fail
Headers:

Content-Type: application/json

Body (JSON):

{
  "zone": "{{ $json.zone }}",
  "job_nm": "{{ $json.job_nm }}",
  "tasksgroup_nm": "{{ $json.tasksgroup_nm }}",
  "job_start_time": "{{ $json.job_start_time }}",
  "duration": "{{ $json.duration }}",
  "duration_sec": {{ $json.duration_sec }},
  "status": "{{ $json.status }}",
  "err_msg": "{{ $json.err_msg }}",
  "zeppelin": "{{ $json.zeppelin }}",
  "explain": true
}

IF Node Conditions

WARNING Condition (Medium Risk):

{{ $json.fail_probability >= 0.5 && $json.fail_probability < 0.8 }}

CRITICAL Condition (High Risk):

{{ $json.fail_probability >= 0.8 }}

Combined Alert Condition (WARNING or CRITICAL):

{{ $json.fail_probability >= 0.5 }}

Example Response Handling

The response will look like:

{
  "fail_probability": 0.79,
  "risk_level": "MEDIUM",
  "top_drivers": [
    {
      "feature": "failure_rate_7",
      "shap_value": 0.30,
      "effect": "increase"
    },
    {
      "feature": "duration_zscore",
      "shap_value": 0.18,
      "effect": "increase"
    }
  ],
  "recommended_actions": [
    "Monitor upstream dependencies and recent job history",
    "Check for resource constraints or data volume spikes"
  ]
}

Slack Alert Example

Slack Node Configuration:

• Channel: #job-alerts
• Text:

🚨 Job Failure Alert

Job: {{ $json.job_nm }}
Zone: {{ $json.zone }}
Risk Level: *{{ $('HTTP Request').item.json.risk_level }}*
Failure Probability: {{ $('HTTP Request').item.json.fail_probability * 100 }}%

Top Risk Factors:
{{ $('HTTP Request').item.json.top_drivers.map(d => `• ${d.feature}: ${d.effect}`).join('\n') }}

Recommended Actions:
{{ $('HTTP Request').item.json.recommended_actions.map(a => `• ${a}`).join('\n') }}

---

Example 2: Anomaly Detection Workflow

HTTP Request Node Configuration

Method: POST
URL: http://job-ml:8000/detect/anomaly
Headers:

Content-Type: application/json

Body (JSON):

{
  "features": {
    "duration_sec": {{ $json.duration_sec }},
    "duration_zscore": {{ $json.duration_zscore }},
    "avg_duration_7": {{ $json.avg_duration_7 }},
    "failure_rate_7": {{ $json.failure_rate_7 }},
    "err_msg_len": {{ $json.err_msg_len || 0 }},
    "hour_sin": {{ $json.hour_sin || 0 }},
    "hour_cos": {{ $json.hour_cos || 0 }}
  },
  "threshold": 0.01
}

IF Node Condition

Anomaly Detected:

{{ $json.is_anomaly === true }}

High Severity Anomaly (reconstruction error > 3x threshold):

{{ $json.is_anomaly === true && $json.reconstruction_error > ($json.threshold * 3) }}

Example Response

{
  "reconstruction_error": 0.0235,
  "is_anomaly": true,
  "threshold": 0.01,
  "top_drivers": [
    {
      "feature": "duration_zscore",
      "error": 0.0142
    },
    {
      "feature": "duration_sec",
      "error": 0.0068
    }
  ]
}

---

Example 3: Combined Workflow (Prediction + Anomaly)

Workflow Structure

1. Trigger: Job completion event
2. HTTP Request 1: Job failure prediction
3. HTTP Request 2: Anomaly detection
4. IF Node: Combined alert logic
5. Action: Send alert

Combined Alert Condition

Alert if EITHER prediction is risky OR anomaly is detected:

{{ 
  ($('Predict').item.json.fail_probability >= 0.5) || 
  ($('Anomaly').item.json.is_anomaly === true) 
}}

High Priority Alert (both conditions):

{{
  ($('Predict').item.json.fail_probability >= 0.8) ||
  ($('Anomaly').item.json.is_anomaly === true && $('Anomaly').item.json.reconstruction_error > ($('Anomaly').item.json.threshold * 3))
}}

---

Example 4: Scheduled Monitoring Workflow

Workflow Structure

1. Schedule Trigger: Every 15 minutes
2. Database Query: Get recent job runs
3. Loop: For each job
   • HTTP Request: Predict failure
   • HTTP Request: Detect anomaly
   • IF: Check conditions
   • Action: Alert if needed

Code Node for Batch Processing

// Process multiple jobs
const jobs = $input.all();
const results = [];

for (const job of jobs) {
  // Call prediction API
  const predictResponse = await $http.post('http://job-ml:8000/predict/job-fail', {
    zone: job.json.zone,
    job_nm: job.json.job_nm,
    job_start_time: job.json.job_start_time,
    duration_sec: job.json.duration_sec,
    status: job.json.status,
    err_msg: job.json.err_msg,
    explain: true
  });
  
  // Call anomaly API
  const anomalyResponse = await $http.post('http://job-ml:8000/detect/anomaly', {
    features: {
      duration_sec: job.json.duration_sec,
      duration_zscore: job.json.duration_zscore || 0,
      err_msg_len: (job.json.err_msg || '').length
    }
  });
  
  results.push({
    job: job.json.job_nm,
    prediction: predictResponse.data,
    anomaly: anomalyResponse.data,
    should_alert: predictResponse.data.fail_probability >= 0.5 || anomalyResponse.data.is_anomaly
  });
}

return results.map(r => ({ json: r }));

---

Alert Severity Levels

INFO

• fail_probability < 0.3 and no anomaly
• Normal operations

LOW

• 0.3 <= fail_probability < 0.5
• Minor deviations detected

WARNING (MEDIUM)

• 0.5 <= fail_probability < 0.8
• OR is_anomaly === true with reconstruction_error < threshold * 3
• Requires monitoring

CRITICAL

• fail_probability >= 0.8
• OR is_anomaly === true with reconstruction_error >= threshold * 3
• Immediate action required

---

Error Handling

HTTP Request Error Handling

In n8n, configure the HTTP Request node to:

• Continue on Error: Enabled
• Response Format: JSON

Add an IF node after HTTP Request to check for errors:

{{ $json.error !== undefined && $json.error !== null }}

Retry Logic

For critical predictions, add a retry mechanism:

1. HTTP Request node
2. IF node: Check if response has error
3. Wait node: 5 seconds
4. HTTP Request node: Retry (max 3 times)

---

Testing the Workflow

Test Payload (Job Failure Prediction)

{
  "zone": "prod",
  "job_nm": "daily_export_customer",
  "tasksgroup_nm": "export_group",
  "job_start_time": "2026-01-21T01:00:00",
  "duration": "01:30:00",
  "duration_sec": 5400,
  "status": "SUCCESS",
  "err_msg": "",
  "zeppelin": null
}

Test Payload (Anomaly Detection)

{
  "features": {
    "duration_sec": 5400,
    "duration_zscore": 1.6,
    "avg_duration_7": 3000,
    "failure_rate_7": 0.15,
    "err_msg_len": 0,
    "hour_sin": 0.2588,
    "hour_cos": 0.9659
  },
  "threshold": 0.01
}

---

n8n Workflow JSON Export

See n8n_workflow_job_monitoring.json for a complete importable workflow.