README.md

---
title: Fraud Model Explainability Assistant
emoji: 👀
colorFrom: gray
colorTo: indigo
sdk: docker
sdk_version: 7.1.0
secrets:
  - GITHUB_TOKEN
app_file: app.py
pinned: false
license: apache-2.0
short_description: Fraud Model Explainability Assistant using Strands Agents
---

# 🔍 Fraud Model Explainability Assistant

An AI-powered assistant built with **Amazon Strands Agents SDK** that helps fraud analysts, data scientists, and executives understand fraud model decisions and ensure fair lending compliance.

## 🎯 Use Case

**Target Role:** VP, Fraud Model Data Science  
**Industry:** Credit Card / Consumer Finance (e.g., Synchrony Financial)

This tool addresses critical needs in fraud model governance:
- **Executive Communication**: Translate complex model outputs for C-suite presentations
- **Fair Lending Compliance**: Document model decisions for regulatory examinations
- **Analyst Efficiency**: Reduce time spent answering "why was this flagged?" questions
- **Audit Readiness**: Provide consistent, thorough explanations for model decisions

## ✨ Features

| Tool | Description |
|------|-------------|
| `get_application_summary` | Basic application info, fraud score, and decision |
| `explain_fraud_score` | SHAP-style feature attribution showing why an app was flagged |
| `compare_to_population` | Statistical comparison to approved/denied populations |
| `check_fair_lending_flags` | Fair lending compliance review and documentation |
| `get_identity_network` | Analyze linked applications (device, phone, address, etc.) |
| `get_model_performance` | Model metrics, KPIs, and financial impact |

## 🚀 Quick Start

### Installation

```bash
# Clone or download the repository
cd fraud_explainability_agent

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt
```

## Environment Variables

Required environment variables:

```
OPENAI_API_KEY=your-openai-key
CONFLUENCE_URL=https://your-domain.atlassian.net/wiki
CONFLUENCE_EMAIL=your-email@example.com
CONFLUENCE_API_TOKEN=your-api-token

### Configuration

Set your OpenAI API key:
```bash
export OPENAI_API_KEY="your-api-key-here"
```

Or use AWS credentials for Amazon Bedrock (default):
```bash
export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_DEFAULT_REGION="us-west-2"
```

### Run the App

```bash
python app.py
```

Open http://localhost:7860 in your browser.

## 💬 Example Conversations

### Basic Investigation
> **User:** "Why was application APP-78432 flagged as high risk?"
>
> **Agent:** *[calls get_application_summary, explain_fraud_score]*
> 
> Returns detailed breakdown of the fraud score with top contributing factors.

### Executive Briefing
> **User:** "I need to present APP-99999 to the CCO. Give me a complete risk summary with compliance review."
>
> **Agent:** *[calls multiple tools: summary, explanation, population comparison, fair lending check]*
>
> Returns comprehensive analysis suitable for executive presentation.

### Compliance Documentation
> **User:** "Check fair lending compliance for application APP-55555"
>
> **Agent:** *[calls check_fair_lending_flags]*
>
> Returns protected class proxy analysis, disparate impact testing results, and adverse action reason codes.

### Synthetic ID Investigation
> **User:** "Show me the identity network analysis for APP-78432"
>
> **Agent:** *[calls get_identity_network]*
>
> Returns linked applications, connection types, network risk score, and ring pattern detection.

## 🏗️ Architecture

- **Backend**: FastAPI + uvicorn
- **Agent Framework**: Strands Agents
- **LLM**: OpenAI GPT-4o
- **Vector Store**: ChromaDB with HuggingFace embeddings
- **Scheduling**: APScheduler for daily data refresh

```
┌─────────────────────────────────────────────────────────────┐
│                    Gradio Web Interface                     │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                    Strands Agent                            │
│  ┌─────────────────────────────────────────────────────┐    │
│  │              System Prompt                          │    │
│  │  "You are a Fraud Model Explainability Assistant..."│    │
│  └─────────────────────────────────────────────────────┘    │
│                                                             │
│  ┌─────────────────────────────────────────────────────┐    │
│  │                    Tools                            │    │
│  │  • get_application_summary                          │    │
│  │  • explain_fraud_score                              │    │
│  │  • compare_to_population                            │    │
│  │  • check_fair_lending_flags                         │    │
│  │  • get_identity_network                             │    │
│  │  • get_model_performance                            │    │
│  └─────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                  LLM (GPT-4o / Bedrock)                     │
└─────────────────────────────────────────────────────────────┘
```

The app will be available at `http://localhost:7860`

## API Endpoints

- `GET /` - Main UI
- `POST /api/ask` - Submit a question
- `GET /api/metrics` - Get performance metrics
- `GET /api/health` - Health check

## 🔧 Production Integration

In production, the mock data generators would be replaced with real integrations:

```python
@tool
def explain_fraud_score(application_id: str) -> str:
    """Get SHAP explanation for fraud score."""
    
    # Production implementation:
    # 1. Query feature store for application features
    features = feature_store.get_features(application_id)
    
    # 2. Get SHAP values from model serving
    shap_values = model_service.explain(application_id)
    
    # 3. Query data warehouse for context
    app_data = snowflake.query(f"SELECT * FROM applications WHERE id = '{application_id}'")
    
    # 4. Format and return explanation
    return format_explanation(features, shap_values, app_data)
```

### Suggested Integrations

| System | Purpose |
|--------|---------|
| **Snowflake / Redshift** | Application data, fraud outcomes |
| **Feature Store** (Feast, Tecton) | Real-time feature retrieval |
| **MLflow / SageMaker** | Model registry, SHAP computation |
| **Elasticsearch** | Identity linkage network queries |
| **Compliance DB** | Fair lending test results, documentation |

## 📊 Mock Data

The demo generates realistic mock data based on the application ID (deterministic seeding). This allows for:
- Consistent demos with specific application IDs
- Realistic distribution of risk levels
- Representative feature values for high/low risk cases

**Sample High-Risk Features:**
- SSN/Credit age mismatch: 70-95%
- Device velocity: 5-15 applications in 30 days
- Address type: CMRA, PO Box, Vacant
- Synthetic ID score: 75-98%

**Sample Low-Risk Features:**
- SSN/Credit age mismatch: 0-20%
- Device velocity: 1-2 applications
- Address type: Residential
- Synthetic ID score: 5-25%

## 🛡️ Compliance Considerations

This tool is designed with regulatory compliance in mind:

- **ECOA Compliance**: Fair lending checks for protected class proxies
- **SR 11-7**: Model risk management documentation support
- **Adverse Action**: Automated reason code generation
- **Audit Trail**: Consistent, reproducible explanations

## 📁 Project Structure

```
fraud_explainability_agent/
├── app.py              # Main application with tools and Gradio UI
├── requirements.txt    # Python dependencies
└── README.md          # This file
```

## 🧪 Demo Mode

Run demo queries without the web interface:

```bash
python app.py --demo
```

## 📚 Resources

- [Strands Agents Documentation](https://strandsagents.com)
- [Strands Agents GitHub](https://github.com/strands-agents/sdk-python)
- [SHAP (SHapley Additive exPlanations)](https://shap.readthedocs.io/)
- [SR 11-7: Guidance on Model Risk Management](https://www.federalreserve.gov/supervisionreg/srletters/sr1107.htm)

## 📄 License

MIT License

---

*Built for fraud model data science teams who need to explain complex model decisions to stakeholders across the organization.*


Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference