README.md

metadata

title: Fraud Model Explainability Assistant
emoji: 👀
colorFrom: gray
colorTo: indigo
sdk: docker
sdk_version: 7.1.0
secrets:
  - GITHUB_TOKEN
app_file: app.py
pinned: false
license: apache-2.0
short_description: Fraud Model Explainability Assistant using Strands Agents

🔍 Fraud Model Explainability Assistant

An AI-powered assistant built with Amazon Strands Agents SDK that helps fraud analysts, data scientists, and executives understand fraud model decisions and ensure fair lending compliance.

🎯 Use Case

Target Role: VP, Fraud Model Data Science
Industry: Credit Card / Consumer Finance (e.g., Synchrony Financial)

This tool addresses critical needs in fraud model governance:

• Executive Communication: Translate complex model outputs for C-suite presentations
• Fair Lending Compliance: Document model decisions for regulatory examinations
• Analyst Efficiency: Reduce time spent answering "why was this flagged?" questions
• Audit Readiness: Provide consistent, thorough explanations for model decisions

✨ Features

Tool	Description
get_application_summary	Basic application info, fraud score, and decision
explain_fraud_score	SHAP-style feature attribution showing why an app was flagged
compare_to_population	Statistical comparison to approved/denied populations
check_fair_lending_flags	Fair lending compliance review and documentation
get_identity_network	Analyze linked applications (device, phone, address, etc.)
get_model_performance	Model metrics, KPIs, and financial impact

🚀 Quick Start

Installation

# Clone or download the repository
cd fraud_explainability_agent

# Create virtual environment
python -m venv .venv
source .venv/bin/activate  # Windows: .venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

Environment Variables

Required environment variables:

OPENAI_API_KEY=your-openai-key
CONFLUENCE_URL=https://your-domain.atlassian.net/wiki
CONFLUENCE_EMAIL=your-email@example.com
CONFLUENCE_API_TOKEN=your-api-token

### Configuration

Set your OpenAI API key:
```bash
export OPENAI_API_KEY="your-api-key-here"

Or use AWS credentials for Amazon Bedrock (default):

export AWS_ACCESS_KEY_ID="your-access-key"
export AWS_SECRET_ACCESS_KEY="your-secret-key"
export AWS_DEFAULT_REGION="us-west-2"

Run the App

python app.py

Open http://localhost:7860 in your browser.

💬 Example Conversations

Basic Investigation

User: "Why was application APP-78432 flagged as high risk?"

Agent: [calls get_application_summary, explain_fraud_score]

Returns detailed breakdown of the fraud score with top contributing factors.

Executive Briefing

User: "I need to present APP-99999 to the CCO. Give me a complete risk summary with compliance review."

Agent: [calls multiple tools: summary, explanation, population comparison, fair lending check]

Returns comprehensive analysis suitable for executive presentation.

Compliance Documentation

User: "Check fair lending compliance for application APP-55555"

Agent: [calls check_fair_lending_flags]

Returns protected class proxy analysis, disparate impact testing results, and adverse action reason codes.

Synthetic ID Investigation

User: "Show me the identity network analysis for APP-78432"

Agent: [calls get_identity_network]

Returns linked applications, connection types, network risk score, and ring pattern detection.

🏗️ Architecture

• Backend: FastAPI + uvicorn
• Agent Framework: Strands Agents
• LLM: OpenAI GPT-4o
• Vector Store: ChromaDB with HuggingFace embeddings
• Scheduling: APScheduler for daily data refresh

┌─────────────────────────────────────────────────────────────┐
│                    Gradio Web Interface                     │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                    Strands Agent                            │
│  ┌─────────────────────────────────────────────────────┐    │
│  │              System Prompt                          │    │
│  │  "You are a Fraud Model Explainability Assistant..."│    │
│  └─────────────────────────────────────────────────────┘    │
│                                                             │
│  ┌─────────────────────────────────────────────────────┐    │
│  │                    Tools                            │    │
│  │  • get_application_summary                          │    │
│  │  • explain_fraud_score                              │    │
│  │  • compare_to_population                            │    │
│  │  • check_fair_lending_flags                         │    │
│  │  • get_identity_network                             │    │
│  │  • get_model_performance                            │    │
│  └─────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                  LLM (GPT-4o / Bedrock)                     │
└─────────────────────────────────────────────────────────────┘

The app will be available at http://localhost:7860

API Endpoints

• GET / - Main UI
• POST /api/ask - Submit a question
• GET /api/metrics - Get performance metrics
• GET /api/health - Health check

🔧 Production Integration

In production, the mock data generators would be replaced with real integrations:

@tool
def explain_fraud_score(application_id: str) -> str:
    """Get SHAP explanation for fraud score."""
    
    # Production implementation:
    # 1. Query feature store for application features
    features = feature_store.get_features(application_id)
    
    # 2. Get SHAP values from model serving
    shap_values = model_service.explain(application_id)
    
    # 3. Query data warehouse for context
    app_data = snowflake.query(f"SELECT * FROM applications WHERE id = '{application_id}'")
    
    # 4. Format and return explanation
    return format_explanation(features, shap_values, app_data)

Suggested Integrations

System	Purpose
Snowflake / Redshift	Application data, fraud outcomes
Feature Store (Feast, Tecton)	Real-time feature retrieval
MLflow / SageMaker	Model registry, SHAP computation
Elasticsearch	Identity linkage network queries
Compliance DB	Fair lending test results, documentation

📊 Mock Data

The demo generates realistic mock data based on the application ID (deterministic seeding). This allows for:

• Consistent demos with specific application IDs
• Realistic distribution of risk levels
• Representative feature values for high/low risk cases

Sample High-Risk Features:

• SSN/Credit age mismatch: 70-95%
• Device velocity: 5-15 applications in 30 days
• Address type: CMRA, PO Box, Vacant
• Synthetic ID score: 75-98%

Sample Low-Risk Features:

• SSN/Credit age mismatch: 0-20%
• Device velocity: 1-2 applications
• Address type: Residential
• Synthetic ID score: 5-25%

🛡️ Compliance Considerations

This tool is designed with regulatory compliance in mind:

• ECOA Compliance: Fair lending checks for protected class proxies
• SR 11-7: Model risk management documentation support
• Adverse Action: Automated reason code generation
• Audit Trail: Consistent, reproducible explanations

📁 Project Structure

fraud_explainability_agent/
├── app.py              # Main application with tools and Gradio UI
├── requirements.txt    # Python dependencies
└── README.md          # This file

🧪 Demo Mode

Run demo queries without the web interface:

python app.py --demo

📚 Resources

• Strands Agents Documentation
• Strands Agents GitHub
• SHAP (SHapley Additive exPlanations)
• SR 11-7: Guidance on Model Risk Management

📄 License

MIT License

---

Built for fraud model data science teams who need to explain complex model decisions to stakeholders across the organization.

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference