README.md

metadata

title: AdRL Studio
colorFrom: purple
colorTo: blue
sdk: docker
app_port: 7860
pinned: false

🎯 AdRL Studio

🎯 AdRL Studio — A contextual multi-armed bandit platform that simulates a real-world ad recommendation and serving system using reinforcement learning. Benchmarks four bandit algorithms side by side, visualizes online learning and regret curves, runs A/B test simulations with statistical significance testing, and serves real-time ad recommendations from user context input.

---

Table of Contents

• Features
• Architecture
• Getting Started
• Docker Deployment
• Dashboard Modules
• ML Models
• Project Structure
• Author
• Contributing
• Disclaimer
• License

---

✨ Features

🎯 Live Ad Serving	Enter user context (age, device, time, category, region) and get real-time ad recommendations from all 4 algorithms simultaneously
▶ Online Learning Simulation	Run 1K–10K impression simulations with SSE-streamed progress, rolling CTR charts, and per-algorithm summaries
📉 Regret Analysis	Visualize cumulative regret curves — the canonical RL evaluation metric — comparing all four policies
⚖ A/B Test Simulator	Run 50/50 traffic splits with two-proportion z-test, p-value, confidence intervals, and statistical significance verdict
🔒 Secure by Design	Role-based access, audit logs, encrypted data pipelines
🐳 Containerized Deployment	Docker-first architecture, cloud-ready and scalable

---

🏗️ Architecture

┌─────────────────────────────────────────────────────────┐
│                      AdRL Studio                        │
│                                                         │
│  ┌───────────┐    ┌───────────┐    ┌───────────────┐  │
│  │  Simulated│───▶│  Bandit   │───▶│   Flask API   │  │
│  │ Ad Environ│    │ Algorithms│    │   Backend     │  │
│  └───────────┘    └───────────┘    └───────┬───────┘  │
│                                            │           │
│                                   ┌────────▼────────┐  │
│                                   │  Plotly Charts  │  │
│                                   │   Dashboard     │  │
│                                   └─────────────────┘  │
└─────────────────────────────────────────────────────────┘

---

🚀 Getting Started

Prerequisites

• Python 3.10+
• Docker & Docker Compose
• Git

Local Installation

# 1. Clone the repository
git clone https://github.com/mnoorchenar/AdRL-Studio.git
cd AdRL-Studio

# 2. Create a virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# 3. Install dependencies
pip install -r requirements.txt

# 4. Configure environment variables
cp .env.example .env
# Edit .env with your settings

# 5. Run the application
python app.py

Open your browser at http://localhost:7860 🎉

---

🐳 Docker Deployment

# Build and run with Docker Compose
docker compose up --build

# Or pull and run the pre-built image
docker pull mnoorchenar/AdRL-Studio
docker run -p 7860:7860 mnoorchenar/AdRL-Studio

---

📊 Dashboard Modules

Module	Description	Status
🎯 Live Ad Serving	Real-time 4-algorithm recommendation from user context	✅ Live
▶ Online Learning	Simulation with SSE streaming and rolling CTR charts	✅ Live
📉 Regret Analysis	Cumulative regret curves for all four algorithms	✅ Live
⚖ A/B Test Simulator	Statistical significance testing with z-test & CI	✅ Live
🌡 Reward Landscape	5×5 CTR heatmap: user content category × ad category	✅ Live
🔬 Policy Inspector	Per-ad learned weights and posterior distributions	🗓️ Planned

---

🧠 ML Models

# Core Models Used in AdRL Studio
models = {
    "epsilon_greedy": "ε-Greedy Neural Bandit — shared PyTorch MLP (39→32→16→1) with decaying ε",
    "ucb1":           "UCB1 — Upper Confidence Bound non-contextual baseline",
    "thompson":       "Thompson Sampling — Bayesian Beta(α,β) per arm",
    "linucb":         "LinUCB Disjoint — ridge regression contextual bandit (production-grade)",
    "environment":    "Simulated 20-ad inventory, 19-dim one-hot context, Bernoulli reward sampling"
}

---

📁 Project Structure

AdRL-Studio/
│
├── 📄 app.py               # Complete Flask application — all logic, templates, and API
├── 📄 Dockerfile           # Container definition (python:3.10-slim, port 7860)
├── 📄 requirements.txt     # Python dependencies
└── 📄 README.md            # This file

All application logic, HTML templates, CSS, and JavaScript live inside app.py using Flask's render_template_string. There are no external static files.

---

👨‍💻 Author

Mohammad Noorchenarboo Data Scientist  |  AI Researcher  |  Biostatistician 📍  Ontario, Canada    📧  mohammadnoorchenarboo@gmail.com ──────────────────────────────────────

---

🤝 Contributing

Contributions are welcome! Please follow these steps:

1. Fork the repository
2. Create a feature branch: git checkout -b feature/amazing-feature
3. Commit your changes: git commit -m 'Add amazing feature'
4. Push to the branch: git push origin feature/amazing-feature
5. Open a Pull Request

---

Disclaimer

This project is developed strictly for educational and research purposes and does not constitute professional advice of any kind. All datasets used are either synthetically generated or publicly available — no real user data is stored. This software is provided "as is" without warranty of any kind; use at your own risk.

---

📜 License

Distributed under the MIT License. See LICENSE for more information.

---

_{The name "AdRL Studio" is used purely for academic and research purposes. Any similarity to existing company names, products, or trademarks is entirely coincidental and unintentional. This project has no affiliation with any commercial entity.}