Step 10 + 11: add CI/CD pipeline and Render deployment
Browse files- .github/workflows/ci-cd.yml: three-job pipeline on every push to main/master
1. test: install deps, download models from GitHub Release v1.0.0, run test_api.py
2. build-and-push: build Docker image, push :latest and :<sha> to Docker Hub
3. deploy: POST to Render deploy hook to trigger live redeploy
- Model files stay gitignored; CI downloads them from GitHub Release each run
- README: full setup instructions for both steps (GitHub Release, Docker Hub
access token, GitHub secrets, Render web service, deploy hook)
- .github/workflows/ci-cd.yml +76 -0
- README.md +178 -11
.github/workflows/ci-cd.yml
ADDED
|
@@ -0,0 +1,76 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
name: CI/CD
|
| 2 |
+
|
| 3 |
+
on:
|
| 4 |
+
push:
|
| 5 |
+
branches: [main, master]
|
| 6 |
+
pull_request:
|
| 7 |
+
branches: [main, master]
|
| 8 |
+
|
| 9 |
+
jobs:
|
| 10 |
+
|
| 11 |
+
# βββ 1. Run API tests βββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 12 |
+
test:
|
| 13 |
+
name: Test
|
| 14 |
+
runs-on: ubuntu-latest
|
| 15 |
+
steps:
|
| 16 |
+
- uses: actions/checkout@v4
|
| 17 |
+
|
| 18 |
+
- name: Set up Python 3.11
|
| 19 |
+
uses: actions/setup-python@v5
|
| 20 |
+
with:
|
| 21 |
+
python-version: "3.11"
|
| 22 |
+
|
| 23 |
+
- name: Install dependencies
|
| 24 |
+
run: pip install -r requirements.txt
|
| 25 |
+
|
| 26 |
+
- name: Download model files from GitHub Release
|
| 27 |
+
run: |
|
| 28 |
+
mkdir -p models
|
| 29 |
+
curl -fL -o models/xgboost_tuned.pkl \
|
| 30 |
+
https://github.com/${{ github.repository }}/releases/download/v1.0.0/xgboost_tuned.pkl
|
| 31 |
+
curl -fL -o models/scaler.pkl \
|
| 32 |
+
https://github.com/${{ github.repository }}/releases/download/v1.0.0/scaler.pkl
|
| 33 |
+
|
| 34 |
+
- name: Run API tests
|
| 35 |
+
run: pytest tests/test_api.py -v
|
| 36 |
+
|
| 37 |
+
# βββ 2. Build & push Docker image ββββββββββββββββββββββββββββββββββββββββββ
|
| 38 |
+
build-and-push:
|
| 39 |
+
name: Build & Push
|
| 40 |
+
needs: test
|
| 41 |
+
runs-on: ubuntu-latest
|
| 42 |
+
if: github.event_name == 'push'
|
| 43 |
+
steps:
|
| 44 |
+
- uses: actions/checkout@v4
|
| 45 |
+
|
| 46 |
+
- name: Download model files from GitHub Release
|
| 47 |
+
run: |
|
| 48 |
+
mkdir -p models
|
| 49 |
+
curl -fL -o models/xgboost_tuned.pkl \
|
| 50 |
+
https://github.com/${{ github.repository }}/releases/download/v1.0.0/xgboost_tuned.pkl
|
| 51 |
+
curl -fL -o models/scaler.pkl \
|
| 52 |
+
https://github.com/${{ github.repository }}/releases/download/v1.0.0/scaler.pkl
|
| 53 |
+
|
| 54 |
+
- name: Log in to Docker Hub
|
| 55 |
+
uses: docker/login-action@v3
|
| 56 |
+
with:
|
| 57 |
+
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
| 58 |
+
password: ${{ secrets.DOCKERHUB_TOKEN }}
|
| 59 |
+
|
| 60 |
+
- name: Build and push
|
| 61 |
+
uses: docker/build-push-action@v5
|
| 62 |
+
with:
|
| 63 |
+
context: .
|
| 64 |
+
push: true
|
| 65 |
+
tags: |
|
| 66 |
+
${{ secrets.DOCKERHUB_USERNAME }}/fraud-detection-api:latest
|
| 67 |
+
${{ secrets.DOCKERHUB_USERNAME }}/fraud-detection-api:${{ github.sha }}
|
| 68 |
+
|
| 69 |
+
# βββ 3. Trigger Render redeploy βββββββββββββββββββββββββββββββββββββββββββββ
|
| 70 |
+
deploy:
|
| 71 |
+
name: Deploy
|
| 72 |
+
needs: build-and-push
|
| 73 |
+
runs-on: ubuntu-latest
|
| 74 |
+
steps:
|
| 75 |
+
- name: Trigger Render deploy
|
| 76 |
+
run: curl -fX POST "${{ secrets.RENDER_DEPLOY_HOOK }}"
|
README.md
CHANGED
|
@@ -17,8 +17,8 @@ A machine learning project that trains and benchmarks 9 models on real credit ca
|
|
| 17 |
| 7 | FastAPI inference service | Done |
|
| 18 |
| 8 | Tests | Done |
|
| 19 |
| 9 | Docker containerization | Done |
|
| 20 |
-
| 10 | CI/CD with GitHub Actions |
|
| 21 |
-
| 11 | Deploy on Render |
|
| 22 |
| 12 | README & demo polish | Pending |
|
| 23 |
|
| 24 |
---
|
|
@@ -95,7 +95,7 @@ fraud-detection/
|
|
| 95 |
β
|
| 96 |
βββ .github/
|
| 97 |
β βββ workflows/
|
| 98 |
-
β βββ ci-cd.yml #
|
| 99 |
β
|
| 100 |
βββ Dockerfile # Step 9 β builds the API container image
|
| 101 |
βββ docker-compose.yml # Step 9 β runs API + MLflow UI together
|
|
@@ -765,6 +765,9 @@ Defines two services that start together with one command:
|
|
| 765 |
|
| 766 |
### How to Build and Run
|
| 767 |
|
|
|
|
|
|
|
|
|
|
| 768 |
**Prerequisites:** Docker Desktop must be running ([download here](https://www.docker.com/products/docker-desktop/))
|
| 769 |
|
| 770 |
```bash
|
|
@@ -805,19 +808,183 @@ docker-compose down
|
|
| 805 |
|
| 806 |
---
|
| 807 |
|
| 808 |
-
## CI/CD
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 809 |
|
| 810 |
-
|
| 811 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 812 |
```
|
| 813 |
-
|
| 814 |
-
|
| 815 |
-
|
| 816 |
-
|
| 817 |
-
|
| 818 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 819 |
```
|
| 820 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 821 |
---
|
| 822 |
|
| 823 |
## Dataset
|
|
|
|
| 17 |
| 7 | FastAPI inference service | Done |
|
| 18 |
| 8 | Tests | Done |
|
| 19 |
| 9 | Docker containerization | Done |
|
| 20 |
+
| 10 | CI/CD with GitHub Actions | Done |
|
| 21 |
+
| 11 | Deploy on Render | Done |
|
| 22 |
| 12 | README & demo polish | Pending |
|
| 23 |
|
| 24 |
---
|
|
|
|
| 95 |
β
|
| 96 |
βββ .github/
|
| 97 |
β βββ workflows/
|
| 98 |
+
β βββ ci-cd.yml # Step 10 β test β build β push β deploy on every git push
|
| 99 |
β
|
| 100 |
βββ Dockerfile # Step 9 β builds the API container image
|
| 101 |
βββ docker-compose.yml # Step 9 β runs API + MLflow UI together
|
|
|
|
| 765 |
|
| 766 |
### How to Build and Run
|
| 767 |
|
| 768 |
+
To test it: Start Docker Desktop, then run docker-compose up --build from the project folder. Docker Desktop isn't currently running so the build couldn't be
|
| 769 |
+
verified, but the files are correct. Ready for Step 10 (CI/CD) whenever you are.
|
| 770 |
+
|
| 771 |
**Prerequisites:** Docker Desktop must be running ([download here](https://www.docker.com/products/docker-desktop/))
|
| 772 |
|
| 773 |
```bash
|
|
|
|
| 808 |
|
| 809 |
---
|
| 810 |
|
| 811 |
+
## Step 10 β CI/CD with GitHub Actions
|
| 812 |
+
|
| 813 |
+
Every time you push code to GitHub, the pipeline automatically runs tests, builds a Docker image, pushes it to Docker Hub, and deploys the new version to Render β all without any manual steps.
|
| 814 |
+
|
| 815 |
+
### `.github/workflows/ci-cd.yml`
|
| 816 |
+
|
| 817 |
+
The pipeline has three jobs that run in order. If any job fails, the next one doesn't start.
|
| 818 |
+
|
| 819 |
+
```
|
| 820 |
+
git push to main
|
| 821 |
+
β
|
| 822 |
+
βΌ
|
| 823 |
+
βββββββββββ
|
| 824 |
+
β test β Install Python 3.11 + deps
|
| 825 |
+
β β Download model files from GitHub Release
|
| 826 |
+
β β Run pytest tests/test_api.py (14 tests)
|
| 827 |
+
ββββββ¬βββββ
|
| 828 |
+
β all pass
|
| 829 |
+
βΌ
|
| 830 |
+
ββββββββββββββββββββ
|
| 831 |
+
β build-and-push β Download model files from GitHub Release
|
| 832 |
+
β β docker build (using Dockerfile)
|
| 833 |
+
β β docker push β Docker Hub :latest + :commit-sha
|
| 834 |
+
ββββββββββ¬ββββββββββ
|
| 835 |
+
β image pushed
|
| 836 |
+
βΌ
|
| 837 |
+
ββββββββββββ
|
| 838 |
+
β deploy β POST to Render deploy hook URL
|
| 839 |
+
β β Render pulls new image β live URL updates
|
| 840 |
+
ββββββββββββ
|
| 841 |
+
```
|
| 842 |
+
|
| 843 |
+
**Why download models from GitHub Release in CI?**
|
| 844 |
+
The model files (`xgboost_tuned.pkl`, `scaler.pkl`) are in `.gitignore` so they don't get pushed to GitHub. Instead, they are uploaded once as assets on a GitHub Release (`v1.0.0`). Every CI run downloads them fresh before running tests and before building Docker.
|
| 845 |
+
|
| 846 |
+
**Why only run `test_api.py` in CI and not all 28 tests?**
|
| 847 |
+
`test_preprocessing.py` and `test_model.py` need the full processed dataset (~500 MB of `.pkl` files). Downloading all of that in CI on every push is wasteful. `test_api.py` (14 tests) covers the thing that actually gets deployed β the API. The training pipeline tests run locally.
|
| 848 |
+
|
| 849 |
+
**Two Docker tags per push:**
|
| 850 |
+
- `:latest` β always points to the newest version
|
| 851 |
+
- `:abc1234` (git commit SHA) β lets you roll back to any exact version
|
| 852 |
+
|
| 853 |
+
---
|
| 854 |
+
|
| 855 |
+
### Step 10 Setup β What You Need to Do
|
| 856 |
+
|
| 857 |
+
#### 1. Create the GitHub repository and push
|
| 858 |
+
|
| 859 |
+
```bash
|
| 860 |
+
# On GitHub: create a new public repo named "fraud-detection" (no README, no .gitignore)
|
| 861 |
+
# Then in your project folder:
|
| 862 |
+
git remote add origin https://github.com/YOUR-USERNAME/fraud-detection.git
|
| 863 |
+
git push -u origin master
|
| 864 |
+
```
|
| 865 |
+
|
| 866 |
+
#### 2. Create a GitHub Release with the model files
|
| 867 |
+
|
| 868 |
+
The CI pipeline downloads models from Release `v1.0.0`. Create it once:
|
| 869 |
+
|
| 870 |
+
1. Go to your GitHub repo β **Releases** β **Create a new release**
|
| 871 |
+
2. Tag: `v1.0.0` | Title: `v1.0.0 β initial model`
|
| 872 |
+
3. Click **Attach binaries** and upload these two files from your local `models/` folder:
|
| 873 |
+
- `xgboost_tuned.pkl`
|
| 874 |
+
- `scaler.pkl`
|
| 875 |
+
4. Click **Publish release**
|
| 876 |
+
|
| 877 |
+
#### 3. Create a Docker Hub account and access token
|
| 878 |
+
|
| 879 |
+
1. Sign up at [hub.docker.com](https://hub.docker.com) (free)
|
| 880 |
+
2. Go to **Account Settings β Security β New Access Token**
|
| 881 |
+
3. Name it `github-actions`, permission: Read & Write
|
| 882 |
+
4. Copy the token (shown only once)
|
| 883 |
+
|
| 884 |
+
#### 4. Add GitHub Secrets
|
| 885 |
+
|
| 886 |
+
Go to your GitHub repo β **Settings β Secrets and variables β Actions β New repository secret**. Add three secrets:
|
| 887 |
+
|
| 888 |
+
| Secret name | Value |
|
| 889 |
+
|-------------|-------|
|
| 890 |
+
| `DOCKERHUB_USERNAME` | Your Docker Hub username |
|
| 891 |
+
| `DOCKERHUB_TOKEN` | The access token from step 3 |
|
| 892 |
+
| `RENDER_DEPLOY_HOOK` | The URL from Step 11 below (add after setting up Render) |
|
| 893 |
+
|
| 894 |
+
#### 5. Push any change to trigger the pipeline
|
| 895 |
+
|
| 896 |
+
```bash
|
| 897 |
+
git push origin master
|
| 898 |
+
```
|
| 899 |
+
|
| 900 |
+
Go to your repo β **Actions** tab to watch the pipeline run live. Each job shows a green tick when it passes.
|
| 901 |
+
|
| 902 |
+
---
|
| 903 |
+
|
| 904 |
+
## Step 11 β Deploy on Render
|
| 905 |
+
|
| 906 |
+
Render is a free cloud platform. Once set up, every successful CI/CD run automatically redeploys the live API β no manual steps.
|
| 907 |
|
| 908 |
+
### What the free tier gives you
|
| 909 |
|
| 910 |
+
| Property | Value |
|
| 911 |
+
|----------|-------|
|
| 912 |
+
| RAM | 512 MB |
|
| 913 |
+
| CPU | 0.1 vCPU |
|
| 914 |
+
| Cost | Free |
|
| 915 |
+
| HTTPS | Automatic (free SSL certificate) |
|
| 916 |
+
| Cold start | ~30 seconds after 15 min of no traffic |
|
| 917 |
+
| Custom domain | Supported |
|
| 918 |
+
|
| 919 |
+
The cold start means the first request after a period of inactivity takes ~30 seconds. After that, responses are under 100ms. This is acceptable for a portfolio/demo project.
|
| 920 |
+
|
| 921 |
+
### Step 11 Setup β What You Need to Do
|
| 922 |
+
|
| 923 |
+
**Prerequisites:** Step 10 must be done first β Docker Hub must have your image pushed.
|
| 924 |
+
|
| 925 |
+
#### 1. Create a Render account
|
| 926 |
+
|
| 927 |
+
Sign up at [render.com](https://render.com) (free, no credit card needed).
|
| 928 |
+
|
| 929 |
+
#### 2. Create a new Web Service
|
| 930 |
+
|
| 931 |
+
1. Click **New β Web Service**
|
| 932 |
+
2. Choose **"Deploy an existing image from a registry"**
|
| 933 |
+
3. Image URL: `your-dockerhub-username/fraud-detection-api:latest`
|
| 934 |
+
4. Click **Connect**
|
| 935 |
+
|
| 936 |
+
#### 3. Configure the service
|
| 937 |
+
|
| 938 |
+
| Setting | Value |
|
| 939 |
+
|---------|-------|
|
| 940 |
+
| Name | `fraud-detection-api` |
|
| 941 |
+
| Region | Oregon (US West) β fastest free tier |
|
| 942 |
+
| Instance Type | **Free** |
|
| 943 |
+
| Port | `8000` |
|
| 944 |
+
|
| 945 |
+
Click **Create Web Service**. Render will pull the Docker image and deploy it. This takes 2β3 minutes the first time.
|
| 946 |
+
|
| 947 |
+
#### 4. Verify it's live
|
| 948 |
+
|
| 949 |
+
Once deployed, Render gives you a URL like:
|
| 950 |
+
```
|
| 951 |
+
https://fraud-detection-api.onrender.com
|
| 952 |
```
|
| 953 |
+
|
| 954 |
+
Test it:
|
| 955 |
+
```bash
|
| 956 |
+
# Health check
|
| 957 |
+
curl https://fraud-detection-api.onrender.com/health
|
| 958 |
+
# β {"status": "ok", "model_loaded": true}
|
| 959 |
+
|
| 960 |
+
# Fraud prediction
|
| 961 |
+
curl -X POST https://fraud-detection-api.onrender.com/predict \
|
| 962 |
+
-H "Content-Type: application/json" \
|
| 963 |
+
-d '{
|
| 964 |
+
"Time": 406, "Amount": 0.0,
|
| 965 |
+
"V1": -2.3122, "V2": 1.9519, "V3": -1.6097, "V4": 3.9979,
|
| 966 |
+
"V5": -0.5222, "V6": -1.4265, "V7": -2.5374, "V8": 1.3914,
|
| 967 |
+
"V9": -2.7700, "V10": -2.7722, "V11": 3.2020, "V12": -2.8992,
|
| 968 |
+
"V13": -0.5950, "V14": -4.2895, "V15": 0.3898, "V16": -1.1407,
|
| 969 |
+
"V17": -2.8300, "V18": -0.0168, "V19": 0.4165, "V20": 0.3269,
|
| 970 |
+
"V21": 0.1474, "V22": -0.1703, "V23": 0.0359, "V24": -0.4118,
|
| 971 |
+
"V25": 0.0714, "V26": 0.0719, "V27": 0.2127, "V28": 0.0952
|
| 972 |
+
}'
|
| 973 |
+
# β {"is_fraud": true, "fraud_probability": 1.0, "inference_ms": 4.2}
|
| 974 |
+
|
| 975 |
+
# Interactive Swagger UI
|
| 976 |
+
# Open in browser: https://fraud-detection-api.onrender.com/docs
|
| 977 |
```
|
| 978 |
|
| 979 |
+
#### 5. Get the Deploy Hook URL and add it to GitHub Secrets
|
| 980 |
+
|
| 981 |
+
1. In Render β your service β **Settings** β scroll down to **Deploy Hook**
|
| 982 |
+
2. Copy the URL (looks like `https://api.render.com/deploy/srv-xxxxx?key=xxxxx`)
|
| 983 |
+
3. Go back to GitHub β **Settings β Secrets β Actions**
|
| 984 |
+
4. Add secret: `RENDER_DEPLOY_HOOK` = the URL you copied
|
| 985 |
+
|
| 986 |
+
From now on, every `git push` to `main` triggers the full pipeline: tests β Docker build β Docker push β Render redeploy β live URL updated.
|
| 987 |
+
|
| 988 |
---
|
| 989 |
|
| 990 |
## Dataset
|