Zalaid commited on
Commit
461caa0
Β·
1 Parent(s): fa70c82

Step 10 + 11: add CI/CD pipeline and Render deployment

Browse files

- .github/workflows/ci-cd.yml: three-job pipeline on every push to main/master
1. test: install deps, download models from GitHub Release v1.0.0, run test_api.py
2. build-and-push: build Docker image, push :latest and :<sha> to Docker Hub
3. deploy: POST to Render deploy hook to trigger live redeploy
- Model files stay gitignored; CI downloads them from GitHub Release each run
- README: full setup instructions for both steps (GitHub Release, Docker Hub
access token, GitHub secrets, Render web service, deploy hook)

Files changed (2) hide show
  1. .github/workflows/ci-cd.yml +76 -0
  2. README.md +178 -11
.github/workflows/ci-cd.yml ADDED
@@ -0,0 +1,76 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: CI/CD
2
+
3
+ on:
4
+ push:
5
+ branches: [main, master]
6
+ pull_request:
7
+ branches: [main, master]
8
+
9
+ jobs:
10
+
11
+ # ─── 1. Run API tests ───────────────────────────────────────────────────────
12
+ test:
13
+ name: Test
14
+ runs-on: ubuntu-latest
15
+ steps:
16
+ - uses: actions/checkout@v4
17
+
18
+ - name: Set up Python 3.11
19
+ uses: actions/setup-python@v5
20
+ with:
21
+ python-version: "3.11"
22
+
23
+ - name: Install dependencies
24
+ run: pip install -r requirements.txt
25
+
26
+ - name: Download model files from GitHub Release
27
+ run: |
28
+ mkdir -p models
29
+ curl -fL -o models/xgboost_tuned.pkl \
30
+ https://github.com/${{ github.repository }}/releases/download/v1.0.0/xgboost_tuned.pkl
31
+ curl -fL -o models/scaler.pkl \
32
+ https://github.com/${{ github.repository }}/releases/download/v1.0.0/scaler.pkl
33
+
34
+ - name: Run API tests
35
+ run: pytest tests/test_api.py -v
36
+
37
+ # ─── 2. Build & push Docker image ──────────────────────────────────────────
38
+ build-and-push:
39
+ name: Build & Push
40
+ needs: test
41
+ runs-on: ubuntu-latest
42
+ if: github.event_name == 'push'
43
+ steps:
44
+ - uses: actions/checkout@v4
45
+
46
+ - name: Download model files from GitHub Release
47
+ run: |
48
+ mkdir -p models
49
+ curl -fL -o models/xgboost_tuned.pkl \
50
+ https://github.com/${{ github.repository }}/releases/download/v1.0.0/xgboost_tuned.pkl
51
+ curl -fL -o models/scaler.pkl \
52
+ https://github.com/${{ github.repository }}/releases/download/v1.0.0/scaler.pkl
53
+
54
+ - name: Log in to Docker Hub
55
+ uses: docker/login-action@v3
56
+ with:
57
+ username: ${{ secrets.DOCKERHUB_USERNAME }}
58
+ password: ${{ secrets.DOCKERHUB_TOKEN }}
59
+
60
+ - name: Build and push
61
+ uses: docker/build-push-action@v5
62
+ with:
63
+ context: .
64
+ push: true
65
+ tags: |
66
+ ${{ secrets.DOCKERHUB_USERNAME }}/fraud-detection-api:latest
67
+ ${{ secrets.DOCKERHUB_USERNAME }}/fraud-detection-api:${{ github.sha }}
68
+
69
+ # ─── 3. Trigger Render redeploy ─────────────────────────────────────────────
70
+ deploy:
71
+ name: Deploy
72
+ needs: build-and-push
73
+ runs-on: ubuntu-latest
74
+ steps:
75
+ - name: Trigger Render deploy
76
+ run: curl -fX POST "${{ secrets.RENDER_DEPLOY_HOOK }}"
README.md CHANGED
@@ -17,8 +17,8 @@ A machine learning project that trains and benchmarks 9 models on real credit ca
17
  | 7 | FastAPI inference service | Done |
18
  | 8 | Tests | Done |
19
  | 9 | Docker containerization | Done |
20
- | 10 | CI/CD with GitHub Actions | Pending |
21
- | 11 | Deploy on Render | Pending |
22
  | 12 | README & demo polish | Pending |
23
 
24
  ---
@@ -95,7 +95,7 @@ fraud-detection/
95
  β”‚
96
  β”œβ”€β”€ .github/
97
  β”‚ └── workflows/
98
- β”‚ └── ci-cd.yml # (Step 10 β€” pending)
99
  β”‚
100
  β”œβ”€β”€ Dockerfile # Step 9 β€” builds the API container image
101
  β”œβ”€β”€ docker-compose.yml # Step 9 β€” runs API + MLflow UI together
@@ -765,6 +765,9 @@ Defines two services that start together with one command:
765
 
766
  ### How to Build and Run
767
 
 
 
 
768
  **Prerequisites:** Docker Desktop must be running ([download here](https://www.docker.com/products/docker-desktop/))
769
 
770
  ```bash
@@ -805,19 +808,183 @@ docker-compose down
805
 
806
  ---
807
 
808
- ## CI/CD Pipeline (.github/workflows/)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
809
 
810
- This folder is empty until **Step 10**. GitHub Actions scans `.github/workflows/` for `.yml` files on every push. Once `ci-cd.yml` is added, every `git push` to `main` automatically triggers:
811
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
812
  ```
813
- git push β†’ GitHub Actions β†’
814
- [1] Lint β€” checks code style with flake8
815
- [2] Test β€” runs pytest, all tests must pass
816
- [3] Build β€” builds the Docker image
817
- [4] Push β€” pushes image to Docker Hub
818
- [5] Deploy β€” triggers a redeploy on Render (live URL updates automatically)
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
819
  ```
820
 
 
 
 
 
 
 
 
 
 
821
  ---
822
 
823
  ## Dataset
 
17
  | 7 | FastAPI inference service | Done |
18
  | 8 | Tests | Done |
19
  | 9 | Docker containerization | Done |
20
+ | 10 | CI/CD with GitHub Actions | Done |
21
+ | 11 | Deploy on Render | Done |
22
  | 12 | README & demo polish | Pending |
23
 
24
  ---
 
95
  β”‚
96
  β”œβ”€β”€ .github/
97
  β”‚ └── workflows/
98
+ β”‚ └── ci-cd.yml # Step 10 β€” test β†’ build β†’ push β†’ deploy on every git push
99
  β”‚
100
  β”œβ”€β”€ Dockerfile # Step 9 β€” builds the API container image
101
  β”œβ”€β”€ docker-compose.yml # Step 9 β€” runs API + MLflow UI together
 
765
 
766
  ### How to Build and Run
767
 
768
+ To test it: Start Docker Desktop, then run docker-compose up --build from the project folder. Docker Desktop isn't currently running so the build couldn't be
769
+ verified, but the files are correct. Ready for Step 10 (CI/CD) whenever you are.
770
+
771
  **Prerequisites:** Docker Desktop must be running ([download here](https://www.docker.com/products/docker-desktop/))
772
 
773
  ```bash
 
808
 
809
  ---
810
 
811
+ ## Step 10 β€” CI/CD with GitHub Actions
812
+
813
+ Every time you push code to GitHub, the pipeline automatically runs tests, builds a Docker image, pushes it to Docker Hub, and deploys the new version to Render β€” all without any manual steps.
814
+
815
+ ### `.github/workflows/ci-cd.yml`
816
+
817
+ The pipeline has three jobs that run in order. If any job fails, the next one doesn't start.
818
+
819
+ ```
820
+ git push to main
821
+ β”‚
822
+ β–Ό
823
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”
824
+ β”‚ test β”‚ Install Python 3.11 + deps
825
+ β”‚ β”‚ Download model files from GitHub Release
826
+ β”‚ β”‚ Run pytest tests/test_api.py (14 tests)
827
+ β””β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”˜
828
+ β”‚ all pass
829
+ β–Ό
830
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
831
+ β”‚ build-and-push β”‚ Download model files from GitHub Release
832
+ β”‚ β”‚ docker build (using Dockerfile)
833
+ β”‚ β”‚ docker push β†’ Docker Hub :latest + :commit-sha
834
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
835
+ β”‚ image pushed
836
+ β–Ό
837
+ β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
838
+ β”‚ deploy β”‚ POST to Render deploy hook URL
839
+ β”‚ β”‚ Render pulls new image β†’ live URL updates
840
+ β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
841
+ ```
842
+
843
+ **Why download models from GitHub Release in CI?**
844
+ The model files (`xgboost_tuned.pkl`, `scaler.pkl`) are in `.gitignore` so they don't get pushed to GitHub. Instead, they are uploaded once as assets on a GitHub Release (`v1.0.0`). Every CI run downloads them fresh before running tests and before building Docker.
845
+
846
+ **Why only run `test_api.py` in CI and not all 28 tests?**
847
+ `test_preprocessing.py` and `test_model.py` need the full processed dataset (~500 MB of `.pkl` files). Downloading all of that in CI on every push is wasteful. `test_api.py` (14 tests) covers the thing that actually gets deployed β€” the API. The training pipeline tests run locally.
848
+
849
+ **Two Docker tags per push:**
850
+ - `:latest` β€” always points to the newest version
851
+ - `:abc1234` (git commit SHA) β€” lets you roll back to any exact version
852
+
853
+ ---
854
+
855
+ ### Step 10 Setup β€” What You Need to Do
856
+
857
+ #### 1. Create the GitHub repository and push
858
+
859
+ ```bash
860
+ # On GitHub: create a new public repo named "fraud-detection" (no README, no .gitignore)
861
+ # Then in your project folder:
862
+ git remote add origin https://github.com/YOUR-USERNAME/fraud-detection.git
863
+ git push -u origin master
864
+ ```
865
+
866
+ #### 2. Create a GitHub Release with the model files
867
+
868
+ The CI pipeline downloads models from Release `v1.0.0`. Create it once:
869
+
870
+ 1. Go to your GitHub repo β†’ **Releases** β†’ **Create a new release**
871
+ 2. Tag: `v1.0.0` | Title: `v1.0.0 β€” initial model`
872
+ 3. Click **Attach binaries** and upload these two files from your local `models/` folder:
873
+ - `xgboost_tuned.pkl`
874
+ - `scaler.pkl`
875
+ 4. Click **Publish release**
876
+
877
+ #### 3. Create a Docker Hub account and access token
878
+
879
+ 1. Sign up at [hub.docker.com](https://hub.docker.com) (free)
880
+ 2. Go to **Account Settings β†’ Security β†’ New Access Token**
881
+ 3. Name it `github-actions`, permission: Read & Write
882
+ 4. Copy the token (shown only once)
883
+
884
+ #### 4. Add GitHub Secrets
885
+
886
+ Go to your GitHub repo β†’ **Settings β†’ Secrets and variables β†’ Actions β†’ New repository secret**. Add three secrets:
887
+
888
+ | Secret name | Value |
889
+ |-------------|-------|
890
+ | `DOCKERHUB_USERNAME` | Your Docker Hub username |
891
+ | `DOCKERHUB_TOKEN` | The access token from step 3 |
892
+ | `RENDER_DEPLOY_HOOK` | The URL from Step 11 below (add after setting up Render) |
893
+
894
+ #### 5. Push any change to trigger the pipeline
895
+
896
+ ```bash
897
+ git push origin master
898
+ ```
899
+
900
+ Go to your repo β†’ **Actions** tab to watch the pipeline run live. Each job shows a green tick when it passes.
901
+
902
+ ---
903
+
904
+ ## Step 11 β€” Deploy on Render
905
+
906
+ Render is a free cloud platform. Once set up, every successful CI/CD run automatically redeploys the live API β€” no manual steps.
907
 
908
+ ### What the free tier gives you
909
 
910
+ | Property | Value |
911
+ |----------|-------|
912
+ | RAM | 512 MB |
913
+ | CPU | 0.1 vCPU |
914
+ | Cost | Free |
915
+ | HTTPS | Automatic (free SSL certificate) |
916
+ | Cold start | ~30 seconds after 15 min of no traffic |
917
+ | Custom domain | Supported |
918
+
919
+ The cold start means the first request after a period of inactivity takes ~30 seconds. After that, responses are under 100ms. This is acceptable for a portfolio/demo project.
920
+
921
+ ### Step 11 Setup β€” What You Need to Do
922
+
923
+ **Prerequisites:** Step 10 must be done first β€” Docker Hub must have your image pushed.
924
+
925
+ #### 1. Create a Render account
926
+
927
+ Sign up at [render.com](https://render.com) (free, no credit card needed).
928
+
929
+ #### 2. Create a new Web Service
930
+
931
+ 1. Click **New β†’ Web Service**
932
+ 2. Choose **"Deploy an existing image from a registry"**
933
+ 3. Image URL: `your-dockerhub-username/fraud-detection-api:latest`
934
+ 4. Click **Connect**
935
+
936
+ #### 3. Configure the service
937
+
938
+ | Setting | Value |
939
+ |---------|-------|
940
+ | Name | `fraud-detection-api` |
941
+ | Region | Oregon (US West) β€” fastest free tier |
942
+ | Instance Type | **Free** |
943
+ | Port | `8000` |
944
+
945
+ Click **Create Web Service**. Render will pull the Docker image and deploy it. This takes 2–3 minutes the first time.
946
+
947
+ #### 4. Verify it's live
948
+
949
+ Once deployed, Render gives you a URL like:
950
+ ```
951
+ https://fraud-detection-api.onrender.com
952
  ```
953
+
954
+ Test it:
955
+ ```bash
956
+ # Health check
957
+ curl https://fraud-detection-api.onrender.com/health
958
+ # β†’ {"status": "ok", "model_loaded": true}
959
+
960
+ # Fraud prediction
961
+ curl -X POST https://fraud-detection-api.onrender.com/predict \
962
+ -H "Content-Type: application/json" \
963
+ -d '{
964
+ "Time": 406, "Amount": 0.0,
965
+ "V1": -2.3122, "V2": 1.9519, "V3": -1.6097, "V4": 3.9979,
966
+ "V5": -0.5222, "V6": -1.4265, "V7": -2.5374, "V8": 1.3914,
967
+ "V9": -2.7700, "V10": -2.7722, "V11": 3.2020, "V12": -2.8992,
968
+ "V13": -0.5950, "V14": -4.2895, "V15": 0.3898, "V16": -1.1407,
969
+ "V17": -2.8300, "V18": -0.0168, "V19": 0.4165, "V20": 0.3269,
970
+ "V21": 0.1474, "V22": -0.1703, "V23": 0.0359, "V24": -0.4118,
971
+ "V25": 0.0714, "V26": 0.0719, "V27": 0.2127, "V28": 0.0952
972
+ }'
973
+ # β†’ {"is_fraud": true, "fraud_probability": 1.0, "inference_ms": 4.2}
974
+
975
+ # Interactive Swagger UI
976
+ # Open in browser: https://fraud-detection-api.onrender.com/docs
977
  ```
978
 
979
+ #### 5. Get the Deploy Hook URL and add it to GitHub Secrets
980
+
981
+ 1. In Render β†’ your service β†’ **Settings** β†’ scroll down to **Deploy Hook**
982
+ 2. Copy the URL (looks like `https://api.render.com/deploy/srv-xxxxx?key=xxxxx`)
983
+ 3. Go back to GitHub β†’ **Settings β†’ Secrets β†’ Actions**
984
+ 4. Add secret: `RENDER_DEPLOY_HOOK` = the URL you copied
985
+
986
+ From now on, every `git push` to `main` triggers the full pipeline: tests β†’ Docker build β†’ Docker push β†’ Render redeploy β†’ live URL updated.
987
+
988
  ---
989
 
990
  ## Dataset