gary-boon commited on
Commit
0e48dc7
·
1 Parent(s): 767a3fd

Add GitHub Actions workflow for security scanning and automated deployment

Browse files

- Implement Snyk security scanning for Python dependencies
- Add Bandit for Python code security analysis
- Include Safety checks for known vulnerabilities
- Configure automated deployment to HuggingFace Spaces after security checks
- Create comprehensive security documentation
- Address known issue: transformers==4.35.0 has vulnerabilities (needs update)

.github/workflows/security-and-deploy.yml ADDED
@@ -0,0 +1,141 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ name: Security Check and Deploy to HuggingFace
2
+
3
+ on:
4
+ push:
5
+ branches: [ main ]
6
+ pull_request:
7
+ branches: [ main ]
8
+ workflow_dispatch:
9
+
10
+ jobs:
11
+ security-check:
12
+ runs-on: ubuntu-latest
13
+
14
+ steps:
15
+ - uses: actions/checkout@v3
16
+
17
+ - name: Set up Python
18
+ uses: actions/setup-python@v4
19
+ with:
20
+ python-version: '3.10'
21
+
22
+ - name: Install dependencies
23
+ run: |
24
+ python -m pip install --upgrade pip
25
+ pip install -r requirements.txt
26
+
27
+ # Run Snyk security scan
28
+ - name: Run Snyk Security Scan
29
+ env:
30
+ SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
31
+ run: |
32
+ # Install Snyk CLI
33
+ curl -Lo snyk https://static.snyk.io/cli/latest/snyk-linux
34
+ chmod +x snyk
35
+
36
+ # Authenticate with Snyk
37
+ ./snyk auth $SNYK_TOKEN
38
+
39
+ echo "===== Python Dependency Security Scan ====="
40
+ # Test for vulnerabilities (non-blocking initially)
41
+ ./snyk test --severity-threshold=high --file=requirements.txt || true
42
+
43
+ # Generate SARIF report for GitHub Security tab
44
+ ./snyk test --severity-threshold=low --file=requirements.txt --sarif-file-output=snyk.sarif || true
45
+
46
+ echo "===== Security Scan Complete ====="
47
+ continue-on-error: true
48
+
49
+ # Upload results to GitHub Security tab
50
+ - name: Upload Snyk results to GitHub Code Scanning
51
+ if: always()
52
+ uses: github/codeql-action/upload-sarif@v3
53
+ continue-on-error: true
54
+ with:
55
+ sarif_file: snyk.sarif
56
+ category: snyk-python
57
+
58
+ # Monitor project with Snyk (updates dashboard)
59
+ - name: Monitor with Snyk
60
+ if: github.event_name == 'push' && github.ref == 'refs/heads/main'
61
+ env:
62
+ SNYK_TOKEN: ${{ secrets.SNYK_TOKEN }}
63
+ run: |
64
+ ./snyk monitor --file=requirements.txt --project-name=visualisable-ai-backend || true
65
+ continue-on-error: true
66
+
67
+ # Run Python security checks with bandit
68
+ - name: Run Bandit Security Linter
69
+ run: |
70
+ pip install bandit
71
+ echo "===== Python Code Security Analysis ====="
72
+ bandit -r backend/ -f json -o bandit-report.json || true
73
+ bandit -r backend/ || true
74
+ echo "===== Code Analysis Complete ====="
75
+ continue-on-error: true
76
+
77
+ # Run safety check for known security vulnerabilities
78
+ - name: Run Safety Check
79
+ run: |
80
+ pip install safety
81
+ echo "===== Safety Vulnerability Check ====="
82
+ safety check --json > safety-report.json || true
83
+ safety check || true
84
+ echo "===== Safety Check Complete ====="
85
+ continue-on-error: true
86
+
87
+ - name: Security Summary
88
+ if: always()
89
+ run: |
90
+ echo "## Security Scan Summary" >> $GITHUB_STEP_SUMMARY
91
+ echo "" >> $GITHUB_STEP_SUMMARY
92
+ echo "### Checks Performed:" >> $GITHUB_STEP_SUMMARY
93
+ echo "- ✅ Snyk dependency vulnerability scan" >> $GITHUB_STEP_SUMMARY
94
+ echo "- ✅ Bandit Python security linter" >> $GITHUB_STEP_SUMMARY
95
+ echo "- ✅ Safety known vulnerability check" >> $GITHUB_STEP_SUMMARY
96
+ echo "" >> $GITHUB_STEP_SUMMARY
97
+ echo "### Known Issues to Address:" >> $GITHUB_STEP_SUMMARY
98
+ echo "- transformers==4.35.0 has known vulnerabilities" >> $GITHUB_STEP_SUMMARY
99
+ echo "- Consider upgrading to transformers>=4.36.0" >> $GITHUB_STEP_SUMMARY
100
+ echo "" >> $GITHUB_STEP_SUMMARY
101
+ echo "Check the logs above for detailed findings." >> $GITHUB_STEP_SUMMARY
102
+
103
+ deploy-to-huggingface:
104
+ runs-on: ubuntu-latest
105
+ needs: security-check
106
+ if: github.event_name == 'push' && github.ref == 'refs/heads/main'
107
+
108
+ steps:
109
+ - uses: actions/checkout@v3
110
+ with:
111
+ fetch-depth: 0
112
+ lfs: true
113
+
114
+ - name: Configure Git
115
+ run: |
116
+ git config --global user.email "github-actions[bot]@users.noreply.github.com"
117
+ git config --global user.name "github-actions[bot]"
118
+
119
+ - name: Push to HuggingFace Space
120
+ env:
121
+ HF_TOKEN: ${{ secrets.HF_TOKEN }}
122
+ run: |
123
+ # Add HuggingFace remote with authentication
124
+ git remote add huggingface https://visualisable-ai:$HF_TOKEN@huggingface.co/spaces/visualisable-ai/api
125
+
126
+ # Push to HuggingFace
127
+ git push huggingface main:main --force
128
+
129
+ echo "✅ Deployed to HuggingFace Spaces successfully!" >> $GITHUB_STEP_SUMMARY
130
+ echo "🔗 View at: https://huggingface.co/spaces/visualisable-ai/api" >> $GITHUB_STEP_SUMMARY
131
+ continue-on-error: true
132
+
133
+ - name: Deployment Status
134
+ if: always()
135
+ run: |
136
+ if [ $? -eq 0 ]; then
137
+ echo "### Deployment Status: ✅ Success" >> $GITHUB_STEP_SUMMARY
138
+ else
139
+ echo "### Deployment Status: ⚠️ Manual deployment may be needed" >> $GITHUB_STEP_SUMMARY
140
+ echo "Run 'git push origin main' locally if automatic deployment failed" >> $GITHUB_STEP_SUMMARY
141
+ fi
SECURITY.md ADDED
@@ -0,0 +1,149 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Security Configuration for Visualisable AI Backend
2
+
3
+ This document explains the security scanning setup for the Python backend that powers the HuggingFace Spaces API.
4
+
5
+ ## Overview
6
+
7
+ The backend repository now includes automated security scanning via GitHub Actions that runs before deployment to HuggingFace Spaces. This ensures that security vulnerabilities are identified and can be addressed before code reaches production.
8
+
9
+ ## Security Tools Used
10
+
11
+ 1. **Snyk** - Comprehensive vulnerability scanning for Python dependencies
12
+ 2. **Bandit** - Python-specific security linter for common security issues
13
+ 3. **Safety** - Checks Python dependencies against known security vulnerabilities database
14
+
15
+ ## Setup Instructions
16
+
17
+ ### 1. GitHub Repository Setup
18
+
19
+ Since this repository currently only has HuggingFace as a remote, you'll need to:
20
+
21
+ 1. Create a GitHub repository for the backend:
22
+ ```bash
23
+ # Add GitHub as a remote
24
+ git remote add github https://github.com/YOUR_USERNAME/visualisable-ai-backend.git
25
+
26
+ # Push to GitHub
27
+ git push github main
28
+ ```
29
+
30
+ 2. Enable GitHub Actions in the repository settings
31
+
32
+ ### 2. Required Secrets
33
+
34
+ Add the following secrets to your GitHub repository (Settings → Secrets and variables → Actions):
35
+
36
+ #### SNYK_TOKEN
37
+ 1. Sign up for free at https://snyk.io
38
+ 2. Go to Account Settings → Auth Token
39
+ 3. Copy your personal auth token
40
+ 4. Add as `SNYK_TOKEN` in GitHub secrets
41
+
42
+ #### HF_TOKEN (for automated deployment)
43
+ 1. Go to https://huggingface.co/settings/tokens
44
+ 2. Create a new token with write access
45
+ 3. Add as `HF_TOKEN` in GitHub secrets
46
+
47
+ ## Workflow Features
48
+
49
+ The `security-and-deploy.yml` workflow:
50
+
51
+ 1. **Runs on every push and PR** to the main branch
52
+ 2. **Security scanning includes:**
53
+ - Dependency vulnerability scanning with Snyk
54
+ - Code security analysis with Bandit
55
+ - Known vulnerability checking with Safety
56
+ - Results uploaded to GitHub Security tab
57
+ - Project monitoring in Snyk dashboard
58
+
59
+ 3. **Automated deployment** (only on main branch):
60
+ - After security checks pass
61
+ - Pushes directly to HuggingFace Spaces
62
+ - Maintains deployment history in GitHub
63
+
64
+ ## Current Security Status
65
+
66
+ ### Known Issues
67
+
68
+ **transformers==4.35.0** has known vulnerabilities:
69
+ - Multiple security issues have been fixed in newer versions
70
+ - Recommended upgrade: `transformers>=4.36.0`
71
+
72
+ ### To Fix Vulnerabilities
73
+
74
+ 1. Update `requirements.txt`:
75
+ ```txt
76
+ transformers==4.36.2 # or latest stable version
77
+ ```
78
+
79
+ 2. Test locally:
80
+ ```bash
81
+ pip install -r requirements.txt
82
+ python -m pytest # if you have tests
83
+ python app.py # test the application
84
+ ```
85
+
86
+ 3. Commit and push:
87
+ ```bash
88
+ git add requirements.txt
89
+ git commit -m "Security: Update transformers to fix vulnerabilities"
90
+ git push github main # triggers security scan and deployment
91
+ ```
92
+
93
+ ## Local Security Testing
94
+
95
+ Run security checks locally before pushing:
96
+
97
+ ```bash
98
+ # Install tools
99
+ pip install snyk bandit safety
100
+
101
+ # Run Snyk (requires authentication)
102
+ snyk auth
103
+ snyk test
104
+
105
+ # Run Bandit
106
+ bandit -r backend/
107
+
108
+ # Run Safety
109
+ safety check
110
+ ```
111
+
112
+ ## Monitoring
113
+
114
+ - **GitHub Security Tab**: View SARIF reports and security alerts
115
+ - **Snyk Dashboard**: https://app.snyk.io - Monitor all vulnerabilities
116
+ - **GitHub Actions**: Check workflow runs for security scan results
117
+
118
+ ## Security Best Practices
119
+
120
+ 1. **Regular Updates**: Keep dependencies updated to latest secure versions
121
+ 2. **Monitor Alerts**: Check Snyk dashboard weekly for new vulnerabilities
122
+ 3. **Test Before Deploy**: Always test locally after updating dependencies
123
+ 4. **Review PR Scans**: Security scans run on PRs - review before merging
124
+
125
+ ## Troubleshooting
126
+
127
+ ### Workflow not running
128
+ - Ensure GitHub Actions is enabled in repository settings
129
+ - Check that secrets are properly configured
130
+ - Verify the workflow file is in `.github/workflows/`
131
+
132
+ ### Deployment failing
133
+ - Check HF_TOKEN has write permissions
134
+ - Ensure HuggingFace Space name matches in workflow
135
+ - Manual deployment fallback: `git push origin main`
136
+
137
+ ### Security scan failures
138
+ - Non-blocking by default (continue-on-error: true)
139
+ - Review logs for specific vulnerabilities
140
+ - Update dependencies to fix issues
141
+ - Can be made blocking by removing continue-on-error
142
+
143
+ ## Next Steps
144
+
145
+ 1. Push this repository to GitHub
146
+ 2. Configure the required secrets
147
+ 3. Run the workflow to establish baseline security status
148
+ 4. Address the transformers vulnerability
149
+ 5. Consider making security checks blocking after initial setup