Wuhp commited on
Commit ·
d663d8f
1
Parent(s): 6635b77
Build HackerOne scope workspace and ZAP passive scanner API
Browse files- .gitignore +8 -0
- Dockerfile +11 -0
- README.md +83 -1
- app/__init__.py +0 -0
- app/index.html +28 -0
- app/main.py +214 -0
- app/scope.py +103 -0
- requirements.txt +5 -0
- start.sh +6 -0
- tests/test_scope.py +25 -0
.gitignore
ADDED
|
@@ -0,0 +1,8 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
__pycache__/
|
| 2 |
+
.pytest_cache/
|
| 3 |
+
*.pyc
|
| 4 |
+
.env
|
| 5 |
+
*.sqlite
|
| 6 |
+
*.db
|
| 7 |
+
results/
|
| 8 |
+
node_modules/
|
Dockerfile
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
FROM zaproxy/zap-stable
|
| 2 |
+
USER root
|
| 3 |
+
RUN python3 -m venv /opt/asfb-venv && /opt/asfb-venv/bin/pip install --no-cache-dir fastapi==0.115.12 uvicorn==0.34.2 python-multipart==0.0.20 httpx==0.28.1
|
| 4 |
+
WORKDIR /app
|
| 5 |
+
COPY --chown=zap:zap app /app/app
|
| 6 |
+
COPY --chown=zap:zap start.sh /app/start.sh
|
| 7 |
+
RUN chmod +x /app/start.sh
|
| 8 |
+
USER zap
|
| 9 |
+
ENV PATH="/opt/asfb-venv/bin:$PATH" PYTHONUNBUFFERED=1
|
| 10 |
+
EXPOSE 7860
|
| 11 |
+
CMD ["/app/start.sh"]
|
README.md
CHANGED
|
@@ -1 +1,83 @@
|
|
| 1 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: ASFB Bug Bounty Scanner
|
| 3 |
+
emoji: 🔎
|
| 4 |
+
sdk: docker
|
| 5 |
+
app_port: 7860
|
| 6 |
+
---
|
| 7 |
+
|
| 8 |
+
# ASFB — Bug bounty scope and ZAP workspace
|
| 9 |
+
|
| 10 |
+
A self-hosted, API-driven workbench for importing HackerOne scope exports, prioritizing web targets, configuring testing identities, and running **rate-limited passive checks** using the standard OWASP ZAP daemon.
|
| 11 |
+
|
| 12 |
+
> This is an initial working release, not a comprehensive automated vulnerability detector. It does not prove IDORs or RSA/JWT weaknesses, run Nuclei, or autonomously scan wildcards. Those are future modules.
|
| 13 |
+
|
| 14 |
+
## Features
|
| 15 |
+
- Upload CSV scope exports; recognizes common HackerOne column names (asset name/identifier, type, bounty, severity, resolved reports, last update, instructions).
|
| 16 |
+
- API and web UI filtering by reward eligibility, severity, scanner compatibility, text and report volume.
|
| 17 |
+
- Preserves non-web assets and wildcard entries for manual review instead of automatically treating them as HTTP scan targets.
|
| 18 |
+
- Program-specific scan-policy approval, exact-host scope enforcement, SSRF checks, URL/path exclusions, and small default request budgets.
|
| 19 |
+
- Request profiles with custom User-Agent and account-specific cookies/headers. Credentials exist **in process memory only** and are not serialized to files, databases or reports.
|
| 20 |
+
- Sequential background jobs, stop/status endpoints, and ZAP passive alerts.
|
| 21 |
+
- API-key authentication on the backend; API docs at `/docs`.
|
| 22 |
+
|
| 23 |
+
## Deployment
|
| 24 |
+
Create a **private** Hugging Face Docker Space from this repository (or run Docker locally). Set secrets:
|
| 25 |
+
- `ASFB_API_KEY`: a long random API key **required on startup**.
|
| 26 |
+
- `ZAP_API_KEY`: a separate random API key **required on startup**.
|
| 27 |
+
|
| 28 |
+
Configure the frontend by opening the Space, entering `ASFB_API_KEY` into the UI. The browser keeps the key only in memory until the page is reloaded. Do not distribute this key publicly.
|
| 29 |
+
|
| 30 |
+
```bash
|
| 31 |
+
docker build -t asfbb .
|
| 32 |
+
docker run --rm -p 7860:7860 -e ASFB_API_KEY=change-me-long-random-value -e ZAP_API_KEY=another-long-random-value asfbb
|
| 33 |
+
```
|
| 34 |
+
|
| 35 |
+
A free Hugging Face Space may sleep and its working disk is ephemeral. This release keeps all data in memory. Do not rely on it for durable findings/history. For public/multiuser deployment, add per-user authentication, a durable encrypted database and stricter queue isolation first.
|
| 36 |
+
|
| 37 |
+
## Workflow
|
| 38 |
+
1. Import a HackerOne CSV via the dashboard or `POST /api/import` multipart upload.
|
| 39 |
+
2. Review the parsed assets, their eligibility and descriptions.
|
| 40 |
+
3. Add a policy acknowledgement and verify the program permits the proposed automation.
|
| 41 |
+
4. Choose an individual, unambiguous domain/URL row. Wildcards, category/group rows, mobile/source-code assets are not auto-scannable.
|
| 42 |
+
5. Configure an optional account identity in the UI. Login credentials are not required for unauthenticated scans.
|
| 43 |
+
6. Start a passive ZAP scan. The job makes up to the configured number of scoped GET requests; no active scanner payloads are sent.
|
| 44 |
+
7. Review the findings and investigate manually.
|
| 45 |
+
|
| 46 |
+
### API examples
|
| 47 |
+
```bash
|
| 48 |
+
curl -H "X-ASFB-Key: $ASFB_API_KEY" -F "file=@scope.csv" http://localhost:7860/api/import
|
| 49 |
+
curl -H "X-ASFB-Key: $ASFB_API_KEY" http://localhost:7860/api/assets
|
| 50 |
+
```
|
| 51 |
+
|
| 52 |
+
`POST /api/scans` example (replace asset_id with an imported ID):
|
| 53 |
+
```json
|
| 54 |
+
{
|
| 55 |
+
"asset_id": "ASSET_ID",
|
| 56 |
+
"policy_confirmed": true,
|
| 57 |
+
"user_agent": "ASFB-research/0.1",
|
| 58 |
+
"max_requests": 3,
|
| 59 |
+
"delay_seconds": 1.5,
|
| 60 |
+
"account_id": null
|
| 61 |
+
}
|
| 62 |
+
```
|
| 63 |
+
|
| 64 |
+
`POST /api/accounts` accepts `{ "name": "A", "host": "example.org", "cookie": "session=...", "headers": {"X-Testing": "yes"} }`. Accounts are stored only until the process restarts, not included in API responses, and restricted to their exact host.
|
| 65 |
+
|
| 66 |
+
## Limitations and safety
|
| 67 |
+
- Imported scope does not override a program's rules. Scope tables do not consistently include automation prohibitions; check them manually before confirming.
|
| 68 |
+
- A listed source-code path or internal-service wildcard is not interpreted as authorization for external web scans.
|
| 69 |
+
- DNS is checked before a job; network changes, redirects, proxy behavior and complex authorization still require hardened egress controls. The scan fetcher disallows redirects and rechecks resolved addresses. Deploy on isolated infrastructure for high-assurance SSRF control.
|
| 70 |
+
- This MVP uses one backend, one ZAP daemon, one active job at a time; it is not a public SaaS or multi-tenant service.
|
| 71 |
+
- Only HTTP(S) ports 80, 443, 8080 are supported by the default target validator (matching common HF egress ports).
|
| 72 |
+
- Authentication profiles can become stale; do not use production customer credentials.
|
| 73 |
+
- Findings from passive rules are signals, not verified vulnerabilities.
|
| 74 |
+
- The ZAP upstream base image is tagged `stable`; pin it to a digest before production.
|
| 75 |
+
|
| 76 |
+
## Tests
|
| 77 |
+
```bash
|
| 78 |
+
python -m pip install -r requirements.txt
|
| 79 |
+
python -m pytest -q
|
| 80 |
+
```
|
| 81 |
+
|
| 82 |
+
## Next milestones
|
| 83 |
+
OpenAPI discovery, per-program policy manifests, scoped crawling, differential two-account IDOR replay, verified JWT rule pack, vulnerability triage, persistence and scan-history diffing.
|
app/__init__.py
ADDED
|
File without changes
|
app/index.html
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
<!doctype html>
|
| 2 |
+
<html lang="en"><head><meta charset="utf-8"/><meta name="viewport" content="width=device-width,initial-scale=1"/>
|
| 3 |
+
<title>ASFB — Scope & Scan Workspace</title>
|
| 4 |
+
<style>
|
| 5 |
+
:root{font-family:Inter,system-ui,Arial,sans-serif;color:#24223a;background:#f8f7fc}*{box-sizing:border-box}body{margin:0}.shell{max-width:1240px;margin:auto;padding:26px 18px}header{display:flex;justify-content:space-between;align-items:center;gap:16px;flex-wrap:wrap}h1{margin:0;font-size:27px;letter-spacing:-1px}h2{font-size:17px;margin:0 0 15px}small,.muted{color:#77738b}.tag{background:#ede8fb;color:#684ba9;border-radius:30px;padding:5px 10px;font-size:12px}.card{background:white;border:1px solid #e9e4f0;border-radius:16px;padding:20px;margin-top:18px;box-shadow:0 3px 20px #33265708}.grid{display:grid;grid-template-columns:1.1fr 1fr;gap:18px}.controls{display:flex;flex-wrap:wrap;gap:9px;align-items:center}button{background:#7959c8;color:#fff;border:0;border-radius:9px;padding:10px 15px;cursor:pointer;font-weight:650}button:disabled{opacity:.45;cursor:not-allowed}button.secondary{background:#f1edfa;color:#6545aa}input,select,textarea{padding:10px;border:1px solid #dcd5eb;border-radius:9px;font:inherit;background:white;max-width:100%}label{font-size:13px;color:#56506f}.field{display:flex;flex-direction:column;gap:5px;min-width:140px;flex:1}.field input{width:100%}table{border-collapse:collapse;width:100%;font-size:13px}th,td{text-align:left;padding:11px 9px;border-bottom:1px solid #eeebf2;vertical-align:top}th{color:#736e85;font-size:11px;text-transform:uppercase}table td:first-child{overflow-wrap:anywhere}tr:hover td{background:#faf8ff}.scroll{max-height:480px;overflow:auto}.stats{display:flex;gap:10px;flex-wrap:wrap}.stat{padding:13px 20px;background:#f6f2ff;border-radius:12px;min-width:120px}.stat b{font-size:22px;display:block}.notice{padding:12px 14px;border-radius:9px;background:#f8f4e8;font-size:13px;margin-top:12px}.error{color:#a3344c}.ok{color:#376c52}.topbar{display:flex;align-items:center;gap:8px}.results{white-space:pre-wrap;overflow-wrap:anywhere;font-size:12px;background:#f8f7fc;padding:12px;border-radius:8px;max-height:360px;overflow:auto}@media(max-width:780px){.grid{grid-template-columns:1fr}.shell{padding:14px}.card{padding:14px}table{font-size:12px}th,td{padding:7px 4px}}
|
| 6 |
+
</style></head><body><main class="shell">
|
| 7 |
+
<header><div><h1>ASFB <span class="tag">v0.1</span></h1><div class="muted">Bug bounty scope manager · OWASP ZAP passive analysis</div></div><div class="topbar"><input id="apiKey" type="password" placeholder="ASFB API key" autocomplete="off"/><button onclick="refresh()">Connect</button></div></header>
|
| 8 |
+
<div class="notice">Importing scope is not approval to scan. Check each program's automation rules and test only accounts/assets you are authorized to use. Scanning is passive, one asset at a time, and disabled until you confirm policy.</div>
|
| 9 |
+
<div class="grid"><section class="card"><h2>1. Import HackerOne scope</h2><div class="controls"><input id="file" type="file" accept=".csv,text/csv"/><button onclick="upload()">Import CSV</button></div><p class="muted">Headers supported: Asset name, Type, Coverage, Max. severity, Bounty, Last update, Resolved Reports, plus common API export variants.</p></section>
|
| 10 |
+
<section class="card"><h2>2. Scan settings</h2><div class="controls"><div class="field"><label>Custom User-Agent</label><input id="agent" value="ASFB-Research/0.1"/></div><div class="field"><label>Requests (max 10)</label><input id="requests" type="number" min="1" max="10" value="1"/></div></div><div class="controls" style="margin-top:10px"><div class="field"><label>Identity</label><select id="account"><option value="">Anonymous</option></select></div><div class="field"><label>Delay, seconds</label><input id="delay" type="number" value="1.5" min="1" max="30" step=".5"/></div></div><div style="margin-top:15px"><label><input type="checkbox" id="policy"/> I have checked this program's rules and confirm this passive automated testing is permitted.</label></div><p id="selected" class="muted">Select a compatible asset below.</p><button id="start" disabled onclick="scan()">Start passive scan</button></section></div>
|
| 11 |
+
<section class="card"><h2>Scope assets</h2><div class="controls"><div class="field"><label>Search</label><input id="search" placeholder="Hostname / description" oninput="render()"/></div><div class="field"><label>Minimum max. severity</label><select id="severity" onchange="render()"><option value="all">All</option><option value="medium">Medium+</option><option value="high">High+</option><option value="critical">Critical only</option></select></div><div class="field"><label>Maximum resolved reports</label><input id="reports" type="number" min="0" placeholder="Any" oninput="render()"/></div></div><div class="controls" style="margin-top:13px"><label><input id="bounty" type="checkbox" onchange="render()"/> Reward eligible only</label><label><input id="compatible" type="checkbox" onchange="render()"/> ZAP-compatible only</label></div><div class="stats" style="margin:15px 0"><div class="stat"><b id="total">0</b><small>Imported assets</small></div><div class="stat"><b id="filtered">0</b><small>Filtered assets</small></div></div><div class="scroll"><table><thead><tr><th>Choose</th><th>Asset / description</th><th>Type</th><th>Max. severity</th><th>Bounty</th><th>Resolved</th></tr></thead><tbody id="rows"></tbody></table></div></section>
|
| 12 |
+
<div class="grid"><section class="card"><h2>Account profiles (session only)</h2><div class="controls"><div class="field"><label>Account label</label><input id="aname" placeholder="Test account A"/></div><div class="field"><label>Exact host</label><input id="ahost" placeholder="api.example.org"/></div></div><div class="field" style="margin-top:10px"><label>Cookie header (not persisted)</label><input id="acookie" type="password" autocomplete="off" placeholder="session=..."/></div><button style="margin-top:12px" onclick="addAccount()">Add account</button><p class="muted">Do not use other users' credentials. Cookies stay in backend process memory, not disk.</p></section>
|
| 13 |
+
<section class="card"><h2>Scan queue and findings</h2><button class="secondary" onclick="loadScans()">Refresh scans</button><div id="scans" class="results">No scans yet.</div></section></div>
|
| 14 |
+
<p id="message" class="muted"></p></main>
|
| 15 |
+
<script>
|
| 16 |
+
let assets=[],selectedId=null;
|
| 17 |
+
const el=id=>document.getElementById(id);
|
| 18 |
+
const esc=s=>String(s??"").replace(/[&<>"']/g,c=>({'&':'&','<':'<','>':'>','"':'"',"'":'''}[c]));
|
| 19 |
+
async function api(path,options={}){const h={'X-ASFB-Key':el('apiKey').value,...(options.headers||{})};const r=await fetch('/api'+path,{...options,headers:h});const text=await r.text();let data;try{data=JSON.parse(text)}catch{data={detail:text}}if(!r.ok)throw Error(typeof data.detail==='string'?data.detail:JSON.stringify(data.detail||data));return data}
|
| 20 |
+
function message(text,error=false){el('message').textContent=text;el('message').className=error?'error':'muted'}
|
| 21 |
+
async function refresh(){try{const r=await api('/assets');assets=r.assets;render();await loadAccounts();await loadScans();message('Connected.')}catch(e){message(e.message,true)}}
|
| 22 |
+
function render(){const ranks={unknown:-1,none:0,low:1,medium:2,high:3,critical:4}, min=el('severity').value, q=el('search').value.toLowerCase(),max=el('reports').value;const rows=assets.filter(a=>(!el('bounty').checked||a.bounty)&&(!el('compatible').checked||a.compatibility==='web')&&(min==='all'||ranks[a.max_severity]>=ranks[min])&&(max===''||a.resolved_reports!==null&&a.resolved_reports<=Number(max))&&((a.name+' '+a.description).toLowerCase().includes(q)));el('total').textContent=assets.length;el('filtered').textContent=rows.length;el('rows').innerHTML=rows.map(a=>'<tr><td><input type="radio" name="asset" '+(selectedId===a.id?'checked ':'')+(a.compatibility!=='web'?'disabled ':'')+'data-id="'+esc(a.id)+'"></td><td><b>'+esc(a.name)+'</b><br><small>'+esc(a.description)+'</small></td><td>'+esc(a.type)+'<br><small>'+esc(a.compatibility)+'</small></td><td>'+esc(a.max_severity)+'</td><td>'+(a.bounty?'Eligible':'No')+'</td><td>'+esc(a.resolved_reports??'—')+'</td></tr>').join('');document.querySelectorAll('[data-id]').forEach(n=>n.addEventListener('change',()=>{selectedId=n.dataset.id;el('selected').textContent=assets.find(a=>a.id===selectedId)?.name||'';el('start').disabled=false}))}
|
| 23 |
+
async function upload(){const f=el('file').files[0];if(!f)return message('Choose a CSV file',true);const d=new FormData();d.append('file',f);try{const r=await api('/import',{method:'POST',body:d});message('Imported '+r.imported+' assets ('+r.web+' web-compatible).');await refresh()}catch(e){message(e.message,true)}}
|
| 24 |
+
async function addAccount(){const body={name:el('aname').value,host:el('ahost').value,cookie:el('acookie').value};try{await api('/accounts',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(body)});el('acookie').value='';await loadAccounts();message('Account profile added (memory only).')}catch(e){message(e.message,true)}}
|
| 25 |
+
async function loadAccounts(){const a=await api('/accounts');el('account').innerHTML='<option value="">Anonymous</option>'+a.map(v=>'<option value="'+esc(v.id)+'">'+esc(v.name+' — '+v.host)+'</option>').join('')}
|
| 26 |
+
async function scan(){if(!selectedId)return;const body={asset_id:selectedId,policy_confirmed:el('policy').checked,user_agent:el('agent').value,max_requests:Number(el('requests').value),delay_seconds:Number(el('delay').value),account_id:el('account').value||null};try{const r=await api('/scans',{method:'POST',headers:{'Content-Type':'application/json'},body:JSON.stringify(body)});message('Scan queued: '+r.id);await loadScans()}catch(e){message(e.message,true)}}
|
| 27 |
+
async function loadScans(){try{const r=await api('/scans');el('scans').textContent=JSON.stringify(r.map(x=>({id:x.id,status:x.status,target:x.target,requests:x.request_count,alerts:x.alerts,error:x.error})),null,2)}catch(e){message(e.message,true)}}
|
| 28 |
+
</script></body></html>
|
app/main.py
ADDED
|
@@ -0,0 +1,214 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import asyncio
|
| 2 |
+
import os
|
| 3 |
+
import secrets
|
| 4 |
+
import time
|
| 5 |
+
import uuid
|
| 6 |
+
from threading import Lock
|
| 7 |
+
from urllib.parse import urlsplit
|
| 8 |
+
|
| 9 |
+
import httpx
|
| 10 |
+
from fastapi import FastAPI, File, Header, HTTPException, UploadFile, Depends
|
| 11 |
+
from fastapi.responses import FileResponse
|
| 12 |
+
from pydantic import BaseModel, Field
|
| 13 |
+
|
| 14 |
+
from .scope import LEVELS, parse_csv, public_host, target_url
|
| 15 |
+
|
| 16 |
+
API_KEY = os.environ.get("ASFB_API_KEY", "")
|
| 17 |
+
ZAP_KEY = os.environ.get("ZAP_API_KEY", "")
|
| 18 |
+
ZAP_URL = "http://127.0.0.1:8090"
|
| 19 |
+
app = FastAPI(title="ASFB Scanner API", version="0.1.0", docs_url="/docs")
|
| 20 |
+
assets = {}
|
| 21 |
+
accounts = {}
|
| 22 |
+
jobs = {}
|
| 23 |
+
queue = asyncio.Queue()
|
| 24 |
+
zap_lock = Lock()
|
| 25 |
+
|
| 26 |
+
def authorize(x_asfb_key: str | None = Header(None)):
|
| 27 |
+
if not API_KEY or not secrets.compare_digest(x_asfb_key or "", API_KEY):
|
| 28 |
+
raise HTTPException(401, "Invalid API key")
|
| 29 |
+
|
| 30 |
+
class Account(BaseModel):
|
| 31 |
+
name: str = Field(min_length=1, max_length=100)
|
| 32 |
+
host: str = Field(min_length=4, max_length=253)
|
| 33 |
+
cookie: str = Field(default="", max_length=8000)
|
| 34 |
+
headers: dict[str, str] = Field(default_factory=dict)
|
| 35 |
+
|
| 36 |
+
class ScanRequest(BaseModel):
|
| 37 |
+
asset_id: str
|
| 38 |
+
policy_confirmed: bool = False
|
| 39 |
+
user_agent: str = Field(default="ASFB-Research/0.1", max_length=300)
|
| 40 |
+
max_requests: int = Field(default=1, ge=1, le=10)
|
| 41 |
+
delay_seconds: float = Field(default=1.5, ge=1, le=30)
|
| 42 |
+
account_id: str | None = None
|
| 43 |
+
|
| 44 |
+
@app.on_event("startup")
|
| 45 |
+
async def startup():
|
| 46 |
+
if not API_KEY or not ZAP_KEY:
|
| 47 |
+
raise RuntimeError("ASFB_API_KEY and ZAP_API_KEY are mandatory")
|
| 48 |
+
app.state.worker = asyncio.create_task(worker())
|
| 49 |
+
|
| 50 |
+
@app.on_event("shutdown")
|
| 51 |
+
async def shutdown():
|
| 52 |
+
app.state.worker.cancel()
|
| 53 |
+
|
| 54 |
+
@app.get("/")
|
| 55 |
+
async def index():
|
| 56 |
+
return FileResponse("/app/app/index.html")
|
| 57 |
+
|
| 58 |
+
@app.get("/api/health")
|
| 59 |
+
async def health():
|
| 60 |
+
return {"status": "ok", "zap": bool(ZAP_KEY)}
|
| 61 |
+
|
| 62 |
+
@app.post("/api/import", dependencies=[Depends(authorize)])
|
| 63 |
+
async def import_file(file: UploadFile = File(...)):
|
| 64 |
+
blob = await file.read(2_000_001)
|
| 65 |
+
try:
|
| 66 |
+
rows = parse_csv(blob)
|
| 67 |
+
except (UnicodeDecodeError, ValueError) as exc:
|
| 68 |
+
raise HTTPException(422, str(exc))
|
| 69 |
+
prefix = str(uuid.uuid4())[:8]
|
| 70 |
+
for r in rows:
|
| 71 |
+
r["id"] = prefix + "-" + r["id"]
|
| 72 |
+
assets[r["id"]] = r
|
| 73 |
+
return {"imported": len(rows), "web": sum(r["compatibility"] == "web" for r in rows)}
|
| 74 |
+
|
| 75 |
+
@app.get("/api/assets", dependencies=[Depends(authorize)])
|
| 76 |
+
def list_assets(bounty_only: bool = False, severity: str = "all", compatible_only: bool = False, search: str = "", max_reports: int | None = None):
|
| 77 |
+
rows = list(assets.values())
|
| 78 |
+
if bounty_only:
|
| 79 |
+
rows = [r for r in rows if r["bounty"]]
|
| 80 |
+
if severity != "all":
|
| 81 |
+
if severity not in LEVELS:
|
| 82 |
+
raise HTTPException(422, "Unsupported severity")
|
| 83 |
+
rows = [r for r in rows if LEVELS.get(r["max_severity"], -1) >= LEVELS[severity]]
|
| 84 |
+
if compatible_only:
|
| 85 |
+
rows = [r for r in rows if r["compatibility"] == "web"]
|
| 86 |
+
if search:
|
| 87 |
+
rows = [r for r in rows if search.lower() in r["name"].lower() or search.lower() in r["description"].lower()]
|
| 88 |
+
if max_reports is not None:
|
| 89 |
+
rows = [r for r in rows if r["resolved_reports"] is not None and r["resolved_reports"] <= max_reports]
|
| 90 |
+
return {"total": len(rows), "assets": rows}
|
| 91 |
+
|
| 92 |
+
@app.post("/api/accounts", dependencies=[Depends(authorize)])
|
| 93 |
+
def add_account(account: Account):
|
| 94 |
+
h = account.host.lower().rstrip(".")
|
| 95 |
+
if urlsplit("https://" + h).hostname != h or any(c in h for c in "/:@*"):
|
| 96 |
+
raise HTTPException(422, "Account must specify a single hostname")
|
| 97 |
+
public_host(h, os.environ.get("ALLOW_PRIVATE_TARGETS") == "1")
|
| 98 |
+
if any(k.lower() in {"host", "cookie", "content-length", "authorization", "proxy-authorization"} for k in account.headers):
|
| 99 |
+
raise HTTPException(422, "Use cookie field for cookies; protected headers cannot be overridden")
|
| 100 |
+
identifier = uuid.uuid4().hex
|
| 101 |
+
accounts[identifier] = account.model_dump() if hasattr(account, "model_dump") else account.dict()
|
| 102 |
+
accounts[identifier]["host"] = h
|
| 103 |
+
return {"id": identifier, "name": account.name, "host": h}
|
| 104 |
+
|
| 105 |
+
@app.get("/api/accounts", dependencies=[Depends(authorize)])
|
| 106 |
+
def list_accounts():
|
| 107 |
+
return [{"id": k, "name": v["name"], "host": v["host"]} for k, v in accounts.items()]
|
| 108 |
+
|
| 109 |
+
@app.delete("/api/accounts/{account_id}", dependencies=[Depends(authorize)])
|
| 110 |
+
def remove_account(account_id: str):
|
| 111 |
+
accounts.pop(account_id, None)
|
| 112 |
+
return {"removed": True}
|
| 113 |
+
|
| 114 |
+
@app.post("/api/scans", dependencies=[Depends(authorize)])
|
| 115 |
+
async def create_scan(spec: ScanRequest):
|
| 116 |
+
if not spec.policy_confirmed:
|
| 117 |
+
raise HTTPException(422, "You must explicitly confirm program automation policy")
|
| 118 |
+
item = assets.get(spec.asset_id)
|
| 119 |
+
if not item:
|
| 120 |
+
raise HTTPException(404, "Asset not found")
|
| 121 |
+
try:
|
| 122 |
+
url, host = target_url(item)
|
| 123 |
+
public_host(host, os.environ.get("ALLOW_PRIVATE_TARGETS") == "1")
|
| 124 |
+
except ValueError as exc:
|
| 125 |
+
raise HTTPException(422, str(exc))
|
| 126 |
+
if spec.account_id:
|
| 127 |
+
profile = accounts.get(spec.account_id)
|
| 128 |
+
if not profile or profile["host"] != host:
|
| 129 |
+
raise HTTPException(422, "Account profile not found or host mismatch")
|
| 130 |
+
identifier = uuid.uuid4().hex
|
| 131 |
+
jobs[identifier] = {"id": identifier, "asset_id": spec.asset_id, "target": url,
|
| 132 |
+
"status": "queued", "alerts": [], "error": None, "created": time.time(),
|
| 133 |
+
"request_count": 0, "cancelled": False}
|
| 134 |
+
await queue.put((identifier, spec))
|
| 135 |
+
return {"id": identifier, "status": "queued"}
|
| 136 |
+
|
| 137 |
+
@app.get("/api/scans", dependencies=[Depends(authorize)])
|
| 138 |
+
def scan_list():
|
| 139 |
+
return list(jobs.values())[-100:]
|
| 140 |
+
|
| 141 |
+
@app.get("/api/scans/{scan_id}", dependencies=[Depends(authorize)])
|
| 142 |
+
def scan_status(scan_id: str):
|
| 143 |
+
if scan_id not in jobs:
|
| 144 |
+
raise HTTPException(404, "Scan not found")
|
| 145 |
+
return jobs[scan_id]
|
| 146 |
+
|
| 147 |
+
@app.post("/api/scans/{scan_id}/cancel", dependencies=[Depends(authorize)])
|
| 148 |
+
def cancel_scan(scan_id: str):
|
| 149 |
+
if scan_id not in jobs:
|
| 150 |
+
raise HTTPException(404, "Scan not found")
|
| 151 |
+
jobs[scan_id]["cancelled"] = True
|
| 152 |
+
return {"cancel_requested": True}
|
| 153 |
+
|
| 154 |
+
async def zap_call(client, component, action, params=None):
|
| 155 |
+
response = await client.get(f"{ZAP_URL}/JSON/{component}/{action}/", params={**(params or {}), "apikey": ZAP_KEY})
|
| 156 |
+
response.raise_for_status()
|
| 157 |
+
data = response.json()
|
| 158 |
+
if "code" in data:
|
| 159 |
+
raise RuntimeError(str(data.get("message") or data["code"]))
|
| 160 |
+
return data
|
| 161 |
+
|
| 162 |
+
async def worker():
|
| 163 |
+
while True:
|
| 164 |
+
identifier, spec = await queue.get()
|
| 165 |
+
job = jobs[identifier]
|
| 166 |
+
try:
|
| 167 |
+
if job["cancelled"]:
|
| 168 |
+
job["status"] = "cancelled"
|
| 169 |
+
continue
|
| 170 |
+
job["status"] = "running"
|
| 171 |
+
url, host = target_url(assets[spec.asset_id])
|
| 172 |
+
profile = accounts.get(spec.account_id) if spec.account_id else None
|
| 173 |
+
headers = {"User-Agent": spec.user_agent}
|
| 174 |
+
if profile:
|
| 175 |
+
headers.update(profile["headers"])
|
| 176 |
+
if profile["cookie"]:
|
| 177 |
+
headers["Cookie"] = profile["cookie"]
|
| 178 |
+
# Every job uses ZAP's HTTP proxy to populate passive alerts. No active scanning.
|
| 179 |
+
async with httpx.AsyncClient(timeout=15, follow_redirects=False, trust_env=False,
|
| 180 |
+
proxy=ZAP_URL, verify=False) as client:
|
| 181 |
+
for i in range(spec.max_requests):
|
| 182 |
+
if job["cancelled"]:
|
| 183 |
+
break
|
| 184 |
+
# Re-check every request, including DNS answers. Egress firewall is still recommended.
|
| 185 |
+
public_host(host, os.environ.get("ALLOW_PRIVATE_TARGETS") == "1")
|
| 186 |
+
try:
|
| 187 |
+
response = await client.get(url, headers=headers)
|
| 188 |
+
job["request_count"] += 1
|
| 189 |
+
job["last_status"] = response.status_code
|
| 190 |
+
except httpx.HTTPError as exc:
|
| 191 |
+
job["error"] = f"Target fetch failed: {exc.__class__.__name__}"
|
| 192 |
+
break
|
| 193 |
+
if i + 1 < spec.max_requests:
|
| 194 |
+
await asyncio.sleep(spec.delay_seconds)
|
| 195 |
+
if job["cancelled"]:
|
| 196 |
+
job["status"] = "cancelled"
|
| 197 |
+
continue
|
| 198 |
+
async with httpx.AsyncClient(timeout=20, trust_env=False) as z:
|
| 199 |
+
await asyncio.sleep(2)
|
| 200 |
+
# Filter results strictly to the requested URL/host, because ZAP session is shared.
|
| 201 |
+
info = await zap_call(z, "core", "view/alerts", {"baseurl": url, "start": "0", "count": "100"})
|
| 202 |
+
alerts = info.get("alerts", [])
|
| 203 |
+
job["alerts"] = [{
|
| 204 |
+
"name": a.get("name"), "risk": a.get("risk"),
|
| 205 |
+
"confidence": a.get("confidence"), "url": a.get("url"),
|
| 206 |
+
"description": (a.get("description") or "")[:1000]
|
| 207 |
+
} for a in alerts if urlsplit(a.get("url") or "").hostname == host]
|
| 208 |
+
job["status"] = "completed"
|
| 209 |
+
except Exception as exc:
|
| 210 |
+
job["status"] = "failed"
|
| 211 |
+
job["error"] = str(exc)[:250]
|
| 212 |
+
finally:
|
| 213 |
+
# Never retain authenticated headers in logs or job history.
|
| 214 |
+
queue.task_done()
|
app/scope.py
ADDED
|
@@ -0,0 +1,103 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
import csv
|
| 2 |
+
import io
|
| 3 |
+
import ipaddress
|
| 4 |
+
import re
|
| 5 |
+
import socket
|
| 6 |
+
from urllib.parse import urlsplit
|
| 7 |
+
|
| 8 |
+
WEB_TYPES = {"domain", "url", "website", "web application", "api"}
|
| 9 |
+
LEVELS = {"none": 0, "low": 1, "medium": 2, "high": 3, "critical": 4}
|
| 10 |
+
|
| 11 |
+
def yes(value):
|
| 12 |
+
s = str(value or "").strip().lower()
|
| 13 |
+
return s in {"true", "yes", "eligible", "1", "y"}
|
| 14 |
+
|
| 15 |
+
def field(row, *aliases):
|
| 16 |
+
normalized = {re.sub(r"[^a-z0-9]", "", str(k).lower()): v for k, v in row.items() if k is not None}
|
| 17 |
+
for alias in aliases:
|
| 18 |
+
key = re.sub(r"[^a-z0-9]", "", alias.lower())
|
| 19 |
+
if key in normalized:
|
| 20 |
+
return str(normalized[key] or "").strip()
|
| 21 |
+
return ""
|
| 22 |
+
|
| 23 |
+
def classify(identifier, asset_type):
|
| 24 |
+
name = identifier.strip()
|
| 25 |
+
kind = asset_type.lower().strip()
|
| 26 |
+
if "*" in name or kind in {"other", "android", "android: play store", "ios", "source code", "executable", "hardware"}:
|
| 27 |
+
return "manual"
|
| 28 |
+
if kind and kind not in WEB_TYPES:
|
| 29 |
+
return "manual"
|
| 30 |
+
try:
|
| 31 |
+
p = urlsplit(name if "://" in name else "https://" + name)
|
| 32 |
+
host = (p.hostname or "").lower().rstrip(".")
|
| 33 |
+
if p.scheme not in {"http", "https"} or not host or p.username or p.password:
|
| 34 |
+
return "manual"
|
| 35 |
+
if not re.fullmatch(r"[a-z0-9.-]+", host) or "." not in host:
|
| 36 |
+
return "manual"
|
| 37 |
+
if p.port not in (None, 80, 443, 8080):
|
| 38 |
+
return "manual"
|
| 39 |
+
return "web"
|
| 40 |
+
except ValueError:
|
| 41 |
+
return "manual"
|
| 42 |
+
|
| 43 |
+
def parse_csv(content):
|
| 44 |
+
if len(content) > 2_000_000:
|
| 45 |
+
raise ValueError("CSV too large (max 2 MB)")
|
| 46 |
+
decoded = content.decode("utf-8-sig")
|
| 47 |
+
reader = csv.DictReader(io.StringIO(decoded))
|
| 48 |
+
if not reader.fieldnames:
|
| 49 |
+
raise ValueError("CSV is empty")
|
| 50 |
+
assets = []
|
| 51 |
+
for i, row in enumerate(reader):
|
| 52 |
+
if i >= 3000:
|
| 53 |
+
raise ValueError("Scope contains over 3000 entries")
|
| 54 |
+
name = field(row, "asset_identifier", "asset name", "asset", "identifier")
|
| 55 |
+
if not name:
|
| 56 |
+
continue
|
| 57 |
+
kind = field(row, "asset_type", "type")
|
| 58 |
+
bounty = field(row, "eligible_for_bounty", "bounty", "bounty eligible")
|
| 59 |
+
submission = field(row, "eligible_for_submission")
|
| 60 |
+
severity = field(row, "max_severity", "max. severity", "maximum severity").lower()
|
| 61 |
+
reports = field(row, "resolved reports", "resolved_reports", "resolved_reports_count")
|
| 62 |
+
m = re.search(r"\d+", reports)
|
| 63 |
+
assets.append({
|
| 64 |
+
"id": "asset-" + str(i + 1),
|
| 65 |
+
"name": name[:1000],
|
| 66 |
+
"type": kind[:100],
|
| 67 |
+
"compatibility": classify(name, kind),
|
| 68 |
+
"bounty": yes(bounty),
|
| 69 |
+
"submission": yes(submission) if submission else None,
|
| 70 |
+
"max_severity": severity if severity in LEVELS else "unknown",
|
| 71 |
+
"last_update": field(row, "last update", "last_updated", "updated_at")[:80],
|
| 72 |
+
"resolved_reports": int(m.group()) if m else None,
|
| 73 |
+
"description": field(row, "description", "instruction", "instructions", "asset description")[:3000],
|
| 74 |
+
"coverage": field(row, "coverage")[:200]
|
| 75 |
+
})
|
| 76 |
+
if not assets:
|
| 77 |
+
raise ValueError("No recognized asset names. Expected asset_identifier or Asset name column")
|
| 78 |
+
return assets
|
| 79 |
+
|
| 80 |
+
def target_url(asset):
|
| 81 |
+
if asset["compatibility"] != "web":
|
| 82 |
+
raise ValueError("Only explicit web assets are scannable")
|
| 83 |
+
raw = asset["name"].strip()
|
| 84 |
+
p = urlsplit(raw if "://" in raw else "https://" + raw)
|
| 85 |
+
if p.query or p.fragment:
|
| 86 |
+
raise ValueError("Scope URL with query/fragment requires manual review")
|
| 87 |
+
return p.geturl(), (p.hostname or "").lower().rstrip(".")
|
| 88 |
+
|
| 89 |
+
def public_host(host, allow_private=False):
|
| 90 |
+
if not re.fullmatch(r"[a-z0-9.-]+", host):
|
| 91 |
+
raise ValueError("Invalid hostname")
|
| 92 |
+
try:
|
| 93 |
+
addresses = {x[4][0] for x in socket.getaddrinfo(host, None, proto=socket.IPPROTO_TCP)}
|
| 94 |
+
except socket.gaierror as exc:
|
| 95 |
+
raise ValueError("DNS resolution failed") from exc
|
| 96 |
+
if not addresses:
|
| 97 |
+
raise ValueError("No DNS results")
|
| 98 |
+
if not allow_private:
|
| 99 |
+
for address in addresses:
|
| 100 |
+
ip = ipaddress.ip_address(address)
|
| 101 |
+
if not ip.is_global:
|
| 102 |
+
raise ValueError("Private, reserved, or non-global address not allowed")
|
| 103 |
+
return addresses
|
requirements.txt
ADDED
|
@@ -0,0 +1,5 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
fastapi==0.115.12
|
| 2 |
+
uvicorn==0.34.2
|
| 3 |
+
python-multipart==0.0.20
|
| 4 |
+
httpx==0.28.1
|
| 5 |
+
pytest==8.3.5
|
start.sh
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
#!/bin/sh
|
| 2 |
+
set -eu
|
| 3 |
+
: "${ASFB_API_KEY:?Set ASFB_API_KEY in Space secrets}"
|
| 4 |
+
: "${ZAP_API_KEY:?Set ZAP_API_KEY in Space secrets}"
|
| 5 |
+
/zap/zap.sh -daemon -host 127.0.0.1 -port 8090 -config "api.key=${ZAP_API_KEY}" -config api.addrs.addr.name=127.0.0.1 -config api.addrs.addr.regex=false > /tmp/asfb-zap.log 2>&1 &
|
| 6 |
+
exec uvicorn app.main:app --host 0.0.0.0 --port 7860
|
tests/test_scope.py
ADDED
|
@@ -0,0 +1,25 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
from app.scope import parse_csv, classify, target_url
|
| 2 |
+
import pytest
|
| 3 |
+
|
| 4 |
+
def test_hackerone_table():
|
| 5 |
+
csv = b"Asset name,Type,Max. severity,Bounty,Resolved Reports,Description\nweb.example.org,Domain,Critical,Eligible,24 (3%),Main app\n*.example.org,Other,High,Eligible,7 (1%),internal services\ncom.example.android,Android: Play Store,Critical,Eligible,72,app\n"
|
| 6 |
+
a = parse_csv(csv)
|
| 7 |
+
assert a[0]["compatibility"] == "web"
|
| 8 |
+
assert a[0]["resolved_reports"] == 24
|
| 9 |
+
assert a[0]["bounty"] is True
|
| 10 |
+
assert a[1]["compatibility"] == "manual"
|
| 11 |
+
assert a[2]["compatibility"] == "manual"
|
| 12 |
+
|
| 13 |
+
def test_url_and_credential_handling():
|
| 14 |
+
assert classify("https://api.example.org/v1/", "URL") == "web"
|
| 15 |
+
assert classify("https://name:pass@example.org", "URL") == "manual"
|
| 16 |
+
assert classify("localhost", "Domain") == "manual"
|
| 17 |
+
assert classify("*.example.org", "Domain") == "manual"
|
| 18 |
+
with pytest.raises(ValueError):
|
| 19 |
+
target_url({"name": "*.example.org", "compatibility": "manual"})
|
| 20 |
+
|
| 21 |
+
def test_unknown_and_headers():
|
| 22 |
+
a = parse_csv(b"asset_identifier,asset_type,eligible_for_bounty,eligible_for_submission,max_severity\na.example.org,DOMAIN,true,false,high\n")
|
| 23 |
+
assert a[0]["bounty"] is True
|
| 24 |
+
assert a[0]["submission"] is False
|
| 25 |
+
assert a[0]["max_severity"] == "high"
|