Spaces:
Sleeping
Sleeping
File size: 3,339 Bytes
369e64d | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | # URLScan.io API Integration
This module provides a Python client for interacting with the URLScan.io API to analyze URLs for phishing detection.
## Setup
1. **Get an API key:**
- Sign up at https://urlscan.io/user/signup
- Get your API key from your account settings
2. **Set environment variable:**
```bash
export URLSCAN_API_KEY="your_api_key_here"
```
Or create a `.env` file:
```bash
cp .env.example .env
# Edit .env and add your API key
```
## Basic Usage
### Initialize Client
```python
from phising_detection.api import URLScanClient
# Using environment variable
client = URLScanClient()
# Or pass API key directly
client = URLScanClient(api_key="your_api_key")
```
### Submit a URL for Scanning
```python
# Submit URL
result = client.submit_url(
url="https://suspicious-site.com",
visibility="public", # or "unlisted" or "private"
tags=["phishing", "test"]
)
uuid = result["uuid"]
print(f"Scan UUID: {uuid}")
```
### Retrieve Results
```python
# Get results by UUID
scan_result = client.get_result(uuid)
# Access scan data
page_title = scan_result["page"]["title"]
screenshot = scan_result["task"]["screenshotURL"]
```
### Submit and Wait for Results
```python
# Submit and automatically wait for completion
result = client.submit_and_wait(
url="https://example.com",
max_wait=60, # seconds
poll_interval=5 # seconds between checks
)
```
### Get Verdict
```python
# Get simple verdict (malicious/safe)
verdict = client.get_verdict(uuid)
print(f"Verdict: {verdict}") # "malicious" or "safe"
```
### Search Existing Scans
```python
# Search for scans by domain
results = client.search(
query="domain:example.com",
size=10
)
for scan in results["results"]:
print(scan["task"]["url"])
```
## API Response Examples
### Submission Response
```json
{
"uuid": "abc123...",
"result": "https://urlscan.io/result/abc123.../",
"api": "https://urlscan.io/api/v1/result/abc123.../"
}
```
### Result Response
```json
{
"page": {
"url": "https://example.com",
"title": "Example Domain",
"status": "200"
},
"verdicts": {
"overall": {
"score": 0,
"malicious": false
}
},
"task": {
"uuid": "abc123...",
"time": "2024-01-01T12:00:00.000Z",
"screenshotURL": "https://..."
}
}
```
## Error Handling
```python
from phising_detection.api import URLScanClient, URLScanError
try:
client = URLScanClient()
result = client.submit_url("https://example.com")
except URLScanError as e:
print(f"Error: {e}")
```
## Rate Limits
- Free tier: 50 submissions per day
- Paid tier: Higher limits available
- The client handles rate limit errors automatically
## Integration with Phishing Detection
```python
from phising_detection.api import URLScanClient
from phising_detection.data import load_phishing_urls
# Load your phishing URLs
df = load_phishing_urls()
# Analyze URLs
client = URLScanClient()
for idx, row in df.head(10).iterrows(): # Sample first 10
try:
result = client.submit_and_wait(row['url'])
verdict = client.get_verdict(result['task']['uuid'])
print(f"{row['url']}: {verdict}")
except URLScanError as e:
print(f"Error scanning {row['url']}: {e}")
```
## API Documentation
Full API documentation: https://urlscan.io/docs/api/
|