BitCheck Dev Claude Opus 4.7 commited on
Commit Β·
e431e4e
1
Parent(s): e3b35a3
docs: add README and deployment guide for audio service
Browse filesCo-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- bitcheck-audio-service/README.md +237 -14
bitcheck-audio-service/README.md
CHANGED
|
@@ -2,32 +2,255 @@
|
|
| 2 |
|
| 3 |
AI-powered audio verification microservice that detects AI-generated / deepfake speech using a Random Forest classifier trained on 26 spectral and MFCC audio features.
|
| 4 |
|
| 5 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
|
| 7 |
```bash
|
|
|
|
| 8 |
pip install -r requirements.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 9 |
uvicorn main:app --host 0.0.0.0 --port 7860
|
| 10 |
```
|
| 11 |
|
| 12 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 13 |
|
| 14 |
-
|
| 15 |
-
|--------|-------------------|----------------------------|
|
| 16 |
-
| GET | / | Service info |
|
| 17 |
-
| GET | /health | Health check |
|
| 18 |
-
| POST | /verify/audio | Upload and verify audio |
|
| 19 |
|
| 20 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 21 |
|
| 22 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
| 23 |
|
| 24 |
-
##
|
| 25 |
|
| 26 |
```bash
|
| 27 |
-
|
| 28 |
-
|
|
|
|
|
|
|
|
|
|
| 29 |
```
|
| 30 |
|
| 31 |
-
##
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 32 |
|
| 33 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 2 |
|
| 3 |
AI-powered audio verification microservice that detects AI-generated / deepfake speech using a Random Forest classifier trained on 26 spectral and MFCC audio features.
|
| 4 |
|
| 5 |
+
## Overview
|
| 6 |
+
|
| 7 |
+
BitCheck Audio Verification accepts uploaded audio files, extracts 26 handcrafted audio features, and runs them through a saved Random Forest model to estimate whether the speech is likely human-recorded or AI-generated. The service produces a risk-based verification report β not an absolute judgment.
|
| 8 |
+
|
| 9 |
+
## Architecture
|
| 10 |
+
|
| 11 |
+
```
|
| 12 |
+
upload β validate β metadata β preprocess β quality β features β model β trust β report
|
| 13 |
+
```
|
| 14 |
+
|
| 15 |
+
### Pipeline Steps
|
| 16 |
+
|
| 17 |
+
| Step | Module | Description |
|
| 18 |
+
|------|--------|-------------|
|
| 19 |
+
| 1 | `file_validator` | Extension, size, SHA256, safe save |
|
| 20 |
+
| 2 | `audio_metadata_analyzer` | ffprobe/librosa metadata, AI tool markers |
|
| 21 |
+
| 3 | `audio_preprocessor` | Normalize to mono 22.05kHz WAV |
|
| 22 |
+
| 4 | `audio_quality_analyzer` | Silence, clipping, SNR proxy |
|
| 23 |
+
| 5 | `audio_feature_extractor` | Extract exact 26 features |
|
| 24 |
+
| 6 | `audio_model_loader` | Load Random Forest from joblib |
|
| 25 |
+
| 7 | `audio_analyzer` | Run inference, map classes to fake/real |
|
| 26 |
+
| 8 | `trust_scorer` | Weighted risk synthesis |
|
| 27 |
+
| 9 | `report_builder` | Assemble final JSON report |
|
| 28 |
+
|
| 29 |
+
## API Endpoints
|
| 30 |
+
|
| 31 |
+
### `GET /`
|
| 32 |
+
|
| 33 |
+
Service info and status.
|
| 34 |
+
|
| 35 |
+
### `GET /health`
|
| 36 |
+
|
| 37 |
+
Health check with model/ffmpeg availability.
|
| 38 |
+
|
| 39 |
+
```json
|
| 40 |
+
{
|
| 41 |
+
"status": "ok",
|
| 42 |
+
"service": "BitCheck Audio Verification API",
|
| 43 |
+
"version": "1.0.0",
|
| 44 |
+
"audio_model_found": true,
|
| 45 |
+
"ffmpeg_available": true,
|
| 46 |
+
"max_audio_duration_seconds": 30,
|
| 47 |
+
"target_sample_rate": 22050
|
| 48 |
+
}
|
| 49 |
+
```
|
| 50 |
+
|
| 51 |
+
### `POST /verify/audio`
|
| 52 |
+
|
| 53 |
+
Upload and verify an audio file.
|
| 54 |
+
|
| 55 |
+
**Request:** `multipart/form-data`
|
| 56 |
+
|
| 57 |
+
| Field | Type | Default | Description |
|
| 58 |
+
|-------|------|---------|-------------|
|
| 59 |
+
| `file` | file | required | Audio file to verify |
|
| 60 |
+
| `max_duration_seconds` | int | 30 | Max duration to analyze |
|
| 61 |
+
| `strict_duration_limit` | bool | false | Reject if longer than max |
|
| 62 |
+
| `return_features` | bool | false | Include raw feature values |
|
| 63 |
+
| `run_quality_analysis` | bool | true | Run quality signal analysis |
|
| 64 |
+
|
| 65 |
+
**Response:** Full verification report (see example below).
|
| 66 |
+
|
| 67 |
+
**Error responses:**
|
| 68 |
+
- `400` β `unsupported_file_type`, `invalid_upload`, `audio_too_long`
|
| 69 |
+
- `413` β `file_too_large`
|
| 70 |
+
- `422` β `preprocessing_failed`
|
| 71 |
+
|
| 72 |
+
### `GET /outputs/{filename}`
|
| 73 |
+
|
| 74 |
+
Serve generated output files (normalized WAVs).
|
| 75 |
+
|
| 76 |
+
## Supported Audio Formats
|
| 77 |
+
|
| 78 |
+
- `.wav`, `.mp3`, `.m4a`, `.aac`, `.flac`, `.ogg`, `.webm`
|
| 79 |
+
- Video: `.mp4`, `.mov` (audio extracted for analysis)
|
| 80 |
+
|
| 81 |
+
## Model File Placement
|
| 82 |
+
|
| 83 |
+
Place your trained Random Forest model in the `models/` directory:
|
| 84 |
+
|
| 85 |
+
```
|
| 86 |
+
models/BitcheckDeepfake.joblib
|
| 87 |
+
```
|
| 88 |
+
|
| 89 |
+
Fallback paths are checked in order:
|
| 90 |
+
1. `models/BitcheckDeepfake.joblib` (preferred)
|
| 91 |
+
2. `models/BitcheckDeepFake`
|
| 92 |
+
3. `models/BitcheckDeepFake.joblib`
|
| 93 |
+
4. `models/BitcheckDeepfake`
|
| 94 |
+
|
| 95 |
+
The service starts and runs without the model β it returns `completed_with_warnings` with `model_found: false`.
|
| 96 |
+
|
| 97 |
+
## Audio Feature Pipeline
|
| 98 |
+
|
| 99 |
+
### Exact 26 Features
|
| 100 |
+
|
| 101 |
+
Features are computed as the mean over time using librosa at 22050 Hz mono:
|
| 102 |
+
|
| 103 |
+
1. `chroma_stft` β Mean chromagram (12 pitch classes)
|
| 104 |
+
2. `rms` β Root mean square energy
|
| 105 |
+
3. `spectral_centroid` β Center of mass of spectrum
|
| 106 |
+
4. `spectral_bandwidth` β Spread of spectrum
|
| 107 |
+
5. `rolloff` β Frequency below which 85% of energy lies
|
| 108 |
+
6. `zero_crossing_rate` β Rate of sign changes
|
| 109 |
+
7β26. `mfcc1`β`mfcc20` β 20 Mel-frequency cepstral coefficients
|
| 110 |
+
|
| 111 |
+
### Class Mapping
|
| 112 |
+
|
| 113 |
+
The Random Forest model from the training notebook maps:
|
| 114 |
+
|
| 115 |
+
| Model Class | Meaning |
|
| 116 |
+
|-------------|---------|
|
| 117 |
+
| `0` | **FAKE** / AI-generated / deepfake |
|
| 118 |
+
| `1` | **REAL** / human |
|
| 119 |
+
|
| 120 |
+
For string-labeled models, known keywords are matched (`FAKE`, `REAL`, `ai_generated`, `human`, etc.). Unknown labels return raw probabilities with a warning.
|
| 121 |
+
|
| 122 |
+
## Local Setup
|
| 123 |
|
| 124 |
```bash
|
| 125 |
+
# Install dependencies
|
| 126 |
pip install -r requirements.txt
|
| 127 |
+
|
| 128 |
+
# Copy environment config
|
| 129 |
+
cp .env.example .env
|
| 130 |
+
|
| 131 |
+
# Place your model
|
| 132 |
+
cp /path/to/BitcheckDeepfake.joblib models/
|
| 133 |
+
|
| 134 |
+
# Run the server
|
| 135 |
uvicorn main:app --host 0.0.0.0 --port 7860
|
| 136 |
```
|
| 137 |
|
| 138 |
+
## Environment Variables
|
| 139 |
+
|
| 140 |
+
| Variable | Default | Description |
|
| 141 |
+
|----------|---------|-------------|
|
| 142 |
+
| `APP_NAME` | BitCheck Audio Verification API | App name |
|
| 143 |
+
| `VERSION` | 1.0.0 | App version |
|
| 144 |
+
| `UPLOAD_DIR` | uploads | Upload storage |
|
| 145 |
+
| `OUTPUT_DIR` | outputs | Normalized audio output |
|
| 146 |
+
| `MODEL_DIR` | models | Model directory |
|
| 147 |
+
| `AUDIO_MODEL_PATH` | models/BitcheckDeepfake.joblib | Primary model path |
|
| 148 |
+
| `MAX_AUDIO_DURATION_SECONDS` | 30 | Max duration to analyze |
|
| 149 |
+
| `MAX_AUDIO_SIZE_MB` | 25 | Max upload size |
|
| 150 |
+
| `TARGET_SAMPLE_RATE` | 22050 | Target sample rate |
|
| 151 |
+
| `STRICT_DURATION_LIMIT` | false | Reject over-limit audio |
|
| 152 |
+
| `LOG_LEVEL` | INFO | Logging level |
|
| 153 |
+
|
| 154 |
+
## Example curl Requests
|
| 155 |
+
|
| 156 |
+
```bash
|
| 157 |
+
# Verify an audio file
|
| 158 |
+
curl -X POST http://localhost:7860/verify/audio \
|
| 159 |
+
-F "file=@voice_note.mp3" \
|
| 160 |
+
-F "max_duration_seconds=30" \
|
| 161 |
+
-F "return_features=false"
|
| 162 |
+
|
| 163 |
+
# With raw features
|
| 164 |
+
curl -X POST http://localhost:7860/verify/audio \
|
| 165 |
+
-F "file=@recording.wav" \
|
| 166 |
+
-F "return_features=true"
|
| 167 |
+
|
| 168 |
+
# Strict duration limit
|
| 169 |
+
curl -X POST http://localhost:7860/verify/audio \
|
| 170 |
+
-F "file=@long_audio.mp3" \
|
| 171 |
+
-F "strict_duration_limit=true"
|
| 172 |
+
```
|
| 173 |
+
|
| 174 |
+
## Example Response
|
| 175 |
+
|
| 176 |
+
```json
|
| 177 |
+
{
|
| 178 |
+
"verification_id": "a1b2c3d4-...",
|
| 179 |
+
"service": "BitCheck",
|
| 180 |
+
"file_type": "audio",
|
| 181 |
+
"status": "completed_with_warnings",
|
| 182 |
+
"processing_time_ms": 1234,
|
| 183 |
+
"input": {
|
| 184 |
+
"filename": "voice_note.mp3",
|
| 185 |
+
"sha256": "abc123...",
|
| 186 |
+
"extension": ".mp3",
|
| 187 |
+
"mime_type": "audio/mpeg",
|
| 188 |
+
"file_size_bytes": 123456
|
| 189 |
+
},
|
| 190 |
+
"file_validation": {"valid": true, "warnings": []},
|
| 191 |
+
"audio_metadata": {
|
| 192 |
+
"duration_seconds": 5.2,
|
| 193 |
+
"sample_rate": 44100,
|
| 194 |
+
"channels": 2,
|
| 195 |
+
"codec": "mp3",
|
| 196 |
+
"bitrate": 128000,
|
| 197 |
+
"format": "mp3",
|
| 198 |
+
"metadata_risk_score": 0.05
|
| 199 |
+
},
|
| 200 |
+
"trust": {
|
| 201 |
+
"trust_score": 78,
|
| 202 |
+
"risk_score": 0.22,
|
| 203 |
+
"risk_level": "Low Risk",
|
| 204 |
+
"decision": "approve",
|
| 205 |
+
"summary": "Overall risk signals are low."
|
| 206 |
+
},
|
| 207 |
+
"limitations": ["BitCheck audio verification is probabilistic..."],
|
| 208 |
+
"recommended_actions": ["Request original uncompressed audio..."]
|
| 209 |
+
}
|
| 210 |
+
```
|
| 211 |
|
| 212 |
+
## Hugging Face Spaces Deployment
|
|
|
|
|
|
|
|
|
|
|
|
|
| 213 |
|
| 214 |
+
1. Create a new Space at [huggingface.co/new-space](https://huggingface.co/new-space)
|
| 215 |
+
2. Choose the **Docker** SDK
|
| 216 |
+
3. Clone the Space repo and copy all files from this project
|
| 217 |
+
4. Place `BitcheckDeepfake.joblib` in the `models/` directory
|
| 218 |
+
5. Push to the Space β Hugging Face will build and deploy automatically
|
| 219 |
|
| 220 |
+
The Dockerfile is pre-configured:
|
| 221 |
+
- Python 3.11-slim
|
| 222 |
+
- ffmpeg, libsndfile1, libgl1 pre-installed
|
| 223 |
+
- Exposes port 7860
|
| 224 |
+
- Runs `uvicorn main:app --host 0.0.0.0 --port 7860`
|
| 225 |
|
| 226 |
+
## Testing
|
| 227 |
|
| 228 |
```bash
|
| 229 |
+
# Run all tests
|
| 230 |
+
python -m pytest tests/ -q
|
| 231 |
+
|
| 232 |
+
# Run specific test file
|
| 233 |
+
python -m pytest tests/test_audio_analyzer.py -q
|
| 234 |
```
|
| 235 |
|
| 236 |
+
## Limitations
|
| 237 |
+
|
| 238 |
+
- BitCheck audio verification is probabilistic, not absolute proof
|
| 239 |
+
- The Random Forest model depends on matching the exact 26-feature extraction pipeline used during training
|
| 240 |
+
- Noisy, compressed, very short, or silent audio can reduce reliability
|
| 241 |
+
- The model was trained on a specific dataset and may not generalize to all AI voice generators
|
| 242 |
+
- Audio from social media apps may be compressed and harder to analyze
|
| 243 |
+
- High-stakes decisions should involve human review
|
| 244 |
+
|
| 245 |
+
## Future Improvements
|
| 246 |
|
| 247 |
+
- Stronger deep learning model (spectrogram CNN or audio transformer)
|
| 248 |
+
- Audio foundation model (Wav2Vec2, HuBERT, Whisper-based)
|
| 249 |
+
- SHAP-based explanation for individual feature contributions
|
| 250 |
+
- Speaker consistency and voice similarity analysis
|
| 251 |
+
- Robustness testing against common compression/codec pipelines
|
| 252 |
+
- Multilingual and accent-robust evaluation
|
| 253 |
+
- Nigerian / West African voice dataset fine-tuning
|
| 254 |
+
- Integration with BitCheck image and video verification services
|
| 255 |
+
- Real-time streaming audio verification
|
| 256 |
+
- Dashboard and visualization UI
|