Jimisaq commited on
Commit
7dcc395
·
1 Parent(s): d352f58

Initial commit: Added AUD relapse risk detection app

Browse files
.dockerignore ADDED
@@ -0,0 +1,12 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ .DS_Store
2
+ .git
3
+ .jupyter_config
4
+ .jupyter_data
5
+ .jupyter_runtime
6
+ .venv
7
+ __pycache__
8
+ *.pyc
9
+ *.pyo
10
+ *.pyd
11
+ *.log
12
+ tmp*.wav
Dockerfile CHANGED
@@ -1,20 +1,24 @@
1
- FROM python:3.13.5-slim
 
 
 
 
 
 
 
2
 
3
  WORKDIR /app
4
 
5
- RUN apt-get update && apt-get install -y \
6
- build-essential \
7
- curl \
8
- git \
9
  && rm -rf /var/lib/apt/lists/*
10
 
11
- COPY requirements.txt ./
12
- COPY src/ ./src/
13
 
14
- RUN pip3 install -r requirements.txt
15
 
16
  EXPOSE 8501
17
 
18
- HEALTHCHECK CMD curl --fail http://localhost:8501/_stcore/health
19
-
20
- ENTRYPOINT ["streamlit", "run", "src/streamlit_app.py", "--server.port=8501", "--server.address=0.0.0.0"]
 
1
+ FROM python:3.11-slim
2
+
3
+ ENV PYTHONDONTWRITEBYTECODE=1
4
+ ENV PYTHONUNBUFFERED=1
5
+ ENV STREAMLIT_SERVER_HEADLESS=true
6
+ ENV STREAMLIT_BROWSER_GATHER_USAGE_STATS=false
7
+ ENV STREAMLIT_SERVER_PORT=8501
8
+ ENV STREAMLIT_SERVER_ADDRESS=0.0.0.0
9
 
10
  WORKDIR /app
11
 
12
+ RUN apt-get update && apt-get install -y --no-install-recommends \
13
+ ffmpeg \
14
+ libsndfile1 \
 
15
  && rm -rf /var/lib/apt/lists/*
16
 
17
+ COPY requirements.txt .
18
+ RUN pip install --no-cache-dir -r requirements.txt
19
 
20
+ COPY . .
21
 
22
  EXPOSE 8501
23
 
24
+ CMD ["python", "-m", "streamlit", "run", "streamlit_app.py", "--server.port", "8501", "--server.address", "0.0.0.0"]
 
 
README.md CHANGED
@@ -1,20 +1,54 @@
1
  ---
2
- title: Aud Relapse Risk Detector
3
- emoji: 🚀
4
- colorFrom: red
5
- colorTo: red
6
  sdk: docker
7
  app_port: 8501
8
- tags:
9
- - streamlit
10
  pinned: false
11
- short_description: 'Speech-based AUD relapse-risk MVP using CNN-BiLSTM and SVM '
12
  license: mit
13
  ---
14
 
15
- # Welcome to Streamlit!
16
 
17
- Edit `/src/streamlit_app.py` to customize this app to your heart's desire. :heart:
18
 
19
- If you have any questions, checkout our [documentation](https://docs.streamlit.io) and [community
20
- forums](https://discuss.streamlit.io).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: AUD Relapse Risk Demo
3
+ emoji: 🎙️
4
+ colorFrom: blue
5
+ colorTo: green
6
  sdk: docker
7
  app_port: 8501
 
 
8
  pinned: false
9
+ short_description: Speech-based AUD relapse-risk MVP using CNN-BiLSTM and SVM
10
  license: mit
11
  ---
12
 
13
+ # AUD Relapse Risk Demo
14
 
15
+ Minimal Hugging Face Space for the speech-based Alcohol Use Disorder relapse-risk MVP.
16
 
17
+ This Space serves the Streamlit UI and lets the user choose between:
18
+
19
+ - `cnn_bilstm`
20
+ - `svm`
21
+
22
+ Each model uses its own matching inference pipeline:
23
+
24
+ - `cnn_bilstm`:
25
+ - loads `outputs/cnn_bilstm_adversarial_model.pth`
26
+ - preprocesses audio into a normalized `128x128` log-mel spectrogram
27
+ - `svm`:
28
+ - loads `models/b1_svm_imp_best.pkl`
29
+ - preprocesses audio into the `194`-feature flat vector inferred from the training notebook
30
+
31
+ The returned Relapse Risk Index is a preliminary speech-based relapse-risk proxy and not a medical diagnosis.
32
+
33
+ ## Local Run
34
+
35
+ ```bash
36
+ python -m streamlit run streamlit_app.py
37
+ ```
38
+
39
+ ## Hugging Face Deploy
40
+
41
+ 1. Create a new Space on Hugging Face.
42
+ 2. Choose `Docker` as the SDK.
43
+ 3. Push this repository to the Space.
44
+ 4. Wait for the image build to finish.
45
+ 5. Open the Space URL.
46
+
47
+ ## Included Model Files
48
+
49
+ - `outputs/cnn_bilstm_adversarial_model.pth`
50
+ - `models/b1_svm_imp_best.pkl`
51
+
52
+ ## Public App
53
+
54
+ The Space exposes the Streamlit UI on port `8501`.
inference.py ADDED
@@ -0,0 +1,271 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Minimal inference entry point.
3
+
4
+ Supports two artifact types:
5
+
6
+ 1. CNN-BiLSTM PyTorch state_dict:
7
+ `./outputs/cnn_bilstm_adversarial_model.pth`
8
+
9
+ 2. Sklearn impairment SVM pipeline:
10
+ `./models/b1_svm_imp_best.pkl`
11
+
12
+ Or set:
13
+ `AUD_MODEL_PATH=/full/path/to/your_state_dict.pth`
14
+
15
+ Example:
16
+ `python3 -c "from inference import predict_rri; print(predict_rri('sample.wav'))"`
17
+ """
18
+
19
+ import os
20
+ import pickle
21
+ import logging
22
+ from functools import lru_cache
23
+ from pathlib import Path
24
+ from typing import Any
25
+
26
+ import numpy as np
27
+ import torch
28
+
29
+ from model import CnnBiLstmAdversarialModel
30
+ from preprocess import extract_svm_features, preprocess_audio_file
31
+
32
+
33
+ MODEL_CANDIDATES = {
34
+ "cnn_bilstm": [
35
+ Path("outputs/cnn_bilstm_adversarial_model.pth"),
36
+ Path("models/cnn_bilstm_adversarial_state_dict.pth"),
37
+ ],
38
+ "svm": [
39
+ Path("models/b1_svm_imp_best.pkl"),
40
+ ],
41
+ }
42
+ CLASS_NAMES = {0: "not_impaired", 1: "impaired"}
43
+ TORCH_INTERPRETATION = "Preliminary speech-based relapse-risk proxy derived from the impairment head."
44
+ SKLEARN_INTERPRETATION = "Preliminary speech-based relapse-risk proxy derived from the impairment classifier."
45
+ logger = logging.getLogger(__name__)
46
+
47
+
48
+ def get_device() -> torch.device:
49
+ return torch.device("cuda" if torch.cuda.is_available() else "cpu")
50
+
51
+
52
+ def get_risk_level(relapse_risk_index: float) -> str:
53
+ if relapse_risk_index <= 0.33:
54
+ return "Low"
55
+ if relapse_risk_index <= 0.66:
56
+ return "Moderate"
57
+ return "High"
58
+
59
+
60
+ def get_supported_model_names() -> tuple[str, ...]:
61
+ return tuple(MODEL_CANDIDATES.keys())
62
+
63
+
64
+ def get_available_models() -> dict[str, str]:
65
+ available_models: dict[str, str] = {}
66
+ for model_name, candidates in MODEL_CANDIDATES.items():
67
+ existing_path = next((candidate for candidate in candidates if candidate.exists()), candidates[0])
68
+ available_models[model_name] = str(existing_path)
69
+ return available_models
70
+
71
+
72
+ def resolve_default_model_name() -> str:
73
+ for model_name, candidates in MODEL_CANDIDATES.items():
74
+ if any(candidate.exists() for candidate in candidates):
75
+ return model_name
76
+ return "cnn_bilstm"
77
+
78
+
79
+ def resolve_model_path(
80
+ model_name: str | None = None,
81
+ model_path: str | None = None,
82
+ ) -> str:
83
+ if model_path:
84
+ return model_path
85
+
86
+ env_model_path = os.getenv("AUD_MODEL_PATH")
87
+ if env_model_path:
88
+ return env_model_path
89
+
90
+ selected_model_name = model_name or resolve_default_model_name()
91
+ if selected_model_name not in MODEL_CANDIDATES:
92
+ supported_names = ", ".join(get_supported_model_names())
93
+ raise ValueError(f"Unsupported model_name '{selected_model_name}'. Choose one of: {supported_names}.")
94
+
95
+ candidates = MODEL_CANDIDATES[selected_model_name]
96
+ existing_path = next((candidate for candidate in candidates if candidate.exists()), candidates[0])
97
+ return str(existing_path)
98
+
99
+
100
+ def _clean_state_dict_keys(state_dict: dict) -> dict:
101
+ if state_dict and all(key.startswith("module.") for key in state_dict):
102
+ return {key.removeprefix("module."): value for key, value in state_dict.items()}
103
+ return state_dict
104
+
105
+
106
+ def _extract_state_dict(checkpoint) -> dict:
107
+ if isinstance(checkpoint, dict):
108
+ for key in ("state_dict", "model_state_dict"):
109
+ if key in checkpoint and isinstance(checkpoint[key], dict):
110
+ return _clean_state_dict_keys(checkpoint[key])
111
+ if isinstance(checkpoint, dict):
112
+ return _clean_state_dict_keys(checkpoint)
113
+ raise ValueError("Checkpoint format is not a valid state_dict or wrapped state_dict.")
114
+
115
+
116
+ def _summarize_array(values: np.ndarray, preview_count: int = 8) -> dict[str, Any]:
117
+ flat_values = np.asarray(values, dtype=np.float32).reshape(-1)
118
+ preview = [round(float(value), 4) for value in flat_values[:preview_count]]
119
+ return {
120
+ "shape": tuple(int(dim) for dim in np.asarray(values).shape),
121
+ "dtype": str(np.asarray(values).dtype),
122
+ "min": round(float(flat_values.min()), 4),
123
+ "max": round(float(flat_values.max()), 4),
124
+ "mean": round(float(flat_values.mean()), 4),
125
+ "preview": preview,
126
+ }
127
+
128
+
129
+ def _summarize_tensor(tensor: torch.Tensor, preview_count: int = 8) -> dict[str, Any]:
130
+ cpu_tensor = tensor.detach().cpu()
131
+ flat_values = cpu_tensor.reshape(-1)
132
+ preview = [round(float(value), 4) for value in flat_values[:preview_count].tolist()]
133
+ return {
134
+ "shape": tuple(int(dim) for dim in cpu_tensor.shape),
135
+ "dtype": str(cpu_tensor.dtype),
136
+ "device": str(tensor.device),
137
+ "min": round(float(flat_values.min().item()), 4),
138
+ "max": round(float(flat_values.max().item()), 4),
139
+ "mean": round(float(flat_values.mean().item()), 4),
140
+ "preview": preview,
141
+ }
142
+
143
+
144
+ @lru_cache(maxsize=4)
145
+ def load_model(model_path: str, device_name: str) -> Any:
146
+ checkpoint_path = Path(model_path)
147
+ if not checkpoint_path.exists():
148
+ raise FileNotFoundError(
149
+ f"Model weights not found at '{checkpoint_path}'. "
150
+ "Place the saved model there or set AUD_MODEL_PATH."
151
+ )
152
+
153
+ if checkpoint_path.suffix == ".pkl":
154
+ logger.info("Loading sklearn model from %s", checkpoint_path)
155
+ with checkpoint_path.open("rb") as file:
156
+ return pickle.load(file)
157
+
158
+ device = torch.device(device_name)
159
+ logger.info("Loading torch model from %s on %s", checkpoint_path, device)
160
+ checkpoint = torch.load(checkpoint_path, map_location=device)
161
+ state_dict = _extract_state_dict(checkpoint)
162
+
163
+ model = CnnBiLstmAdversarialModel()
164
+ model.load_state_dict(state_dict)
165
+ model.to(device)
166
+ model.eval()
167
+ return model
168
+
169
+
170
+ def _predict_with_torch_model(
171
+ model: CnnBiLstmAdversarialModel,
172
+ audio_file_path: str,
173
+ resolved_device: str,
174
+ ) -> tuple[float, int]:
175
+ input_tensor = preprocess_audio_file(audio_file_path).to(torch.device(resolved_device))
176
+ logger.info(
177
+ "CNN-BiLSTM input summary for %s: %s",
178
+ audio_file_path,
179
+ _summarize_tensor(input_tensor),
180
+ )
181
+ outputs = model(mel_spectrogram=input_tensor, return_domain_output=False)
182
+ impairment_logits = outputs["impairment_logits"]
183
+ impairment_probabilities = torch.softmax(impairment_logits, dim=1)
184
+
185
+ relapse_risk_index = float(impairment_probabilities[0, 1].item())
186
+ predicted_impairment_index = int(torch.argmax(impairment_probabilities, dim=1).item())
187
+ logger.info(
188
+ "CNN-BiLSTM output for %s: logits=%s probabilities=%s predicted_class=%s rri=%.4f",
189
+ audio_file_path,
190
+ [round(float(value), 4) for value in impairment_logits[0].detach().cpu().tolist()],
191
+ [round(float(value), 4) for value in impairment_probabilities[0].detach().cpu().tolist()],
192
+ CLASS_NAMES[predicted_impairment_index],
193
+ relapse_risk_index,
194
+ )
195
+ return relapse_risk_index, predicted_impairment_index
196
+
197
+
198
+ def _predict_with_sklearn_model(model: Any, audio_file_path: str) -> tuple[float, int]:
199
+ features = extract_svm_features(audio_file_path)
200
+ logger.info(
201
+ "SVM feature summary for %s: %s",
202
+ audio_file_path,
203
+ _summarize_array(features),
204
+ )
205
+
206
+ expected_feature_count = getattr(model, "n_features_in_", features.shape[1])
207
+ if features.shape[1] != expected_feature_count:
208
+ raise ValueError(
209
+ f"SVM feature mismatch: extracted {features.shape[1]} features, "
210
+ f"but model expects {expected_feature_count}."
211
+ )
212
+
213
+ if not hasattr(model, "predict_proba"):
214
+ raise ValueError("Sklearn model does not expose predict_proba().")
215
+
216
+ predicted_impairment_index = int(np.asarray(model.predict(features))[0])
217
+ impairment_probabilities = np.asarray(model.predict_proba(features), dtype=np.float32)
218
+ relapse_risk_index = float(impairment_probabilities[0, 1])
219
+ logger.info(
220
+ "SVM output for %s: probabilities=%s predicted_class=%s rri=%.4f",
221
+ audio_file_path,
222
+ [round(float(value), 4) for value in impairment_probabilities[0].tolist()],
223
+ CLASS_NAMES[predicted_impairment_index],
224
+ relapse_risk_index,
225
+ )
226
+ return relapse_risk_index, predicted_impairment_index
227
+
228
+
229
+ @torch.no_grad()
230
+ def predict_rri(
231
+ audio_file_path: str,
232
+ model_name: str | None = None,
233
+ model_path: str | None = None,
234
+ device: str | None = None,
235
+ ) -> dict[str, float | str]:
236
+ resolved_model_path = resolve_model_path(model_name=model_name, model_path=model_path)
237
+ resolved_device = device or str(get_device())
238
+ selected_model_name = model_name or resolve_default_model_name()
239
+
240
+ logger.info(
241
+ "Starting inference: model_name=%s model_path=%s audio_file=%s device=%s",
242
+ selected_model_name,
243
+ resolved_model_path,
244
+ audio_file_path,
245
+ resolved_device,
246
+ )
247
+
248
+ model = load_model(resolved_model_path, resolved_device)
249
+ if isinstance(model, CnnBiLstmAdversarialModel):
250
+ relapse_risk_index, predicted_impairment_index = _predict_with_torch_model(
251
+ model=model,
252
+ audio_file_path=audio_file_path,
253
+ resolved_device=resolved_device,
254
+ )
255
+ interpretation = TORCH_INTERPRETATION
256
+ else:
257
+ relapse_risk_index, predicted_impairment_index = _predict_with_sklearn_model(
258
+ model=model,
259
+ audio_file_path=audio_file_path,
260
+ )
261
+ interpretation = SKLEARN_INTERPRETATION
262
+
263
+ rounded_rri = round(relapse_risk_index, 4)
264
+
265
+ return {
266
+ "relapse_risk_index": rounded_rri,
267
+ "risk_level": get_risk_level(relapse_risk_index),
268
+ "impairment_probability": rounded_rri,
269
+ "predicted_impairment_class": CLASS_NAMES[predicted_impairment_index],
270
+ "interpretation": interpretation,
271
+ }
main.py ADDED
@@ -0,0 +1,96 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Run locally:
3
+
4
+ 1. Put your saved model in the local project:
5
+ `outputs/cnn_bilstm_adversarial_model.pth`
6
+ or
7
+ `models/b1_svm_imp_best.pkl`
8
+ or set `AUD_MODEL_PATH=/full/path/to/your/model`
9
+
10
+ 2. Install the minimal dependencies:
11
+ `pip install fastapi uvicorn torch librosa scikit-image python-multipart`
12
+
13
+ 3. Start the API:
14
+ `uvicorn main:app --reload`
15
+
16
+ 4. Check health:
17
+ `curl http://127.0.0.1:8000/health`
18
+
19
+ 5. Call prediction:
20
+ `curl -X POST "http://127.0.0.1:8000/predict-rri" -F "model_name=cnn_bilstm" -F "file=@sample.wav"`
21
+ """
22
+
23
+ import os
24
+ import tempfile
25
+ import logging
26
+ from pathlib import Path
27
+
28
+ from fastapi import FastAPI, File, Form, HTTPException, UploadFile
29
+ import uvicorn
30
+
31
+ from inference import (
32
+ get_available_models,
33
+ get_supported_model_names,
34
+ predict_rri,
35
+ resolve_default_model_name,
36
+ resolve_model_path,
37
+ )
38
+
39
+
40
+ app = FastAPI(title="AUD Relapse Risk MVP", version="0.1.0")
41
+ logging.basicConfig(
42
+ level=logging.INFO,
43
+ format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
44
+ )
45
+
46
+
47
+ @app.get("/health")
48
+ def health() -> dict[str, str | bool | dict[str, str]]:
49
+ default_model_name = resolve_default_model_name()
50
+ model_path = resolve_model_path(model_name=default_model_name)
51
+ return {
52
+ "status": "ok",
53
+ "default_model": default_model_name,
54
+ "model_path": model_path,
55
+ "model_found": Path(model_path).exists(),
56
+ "available_models": get_available_models(),
57
+ }
58
+
59
+
60
+ @app.post("/predict-rri")
61
+ async def predict_rri_endpoint(
62
+ model_name: str | None = Form(None),
63
+ file: UploadFile = File(...),
64
+ ) -> dict[str, float | str]:
65
+ selected_model_name = model_name or resolve_default_model_name()
66
+
67
+ if selected_model_name not in get_supported_model_names():
68
+ supported_names = ", ".join(get_supported_model_names())
69
+ raise HTTPException(
70
+ status_code=400,
71
+ detail=f"Unsupported model_name '{selected_model_name}'. Choose one of: {supported_names}.",
72
+ )
73
+
74
+ suffix = Path(file.filename or "upload.wav").suffix or ".wav"
75
+
76
+ with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as temporary_file:
77
+ temporary_file.write(await file.read())
78
+ temporary_audio_path = temporary_file.name
79
+
80
+ try:
81
+ return predict_rri(temporary_audio_path, model_name=selected_model_name)
82
+ except FileNotFoundError as error:
83
+ raise HTTPException(status_code=500, detail=str(error)) from error
84
+ except ValueError as error:
85
+ raise HTTPException(status_code=400, detail=str(error)) from error
86
+ except Exception as error:
87
+ raise HTTPException(status_code=500, detail=f"Inference failed: {error}") from error
88
+ finally:
89
+ try:
90
+ os.remove(temporary_audio_path)
91
+ except OSError:
92
+ pass
93
+
94
+
95
+ if __name__ == "__main__":
96
+ uvicorn.run("main:app", host="127.0.0.1", port=8110, reload=False)
model.py ADDED
@@ -0,0 +1,142 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import torch
2
+ import torch.nn as nn
3
+ from torch.autograd import Function
4
+
5
+
6
+ class GradientReversalFunction(Function):
7
+ @staticmethod
8
+ def forward(ctx, input_tensor, lambda_value):
9
+ ctx.lambda_value = lambda_value
10
+ return input_tensor.view_as(input_tensor)
11
+
12
+ @staticmethod
13
+ def backward(ctx, grad_output):
14
+ return -ctx.lambda_value * grad_output, None
15
+
16
+
17
+ class GradientReversalLayer(nn.Module):
18
+ def __init__(self, lambda_value: float = 1.0):
19
+ super().__init__()
20
+ self.lambda_value = lambda_value
21
+
22
+ def forward(self, input_tensor: torch.Tensor) -> torch.Tensor:
23
+ return GradientReversalFunction.apply(input_tensor, self.lambda_value)
24
+
25
+
26
+ class DomainDiscriminator(nn.Module):
27
+ def __init__(self, input_size: int, hidden_size: int = 128):
28
+ super().__init__()
29
+ self.network = nn.Sequential(
30
+ nn.Linear(input_size, hidden_size),
31
+ nn.ReLU(),
32
+ nn.Dropout(0.3),
33
+ nn.Linear(hidden_size, 2),
34
+ )
35
+
36
+ def forward(self, input_tensor: torch.Tensor) -> torch.Tensor:
37
+ return self.network(input_tensor)
38
+
39
+
40
+ class ConvolutionalFeatureExtractor(nn.Module):
41
+ def __init__(self):
42
+ super().__init__()
43
+ self.block_1 = nn.Sequential(
44
+ nn.Conv2d(1, 32, kernel_size=3, padding=1),
45
+ nn.BatchNorm2d(32),
46
+ nn.ReLU(),
47
+ nn.MaxPool2d(2),
48
+ )
49
+ self.block_2 = nn.Sequential(
50
+ nn.Conv2d(32, 64, kernel_size=3, padding=1),
51
+ nn.BatchNorm2d(64),
52
+ nn.ReLU(),
53
+ nn.MaxPool2d(2),
54
+ )
55
+ self.block_3 = nn.Sequential(
56
+ nn.Conv2d(64, 128, kernel_size=3, padding=1),
57
+ nn.BatchNorm2d(128),
58
+ nn.ReLU(),
59
+ nn.MaxPool2d(2),
60
+ )
61
+
62
+ def forward(self, input_tensor: torch.Tensor) -> torch.Tensor:
63
+ x = self.block_1(input_tensor)
64
+ x = self.block_2(x)
65
+ x = self.block_3(x)
66
+ return x
67
+
68
+
69
+ class AttentionPooling(nn.Module):
70
+ def __init__(self, feature_size: int):
71
+ super().__init__()
72
+ self.attention_layer = nn.Linear(feature_size, 1)
73
+
74
+ def forward(self, sequence_tensor: torch.Tensor) -> torch.Tensor:
75
+ attention_scores = self.attention_layer(sequence_tensor).squeeze(-1)
76
+ attention_weights = torch.softmax(attention_scores, dim=1).unsqueeze(-1)
77
+ pooled_output = torch.sum(sequence_tensor * attention_weights, dim=1)
78
+ return pooled_output
79
+
80
+
81
+ class CnnBiLstmAdversarialModel(nn.Module):
82
+ def __init__(self, num_emotions: int = 7):
83
+ super().__init__()
84
+ self.cnn_feature_extractor = ConvolutionalFeatureExtractor()
85
+ self.bi_lstm = nn.LSTM(
86
+ input_size=128 * 16,
87
+ hidden_size=128,
88
+ num_layers=1,
89
+ batch_first=True,
90
+ bidirectional=True,
91
+ )
92
+ self.attention_pooling = AttentionPooling(feature_size=256)
93
+ self.shared_projection = nn.Sequential(
94
+ nn.Linear(256, 256),
95
+ nn.LayerNorm(256),
96
+ nn.ReLU(),
97
+ nn.Dropout(0.3),
98
+ )
99
+ self.emotion_head = nn.Sequential(
100
+ nn.Linear(256, 128),
101
+ nn.ReLU(),
102
+ nn.Dropout(0.3),
103
+ nn.Linear(128, num_emotions),
104
+ )
105
+ self.impairment_head = nn.Sequential(
106
+ nn.Linear(256, 128),
107
+ nn.ReLU(),
108
+ nn.Dropout(0.3),
109
+ nn.Linear(128, 2),
110
+ )
111
+ self.gradient_reversal = GradientReversalLayer(lambda_value=1.0)
112
+ self.domain_discriminator = DomainDiscriminator(256, hidden_size=128)
113
+
114
+ def forward(
115
+ self,
116
+ mel_spectrogram: torch.Tensor,
117
+ return_domain_output: bool = True,
118
+ ) -> dict[str, torch.Tensor]:
119
+ cnn_output = self.cnn_feature_extractor(mel_spectrogram)
120
+ batch_size, channels, height, width = cnn_output.shape
121
+ sequence_input = (
122
+ cnn_output.permute(0, 3, 1, 2).contiguous().view(batch_size, width, channels * height)
123
+ )
124
+
125
+ lstm_output, _ = self.bi_lstm(sequence_input)
126
+ shared_representation = self.attention_pooling(lstm_output)
127
+ shared_representation = self.shared_projection(shared_representation)
128
+
129
+ emotion_logits = self.emotion_head(shared_representation)
130
+ impairment_logits = self.impairment_head(shared_representation)
131
+
132
+ outputs = {
133
+ "shared_representation": shared_representation,
134
+ "emotion_logits": emotion_logits,
135
+ "impairment_logits": impairment_logits,
136
+ }
137
+
138
+ if return_domain_output:
139
+ reversed_features = self.gradient_reversal(shared_representation)
140
+ outputs["domain_logits"] = self.domain_discriminator(reversed_features)
141
+
142
+ return outputs
models/b1_svm_imp_best.pkl ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:efdd89f0501458ea6b86503e9fc652df7e8e0e1e04c0012215aae602880be72e
3
+ size 1092531
outputs/cnn_bilstm_adversarial_model.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c9e39f271cbebf59c621ff36b532c6a123bcc20c15d1950b64947307b0bc27ca
3
+ size 9980143
preprocess.py ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import numpy as np
2
+ import torch
3
+ import librosa
4
+ from skimage.transform import resize as sk_resize
5
+ from scipy.signal import find_peaks
6
+
7
+
8
+ TARGET_SAMPLE_RATE = 16000
9
+ CLIP_DURATION_SECONDS = 4.0
10
+ CLIP_NUM_SAMPLES = int(TARGET_SAMPLE_RATE * CLIP_DURATION_SECONDS)
11
+ FFT_SIZE = 1024
12
+ HOP_LENGTH = 160
13
+ NUM_MELS = 128
14
+ MEL_IMAGE_SIZE = (128, 128)
15
+ SVM_NUM_MELS = 40
16
+ SVM_NUM_MFCC = 40
17
+
18
+
19
+ def standardize_waveform_length(
20
+ waveform: np.ndarray,
21
+ required_num_samples: int = CLIP_NUM_SAMPLES,
22
+ ) -> np.ndarray:
23
+ if len(waveform) < required_num_samples:
24
+ waveform = np.pad(waveform, (0, required_num_samples - len(waveform)))
25
+ else:
26
+ waveform = waveform[:required_num_samples]
27
+ return waveform.astype(np.float32)
28
+
29
+
30
+ def load_audio_file(
31
+ audio_path: str,
32
+ sample_rate: int = TARGET_SAMPLE_RATE,
33
+ duration_seconds: float = CLIP_DURATION_SECONDS,
34
+ ) -> np.ndarray:
35
+ waveform, _ = librosa.load(
36
+ audio_path,
37
+ sr=sample_rate,
38
+ mono=True,
39
+ duration=duration_seconds,
40
+ )
41
+ return standardize_waveform_length(waveform, int(sample_rate * duration_seconds))
42
+
43
+
44
+ def create_log_mel_spectrogram(
45
+ waveform: np.ndarray,
46
+ sample_rate: int = TARGET_SAMPLE_RATE,
47
+ ) -> np.ndarray:
48
+ mel_spectrogram = librosa.feature.melspectrogram(
49
+ y=waveform,
50
+ sr=sample_rate,
51
+ n_fft=FFT_SIZE,
52
+ hop_length=HOP_LENGTH,
53
+ n_mels=NUM_MELS,
54
+ )
55
+ log_mel = librosa.power_to_db(mel_spectrogram, ref=np.max)
56
+ log_mel = sk_resize(log_mel, MEL_IMAGE_SIZE, anti_aliasing=True)
57
+ log_mel = (log_mel - log_mel.min()) / (log_mel.max() - log_mel.min() + 1e-8)
58
+ return log_mel.astype(np.float32)
59
+
60
+
61
+ def preprocess_audio_file(audio_path: str) -> torch.Tensor:
62
+ waveform = load_audio_file(audio_path)
63
+ log_mel = create_log_mel_spectrogram(waveform)
64
+ return torch.tensor(log_mel[np.newaxis, np.newaxis, :, :], dtype=torch.float32)
65
+
66
+
67
+ def load_fixed_clip(
68
+ audio_path: str,
69
+ sample_rate: int = TARGET_SAMPLE_RATE,
70
+ duration_seconds: float = CLIP_DURATION_SECONDS,
71
+ ) -> np.ndarray:
72
+ return load_audio_file(
73
+ audio_path=audio_path,
74
+ sample_rate=sample_rate,
75
+ duration_seconds=duration_seconds,
76
+ )
77
+
78
+
79
+ def extract_svm_features(audio_path: str) -> np.ndarray:
80
+ """
81
+ Reproduce the flat feature pipeline used for the saved sklearn SVM artifact.
82
+
83
+ Source notebook: `_Traing.ipynb`, function `extract_flat(...)`
84
+
85
+ Feature layout:
86
+ - MFCC mean + delta mean + delta-delta mean: 120
87
+ - spectral statistics + spectral contrast mean: 13
88
+ - log-mel mean: 40
89
+ - tonnetz mean: 6
90
+ - chroma mean: 12
91
+ - prosodic features (mean F0, std F0, speech rate): 3
92
+ Total: 194
93
+ """
94
+ waveform = load_fixed_clip(audio_path)
95
+
96
+ mfcc = librosa.feature.mfcc(
97
+ y=waveform,
98
+ sr=TARGET_SAMPLE_RATE,
99
+ n_mfcc=SVM_NUM_MFCC,
100
+ hop_length=HOP_LENGTH,
101
+ )
102
+ mfcc_delta = librosa.feature.delta(mfcc)
103
+ mfcc_delta2 = librosa.feature.delta(mfcc, order=2)
104
+ mfcc_features = np.concatenate(
105
+ [
106
+ np.mean(mfcc, axis=1),
107
+ np.mean(mfcc_delta, axis=1),
108
+ np.mean(mfcc_delta2, axis=1),
109
+ ]
110
+ )
111
+
112
+ spectral_contrast = librosa.feature.spectral_contrast(
113
+ y=waveform,
114
+ sr=TARGET_SAMPLE_RATE,
115
+ hop_length=HOP_LENGTH,
116
+ n_bands=6,
117
+ )
118
+ spectral_features = np.array(
119
+ [
120
+ librosa.feature.spectral_centroid(
121
+ y=waveform,
122
+ sr=TARGET_SAMPLE_RATE,
123
+ hop_length=HOP_LENGTH,
124
+ ).mean(),
125
+ librosa.feature.spectral_bandwidth(
126
+ y=waveform,
127
+ sr=TARGET_SAMPLE_RATE,
128
+ hop_length=HOP_LENGTH,
129
+ ).mean(),
130
+ librosa.feature.spectral_rolloff(
131
+ y=waveform,
132
+ sr=TARGET_SAMPLE_RATE,
133
+ hop_length=HOP_LENGTH,
134
+ ).mean(),
135
+ librosa.onset.onset_strength(
136
+ y=waveform,
137
+ sr=TARGET_SAMPLE_RATE,
138
+ hop_length=HOP_LENGTH,
139
+ ).mean(),
140
+ librosa.feature.zero_crossing_rate(
141
+ waveform,
142
+ hop_length=HOP_LENGTH,
143
+ ).mean(),
144
+ librosa.feature.rms(
145
+ y=waveform,
146
+ hop_length=HOP_LENGTH,
147
+ ).mean(),
148
+ *np.mean(spectral_contrast, axis=1),
149
+ ],
150
+ dtype=np.float32,
151
+ )
152
+
153
+ mel = librosa.feature.melspectrogram(
154
+ y=waveform,
155
+ sr=TARGET_SAMPLE_RATE,
156
+ n_mels=SVM_NUM_MELS,
157
+ hop_length=HOP_LENGTH,
158
+ )
159
+ mel_features = np.mean(librosa.power_to_db(mel, ref=np.max), axis=1)
160
+
161
+ harmonic = librosa.effects.harmonic(waveform)
162
+ tonnetz_features = np.mean(
163
+ librosa.feature.tonnetz(y=harmonic, sr=TARGET_SAMPLE_RATE),
164
+ axis=1,
165
+ )
166
+ chroma_features = np.mean(
167
+ librosa.feature.chroma_stft(
168
+ y=waveform,
169
+ sr=TARGET_SAMPLE_RATE,
170
+ hop_length=HOP_LENGTH,
171
+ ),
172
+ axis=1,
173
+ )
174
+
175
+ f0, voiced_flags, _ = librosa.pyin(
176
+ waveform,
177
+ fmin=60,
178
+ fmax=500,
179
+ sr=TARGET_SAMPLE_RATE,
180
+ )
181
+ voiced_f0 = f0[voiced_flags & ~np.isnan(f0)]
182
+ mean_f0 = float(np.mean(voiced_f0)) if len(voiced_f0) > 5 else 0.0
183
+ std_f0 = float(np.std(voiced_f0)) if len(voiced_f0) > 5 else 0.0
184
+
185
+ rms = librosa.feature.rms(y=waveform, hop_length=HOP_LENGTH)[0]
186
+ peaks, _ = find_peaks(
187
+ rms,
188
+ height=rms.mean() * 0.8,
189
+ distance=int(0.15 / (HOP_LENGTH / TARGET_SAMPLE_RATE)),
190
+ )
191
+ prosodic_features = np.array(
192
+ [mean_f0, std_f0, len(peaks) / CLIP_DURATION_SECONDS],
193
+ dtype=np.float32,
194
+ )
195
+
196
+ features = np.concatenate(
197
+ [
198
+ mfcc_features,
199
+ spectral_features,
200
+ mel_features,
201
+ tonnetz_features,
202
+ chroma_features,
203
+ prosodic_features,
204
+ ],
205
+ dtype=np.float32,
206
+ )
207
+
208
+ features = np.nan_to_num(features, nan=0.0, posinf=0.0, neginf=0.0)
209
+ if features.shape[0] != 194:
210
+ raise ValueError(f"SVM feature mismatch during extraction: expected 194, got {features.shape[0]}.")
211
+
212
+ return features[np.newaxis, :]
requirements.txt CHANGED
@@ -1,3 +1,10 @@
1
- altair
2
- pandas
3
- streamlit
 
 
 
 
 
 
 
 
1
+ streamlit==1.46.1
2
+ fastapi==0.116.1
3
+ uvicorn==0.35.0
4
+ torch==2.8.0
5
+ librosa==0.11.0
6
+ scikit-image==0.25.2
7
+ python-multipart==0.0.20
8
+ scikit-learn==1.6.1
9
+ scipy==1.16.2
10
+ numpy==2.3.3
src/streamlit_app.py DELETED
@@ -1,40 +0,0 @@
1
- import altair as alt
2
- import numpy as np
3
- import pandas as pd
4
- import streamlit as st
5
-
6
- """
7
- # Welcome to Streamlit!
8
-
9
- Edit `/streamlit_app.py` to customize this app to your heart's desire :heart:.
10
- If you have any questions, checkout our [documentation](https://docs.streamlit.io) and [community
11
- forums](https://discuss.streamlit.io).
12
-
13
- In the meantime, below is an example of what you can do with just a few lines of code:
14
- """
15
-
16
- num_points = st.slider("Number of points in spiral", 1, 10000, 1100)
17
- num_turns = st.slider("Number of turns in spiral", 1, 300, 31)
18
-
19
- indices = np.linspace(0, 1, num_points)
20
- theta = 2 * np.pi * num_turns * indices
21
- radius = indices
22
-
23
- x = radius * np.cos(theta)
24
- y = radius * np.sin(theta)
25
-
26
- df = pd.DataFrame({
27
- "x": x,
28
- "y": y,
29
- "idx": indices,
30
- "rand": np.random.randn(num_points),
31
- })
32
-
33
- st.altair_chart(alt.Chart(df, height=700, width=700)
34
- .mark_point(filled=True)
35
- .encode(
36
- x=alt.X("x", axis=None),
37
- y=alt.Y("y", axis=None),
38
- color=alt.Color("idx", legend=None, scale=alt.Scale()),
39
- size=alt.Size("rand", legend=None, scale=alt.Scale(range=[1, 150])),
40
- ))
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
streamlit_app.py ADDED
@@ -0,0 +1,127 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """
2
+ Simple Streamlit interface for local testing.
3
+
4
+ Run:
5
+ `streamlit run streamlit_app.py`
6
+ """
7
+
8
+ import os
9
+ import tempfile
10
+ import logging
11
+ from pathlib import Path
12
+
13
+ import streamlit as st
14
+
15
+ from inference import get_supported_model_names, predict_rri, resolve_default_model_name
16
+
17
+ logging.basicConfig(
18
+ level=logging.INFO,
19
+ format="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
20
+ )
21
+
22
+ st.set_page_config(
23
+ page_title="AUD Relapse Risk Demo",
24
+ page_icon="🎙️",
25
+ layout="centered",
26
+ menu_items={
27
+ "Get Help": None,
28
+ "Report a Bug": None,
29
+ "About": None,
30
+ },
31
+ )
32
+
33
+
34
+ def risk_color(risk_level: str) -> str:
35
+ if risk_level == "Low":
36
+ return "#1f7a4d"
37
+ if risk_level == "Moderate":
38
+ return "#b26a00"
39
+ return "#b42318"
40
+
41
+
42
+ def main() -> None:
43
+ st.title("Speech-Based Relapse Risk Demo")
44
+ st.caption("Upload or record one audio sample to generate a preliminary speech-based relapse-risk proxy.")
45
+
46
+ model_labels = {
47
+ "cnn_bilstm": "CNN-BiLSTM",
48
+ "svm": "SVM",
49
+ }
50
+ supported_model_names = list(get_supported_model_names())
51
+ default_model_name = resolve_default_model_name()
52
+ default_index = supported_model_names.index(default_model_name)
53
+
54
+ selected_model_name = st.selectbox(
55
+ "Model",
56
+ options=supported_model_names,
57
+ index=default_index,
58
+ format_func=lambda model_name: model_labels.get(model_name, model_name),
59
+ )
60
+
61
+ input_mode = st.radio(
62
+ "Audio source",
63
+ options=["Upload file", "Record in app"],
64
+ horizontal=True,
65
+ )
66
+
67
+ audio_file = None
68
+
69
+ if input_mode == "Upload file":
70
+ audio_file = st.file_uploader(
71
+ "Audio file",
72
+ type=["wav", "mp3", "m4a", "ogg", "flac"],
73
+ accept_multiple_files=False,
74
+ )
75
+ else:
76
+ audio_file = st.audio_input("Record audio")
77
+
78
+ st.info(
79
+ "This output is a preliminary speech-based relapse-risk proxy. It is not a medical diagnosis."
80
+ )
81
+
82
+ if audio_file is None:
83
+ return
84
+
85
+ st.audio(audio_file, format=getattr(audio_file, "type", None) or "audio/wav")
86
+
87
+ if not st.button("Predict RRI", type="primary"):
88
+ return
89
+
90
+ suffix = Path(getattr(audio_file, "name", "recording.wav")).suffix or ".wav"
91
+
92
+ with tempfile.NamedTemporaryFile(delete=False, suffix=suffix) as temporary_file:
93
+ temporary_file.write(audio_file.getbuffer())
94
+ temporary_audio_path = temporary_file.name
95
+
96
+ try:
97
+ with st.spinner("Running inference..."):
98
+ result = predict_rri(
99
+ temporary_audio_path,
100
+ model_name=selected_model_name,
101
+ )
102
+ except Exception as error:
103
+ st.error(f"Inference failed: {error}")
104
+ return
105
+ finally:
106
+ try:
107
+ os.remove(temporary_audio_path)
108
+ except OSError:
109
+ pass
110
+
111
+ color = risk_color(result["risk_level"])
112
+
113
+ st.markdown(
114
+ f"""
115
+ <div style="padding: 1rem 1.1rem; border-radius: 0.9rem; background: #f6f8fb; border-left: 6px solid {color}; margin: 1rem 0;">
116
+ <div style="font-size: 0.9rem; color: #475467;">Risk Level</div>
117
+ <div style="font-size: 1.6rem; font-weight: 700; color: {color};">{result["risk_level"]}</div>
118
+ </div>
119
+ """,
120
+ unsafe_allow_html=True,
121
+ )
122
+
123
+ st.metric("RRI", f'{result["relapse_risk_index"]:.4f}')
124
+
125
+
126
+ if __name__ == "__main__":
127
+ main()