Each core returns the probability that the input is fake. The unified script combines
them as a weighted sum:

```
final_fakeness = 0.5·lipsync + 0.4·audio + 0.1·video
```

(weights are renormalised over whichever cores actually produced a score).

---

## How it routes input

```
        ┌──────────── audio file ───────────►  audio core  ──►  final = audio score
INPUT ──┤
        └──────────── video file ──┬─ extract wav ─► audio core  ┐
                                    ├──────────────► video core  ├─► weighted final
                                    └──────────────► lipsync core ┘
```

Audio-only input only the audio core runs; the final score *is* the audio score.
Video input audio is extracted with `ffmpeg` and sent to the audio core; the video
  file goes to the video and lipsync cores.




## Requirements

Linux `ffmpeg` / `ffprobe`** on `PATH`
  (the scripts default to `/usr/bin/ffmpeg`, `/usr/bin/ffprobe`).
 Three isolated Python environments, since the cores have mutually incompatible
  dependencies, so they cannot share one environment:

  | core | interpreter (default) | key pins |
  |------|-----------------------|----------|
  | audio   | `/opt/dfdetect-envs/audio/bin/python` | torch 2.6.0, transformers 4.44.0, **scikit-learn 1.3.2**, scipy 1.13.1, joblib 1.4.2, librosa, soundfile |
  | video   | `/opt/dfdetect-envs/video/bin/python` | torch 2.2.2, torchvision 0.17.2, transformers 4.51.2, numpy 1.26.4, opencv-python 4.11.0.86 |
  | lipsync | `/opt/conda/envs/avh/bin/python`      | local fairseq + avhubert, dlib, skvideo, python_speech_features, numpy 1.25 |

Put the environments on LOCAL disk (e.g. `/opt`)


### Recreating the environments

```bash
# audio
python -m venv /opt/dfdetect-envs/audio
/opt/dfdetect-envs/audio/bin/python -m pip install -U pip wheel
/opt/dfdetect-envs/audio/bin/python -m pip install \
  torch==2.6.0 torchvision==0.21.0 transformers==4.44.0 \
  scikit-learn==1.3.2 scipy==1.13.1 joblib==1.4.2 \
  librosa soundfile accelerate "huggingface_hub<0.26" pandas numpy

# video
python -m venv /opt/dfdetect-envs/video
/opt/dfdetect-envs/video/bin/python -m pip install -U pip wheel
/opt/dfdetect-envs/video/bin/python -m pip install \
  torch==2.2.2 torchvision==0.17.2 transformers==4.51.2 numpy==1.26.4 \
  opencv-python==4.11.0.86 huggingface-hub safetensors accelerate pillow scipy pandas

# lipsync — a conda env (Python 3.10) named `avh`
conda create -n avh python=3.10 -y
/opt/conda/envs/avh/bin/python -m pip install -U pip wheel
/opt/conda/envs/avh/bin/python -m pip install -r avh-align_core/requirements.txt

```

> The vendored `avh-align_core/fairseq/` is the *importable package only* (no `setup.py`),
> so it works via `sys.path` at runtime but is **not pip-installable** — that is why the
> recipe above installs fairseq from source.



## Inference — `detect.py`

```bash
python detect.py INPUT_FILE                 # text report
python detect.py INPUT_FILE --json          # machine-readable JSON

```



---

## Retraining

Each script reuses the matching core's frozen feature extractor (so the feature space
stays identical to inference) and retrains only the lightweight head. They auto-relaunch
into the correct environment, so just run them with any Python.

### Label conventions (important — audio is inverted!)

| script | metadata columns | label meaning |
|--------|------------------|---------------|
| `retrain_audio.py`   | `file_path,label` | **`1` = real, `0` = fake**  |
| `retrain_video.py`   | `file_path,label` | **`1` = fake, `0` = real**  |
| `retrain_lipsync.py` | `file_path` (or `path`) | **real videos only** (no label — unsupervised alingment network) |



python retrain_audio.py   --metadata audio_meta.csv


python retrain_video.py   --metadata video_meta.csv  --epochs 40


python retrain_lipsync.py --metadata real_videos.csv --epochs 10 --skip-existing


Each script has `--help` for all options (epochs, lr, batch size, `--out`, etc.).

### Using a retrained model

Point `detect.py` at it — the weights are swapped in after the core loads:

```bash
python detect.py clip.mp4 \
  --audio-model   retrained_models/audio/logreg_retrained.joblib \
  --video-model   retrained_models/video/cnn_retrained.pt \
  --lipsync-model retrained_models/lipsync/fusion_retrained.pt
```

---

## Troubleshooting

A core reports `FAILED — its score is dropped and the remaining cores are
  reweighted; rerun with `--verbose` to see that core's traceback (e.g. lipsync needs a
  detectable face; very short clips may yield no segments).
  No GPU / busy GPU — cores fall back to CPU (much slower). The box is shared, so
  throughput depends on how many of the GPUs are free.