AIAMD: answering machine detection (YAMNet + GRU)

Decides whether an outbound call was answered by a human or a machine (voicemail, call screener, IVR, network tones), from the first ≀ 3 s of the callee's audio.

An implementation of A Recurrent Neural Network Approach to the Answering Machine Detection Problem (Altwlkany et al., 2024, arXiv:2410.08235): a frozen YAMNet turns every 0.96 s of audio (every 0.48 s) into 1,024 numbers, and a small GRU gives p(machine) after each window, so it can decide while the call is still going.

8 kHz audio -> fill digital silence + 3.6 kHz low-pass -> 16 kHz -> YAMNet (frozen)
            -> Dense(32, tanh) -> GRU(48) -> Dense(1, sigmoid) = p(machine) after every 0.48 s

44,657 trainable parameters.

Files

File What
amd_gru.keras the GRU (input: YAMNet embeddings, shape (windows, 1024))
amd_gru_calibration.json temperature (1.156) and decision threshold (0.452), fitted on validation data
inference.py self-contained preprocessing + YAMNet + GRU + decision
config.yaml preprocessing, model and decision settings
training_summary.json, amd_gru_history.csv training run (best epoch 4 by validation loss)
evaluation.json full test / holdout results

Usage

pip install tensorflow tensorflow-hub "setuptools<81" soundfile scipy numpy huggingface_hub
hf download kugu/AIAMD --local-dir AIAMD
python AIAMD/inference.py call.wav
from inference import AmdDetector
detector = AmdDetector()                  # folder with the downloaded files
result = detector.predict(samples, sample_rate)   # mono float audio, any sample rate
# {"decision": "machine", "p_machine": 0.99, "threshold": 0.452, "p_machine_per_window": [...], ...}

Preprocessing must match training: runs of exact-zero samples β‰₯ 10 ms are replaced by -70 dBFS noise, then a causal 8th-order Butterworth low-pass at 3.6 kHz, then resampling to 16 kHz, then the first 3 s. inference.py does all of this.

Training data (not released)

Clips Description
Human 2,175 the callee's first response on outbound calls, ≀ 3 s, 8 kHz
Machine 3,000 for training (+ 7,000 holdout) vendor-labelled recordings: voicemail, call screeners, IVR, network tones, beeps
  • Splits 80 / 10 / 10 by call, so no call appears in two splits.
  • Both classes get the same random leading (0–1 s) and trailing (0–0.5 s) silence, made from the clip's own background noise, and machine clips are cut to lengths drawn from the human clips. Clip length and silence therefore don't reveal the class.
  • Windows quieter than -50 dBFS get zero weight in training.

Results

Test: 212 human + 300 machine clips. Holdout: 7,000 machine clips. Decision after the whole clip.

Threshold Human recall Machine recall (test) Machine recall (holdout) Balanced acc. AUC
0.5 0.972 0.950 0.962 0.961 0.993
0.452 (calibrated, target 99 % human recall on val) 0.962 0.963 0.965 0.963 (95 % CI 0.947–0.979) 0.993

Decision after k windows (test, threshold 0.452):

Audio heard 0.98 s 1.46 s 1.94 s 2.42 s 2.90 s
Balanced accuracy 0.868 0.927 0.940 0.958 0.965

Machine categories (test + holdout): voicemail 0.974, network / tone 0.955, call screening 0.949, IVR 0.948. Hardest: custom voicemail, clicks, name-only greetings (~0.86–0.89).

Limitations

  • Recording channel. All human audio comes from one dialer and all machine audio from one vendor, so part of what the model learned may be the recording channel. Scores on audio from other systems may be lower.
  • Strict human-safety targets. The validation set has 220 humans, too few to set a 99 % human-recall threshold reliably: on test the calibrated threshold gives 96.2 %. A few human clips get confident machine scores, so at 99 % human recall only about 73 % of machines are caught. In the same setting a gradient-boosted model on hand-crafted timing / loudness / MFCC features caught 94 %. Choose the threshold on your own data.
  • At most 3 s of audio (training clips were ≀ 3 s); the paper uses 4 s.
  • Labels were screened automatically, not reviewed by ear.
  • 8 kHz telephony audio only.

Citation

@article{altwlkany2024amd,
  title   = {A Recurrent Neural Network Approach to the Answering Machine Detection Problem},
  author  = {Altwlkany, Kemal and Delali{\'c}, Sead and Selmanovi{\'c}, Elmedin and Alihod{\v{z}}i{\'c}, Adis and Lovri{\'c}, Ivica},
  journal = {arXiv preprint arXiv:2410.08235},
  year    = {2024}
}
Downloads last month
-
Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Paper for kugu/AIAMD