fine_tuning_guide.md

🎓 Guide: Fine-Tuning Your Voice Detection Model

This guide explains how to improve your voice detection model's accuracy by fine-tuning it on specialized datasets like ASVspoof or In-the-Wild.

1. Prerequisites

You will need a GPU-enabled environment. Google Colab (Free Tier) is the easiest way to start.

• Google Colab
• Hugging Face Account

2. The Dataset

For audio deepfake detection, you need a dataset with labeled "Real" and "Fake" audio. Recommended Datasets:

• ASVspoof 2019/2021: The gold standard for voice anti-spoofing.
• WaveFake: A dataset of deepfake audio.
• In-the-Wild: Dataset containing deepfakes of politicians and celebrities.

3. Fine-Tuning Steps (in Google Colab)

Step A: Install Libraries

!pip install transformers datasets torch librosa accelerate

Step B: Load Your Dataset

Assuming you have a folder structure like data/real/*.wav and data/fake/*.wav.

from datasets import load_dataset, Audio

# Load from local folder or a Hugging Face dataset rep
dataset = load_dataset("audiofolder", data_dir="path_to_your_data")
# Split into train/test
dataset = dataset.train_test_split(test_size=0.2)

Step C: Preprocessing

Resample all audio to 16kHz (required by Wav2Vec2).

from transformers import AutoFeatureExtractor

model_id = "MelodyMachine/Deepfake-audio-detection"
feature_extractor = AutoFeatureExtractor.from_pretrained(model_id)

def preprocess_function(examples):
    audio_arrays = [x["array"] for x in examples["audio"]]
    inputs = feature_extractor(
        audio_arrays, 
        sampling_rate=16000, 
        max_length=160000, # 10 seconds
        truncation=True
    )
    return inputs

dataset = dataset.cast_column("audio", Audio(sampling_rate=16000))
encoded_dataset = dataset.map(preprocess_function, remove_columns="audio", batched=True)

Step D: Load Model & Training Config

from transformers import AutoModelForAudioClassification, TrainingArguments, Trainer

num_labels = 2
label2id = {"Fake": 0, "Real": 1}
id2label = {0: "Fake", 1: "Real"}

model = AutoModelForAudioClassification.from_pretrained(
    model_id, 
    num_labels=num_labels,
    label2id=label2id,
    id2label=id2label,
    ignore_mismatched_sizes=True # Important when fine-tuning on new classes
)

training_args = TrainingArguments(
    output_dir="./results",
    evaluation_strategy="epoch",
    learning_rate=3e-5,
    per_device_train_batch_size=8,
    num_train_epochs=5,
)

Step E: Train!

trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=encoded_dataset["train"],
    eval_dataset=encoded_dataset["test"],
    tokenizer=feature_extractor,
)

trainer.train()

Step F: Save & Export

model.save_pretrained("my_finetuned_model")
feature_extractor.save_pretrained("my_finetuned_model")

4. Using Your New Model

Once trained, upload your "my_finetuned_model" folder to Hugging Face Hub. Then, simply update MODEL_NAME in your real_detector.py:

MODEL_NAME = "your-username/my_finetuned_model"

💡 Tips for Accuracy

• Diversity: Ensure your "Fake" data includes many different TTS engines (ElevenLabs, Murf, Coqui, etc.).
• Noise: Add background noise to your training data to make the model robust against real-world recordings.