README.md

metadata

language:
  - ja
license: mit
tags:
  - whisper
  - fine-tuning
  - jdd-topic1
  - speechbrain
  - automatic-speech-recognition
base_model: openai/whisper-base
datasets:
  - noflm/jdd_topic1_batch2
pipeline_tag: automatic-speech-recognition

Whisper Fine-tuning Experiment: jdd_topic1_batch2-whisper-base

Model Description

This model contains a complete Whisper fine-tuning experiment including:

• Training checkpoints (SpeechBrain format)
• Final model (Transformers format)
• Test results and evaluation metrics
• Training history visualizations

Model Information

• Base Model: openai/whisper-base
• Framework: SpeechBrain v1.0.3
• Training Dataset: noflm/jdd_topic1_batch2
• Language: Japanese (ja)
• Task: Automatic Speech Recognition (ASR)

Test Results

• WER: 12.17%
• CER: 9.08%
• Test Loss: 0.0814

Contents

├── checkpoints/          # Training checkpoints
│   ├── CKPT+epoch_*/    # Per-epoch checkpoints
│   ├── CKPT+BEST_WER/   # Best WER checkpoint
│   └── CKPT+FINAL/      # Final checkpoint
├── final_model/          # Transformers-compatible model
│   ├── config.json      # Model configuration
│   ├── model.safetensors # Model weights
│   ├── preprocessor_config.json
│   ├── tokenizer_config.json
│   └── ...
├── test_results.json     # Test metrics
├── detailed_metrics.json # Detailed training history
├── training_history_speechbrain.png  # Training curves
└── training_report_speechbrain.txt   # Summary report

Usage

Load checkpoint (SpeechBrain format)

import torch
checkpoint = torch.load('checkpoints/CKPT+BEST_WER/model.ckpt')

Load final model (Transformers format)

from transformers import WhisperForConditionalGeneration, WhisperProcessor

model = WhisperForConditionalGeneration.from_pretrained("./final_model")
processor = WhisperProcessor.from_pretrained("./final_model")

Citation

If you use this experiment data, please cite the original Whisper paper:

@article{radford2022robust,
  title={Robust speech recognition via large-scale weak supervision},
  author={Radford, Alec and Kim, Jong Wook and Xu, Tao and Brockman, Greg and McLeavey, Christine and Sutskever, Ilya},
  journal={arXiv preprint arXiv:2212.04356},
  year={2022}
}