Instructions to use openai/whisper-large-v3 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use openai/whisper-large-v3 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="openai/whisper-large-v3")# Load model directly from transformers import AutoProcessor, AutoModelForSpeechSeq2Seq processor = AutoProcessor.from_pretrained("openai/whisper-large-v3") model = AutoModelForSpeechSeq2Seq.from_pretrained("openai/whisper-large-v3") - Inference
- Notebooks
- Google Colab
- Kaggle
Currently working on finetuning dataset for japanese anime speech
If anyone is interested in a Japanese language finetuning dataset aimed at anime speech you can find it here:
https://huggingface.co/datasets/joujiboi/japanese-anime-speech
I hope we, the open-source community, do great stuff with datasets like this and others that make models perform more evenly across languages!
Post some niche non-English datasets for finetuning whisper below
hey @joujiboi , i'm currently picking up interests on fine tuning a particular language as well, and i'm very new in this LLM domain, do you have some resource you've used before that i could make use of to start my own journey also?
thanks
Firstly, I just want to make it clear Whisper isn't a Large Language Model (predict the next word model), it's an Automatic Speech Recognition model (speech to text).
If you want to finetune Whisper, openai has a blog on how to do so here. If you've never finetuned a model before it might be quite difficult but not impossible, if you want to start by finetuning something a lot easier Stable Diffusion is a lot easier.
Sorry for the late response, I didn't see the notification!
Good luck!