Automatic Speech Recognition
Transformers
Safetensors
phi4mm
text-generation
nlp
code
audio
speech-summarization
speech-translation
visual-question-answering
phi-4-multimodal
phi
phi-4-mini
custom_code
Eval Results
Instructions to use microsoft/Phi-4-multimodal-instruct with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use microsoft/Phi-4-multimodal-instruct with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("automatic-speech-recognition", model="microsoft/Phi-4-multimodal-instruct", trust_remote_code=True)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("microsoft/Phi-4-multimodal-instruct", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
Streaming audio?
#49
by matthen - opened
I am interested in feeding streaming audio into the model, but it looks like the current ConformerEncoder is not causal.
Is that expected?
Is there any change I can make to the config to make the audio representation causal (and then I could fine-tune)? I have tried setting "chunk_se": 2 in the config.json, but that did not seem to work.
Aha if I set chunk_size from -1 to something positive then it works! chunk_se is not actually used in the code
matthen changed discussion status to closed
Could you share your method?
Very appreciate