Instructions to use THUdyh/Ola-7b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use THUdyh/Ola-7b with Transformers:
# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("THUdyh/Ola-7b", dtype="auto") - Notebooks
- Google Colab
- Kaggle
Low bit version issues
#7
by Jilt - opened
I tried low bit versions using bnb, but having issues making it work, could be a bug in infer.py
See github comment: https://github.com/Ola-Omni/Ola/issues/2#issuecomment-2688126751
Hi, as we have not tested our model in low-bit settings, you may follow the error instructions to solve the issue.