Instructions to use Franzabner/mixed-quant-epi with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Franzabner/mixed-quant-epi with Transformers:
# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Franzabner/mixed-quant-epi", dtype="auto") - Notebooks
- Google Colab
- Kaggle
mixed-quant-epi
Per-layer mixed quantization evaluated by EPI -- accuracy vs energy Pareto frontier
Placeholder -- full content coming soon. See the GitHub repo for current work.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support