Hugging Face's logo

laurent-maille
/
EuroLLM-22B-Instruct-2512-INT8

Text Generation
Transformers
Safetensors
llama
eurollm
int8
bitsandbytes
conversational
8-bit precision
Model card Files
xet
 Community
EuroLLM-22B-Instruct-2512-INT8 / README.md
laurent-maille's picture
laurent-maille
Upload INT8 export (safetensors)
4225f7f verified
preview code
|
raw
history blame contribute delete
798 Bytes
metadata 
license: apache-2.0
tags:
  - eurollm
  - int8
  - bitsandbytes
  - transformers

laurent-maille/EuroLLM-22B-Instruct-2512-INT8

Quantized INT8 (bitsandbytes / LLM.int8) version of utter-project/EuroLLM-22B-Instruct-2512.

Notes

  • This repository contains a Transformers-compatible INT8 export.
  • Load with BitsAndBytesConfig(load_in_8bit=True) and device_map="auto".

Example 

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
q = BitsAndBytesConfig(load_in_8bit=True)
tok = AutoTokenizer.from_pretrained("laurent-maille/EuroLLM-22B-Instruct-2512-INT8", trust_remote_code=True)
mdl = AutoModelForCausalLM.from_pretrained("laurent-maille/EuroLLM-22B-Instruct-2512-INT8", device_map="auto", quantization_config=q, trust_remote_code=True)