Whats the closest modeling code?
#9
by michaelfeil - opened
For frameworks support, from which model should be best start modeling (llama, bert, qwen)
Hello, the easiest model to start from is Llama, as EuroBERT follows the Llama architecture, but without the causal mask.
What are other key differences besides causal=False?
Hey @michaelfeil ,our modelling is available here : https://huggingface.co/EuroBERT/EuroBERT-210m/blob/main/modeling_eurobert.py and the llama one here : https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py
It will be hard to tell you the modifications line by line. If you have a use case, feel free to share it—maybe we can help you better.
Nicolas-BZRD changed discussion status to closed