if you encounter error "keyerror: mistral" update transformers directly from git
#4
by rvian - opened
I had an error in the following code cell when running the 7b model in colab:
"
model_name = "defog/sqlcoder-7b"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
trust_remote_code=True,
# torch_dtype=torch.bfloat16,
# load_in_8bit=True,
load_in_4bit=True,
device_map="auto",
use_cache=True,
)
"
KeyError: 'mistral'
To fix it, install the updated version of transformers running the following code:
!pip install --upgrade git+https://github.com/huggingface/transformers
You may have to restart the kernel machine to make it work.
PS: Great model, working pretty well in my tests without finetuning.