Eval Requests
Unfortunately vllm doesn't have good support for gemma-3n-E4B-it. And for some reason LFM2-8B-A1B-absolute-heresy errors for me but the base model doesn't. I can try retesting it later but yeah idk :/
Thank you for trying. Gemma 3n is rather tricky, I guess. Could you post the error logs for LFM2, if still available? I would like to investigate and see what's the issue.
Currently getting
ValueError: Tokenizer class TokenizersBackend does not exist or is not currently imported.
I noticed tokenizers.json, among other configuration files, weren't aligned properly with the original model. It could have caused by some dependency mismatch (packages e.g. transformers are built from git) or by the WIP code I merged in to heretic. Looking back, even mradermacher team had the same error with the model. Anyways, it should work as intended now. Can you try again whenever you find the most convenient?