How to self-host?
Hello there and thank you for sharing the model!
While the model being open-weight is great, it seems to require extra knowledge to be able to benefit from it. It would be great if you'd share instructions on how to self-host the model and connect it to Zed.
Hello!
I have researched a bit about the topic. Based on (https://zed.dev/docs/ai/edit-prediction), "eager" is the default mode.
prompt_format usally defaults to "infer", this did not really work for me, it always suggested to delete code. "zeta2" as prompt_format gave me the exact experience as the usual TAB predictions.
I did the set up using ollama on Fedora 43 running one of the quantizations available in zed-industries/zeta-2 .
ollama pull hf.co/bartowski/zed-industries_zeta-2-GGUF:Q8_0
And then on Zed's settings.json just add or substitute the "edit_predictions" key with the following values.
"edit_predictions": {
"mode": "eager",
"provider": "open_ai_compatible_api",
"open_ai_compatible_api": {
"api_url": "http://localhost:11434/v1/completions",
"model": "hf.co/bartowski/zed-industries_zeta-2-GGUF:Q8_0",
"prompt_format": "zeta2",
"max_output_tokens": 64,
},
This can all be done on the settings UI too!
