Sweaterdog

/

Andy-4

like 13

Hello every one,
I'm the happy owner of a Mac Studio 4M with 36Gb shared ram, and Tahoe 26.x software plus the latest ollama version.
I was eager to try mindcraft with a local only solution. And Sweater/Andy-4 models seemed to be a good solution.
It was with andy-4:micro-q8_0. With that little nuance that it is hallucinating a lot despite a strict prompt, and doing whatever random, unexpected, undesired things to ruin your minecraft world.
So, having plenty of ram, I tried to give andy-4:latest a chance.
Unfortunately, it ended with a code 500. It couldn't be memory. So I dug into the log :

• Assertion failed: (found), function llama_sampler_dist_apply, file llama-sampling.cpp, line 660. SIGABRT: abort
• load: special_eos_id is not in special_eog_ids - the tokenizer config may be incorrect

As I'm not specialized in ollama model making, whether on MAc or any other system, here's my question :

• Is there a possible fix for this problem, or is it too deep and costly to investigate and correct? (i.e. Mac architecture is Arm64 based and doesn't have Cuda's instruction set.)
• If, hopefully, there is a fix, would you be kind enough to drive me through the process of applying the correction, with as much detail as possible.
  Should you need more informations, just tell me what. I didn't dare copy/pasting the 50k log here not to pollute the server with no need.
  If it is necessary to train the model on MacOs Tahoe 26.x, I understand you don't want to do it for the few people using that configuration. So, if I'm guided to do the job, I'll be pleased to participate.
  Thanks a lot.