sampler settings?
#2
by
doc-acula
- opened
As mlx versions are out already, what are recommended sampler settings for this model?
Thanks
Thanks for the question!
For the MLX versions we’re currently using the following sampler settings as our default for general conversational use:
{
"repetition_penalty": 1.06,
"temperature": 0.7,
"top_p": 0.95,
"top_k": 4
}