README.md

---
license: mit
library_name: mlx
tags:
  - mlx
  - function-calling
  - encoder-decoder
base_model: Cactus-Compute/needle
---

# Needle MLX

MLX port of [Cactus-Compute/needle](https://huggingface.co/Cactus-Compute/needle),
a 26M-parameter encoder-decoder tool-calling model.

## Run

```bash
uv run python - <<'PY'
import mlx.core as mx
from model import SimpleAttentionNetwork
from tokenizer import NeedleTokenizer, build_encoder_input

path = "mlx-model"  # or a downloaded Hugging Face snapshot
model = SimpleAttentionNetwork.from_pretrained(path)
tokenizer = NeedleTokenizer.from_pretrained(path)
tools = '[{"name":"get_weather","parameters":{"location":"string"}}]'
src = mx.array([build_encoder_input(tokenizer, "Weather in Delhi?", tools)])
print(tokenizer.decode(model.generate(src, max_new_tokens=128)).removeprefix("<tool_call>"))
PY
```

Create the upload directory from the original checkpoint:

```bash
uv run convert.py weights mlx-model
uvx --from huggingface_hub hf upload mlx-community/needle-mlx mlx-model .
```

## Compatibility

This repository uses a custom Needle architecture. It runs through the included
MLX implementation; it is not currently supported by `mlx-lm`, Ollama,
llama.cpp, or LM Studio. GGUF or an Ollama `Modelfile` alone cannot add the
missing architecture implementation.

## License

MIT. Original model and weights: Cactus Compute.