Upload README.md with huggingface_hub
Browse files
README.md
CHANGED
|
@@ -88,6 +88,15 @@ Every platform folder in this repo holds an engine under 1 MB that loads `needle
|
|
| 88 |
./needle --model needle3.cact --tools tools.json --serve
|
| 89 |
```
|
| 90 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 91 |
The [devices guide](https://cactuscompute.com/blog/needle-supported-devices) covers the runner flags, the C API, the WASI component and air-gapped setup.
|
| 92 |
|
| 93 |
## Citation
|
|
|
|
| 88 |
./needle --model needle3.cact --tools tools.json --serve
|
| 89 |
```
|
| 90 |
|
| 91 |
+
Tool schemas share the model context with the system prompt and conversation.
|
| 92 |
+
`needle_init` returns the tokenized static-prefix length on success and fails
|
| 93 |
+
when that prefix does not fit the model context. In the C API, call
|
| 94 |
+
`needle_last_error()` after a negative return to get the measured prefix tokens
|
| 95 |
+
and context limit. Reduce tool descriptions/schema text, split large catalogues,
|
| 96 |
+
or declare more than five tools so Needle can keep only retrieved tools in the
|
| 97 |
+
per-turn prefix. At completion time, `max_new_tokens` also reserves context room,
|
| 98 |
+
so larger generation caps leave less space for the current turn and history.
|
| 99 |
+
|
| 100 |
The [devices guide](https://cactuscompute.com/blog/needle-supported-devices) covers the runner flags, the C API, the WASI component and air-gapped setup.
|
| 101 |
|
| 102 |
## Citation
|