henryNdubuaku commited on
Commit
b274efc
·
verified ·
1 Parent(s): 0f51a1a

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +9 -0
README.md CHANGED
@@ -88,6 +88,15 @@ Every platform folder in this repo holds an engine under 1 MB that loads `needle
88
  ./needle --model needle3.cact --tools tools.json --serve
89
  ```
90
 
 
 
 
 
 
 
 
 
 
91
  The [devices guide](https://cactuscompute.com/blog/needle-supported-devices) covers the runner flags, the C API, the WASI component and air-gapped setup.
92
 
93
  ## Citation
 
88
  ./needle --model needle3.cact --tools tools.json --serve
89
  ```
90
 
91
+ Tool schemas share the model context with the system prompt and conversation.
92
+ `needle_init` returns the tokenized static-prefix length on success and fails
93
+ when that prefix does not fit the model context. In the C API, call
94
+ `needle_last_error()` after a negative return to get the measured prefix tokens
95
+ and context limit. Reduce tool descriptions/schema text, split large catalogues,
96
+ or declare more than five tools so Needle can keep only retrieved tools in the
97
+ per-turn prefix. At completion time, `max_new_tokens` also reserves context room,
98
+ so larger generation caps leave less space for the current turn and history.
99
+
100
  The [devices guide](https://cactuscompute.com/blog/needle-supported-devices) covers the runner flags, the C API, the WASI component and air-gapped setup.
101
 
102
  ## Citation