itismax0 commited on
Commit
e9ec967
·
verified ·
1 Parent(s): 388b67c

Add llama.cpp/Ollama usage section (F16 GGUF)

Browse files
Files changed (1) hide show
  1. README.md +21 -0
README.md CHANGED
@@ -115,6 +115,27 @@ response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special
115
  print(response)
116
  ```
117
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
118
  ---
119
 
120
  <a id="обзор-на-русском"></a>
 
115
  print(response)
116
  ```
117
 
118
+ ### Run with llama.cpp / Ollama (GGUF, full precision)
119
+
120
+ A full-precision **F16 GGUF** (`RusNanoML-800M-F16.gguf`) is included in this repo — no quantization loss.
121
+
122
+ ```bash
123
+ # llama.cpp
124
+ llama-cli -hf itismax0/RusNanoML-800M:RusNanoML-800M-F16.gguf --temp 0 -p "Найдите все действительные корни уравнения x^4 - 5x^2 + 4 = 0."
125
+ ```
126
+
127
+ **Ollama** (import the GGUF locally):
128
+
129
+ ```bash
130
+ echo "FROM RusNanoML-800M-F16.gguf
131
+ PARAMETER temperature 0
132
+ TEMPLATE \"{{ .Prompt }}\"" > Modelfile
133
+ ollama create rusnanoml-800m -f Modelfile
134
+ ollama run rusnanoml-800m "Найдите все действительные корни уравнения x^4 - 5x^2 + 4 = 0."
135
+ ```
136
+
137
+ > Note: the model uses the Qwen3.5 hybrid linear-attention architecture. llama.cpp builds from `master` (>= b12413) support it; older builds will refuse to load the file.
138
+
139
  ---
140
 
141
  <a id="обзор-на-русском"></a>