Update README.md
Browse files
README.md
CHANGED
|
@@ -24,7 +24,7 @@ architecture:
|
|
| 24 |
- GIST
|
| 25 |
---
|
| 26 |
|
| 27 |
-
# All models tested with ALLM(AnythingLLM) with LM as server, all models should be work with ollama
|
| 28 |
<b> GPT4All has only one model (nomic), but the setup for local documents described below is the same</b><br>
|
| 29 |
|
| 30 |
(sometimes the results are more truthful if the “chat with document only” option is used)<br>
|
|
@@ -40,8 +40,8 @@ give me a ❤️, if you like ;)<br>
|
|
| 40 |
</ul>
|
| 41 |
Working well, all other its up to you! (jina and qwen based not yet supported)
|
| 42 |
<br>
|
| 43 |
-
|
| 44 |
-
#
|
| 45 |
Set your (Max Tokens)context-lenght 16000t main-model, set your embedder-model (Max Embedding Chunk Length) 1024t,set (Max Context Snippets) 14,
|
| 46 |
but in ALLM its cutting all in 1024 character parts, so aprox two times or bit more ~20!
|
| 47 |
<br>
|
|
@@ -58,36 +58,44 @@ You can play and set for your needs, eg 8-snippets a 2048t, or 28-snippets a 512
|
|
| 58 |
<br>
|
| 59 |
...
|
| 60 |
<br>
|
| 61 |
-
|
|
|
|
|
|
|
| 62 |
You have a txt/pdf file maybe 90000words(~300pages) a book. You ask the model lets say "what is described in chapter called XYZ in relation to person ZYX".
|
| 63 |
Now it searches for keywords or similar semantic terms in the document. if it has found them, lets say word and meaning around “XYZ and ZYX” ,
|
| 64 |
now a piece of text 1024token around this word “XYZ/ZYX” is cut out at this point.
|
| 65 |
-
This text snippet is then used for your answer.
|
| 66 |
-
|
|
|
|
|
|
|
|
|
|
| 67 |
<br>
|
| 68 |
-
If you expect multible search results in your docs try 16-snippets or more, if you expect only 2 than dont use more
|
| 69 |
<br>
|
| 70 |
-
If you use snipets-size ~1024t you receive more content, if you use ~256t you receive more facts
|
| 71 |
<br>
|
| 72 |
-
A question for "summary of the document" is most time not useful, if the document has an introduction or summaries its searching there if you have luck
|
| 73 |
<br>
|
| 74 |
-
If a book has a table of contents or a bibliography, I would delete these pages as they often contain relevant search terms but do not help answer the user's question
|
| 75 |
<br>
|
| 76 |
-
If the documents small like 10-20 Pages, its better you copy the whole text inside the prompt
|
| 77 |
<br>
|
| 78 |
...
|
| 79 |
<br>
|
| 80 |
-
|
|
|
|
|
|
|
| 81 |
Some models can handle 128k or 1M tokens, but even with 16k input the response with the same snippets as input is worse than with other well developed models.<br>
|
| 82 |
<br>
|
| 83 |
-
<
|
|
|
|
| 84 |
You are a helpful assistant who provides an overview of ... under the aspects of ... .
|
| 85 |
You use attached excerpts from the collection to generate your answers!
|
| 86 |
Weight each individual excerpt in order, with the most important excerpts at the top and the less important ones further down.
|
| 87 |
The context of the entire article should not be given too much weight.
|
| 88 |
Answer the user's question!
|
| 89 |
After your answer, briefly explain why you included excerpts (1 to X) in your response and justify briefly if you considered some of them unimportant!<br>
|
| 90 |
-
<i>(change it for your needs, this example works well when I consult a book about a person and a term related to them, the explanation was just a test for myself)</i><br>
|
| 91 |
or:<br>
|
| 92 |
You are an imaginative storyteller who crafts compelling narratives with depth, creativity, and coherence.
|
| 93 |
Your goal is to develop rich, engaging stories that captivate readers, staying true to the themes, tone, and style appropriate for the given prompt.
|
|
@@ -97,9 +105,9 @@ or:<br>
|
|
| 97 |
You are are a warm and engaging companion who loves to talk about cooking, recipes and the joy of food.
|
| 98 |
Your aim is to share delicious recipes, cooking tips and the stories behind different cultures in a personal, welcoming and knowledgeable way.<br>
|
| 99 |
<br>
|
| 100 |
-
The system prompt is weighted with a certain amount of influence around your question. You can easily test it once without or with a nonsensical system prompt
|
| 101 |
<br><br>
|
| 102 |
-
usual models
|
| 103 |
llama3.1, llama3.2, qwen2.5, deepseek-r1-distill, SauerkrautLM-Nemo(german) ... <br>
|
| 104 |
(llama3 or phi3.5 are not working well) <br>
|
| 105 |
|
|
|
|
| 24 |
- GIST
|
| 25 |
---
|
| 26 |
|
| 27 |
+
# <b>All models tested with ALLM(AnythingLLM) with LM as server, all models should be work with ollama</b>
|
| 28 |
<b> GPT4All has only one model (nomic), but the setup for local documents described below is the same</b><br>
|
| 29 |
|
| 30 |
(sometimes the results are more truthful if the “chat with document only” option is used)<br>
|
|
|
|
| 40 |
</ul>
|
| 41 |
Working well, all other its up to you! (jina and qwen based not yet supported)
|
| 42 |
<br>
|
| 43 |
+
<br>
|
| 44 |
+
# Short hints for using (Example for a large context with many expected hits):<br>
|
| 45 |
Set your (Max Tokens)context-lenght 16000t main-model, set your embedder-model (Max Embedding Chunk Length) 1024t,set (Max Context Snippets) 14,
|
| 46 |
but in ALLM its cutting all in 1024 character parts, so aprox two times or bit more ~20!
|
| 47 |
<br>
|
|
|
|
| 58 |
<br>
|
| 59 |
...
|
| 60 |
<br>
|
| 61 |
+
|
| 62 |
+
# How embedding and search works:
|
| 63 |
+
|
| 64 |
You have a txt/pdf file maybe 90000words(~300pages) a book. You ask the model lets say "what is described in chapter called XYZ in relation to person ZYX".
|
| 65 |
Now it searches for keywords or similar semantic terms in the document. if it has found them, lets say word and meaning around “XYZ and ZYX” ,
|
| 66 |
now a piece of text 1024token around this word “XYZ/ZYX” is cut out at this point.
|
| 67 |
+
This text snippet is then used for your answer. <br>
|
| 68 |
+
<ul style="line-height: 1;">
|
| 69 |
+
<li>If, for example, the word “XYZ” occurs 100 times in one file, not all 100 are found.</li>
|
| 70 |
+
<br>
|
| 71 |
+
<li>If only one snippet corresponds to your question all other snippets can negatively influence your answer because they do not fit the topic (usually 4 to 32 snippet are fine)</li>
|
| 72 |
<br>
|
| 73 |
+
<li>If you expect multible search results in your docs try 16-snippets or more, if you expect only 2 than dont use more!</li>
|
| 74 |
<br>
|
| 75 |
+
<li>If you use snipets-size ~1024t you receive more content, if you use ~256t you receive more facts.</li>
|
| 76 |
<br>
|
| 77 |
+
<li>A question for "summary of the document" is most time not useful, if the document has an introduction or summaries its searching there if you have luck.</li>
|
| 78 |
<br>
|
| 79 |
+
<li>If a book has a table of contents or a bibliography, I would delete these pages as they often contain relevant search terms but do not help answer the user's question.</li>
|
| 80 |
<br>
|
| 81 |
+
<li>If the documents small like 10-20 Pages, its better you copy the whole text inside the prompt.</li>
|
| 82 |
<br>
|
| 83 |
...
|
| 84 |
<br>
|
| 85 |
+
|
| 86 |
+
# Nevertheless, the <b>main model is also important</b>!
|
| 87 |
+
Especially to deal with the context length and I don't mean just the theoretical number you can set.
|
| 88 |
Some models can handle 128k or 1M tokens, but even with 16k input the response with the same snippets as input is worse than with other well developed models.<br>
|
| 89 |
<br>
|
| 90 |
+
<br>
|
| 91 |
+
# Important -> The Systemprompt (an example):
|
| 92 |
You are a helpful assistant who provides an overview of ... under the aspects of ... .
|
| 93 |
You use attached excerpts from the collection to generate your answers!
|
| 94 |
Weight each individual excerpt in order, with the most important excerpts at the top and the less important ones further down.
|
| 95 |
The context of the entire article should not be given too much weight.
|
| 96 |
Answer the user's question!
|
| 97 |
After your answer, briefly explain why you included excerpts (1 to X) in your response and justify briefly if you considered some of them unimportant!<br>
|
| 98 |
+
<i>(change it for your needs, this example works well when I consult a book about a person and a term related to them, the explanation part was just a test for myself)</i><br>
|
| 99 |
or:<br>
|
| 100 |
You are an imaginative storyteller who crafts compelling narratives with depth, creativity, and coherence.
|
| 101 |
Your goal is to develop rich, engaging stories that captivate readers, staying true to the themes, tone, and style appropriate for the given prompt.
|
|
|
|
| 105 |
You are are a warm and engaging companion who loves to talk about cooking, recipes and the joy of food.
|
| 106 |
Your aim is to share delicious recipes, cooking tips and the stories behind different cultures in a personal, welcoming and knowledgeable way.<br>
|
| 107 |
<br>
|
| 108 |
+
<li> The system prompt is weighted with a certain amount of influence around your question. You can easily test it once without or with a nonsensical system prompt.</li>
|
| 109 |
<br><br>
|
| 110 |
+
usual models works well:<br>
|
| 111 |
llama3.1, llama3.2, qwen2.5, deepseek-r1-distill, SauerkrautLM-Nemo(german) ... <br>
|
| 112 |
(llama3 or phi3.5 are not working well) <br>
|
| 113 |
|