Updates for v0.6

Main pack of updates:
- web ui
- knowledge import
- FAISS DB
- prompt subfolders

.gitignore

@@ -20,4 +20,14 @@ memory/*
 # Ignore all contents of the directory "logs"
 logs/*
 # But do not ignore the directory itself
-!logs/.gitkeep
+!logs/.gitkeep
+
+# Ignore all contents of the directory "tmp"
+tmp/*
+# But do not ignore the directory itself
+!tmp/.gitkeep
+
+# Ignore all contents of the directory "knowledge"
+knowledge/*
+# But do not ignore the directory itself
+!knowledge/.gitkeep

.vscode/launch.json

@@ -2,10 +2,18 @@
     "version": "0.2.0",
     "configurations": [
         {
-            "name": "Debug main.py",
+            "name": "Debug run_ui.py",
             "type": "debugpy",
             "request": "launch",
-            "program": "./main.py",
+            "program": "./run_ui.py",
+            "console": "integratedTerminal",
+            "args": ["-Xfrozen_modules=off"]
+        },
+        {
+            "name": "Debug run_cli.py",
+            "type": "debugpy",
+            "request": "launch",
+            "program": "./run_cli.py",
             "console": "integratedTerminal",
             "args": ["-Xfrozen_modules=off"]
         },

README.md

@@ -1,6 +1,6 @@
 # Agent Zero
 
-[![Join our Discord](https://img.shields.io/badge/Discord-Join%20our%20server-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/B8KZKNsPpj) [![Subscribe on YouTube](https://img.shields.io/badge/YouTube-Subscribe-red?style=for-the-badge&logo=youtube&logoColor=white)](https://www.youtube.com/@AgentZeroFW) [![Connect on LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-blue?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/jan-tomasek/) [![Follow on X.com](https://img.shields.io/badge/X.com-Follow-1DA1F2?style=for-the-badge&logo=x&logoColor=white)](https://x.com/JanTomasekDev)
+[![Join our Skool Community](https://img.shields.io/badge/Skool-Join%20our%20Community-4A90E2?style=for-the-badge&logo=skool&logoColor=white)](https://www.skool.com/agent-zero) [![Join our Discord](https://img.shields.io/badge/Discord-Join%20our%20server-5865F2?style=for-the-badge&logo=discord&logoColor=white)](https://discord.gg/B8KZKNsPpj) [![Subscribe on YouTube](https://img.shields.io/badge/YouTube-Subscribe-red?style=for-the-badge&logo=youtube&logoColor=white)](https://www.youtube.com/@AgentZeroFW) [![Connect on LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-blue?style=for-the-badge&logo=linkedin&logoColor=white)](https://www.linkedin.com/in/jan-tomasek/) [![Follow on X.com](https://img.shields.io/badge/X.com-Follow-1DA1F2?style=for-the-badge&logo=x&logoColor=white)](https://x.com/JanTomasekDev)
 
 
 
@@ -23,12 +23,12 @@
 
 3. **Multi-agent cooperation**
 - Every agent has a superior agent giving it tasks and instructions. Every agent then reports back to its superior.
-- In the case of the first agent, the superior is the human user; the agent sees no difference.
+- In the case of the first agent in the chain (Agent 0), the superior is the human user; the agent sees no difference.
 - Every agent can create its subordinate agent to help break down and solve subtasks. This helps all agents keep their context clean and focused.
 
 4. **Completely customizable and extensible**
 - Almost nothing in this framework is hard-coded. Nothing is hidden. Everything can be extended or changed by the user.
-- The whole behavior is defined by a system prompt in the **prompts/agent.system.md** file. Change this prompt and change the framework dramatically.
+- The whole behavior is defined by a system prompt in the **prompts/default/agent.system.md** file. Change this prompt and change the framework dramatically.
 - The framework does not guide or limit the agent in any way. There are no hard-coded rails that agents have to follow.
 - Every prompt, every small message template sent to the agent in its communication loop, can be found in the **prompts/** folder and changed.
 - Every default tool can be found in the **python/tools/** folder and changed or copied to create new predefined tools.
@@ -81,10 +81,10 @@ If you cannot provide all the necessary conditions or API keys, just change the
 Update: [Guide by CheezChat for Windows](./docs/win_installation_guide.txt)
 
 1. **Required API keys:**
-- At the moment, the only recommended API key is for https://www.perplexity.ai/ API. Perplexity is used as a convenient web search tool and has not yet been replaced by an open-source alternative. If you do not have an API key for Perplexity, leave it empty in the .env file and Perplexity will not be used.
-- Chat models and embedding models can be executed locally via Ollama and HuggingFace or via API as well.
+- No API keys are required. Models can run locally. The only recommended API key is for https://www.perplexity.ai/ API. Perplexity is used as a convenient web search tool and has not yet been replaced by an open-source alternative in Agent Zero. If you do not have an API key for Perplexity, leave it empty in the .env file and Perplexity will not be used.
+- Chat models and embedding models can be executed locally via Ollama, LMStudio and HuggingFace or via API as well.
 
-2. **Enter your API keys:**
+2. **Enter your API keys if needed:**
 - You can enter your API keys into the **.env** file, which you can copy from **example.env**
 - Or you can export your API keys in the terminal session:
 ~~~bash
@@ -97,9 +97,9 @@ export API_KEY_OPENAI="your-api-key-here"
 pip install -r requirements.txt
 ~~~
 
-3. **Choose your chat, utility and embeddings model:**
-- In the **main.py** file, right at the start of the **chat()** function, you can see how the chat model and embedding model are set.
-- You can choose between online models (OpenAI, Anthropic, Groq) or offline (Ollama, HuggingFace) for both.
+3. **Choose your chat, utility and embeddings model and check other configuration options:**
+- In the **initialize.py** file, you can see how the chat model and embedding model are set.
+- You can choose between online models (OpenAI, Anthropic, Groq) or offline (Ollama, LMStudio, HuggingFace) for both.
 
 4. **run Docker:**
 - Easiest way is to install Docker Desktop application and just run it. The rest will be handled by the framework itself.

agent.py

@@ -9,18 +9,24 @@ from langchain_core.messages import HumanMessage, SystemMessage
 from langchain_core.language_models.chat_models import BaseChatModel
 from langchain_core.language_models.llms import BaseLLM
 from langchain_core.embeddings import Embeddings
+from concurrent.futures import Future
+from python.helpers.log import Log
+from python.helpers.dirty_json import DirtyJson
+
 
 @dataclass
 class AgentConfig: 
     chat_model: BaseChatModel | BaseLLM
     utility_model: BaseChatModel | BaseLLM
     embeddings_model:Embeddings
+    prompts_subdir: str = ""
     memory_subdir: str = ""
+    knowledge_subdir: str = ""
     auto_memory_count: int = 3
     auto_memory_skip: int = 2
     rate_limit_seconds: int = 60
     rate_limit_requests: int = 15
-    rate_limit_input_tokens: int = 1000000
+    rate_limit_input_tokens: int = 0
     rate_limit_output_tokens: int = 0
     msgs_keep_max: int = 25
     msgs_keep_start: int = 5
@@ -54,23 +60,35 @@ class Agent:
         self.number = number
         self.agent_name = f"Agent {self.number}"
 
-        self.system_prompt = files.read_file("./prompts/agent.system.md", agent_name=self.agent_name)
-        self.tools_prompt = files.read_file("./prompts/agent.tools.md")
-
         self.history = []
         self.last_message = ""
         self.intervention_message = ""
         self.intervention_status = False
         self.rate_limiter = rate_limiter.RateLimiter(max_calls=self.config.rate_limit_requests,max_input_tokens=self.config.rate_limit_input_tokens,max_output_tokens=self.config.rate_limit_output_tokens,window_seconds=self.config.rate_limit_seconds)
         self.data = {} # free data object all the tools can use
+        self.future: Future|None = None
+
 
         os.chdir(files.get_abs_path("./work_dir")) #change CWD to work_dir
+
+    def communicate(self, msg: str):
+        Agent.paused=False #unpause if paused
         
+        if not self.future or self.future.done():
+            return self.message_loop(msg)
+        else:
+            if Agent.streaming_agent: current_agent = Agent.streaming_agent
+            else:                     current_agent = self
+                
+            current_agent.intervention_message = msg #intervene current agent
+            if self.future: return self.future.result() #wait for original agent
+            else: return ""
 
     def message_loop(self, msg: str):
         try:
+            self.future = Future()
             printer = PrintStyle(italic=True, font_color="#b3ffd9", padding=False)    
-            user_message = files.read_file("./prompts/fw.user_message.md", message=msg)
+            user_message = self.read_prompt("fw.user_message.md", message=msg)
             self.append_message(user_message, human=True) # Append the user's input to the history                        
             memories = self.fetch_memories(True)
                 
@@ -81,7 +99,7 @@ class Agent:
 
                 try:
 
-                    system = self.system_prompt + "\n\n" + self.tools_prompt
+                    system = self.read_prompt("agent.system.md", agent_name=self.agent_name) + "\n\n" + self.read_prompt("agent.tools.md")
                     memories = self.fetch_memories()
                     if memories: system+= "\n\n"+memories
 
@@ -97,8 +115,9 @@ class Agent:
                     self.rate_limiter.limit_call_and_input(tokens)
                     
                     # output that the agent is starting
-                    PrintStyle(bold=True, font_color="green", padding=True, background_color="white").print(f"{self.agent_name}: Starting a message:")
-                                            
+                    PrintStyle(bold=True, font_color="green", padding=True, background_color="white").print(f"{self.agent_name}: Generating:")
+                    log = Log(type="agent", heading=f"{self.agent_name}: Generating:")
+                              
                     for chunk in chain.stream(inputs):
                         if self.handle_intervention(agent_response): break # wait for intervention and handle it, if paused
 
@@ -107,33 +126,50 @@ class Agent:
                         else: content = str(chunk)
                         
                         if content:
-                            printer.stream(content) # output the agent response stream                
+                            printer.stream(content) # output the agent response stream
                             agent_response += content # concatenate stream into the response
+                            self.log_from_stream(agent_response, log)
 
                     self.rate_limiter.set_output_tokens(int(len(agent_response)/4))
                     
                     if not self.handle_intervention(agent_response):
                         if self.last_message == agent_response: #if assistant_response is the same as last message in history, let him know
                             self.append_message(agent_response) # Append the assistant's response to the history
-                            warning_msg = files.read_file("./prompts/fw.msg_repeat.md")
+                            warning_msg = self.read_prompt("fw.msg_repeat.md")
                             self.append_message(warning_msg, human=True) # Append warning message to the history
                             PrintStyle(font_color="orange", padding=True).print(warning_msg)
+                            Log.log(type="warning", content=warning_msg)
 
                         else: #otherwise proceed with tool
                             self.append_message(agent_response) # Append the assistant's response to the history
                             tools_result = self.process_tools(agent_response) # process tools requested in agent message
-                            if tools_result: return tools_result #break the execution if the task is done
+                            if tools_result: #final response of message loop available
+                                self.future.set_result(tools_result) #set result to future
+                                return tools_result #break the execution if the task is done
 
-                # Forward errors to the LLM, maybe he can fix them
+                # Forward errors to the LLM, maybe it can fix them
                 except Exception as e:
                     error_message = errors.format_error(e)
-                    msg_response = files.read_file("./prompts/fw.error.md", error=error_message) # error message template
+                    msg_response = self.read_prompt("fw.error.md", error=error_message) # error message template
                     self.append_message(msg_response, human=True)
                     PrintStyle(font_color="red", padding=True).print(msg_response)
+                    Log.log(type="error", content=msg_response)
+                    self.future.set_exception(e) #set result to future
                     
         finally:
             Agent.streaming_agent = None # unset current streamer
 
+    def read_prompt(self, file:str, **kwargs):
+        content = ""
+        if self.config.prompts_subdir:
+            try:
+                content = files.read_file(files.get_abs_path(f"./prompts/{self.config.prompts_subdir}/{file}"), **kwargs)
+            except Exception as e:
+                pass
+        if not content:
+            content = files.read_file(files.get_abs_path(f"./prompts/default/{file}"), **kwargs)
+        return content
+
     def get_data(self, field:str):
         return self.data.get(field, None)
 
@@ -162,10 +198,12 @@ class Agent:
         chain = prompt | self.config.utility_model
         response = ""
         printer = None
+        logger = None
 
         if output_label:
             PrintStyle(bold=True, font_color="orange", padding=True, background_color="white").print(f"{self.agent_name}: {output_label}:")
-            printer = PrintStyle(italic=True, font_color="orange", padding=False)                
+            printer = PrintStyle(italic=True, font_color="orange", padding=False)
+            logger = Log(type="adhoc", heading=f"{self.agent_name}: {output_label}:")       
 
         formatted_inputs = prompt.format()
         tokens = int(len(formatted_inputs)/4)     
@@ -180,6 +218,7 @@ class Agent:
 
             if printer: printer.stream(content)
             response+=content
+            if logger: logger.update(content=response)
 
         self.rate_limiter.set_output_tokens(int(len(response)/4))
 
@@ -190,7 +229,7 @@ class Agent:
             return self.history[-1]
 
     def replace_middle_messages(self,middle_messages):
-        cleanup_prompt = files.read_file("./prompts/fw.msg_cleanup.md")
+        cleanup_prompt = self.read_prompt("fw.msg_cleanup.md")
         summary = self.send_adhoc_message(system=cleanup_prompt,msg=self.concat_messages(middle_messages), output_label="Mid messages cleanup summary")
         new_human_message = HumanMessage(content=summary)
         return [new_human_message]
@@ -225,7 +264,7 @@ class Agent:
         while self.paused: time.sleep(0.1) # wait if paused
         if self.intervention_message and not self.intervention_status: # if there is an intervention message, but not yet processed
             if progress.strip(): self.append_message(progress) # append the response generated so far
-            user_msg = files.read_file("./prompts/fw.intervention.md", user_message=self.intervention_message) # format the user intervention template
+            user_msg = self.read_prompt("fw.intervention.md", user_message=self.intervention_message) # format the user intervention template
             self.append_message(user_msg,human=True) # append the intervention message
             self.intervention_message = "" # reset the intervention message
             self.intervention_status = True
@@ -253,9 +292,10 @@ class Agent:
             if self.handle_intervention(): return # wait if paused and handle intervention message if needed
             if response.break_loop: return response.message
         else:
-            msg = files.read_file("prompts/fw.msg_misformat.md")
+            msg = self.read_prompt("fw.msg_misformat.md")
             self.append_message(msg, human=True)
             PrintStyle(font_color="red", padding=True).print(msg)
+            Log.log(type="error", content=f"{self.agent_name}: Message misformat:")
 
 
     def get_tool(self, name: str, args: dict, message: str, **kwargs):
@@ -290,9 +330,17 @@ class Agent:
                 "conversation_history" : messages,
                 "raw_memories": memories
             }
-            cleanup_prompt = files.read_file("./prompts/msg.memory_cleanup.md").replace("{", "{{")       
+            cleanup_prompt = self.read_prompt("msg.memory_cleanup.md").replace("{", "{{")       
             clean_memories = self.send_adhoc_message(cleanup_prompt,json.dumps(input), output_label="Memory injection")
             return clean_memories
 
+    def log_from_stream(self, stream: str, log: Log):
+        try:
+            if len(stream) < 25: return # no reason to try
+            response = DirtyJson.parse_string(stream)
+            if isinstance(response, dict): log.update(content=stream, kvps=response) #log if result is a dictionary already
+        except Exception as e:
+            pass
+
     def call_extension(self, name: str, **kwargs) -> Any:
         pass

docs/win_installation_guide.txt

@@ -49,7 +49,7 @@ pip install -r requirements.txt
 ## Run the program
 - Just run the **main.py** file in Python:
 ~~~bash
-python main.py
+python run_ui.py
 ~~~
 - Or run it in debug mode in VS Code using the **debug** button in the top right corner of the editor. I have provided config files for VS Code for this purpose.
 
@@ -140,7 +140,7 @@ PYDEVD_DISABLE_FILE_VALIDATION=1
 6. Configure API Preferences
 
 If you entered an openAI API key earlier, you may skip this step. If you entered an alternative key, 
-* Right Click on the file **main.py** in the Agent-Zero root folder
+* Right Click on the file **initialize.py** in the Agent-Zero root folder
 * Select **Open With**
 * Select **Notepad** or your preferred text editor
 * Scroll about 20 lines down from the top until you see lines that look like this: *chat_llm = models.get_*
@@ -182,11 +182,11 @@ If all of the requirements installed succesfully, you can proceed to run the pro
 * Activate the Agent-Zero environment by double clicking on its name
 * click Open Terminal
 * Navigate to the agent-zero folder
-* Type **python main.py** and press enter
+* Type **python run_ui.py** and press enter
 
 ```
 (Agent-Zero) C:\Users\yourUserName>cd c:\projects\agent-zero
-(Agent-Zero) C:\projects\agent-zero\python main.py
+(Agent-Zero) C:\projects\agent-zero\python run_ui.py
 Initializing framework...
 
 User message ('e' to leave):

example.env

@@ -9,14 +9,11 @@ API_KEY_OPENAI_AZURE=
 OPENAI_AZURE_ENDPOINT=
 OPENAI_API_VERSION=
 
-TOKENIZERS_PARALLELISM=true
-PYDEVD_DISABLE_FILE_VALIDATION=1 
-
 HF_TOKEN=
 
-EDGE_TTS_MODEL="fr-FR-RemyMultilingualNeural"
-# en-US-AndrewMultilingualNeural
-# de-DE-FlorianMultilingualNeural
-# es-ES-AlvaroNeural
-# zh-CN-shaanxi-XiaoniNeural
-# more check https://huggingface.co/spaces/innoai/Edge-TTS-Text-to-Speech and at bottom use via api
+
+WEB_UI_PORT=50001
+
+
+TOKENIZERS_PARALLELISM=true
+PYDEVD_DISABLE_FILE_VALIDATION=1 

initialize.py

@@ -0,0 +1,59 @@
+import models
+from agent import Agent, AgentConfig
+
+def initialize():
+    
+    # main chat model used by agents (smarter, more accurate)
+    chat_llm = models.get_openai_chat(model_name="gpt-4o-mini", temperature=0)
+    # chat_llm = models.get_ollama_chat(model_name="gemma2:latest", temperature=0)
+    chat_llm = models.get_lmstudio_chat(model_name="TheBloke/Mistral-7B-Instruct-v0.2-GGUF", temperature=0)
+    # chat_llm = models.get_openrouter(model_name="meta-llama/llama-3-8b-instruct:free")
+    # chat_llm = models.get_azure_openai_chat(deployment_name="gpt-4o-mini", temperature=0)
+    # chat_llm = models.get_anthropic_chat(model_name="claude-3-5-sonnet-20240620", temperature=0)
+    # chat_llm = models.get_google_chat(model_name="gemini-1.5-flash", temperature=0)
+    # chat_llm = models.get_groq_chat(model_name="llama-3.1-70b-versatile", temperature=0)
+    
+    # utility model used for helper functions (cheaper, faster)
+    utility_llm = chat_llm # change if you want to use a different utility model
+
+    # embedding model used for memory
+    embedding_llm = models.get_openai_embedding(model_name="text-embedding-3-small")
+    # embedding_llm = models.get_ollama_embedding(model_name="nomic-embed-text")
+    # embedding_llm = models.get_huggingface_embedding(model_name="sentence-transformers/all-MiniLM-L6-v2")
+
+    # agent configuration
+    config = AgentConfig(
+        chat_model = chat_llm,
+        utility_model = utility_llm,
+        embeddings_model = embedding_llm,
+        prompts_subdir = "custom",
+        # memory_subdir = "",
+        auto_memory_count = 0,
+        # auto_memory_skip = 2,
+        # rate_limit_seconds = 60,
+        rate_limit_requests = 15,
+        # rate_limit_input_tokens = 0,
+        # rate_limit_output_tokens = 0,
+        # msgs_keep_max = 25,
+        # msgs_keep_start = 5,
+        # msgs_keep_end = 10,
+        max_tool_response_length = 3000,
+        # response_timeout_seconds = 60,
+        code_exec_docker_enabled = True,
+        # code_exec_docker_name = "agent-zero-exe",
+        # code_exec_docker_image = "frdel/agent-zero-exe:latest",
+        # code_exec_docker_ports = { "22/tcp": 50022 }
+        # code_exec_docker_volumes = { files.get_abs_path("work_dir"): {"bind": "/root", "mode": "rw"} }
+        code_exec_ssh_enabled = True,
+        # code_exec_ssh_addr = "localhost",
+        # code_exec_ssh_port = 50022,
+        # code_exec_ssh_user = "root",
+        # code_exec_ssh_pass = "toor",
+        # additional = {},
+    )
+    
+    # create the first agent
+    agent0 = Agent( number = 0, config = config )
+
+    # return initialized agent
+    return agent0

python/helpers/docker.py

@@ -5,6 +5,7 @@ from typing import Optional
 from python.helpers.files import get_abs_path
 from python.helpers.errors import format_error
 from python.helpers.print_style import PrintStyle
+from python.helpers.log import Log
 
 class DockerContainerManager:
     def __init__(self, image: str, name: str, ports: Optional[dict[str, int]] = None, volumes: Optional[dict[str, dict[str, str]]] = None):
@@ -24,7 +25,9 @@ class DockerContainerManager:
                 err = format_error(e)
                 if ("ConnectionRefusedError(61," in err or "Error while fetching server API version" in err):
                     PrintStyle.hint("Connection to Docker failed. Is docker or Docker Desktop running?") # hint for user
+                    Log.log(type="hint", content="Connection to Docker failed. Is docker or Docker Desktop running?")
                     PrintStyle.error(err)
+                    Log.log(type="error", content=err)
                     time.sleep(5) # try again in 5 seconds
                 else: raise
         return self.client
@@ -35,8 +38,11 @@ class DockerContainerManager:
                 self.container.stop()
                 self.container.remove()
                 print(f"Stopped and removed the container: {self.container.id}")
+                Log.log(type="info", content=f"Stopped and removed the container: {self.container.id}")
             except Exception as e:
                 print(f"Failed to stop and remove the container: {e}")
+                Log.log(type="error", content=f"Failed to stop and remove the container: {e}")
+                
 
     def start_container(self) -> None:
         if not self.client: self.client = self.init_docker()
@@ -49,6 +55,8 @@ class DockerContainerManager:
         if existing_container:
             if existing_container.status != 'running':
                 print(f"Starting existing container: {self.name} for safe code execution...")
+                Log.log(type="info", content=f"Starting existing container: {self.name} for safe code execution...")
+                
                 existing_container.start()
                 self.container = existing_container
                 time.sleep(2) # this helps to get SSH ready
@@ -58,13 +66,16 @@ class DockerContainerManager:
                 # print(f"Container with name '{self.name}' is already running with ID: {existing_container.id}")
         else:
             print(f"Initializing docker container {self.name} for safe code execution...")
+            Log.log(type="info", content=f"Initializing docker container {self.name} for safe code execution...")
+
             self.container = self.client.containers.run(
                 self.image,
                 detach=True,
-                ports=self.ports,
+                ports=self.ports, # type: ignore
                 name=self.name,
-                volumes=self.volumes,
-            )
+                volumes=self.volumes, # type: ignore
+            ) 
             atexit.register(self.cleanup_container)
             print(f"Started container with ID: {self.container.id}")
+            Log.log(type="info", content=f"Started container with ID: {self.container.id}")
             time.sleep(5) # this helps to get SSH ready

python/helpers/extract_tools.py

@@ -39,7 +39,6 @@ def extract_json_string(content):
         # Return the matched JSON string
         return match.group(0)
     else:
-        print("No JSON content found.")
         return ""
 
 def fix_json_string(json_string):

python/helpers/knowledge_import.py

@@ -0,0 +1,87 @@
+import glob
+import os
+import hashlib
+import json
+from typing import Any, Dict, Literal, TypedDict
+from langchain_community.document_loaders import (
+    CSVLoader, JSONLoader, PyPDFLoader, TextLoader, UnstructuredHTMLLoader, 
+    UnstructuredMarkdownLoader
+)
+from python.helpers import files
+from python.helpers.log import Log
+
+text_loader_kwargs = {'autodetect_encoding': True}
+
+
+class KnowledgeImport(TypedDict):
+    file: str
+    checksum: str
+    ids: list[str]
+    state: Literal["changed", "original", "removed"]
+    documents: list[Any]
+
+
+def calculate_checksum(file_path: str) -> str:
+    hasher = hashlib.md5()
+    with open(file_path, 'rb') as f:
+        buf = f.read()
+        hasher.update(buf)
+    return hasher.hexdigest()
+
+def load_knowledge(knowledge_dir: str, index: Dict[str, KnowledgeImport]) -> Dict[str, KnowledgeImport]:
+    knowledge_dir = files.get_abs_path(knowledge_dir)
+    
+
+    # Mapping file extensions to corresponding loader classes
+    file_types_loaders = {
+        'txt': TextLoader,
+        'pdf': PyPDFLoader,
+        'csv': CSVLoader,
+        'html': UnstructuredHTMLLoader,
+        'json': JSONLoader,
+        'md': UnstructuredMarkdownLoader
+    }
+
+    cnt_files = 0
+    cnt_docs = 0
+
+    # Fetch all files in the directory with specified extensions
+    kn_files = glob.glob(knowledge_dir + '/**/*', recursive=True)
+    if kn_files:
+        print(f"Found {len(kn_files)} knowledge files in {knowledge_dir}, processing...")
+        Log.log(type="info", content=f"Found {len(kn_files)} knowledge files in {knowledge_dir}, processing...")
+
+    for file_path in kn_files:
+        ext = file_path.split('.')[-1].lower()
+        if ext in file_types_loaders:
+            checksum = calculate_checksum(file_path)
+            file_key = os.path.relpath(file_path, knowledge_dir)
+            
+            # Load existing data from the index or create a new entry
+            file_data = index.get(file_key, {})
+            
+            if file_data.get('checksum') == checksum:
+                file_data['state'] = 'original'
+            else:
+                file_data['state'] = 'changed'
+            
+            if file_data['state'] == 'changed':
+                file_data['checksum'] = checksum
+                loader_cls = file_types_loaders[ext]
+                loader = loader_cls(file_path, **(text_loader_kwargs if ext in ['txt', 'csv', 'html', 'md'] else {}))
+                file_data['documents'] = loader.load_and_split()
+                cnt_files += 1
+                cnt_docs += len(file_data['documents'])
+                # print(f"Imported {len(file_data['documents'])} documents from {file_path}")
+            
+            # Update the index
+            index[file_key] = file_data # type: ignore
+
+    # loop index where state is not set and mark it as removed
+    for file_key, file_data in index.items():
+        if not file_data.get('state', ''):
+            index[file_key]['state'] = 'removed'
+
+    print(f"Processed {cnt_docs} documents from {cnt_files} files.")
+    Log.log(type="info", content=f"Processed {cnt_docs} documents from {cnt_files} files.")
+    return index

python/helpers/log.py

@@ -0,0 +1,58 @@
+from dataclasses import dataclass
+from typing import Optional, Dict
+import uuid
+
+@dataclass
+class LogItem:
+    no: int
+    type: str
+    heading: str
+    content: str
+    kvps: Optional[Dict] = None
+    
+
+class Log:
+
+    guid = uuid.uuid4()
+    version: int = 0
+    last_updated: int = 0
+    logs: list = []
+    
+    def __init__(self, type: str="placeholder", heading: str="", content: str="", kvps: dict|None = None):
+        self.item = Log.log(type, heading, content, kvps)  # create placeholder log item that will be updated
+
+    def update(self, type: Optional[str] = None, heading: str|None = None, content: str|None = None, kvps: dict|None = None):
+        Log.edit(self.item.no, type=type, heading=heading, content=content, kvps=kvps)
+
+    @staticmethod
+    def reset():
+        Log.guid = uuid.uuid4()
+        Log.version = 0
+        Log.last_updated = 0
+        Log.logs = []
+    
+    @staticmethod    
+    def log(type: str, heading: str|None = None, content: str|None = None, kvps: dict|None = None):
+        item = LogItem(len(Log.logs), type, heading or "", content or "", kvps)
+        Log.logs.append(item)
+        Log.version += 1
+        Log.last_updated = item.no
+        return item
+    
+    @staticmethod
+    def edit(no: int, type: Optional[str] = None, heading: str|None = None, content: str|None = None, kvps: dict|None = None):
+        if 0 <= no < len(Log.logs):
+            item = Log.logs[no]
+            if type is not None:
+                item.type = type
+            if heading is not None:
+                item.heading = heading
+            if content is not None:
+                item.content = content
+            if kvps is not None:
+                item.kvps = kvps
+
+            Log.version += 1
+            Log.last_updated = no
+        else:
+            raise IndexError("Log item number out of range")

python/helpers/messages.py

@@ -1,12 +1,12 @@
-from . import files
+# from . import files
 
-
-def truncate_text(output, threshold=1000):
+def truncate_text(agent, output, threshold=1000):
     if len(output) <= threshold:
         return output
 
     # Adjust the file path as needed
-    placeholder = files.read_file("./prompts/fw.msg_truncated.md", removed_chars=(len(output) - threshold))
+    placeholder = agent.read_prompt("fw.msg_truncated.md", removed_chars=(len(output) - threshold))
+    # placeholder = files.read_file("./prompts/default/fw.msg_truncated.md", removed_chars=(len(output) - threshold))
 
     start_len = (threshold - len(placeholder)) // 2
     end_len = threshold - len(placeholder) - start_len

python/helpers/rate_limiter.py

@@ -3,6 +3,7 @@ from collections import deque
 from dataclasses import dataclass
 from typing import List, Tuple
 from .print_style import PrintStyle
+from .log import Log
 
 @dataclass
 class CallRecord:
@@ -48,6 +49,7 @@ class RateLimiter:
             wait_time = oldest_record.timestamp + self.window_seconds - current_time
             if wait_time > 0:
                 PrintStyle(font_color="yellow", padding=True).print(f"Rate limit exceeded. Waiting for {wait_time:.2f} seconds due to: {', '.join(wait_reasons)}")
+                Log.log("rate_limit","Rate limit exceeded",f"Rate limit exceeded. Waiting for {wait_time:.2f} seconds due to: {', '.join(wait_reasons)}")
                 time.sleep(wait_time)
             current_time = time.time()
 
@@ -62,17 +64,3 @@ class RateLimiter:
         if self.call_records:
             self.call_records[-1].output_tokens += output_token_count
         return self
-
-# Example usage
-rate_limiter = RateLimiter(max_calls=5, max_input_tokens=1000, max_output_tokens=2000)
-
-def rate_limited_function(input_token_count: int, output_token_count: int):
-    # First, limit the call and input tokens (this may wait)
-    rate_limiter.limit_call_and_input(input_token_count)
-    
-    # Your function logic here
-    print(f"Function called with {input_token_count} input tokens")
-    
-    # After processing, set the output tokens (this doesn't wait)
-    rate_limiter.set_output_tokens(output_token_count)
-    print(f"Function completed with {output_token_count} output tokens")

python/helpers/shell_ssh.py

@@ -2,12 +2,13 @@ import paramiko
 import time
 import re
 from typing import Optional, Tuple
+from python.helpers.log import Log
+from python.helpers.strings import calculate_valid_match_lengths
 
 class SSHInteractiveSession:
 
-    end_comment = "# @@==>> SSHInteractiveSession End-of-Command  <<==@@"
-
-    ps1_label = "SSHInteractiveSession CLI>"
+    # end_comment = "# @@==>> SSHInteractiveSession End-of-Command  <<==@@"
+    # ps1_label = "SSHInteractiveSession CLI>"
     
     def __init__(self, hostname: str, port: int, username: str, password: str):
         self.hostname = hostname
@@ -18,6 +19,9 @@ class SSHInteractiveSession:
         self.client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
         self.shell = None
         self.full_output = b''
+        self.last_command = b''
+        self.trimmed_command_length = 0  # Initialize trimmed_command_length
+
 
     def connect(self):
         # try 3 times with wait and then except
@@ -27,15 +31,17 @@ class SSHInteractiveSession:
                 self.client.connect(self.hostname, self.port, self.username, self.password)
                 self.shell = self.client.invoke_shell(width=160,height=48)
                 # self.shell.send(f'PS1="{SSHInteractiveSession.ps1_label}"'.encode())
-                return
-                # while True: # wait for end of initial output
-                #     full, part = self.read_output()
-                #     if full and not part: return
-                #     time.sleep(0.1)
+                # return
+                while True: # wait for end of initial output
+                    full, part = self.read_output()
+                    if full and not part: return
+                    time.sleep(0.1)
             except Exception as e:
                 errors += 1
                 if errors < 3:
                     print(f"SSH Connection attempt {errors}...")
+                    Log.log(type="info", content=f"SSH Connection attempt {errors}...")
+                    
                     time.sleep(5)
                 else:
                     raise e
@@ -50,15 +56,32 @@ class SSHInteractiveSession:
         if not self.shell:
             raise Exception("Shell not connected")
         self.full_output = b""
-        self.shell.send((command + " \\\n" +SSHInteractiveSession.end_comment + "\n").encode())
+        # if len(command) > 10: # if command is long, add end_comment to split output
+        #     command = (command + " \\\n" +SSHInteractiveSession.end_comment + "\n")
+        # else:
+        command = command + "\n"
+        self.last_command = command.encode()
+        self.trimmed_command_length = 0
+        self.shell.send(self.last_command)
 
     def read_output(self) -> Tuple[str, str]:
         if not self.shell:
             raise Exception("Shell not connected")
 
         partial_output = b''
+
         while self.shell.recv_ready():
             data = self.shell.recv(1024)
+
+            # Trim own command from output
+            if self.last_command and len(self.last_command) > self.trimmed_command_length:
+                command_to_trim = self.last_command[self.trimmed_command_length:]
+
+                trim_com, trim_out = calculate_valid_match_lengths(command_to_trim, data, deviation_threshold=8, deviation_reset=2, ignore_patterns=[rb'\x1b\[\?\d{4}[a-zA-Z]',rb'\r',rb'>'])
+                if(trim_com > 0 and trim_out > 0):
+                    data = data[trim_out:]
+                    self.trimmed_command_length += trim_com
+            
             partial_output += data
             self.full_output += data
             time.sleep(0.1)  # Prevent busy waiting
@@ -66,15 +89,15 @@ class SSHInteractiveSession:
         # Decode once at the end
         decoded_partial_output = partial_output.decode('utf-8', errors='replace')
         decoded_full_output = self.full_output.decode('utf-8', errors='replace')
-        
+
         decoded_partial_output = self.clean_string(decoded_partial_output)
         decoded_full_output = self.clean_string(decoded_full_output)
 
-        # Split output at end_comment
-        if SSHInteractiveSession.end_comment in decoded_full_output:
-            decoded_full_output = decoded_full_output.split(SSHInteractiveSession.end_comment)[-1].lstrip("\r\n")
-            decoded_partial_output = decoded_partial_output.split(SSHInteractiveSession.end_comment)[-1].lstrip("\r\n")
-        
+        # # Split output at end_comment
+        # if SSHInteractiveSession.end_comment in decoded_full_output:
+        #     decoded_full_output = decoded_full_output.split(SSHInteractiveSession.end_comment)[-1].lstrip("\r\n")
+        #     decoded_partial_output = decoded_partial_output.split(SSHInteractiveSession.end_comment)[-1].lstrip("\r\n")
+
         return decoded_full_output, decoded_partial_output
 
 
@@ -85,5 +108,14 @@ class SSHInteractiveSession:
         
         # Replace '\r\n' with '\n'
         cleaned = cleaned.replace('\r\n', '\n')
-        
-        return cleaned
+
+        # Split the string by newline characters to process each segment separately
+        lines = cleaned.split('\n')
+
+        for i in range(len(lines)):
+            # Handle carriage returns '\r' by splitting and taking the last part
+            parts = [part for part in lines[i].split('\r') if part.strip()]
+            if parts: lines[i] = parts[-1].rstrip()  # Overwrite with the last part after the last '\r'
+
+        return '\n'.join(lines)
+

python/helpers/strings.py

@@ -0,0 +1,125 @@
+# def calculate_valid_match_lengths(first: bytes | str, second: bytes | str, deviation_threshold: int = 5, deviation_reset: int = 5) -> tuple[int, int]:
+#     first_length = len(first)
+#     second_length = len(second)
+
+#     i, j = 0, 0
+#     deviations = 0
+#     matched_since_deviation = 0
+#     last_matched_i, last_matched_j = 0, 0  # Track the last matched index
+
+#     while i < first_length and j < second_length:
+#         if first[i] == second[j]:
+#             last_matched_i, last_matched_j = i + 1, j + 1  # Update last matched position
+#             i += 1
+#             j += 1
+#             matched_since_deviation += 1
+
+#             # Reset the deviation counter if we've matched enough characters since the last deviation
+#             if matched_since_deviation >= deviation_reset:
+#                 deviations = 0
+#                 matched_since_deviation = 0
+#         else:
+#             # Determine the look-ahead based on the remaining deviation threshold
+#             look_ahead = deviation_threshold - deviations
+
+#             # Look ahead to find the best match within the remaining deviation allowance
+#             best_match = None
+#             for k in range(1, look_ahead + 1):
+#                 if i + k < first_length and first[i + k] == second[j]:
+#                     best_match = ('i', k)
+#                     break
+#                 if j + k < second_length and first[i] == second[j + k]:
+#                     best_match = ('j', k)
+#                     break
+
+#             if best_match:
+#                 if best_match[0] == 'i':
+#                     i += best_match[1]
+#                 elif best_match[0] == 'j':
+#                     j += best_match[1]
+#             else:
+#                 i += 1
+#                 j += 1
+
+#             deviations += 1
+#             matched_since_deviation = 0
+
+#             if deviations > deviation_threshold:
+#                 break
+
+#     # Return the last matched positions instead of the current indices
+#     return last_matched_i, last_matched_j
+
+import re
+
+def calculate_valid_match_lengths(first: bytes | str, second: bytes | str, 
+                                  deviation_threshold: int = 5, 
+                                  deviation_reset: int = 5, 
+                                  ignore_patterns: list[bytes|str] = []) -> tuple[int, int]:
+    
+    first_length = len(first)
+    second_length = len(second)
+
+    i, j = 0, 0
+    deviations = 0
+    matched_since_deviation = 0
+    last_matched_i, last_matched_j = 0, 0  # Track the last matched index
+
+    def skip_ignored_patterns(s, index):
+        """Skip characters in `s` that match any pattern in `ignore_patterns` starting from `index`."""
+        while index < len(s):
+            for pattern in ignore_patterns:
+                match = re.match(pattern, s[index:])
+                if match:
+                    index += len(match.group(0))
+                    break
+            else:
+                break
+        return index
+
+    while i < first_length and j < second_length:
+        # Skip ignored patterns
+        i = skip_ignored_patterns(first, i)
+        j = skip_ignored_patterns(second, j)
+
+        if i < first_length and j < second_length and first[i] == second[j]:
+            last_matched_i, last_matched_j = i + 1, j + 1  # Update last matched position
+            i += 1
+            j += 1
+            matched_since_deviation += 1
+
+            # Reset the deviation counter if we've matched enough characters since the last deviation
+            if matched_since_deviation >= deviation_reset:
+                deviations = 0
+                matched_since_deviation = 0
+        else:
+            # Determine the look-ahead based on the remaining deviation threshold
+            look_ahead = deviation_threshold - deviations
+
+            # Look ahead to find the best match within the remaining deviation allowance
+            best_match = None
+            for k in range(1, look_ahead + 1):
+                if i + k < first_length and first[i + k] == second[j]:
+                    best_match = ('i', k)
+                    break
+                if j + k < second_length and first[i] == second[j + k]:
+                    best_match = ('j', k)
+                    break
+
+            if best_match:
+                if best_match[0] == 'i':
+                    i += best_match[1]
+                elif best_match[0] == 'j':
+                    j += best_match[1]
+            else:
+                i += 1
+                j += 1
+
+            deviations += 1
+            matched_since_deviation = 0
+
+            if deviations > deviation_threshold:
+                break
+
+    # Return the last matched positions instead of the current indices
+    return last_matched_i, last_matched_j

python/helpers/tool.py

@@ -3,6 +3,7 @@ from typing import TypedDict
 from agent import Agent
 from python.helpers.print_style import PrintStyle
 from python.helpers import files, messages
+from python.helpers.log import Log
 
 class Response:
     def __init__(self, message: str, break_loop: bool) -> None:
@@ -24,6 +25,7 @@ class Tool:
     def before_execution(self, **kwargs):
         if self.agent.handle_intervention(): return # wait for intervention and handle it, if paused
         PrintStyle(font_color="#1B4F72", padding=True, background_color="white", bold=True).print(f"{self.agent.agent_name}: Using tool '{self.name}':")
+        self.log = Log(type="tool", heading=f"{self.agent.agent_name}: Using tool '{self.name}':", content="", kvps=self.args)
         if self.args and isinstance(self.args, dict):
             for key, value in self.args.items():
                 PrintStyle(font_color="#85C1E9", bold=True).stream(self.nice_key(key)+": ")
@@ -31,12 +33,13 @@ class Tool:
                 PrintStyle().print()
                     
     def after_execution(self, response: Response, **kwargs):
-        text = messages.truncate_text(response.message.strip(), self.agent.config.max_tool_response_length)
-        msg_response = files.read_file("./prompts/fw.tool_response.md", tool_name=self.name, tool_response=text)
+        text = messages.truncate_text(self.agent, response.message.strip(), self.agent.config.max_tool_response_length)
+        msg_response = self.agent.read_prompt("fw.tool_response.md", tool_name=self.name, tool_response=text)
         if self.agent.handle_intervention(): return # wait for intervention and handle it, if paused
         self.agent.append_message(msg_response, human=True)
         PrintStyle(font_color="#1B4F72", background_color="white", padding=True, bold=True).print(f"{self.agent.agent_name}: Response from tool '{self.name}':")
         PrintStyle(font_color="#85C1E9").print(response.message)
+        self.log.update(content=response.message)
 
     def nice_key(self, key:str):
         words = key.split('_')

python/helpers/vdb.py

@@ -1,69 +0,0 @@
-from langchain.storage import InMemoryByteStore, LocalFileStore
-from langchain.embeddings import CacheBackedEmbeddings
-from langchain_core.embeddings import Embeddings
-
-from langchain_chroma import Chroma
-import chromadb
-from chromadb.config import Settings
-
-from . import files
-from langchain_core.documents import Document
-import uuid
-
-
-class VectorDB:
-
-    def __init__(self, embeddings_model:Embeddings, in_memory=False, cache_dir="./cache"):
-        print("Initializing VectorDB...")
-        self.embeddings_model = embeddings_model
-
-        db_cache = files.get_abs_path(cache_dir,"database")
-
-        self.client =chromadb.PersistentClient(path=db_cache)
-        self.collection = self.client.create_collection("my_collection")
-        self.collection
-
-        
-    def search(self, query:str, results=2):
-        emb = self.embeddings_model.embed_query(query)
-        res = self.collection.query(query_embeddings=[emb],n_results=results)
-        best = res["documents"][0][0] # type: ignore
-        
-    # def delete_documents(self, query):
-    #     score_limit = 1
-    #     k = 2
-    #     tot = 0
-    #     while True:
-    #         # Perform similarity search with score
-    #         docs = self.db.similarity_search_with_score(query, k=k)
-
-    #         # Extract document IDs and filter based on score
-    #         document_ids = [result[0].metadata["id"] for result in docs if result[1] < score_limit]
-
-    #         # Delete documents with IDs over the threshold score
-    #         if document_ids:
-    #             fnd = self.db.get(where={"id": {"$in": document_ids}})
-    #             if fnd["ids"]: self.db.delete(ids=fnd["ids"])
-    #             tot += len(fnd["ids"])
-            
-    #         # If fewer than K document IDs, break the loop
-    #         if len(document_ids) < k:
-    #             break
-        
-    #     return tot
-
-    def insert(self, data:str):
-        
-        id = str(uuid.uuid4())
-        emb = self.embeddings_model.embed_documents([data])[0]
-
-        self.collection.add(
-            ids=[id],
-            embeddings=[emb],
-            documents=[data],
-            )
-
-        return id
-        
-
-

python/helpers/vector_db.py

@@ -1,25 +1,33 @@
 from langchain.storage import InMemoryByteStore, LocalFileStore
 from langchain.embeddings import CacheBackedEmbeddings
-from langchain_chroma import Chroma
+# from langchain_chroma import Chroma
+from langchain_community.vectorstores import FAISS
+import faiss
+from langchain_community.docstore.in_memory import InMemoryDocstore
 
+import os, json
 from . import files
 from langchain_core.documents import Document
 import uuid
-
+from python.helpers import knowledge_import
+from python.helpers.log import Log
 
 class VectorDB:
 
-    def __init__(self, embeddings_model, in_memory=False, cache_dir="./cache"):
+    def __init__(self, embeddings_model, in_memory=False, memory_dir="./memory", knowledge_dir="./knowledge"):
         print("Initializing VectorDB...")
+        Log.log("info", content="Initializing VectorDB...")
+        
         self.embeddings_model = embeddings_model
 
-        em_cache = files.get_abs_path(cache_dir,"embeddings")
-        db_cache = files.get_abs_path(cache_dir,"database")
+        self.em_dir = files.get_abs_path(memory_dir,"embeddings")
+        self.db_dir = files.get_abs_path(memory_dir,"database")
+        self.kn_dir = files.get_abs_path(knowledge_dir) if knowledge_dir else ""
         
         if in_memory:
             self.store = InMemoryByteStore()
         else:
-            self.store = LocalFileStore(em_cache)
+            self.store = LocalFileStore(self.em_dir)
 
 
         #here we setup the embeddings model with the chosen cache storage
@@ -28,9 +36,63 @@ class VectorDB:
             self.store, 
             namespace=getattr(embeddings_model, 'model', getattr(embeddings_model, 'model_name', "default")) )
 
+        # self.db = Chroma(
+        #     embedding_function=self.embedder,
+        #     persist_directory=db_dir)
 
-        self.db = Chroma(embedding_function=self.embedder,persist_directory=db_cache)
         
+        # if db folder exists and is not empty:
+        if os.path.exists(self.db_dir) and len(os.listdir(self.db_dir)) > 0:
+            self.db = FAISS.load_local(
+                folder_path=self.db_dir,
+                embeddings=self.embedder,
+                allow_dangerous_deserialization=True
+            )
+        else:
+            index = faiss.IndexFlatL2(len(self.embedder.embed_query("example text")))
+
+            self.db = FAISS(
+                embedding_function=self.embedder,
+                index=index,
+                docstore=InMemoryDocstore(),
+                index_to_docstore_id={})
+
+        #preload knowledge files
+        if self.kn_dir:
+            self.preload_knowledge(self.kn_dir, self.db_dir)
+        
+
+    def preload_knowledge(self, kn_dir:str, db_dir:str):
+
+        # Load the index file if it exists
+        index_path = files.get_abs_path(db_dir, "knowledge_import.json")
+
+        #make sure directory exists
+        if not os.path.exists(db_dir):
+            os.makedirs(db_dir)
+        
+        index: dict[str, knowledge_import.KnowledgeImport] = {}
+        if os.path.exists(index_path):
+            with open(index_path, 'r') as f:
+                index = json.load(f)
+       
+        index = knowledge_import.load_knowledge(kn_dir,index)
+        
+        for file in index:
+            if index[file]['state'] in ['changed', 'removed'] and index[file].get('ids',[]): # for knowledge files that have been changed or removed and have IDs
+                self.delete_documents_by_ids(index[file]['ids']) # remove original version
+            if index[file]['state'] == 'changed':
+                index[file]['ids'] = self.insert_documents(index[file]['documents']) # insert new version
+
+        # remove index where state="removed"
+        index = {k: v for k, v in index.items() if v['state'] != 'removed'}
+        
+        # strip state and documents from index and save it
+        for file in index:
+            if "documents" in index[file]: del index[file]['documents'] # type: ignore
+            if "state" in index[file]: del index[file]['state'] # type: ignore
+        with open(index_path, 'w') as f:
+            json.dump(index, f)    
         
     def search_similarity(self, query, results=3):
         return self.db.similarity_search(query,results)
@@ -64,7 +126,8 @@ class VectorDB:
             # If fewer than K document IDs, break the loop
             if len(document_ids) < k:
                 break
-        
+
+        if tot: self.db.save_local(folder_path=self.db_dir) # persist
         return tot
 
     def delete_documents_by_ids(self, ids:list[str]):
@@ -72,13 +135,20 @@ class VectorDB:
         self.db.delete(ids=ids)
         # post = self.db.get(ids=ids)["ids"]
         #TODO? compare pre and post
+        if ids: self.db.save_local(folder_path=self.db_dir) #persist
         return len(ids)
         
-    def insert_document(self, data):
+    def insert_text(self, text):
         id = str(uuid.uuid4())
-        self.db.add_documents(documents=[ Document(data, metadata={"id": id}) ], ids=[id])
-        
+        self.db.add_documents(documents=[ Document(text, metadata={"id": id}) ], ids=[id])
+        self.db.save_local(folder_path=self.db_dir) #persist
         return id
-        
+    
+    def insert_documents(self, docs:list[Document]):
+        ids = [str(uuid.uuid4()) for _ in range(len(docs))]
+        for doc, id in zip(docs, ids): doc.metadata["id"] = id  #add ids to documents metadata
+        self.db.add_documents(documents=docs, ids=ids)
+        self.db.save_local(folder_path=self.db_dir) #persist
+        return ids
 
 

python/tools/code_execution_tool.py

@@ -11,6 +11,7 @@ from python.helpers.print_style import PrintStyle
 from python.helpers.shell_local import LocalInteractiveSession
 from python.helpers.shell_ssh import SSHInteractiveSession
 from python.helpers.docker import DockerContainerManager
+from python.helpers.log import Log
 
 @dataclass
 class State:
@@ -38,13 +39,23 @@ class CodeExecution(Tool):
         elif runtime == "output":
             response = self.get_terminal_output()
         else:
-            response = files.read_file("./prompts/fw.code_runtime_wrong.md", runtime=runtime)
+            response = self.agent.read_prompt("fw.code_runtime_wrong.md", runtime=runtime)
 
-        if not response: response = files.read_file("./prompts/fw.code_no_output.md")
+        if not response: response = self.agent.read_prompt("fw.code_no_output.md")
         return Response(message=response, break_loop=False)
 
+    def before_execution(self, **kwargs):
+        if self.agent.handle_intervention(): return # wait for intervention and handle it, if paused
+        PrintStyle(font_color="#1B4F72", padding=True, background_color="white", bold=True).print(f"{self.agent.agent_name}: Using tool '{self.name}':")
+        self.log = Log(type="code_exe", heading=f"{self.agent.agent_name}: Using tool '{self.name}':", content="", kvps=self.args)
+        if self.args and isinstance(self.args, dict):
+            for key, value in self.args.items():
+                PrintStyle(font_color="#85C1E9", bold=True).stream(self.nice_key(key)+": ")
+                PrintStyle(font_color="#85C1E9", padding=isinstance(value,str) and "\n" in value).stream(value)
+                PrintStyle().print()
+
     def after_execution(self, response, **kwargs):
-        msg_response = files.read_file("./prompts/fw.tool_response.md", tool_name=self.name, tool_response=response.message)
+        msg_response = self.agent.read_prompt("fw.tool_response.md", tool_name=self.name, tool_response=response.message)
         self.agent.append_message(msg_response, human=True)
 
     def prepare_state(self):
@@ -98,6 +109,7 @@ class CodeExecution(Tool):
         
             if partial_output:
                 PrintStyle(font_color="#85C1E9").stream(partial_output)
+                self.log.update(content=full_output)
                 idle=0    
             else:
                 idle+=1

python/tools/knowledge_tool.py

@@ -1,6 +1,5 @@
 import os
 from agent import Agent
-from . import online_knowledge_tool
 from python.helpers import perplexity_search
 from python.helpers import duckduckgo_search
 
@@ -10,6 +9,7 @@ import concurrent.futures
 from python.helpers.tool import Tool, Response
 from python.helpers import files
 from python.helpers.print_style import PrintStyle
+from python.helpers.log import Log
 
 class Knowledge(Tool):
     def execute(self, question="", **kwargs):
@@ -21,6 +21,7 @@ class Knowledge(Tool):
                 perplexity = executor.submit(perplexity_search.perplexity_search, question)
             else: 
                 PrintStyle.hint("No API key provided for Perplexity. Skipping Perplexity search.")
+                Log(type="hint", content="No API key provided for Perplexity. Skipping Perplexity search.")
                 perplexity = None
                 
 
@@ -46,7 +47,7 @@ class Knowledge(Tool):
             except Exception as e:
                 memory_result = "Memory search failed: " + str(e)
 
-        msg = files.read_file("prompts/tool.knowledge.response.md", 
+        msg = self.agent.read_prompt("tool.knowledge.response.md", 
                               online_sources = ((perplexity_result + "\n\n") if perplexity else "") + str(duckduckgo_result),
                               memory = memory_result )
 

python/tools/memory_tool.py

@@ -6,9 +6,10 @@ import os, json
 from python.helpers.tool import Tool, Response
 from python.helpers.print_style import PrintStyle
 from chromadb.errors import InvalidDimensionException
+from python.helpers.log import Log
 
-# TODO multiple DBs at once
-db: VectorDB | None= None
+# databases based on subdirectories from agent config
+dbs = {}
 
 class Memory(Tool):
     def execute(self,**kwargs):
@@ -28,39 +29,48 @@ class Memory(Tool):
         except InvalidDimensionException as e:
             # hint about embedding change with existing database
             PrintStyle.hint("If you changed your embedding model, you will need to remove contents of /memory directory.")
+            Log(type="hint", content="If you changed your embedding model, you will need to remove contents of /memory directory.")
             raise   
         
         # result = process_query(self.agent, self.args["memory"],self.args["action"], result_count=self.agent.config.auto_memory_count)
         return Response(message=result, break_loop=False)
             
 def search(agent:Agent, query:str, count:int=5, threshold:float=0.1):
-    initialize(agent)
+    db = get_db(agent)
+    # docs = db.search_similarity(query,count) # type: ignore
     docs = db.search_similarity_threshold(query,count,threshold) # type: ignore
-    if len(docs)==0: return files.read_file("./prompts/fw.memories_not_found.md", query=query)
+    if len(docs)==0: return agent.read_prompt("fw.memories_not_found.md", query=query)
     else: return str(docs)
 
 def save(agent:Agent, text:str):
-    initialize(agent)
-    id = db.insert_document(text) # type: ignore
-    return files.read_file("./prompts/fw.memory_saved.md", memory_id=id)
+    db = get_db(agent)
+    id = db.insert_text(text) # type: ignore
+    return agent.read_prompt("fw.memory_saved.md", memory_id=id)
 
 def delete(agent:Agent, ids_str:str):
-    initialize(agent)
+    db = get_db(agent)
     ids = extract_guids(ids_str)
     deleted = db.delete_documents_by_ids(ids) # type: ignore
-    return files.read_file("./prompts/fw.memories_deleted.md", memory_count=deleted)    
+    return agent.read_prompt("fw.memories_deleted.md", memory_count=deleted)    
 
 def forget(agent:Agent, query:str):
-    initialize(agent)
+    db = get_db(agent)
     deleted = db.delete_documents_by_query(query) # type: ignore
-    return files.read_file("./prompts/fw.memories_deleted.md", memory_count=deleted)
+    return agent.read_prompt("fw.memories_deleted.md", memory_count=deleted)
 
-def initialize(agent:Agent):
-    global db
-    if not db:
-        dir = os.path.join("memory",agent.config.memory_subdir)
-        db = VectorDB(embeddings_model=agent.config.embeddings_model, in_memory=False, cache_dir=dir)
+def get_db(agent: Agent):
+    mem_dir = os.path.join("memory", agent.config.memory_subdir)
+    kn_dir = os.path.join("knowledge", agent.config.knowledge_subdir)
+    key = (mem_dir, kn_dir)
 
+    if key not in dbs:
+        db = VectorDB(embeddings_model=agent.config.embeddings_model, in_memory=False, memory_dir=mem_dir, knowledge_dir=kn_dir)
+        dbs[key] = db
+    else:
+        db = dbs[key]
+
+    return db
+        
 def extract_guids(text):
     pattern = r'\b[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-5][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}\b'
     return re.findall(pattern, text)

python/tools/online_knowledge_tool.py

@@ -1,13 +0,0 @@
-from agent import Agent
-from python.helpers import perplexity_search
-from python.helpers.tool import Tool, Response
-
-class OnlineKnowledge(Tool):
-    def execute(self,**kwargs):
-        return Response(
-            message=process_question(self.args["question"]),
-            break_loop=False,
-        )
-
-def process_question(question):
-    return str(perplexity_search.perplexity_search(question))

python/tools/response.py

@@ -6,6 +6,7 @@ from agent import Agent
 from python.helpers.tool import Tool, Response
 from python.helpers import files
 from python.helpers.print_style import PrintStyle
+from python.helpers.log import Log
 
 class ResponseTool(Tool):
 
@@ -14,7 +15,8 @@ class ResponseTool(Tool):
         return Response(message=self.args["text"], break_loop=True)
 
     def before_execution(self, **kwargs):
-        pass # do not add anything to the history or output
+        self.log = Log(type="response", heading=f"{self.agent.agent_name}: Responding:", content=self.args.get("text", ""))
+
     
     def after_execution(self, response, **kwargs):
         pass # do not add anything to the history or output

python/tools/task_done.py

@@ -6,6 +6,7 @@ from agent import Agent
 from python.helpers.tool import Tool, Response
 from python.helpers import files
 from python.helpers.print_style import PrintStyle
+from python.helpers.log import Log
 
 class TaskDone(Tool):
 
@@ -14,7 +15,7 @@ class TaskDone(Tool):
         return Response(message=self.args["text"], break_loop=True)
 
     def before_execution(self, **kwargs):
-        pass # do not add anything to the history or output
+        self.log = Log(type="response", heading=f"{self.agent.agent_name}: Task done:", content=self.args.get("text", ""))
     
     def after_execution(self, response, **kwargs):
         pass # do add anything to the history or output

python/tools/unknown.py

@@ -4,8 +4,8 @@ from python.helpers import files
 class Unknown(Tool):
     def execute(self, **kwargs):
         return Response(
-                message=files.read_file("prompts/fw.tool_not_found.md",
+                message=self.agent.read_prompt("fw.tool_not_found.md",
                                         tool_name=self.name,
-                                        tools_prompt=files.read_file("prompts/agent.tools.md")), 
+                                        tools_prompt=self.agent.read_prompt("agent.tools.md")), 
                 break_loop=False)
 

requirements.txt

@@ -5,7 +5,6 @@ langchain-huggingface==0.0.3
 langchain-openai==0.1.15
 langchain-community==0.2.7
 langchain-anthropic==0.1.19
-langchain-chroma==0.1.2
 langchain-google-genai==1.0.7
 webcolors==24.6.0
 sentence-transformers==3.0.1
@@ -16,4 +15,8 @@ inputimeout==1.0.4
 newspaper3k==0.2.8
 beautifulsoup4==4.12.3
 lxml_html_clean==0.2.0
-pynput==1.7.7
+pynput==1.7.7
+pypdf==4.3.1
+Flask[async]==3.0.3
+Flask-BasicAuth==0.2.0
+faiss-cpu==1.8.0.post1

main.py → run_cli.py

@@ -1,5 +1,4 @@
-import threading, time, models, tts, stt, os, sys, argparse
-from dotenv import load_dotenv
+import threading, time, models, os
 from ansio import application_keypad, mouse_input, raw_input
 from ansio.input import InputEvent, get_input_event
 from agent import Agent, AgentConfig
@@ -7,105 +6,42 @@ from python.helpers.print_style import PrintStyle
 from python.helpers.files import read_file
 from python.helpers import files
 import python.helpers.timed_input as timed_input
-from pynput import keyboard
+from initialize import initialize
 
-load_dotenv()  # take environment variables from.env.
 
 input_lock = threading.Lock()
 os.chdir(files.get_abs_path("./work_dir")) #change CWD to work_dir
 
-def initialize():
-    
-    # main chat model used by agents (smarter, more accurate)
-    chat_llm = models.get_openai_chat(model_name="gpt-4o-mini", temperature=0)
-    # chat_llm = models.get_ollama_chat(model_name="gemma2:latest", temperature=0)
-    # chat_llm = models.get_lmstudio_chat(model_name="TheBloke/Mistral-7B-Instruct-v0.2-GGUF", temperature=0)
-    # chat_llm = models.get_openrouter(model_name="meta-llama/llama-3-8b-instruct:free")
-    # chat_llm = models.get_azure_openai_chat(deployment_name="gpt-4o-mini", temperature=0)
-    # chat_llm = models.get_anthropic_chat(model_name="claude-3-5-sonnet-20240620", temperature=0)
-    # chat_llm = models.get_google_chat(model_name="gemini-1.5-flash", temperature=0)
-    # chat_llm = models.get_groq_chat(model_name="llama-3.1-70b-versatile", temperature=0)
-    
-    # utility model used for helper functions (cheaper, faster)
-    utility_llm = chat_llm # change if you want to use a different utility model
-
-    # embedding model used for memory
-    embedding_llm = models.get_openai_embedding(model_name="text-embedding-3-small")
-    # embedding_llm = models.get_ollama_embedding(model_name="nomic-embed-text")
-    # embedding_llm = models.get_huggingface_embedding(model_name="sentence-transformers/all-MiniLM-L6-v2")
-
-    # agent configuration
-    config = AgentConfig(
-        chat_model = chat_llm,
-        utility_model = utility_llm,
-        embeddings_model = embedding_llm,
-        # memory_subdir = "",
-        auto_memory_count = 0,
-        # auto_memory_skip = 2,
-        # rate_limit_seconds = 60,
-        # rate_limit_requests = 30,
-        # rate_limit_input_tokens = 0,
-        # rate_limit_output_tokens = 0,
-        # msgs_keep_max = 25,
-        # msgs_keep_start = 5,
-        # msgs_keep_end = 10,
-        # max_tool_response_length = 3000,
-        # response_timeout_seconds = 60,
-        code_exec_docker_enabled = True,
-        # code_exec_docker_name = "agent-zero-exe",
-        # code_exec_docker_image = "frdel/agent-zero-exe:latest",
-        # code_exec_docker_ports = { "22/tcp": 50022 }
-        # code_exec_docker_volumes = { files.get_abs_path("work_dir"): {"bind": "/root", "mode": "rw"} }
-        code_exec_ssh_enabled = True,
-        # code_exec_ssh_addr = "localhost",
-        # code_exec_ssh_port = 50022,
-        # code_exec_ssh_user = "root",
-        # code_exec_ssh_pass = "toor",
-        # additional = {},
-    )
-    
-    # create the first agent
-    agent0 = Agent( number = 0, config = config )
-
-    # start the chat loop
-    chat(agent0)
-
-
 # Main conversation loop
 def chat(agent:Agent):
-
+    
     # start the conversation loop  
     while True:
         # ask user for message
         with input_lock:
             timeout = agent.get_data("timeout") # how long the agent is willing to wait
             if not timeout: # if agent wants to wait for user input forever
-                PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ('alt_gr' for mic, 'e' to leave):")        
+                PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ('e' to leave):")        
                 import readline # this fixes arrow keys in terminal
-                listener = keyboard.Listener(on_press=on_press)
-                listener.start()
                 user_input = input("> ")
-                listener.stop()
                 PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}") 
                 
             else: # otherwise wait for user input with a timeout
-                PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ({timeout}s timeout, 'w' to wait, 'alt_gr' for mic, 'e' to leave):")        
+                PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message ({timeout}s timeout, 'w' to wait, 'e' to leave):")        
                 import readline # this fixes arrow keys in terminal
-                listener = keyboard.Listener(on_press=on_press)
-                listener.start()
+                # user_input = timed_input("> ", timeout=timeout)
                 user_input = timeout_input("> ", timeout=timeout)
-                listener.stop()                    
+                                    
                 if not user_input:
-                    user_input = read_file("prompts/fw.msg_timeout.md")
+                    user_input = agent.read_prompt("fw.msg_timeout.md")
                     PrintStyle(font_color="white", padding=False).stream(f"{user_input}")        
                 else:
                     user_input = user_input.strip()
                     if user_input.lower()=="w": # the user needs more time
-                        listener = keyboard.Listener(on_press=on_press)
-                        listener.start()
-                        user_input = input("> ")
-                        listener.stop()
-                    PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")              
+                        user_input = input("> ").strip()
+                    PrintStyle(font_color="white", padding=False, log_only=True).print(f"> {user_input}")        
+                    
+                    
 
         # exit the conversation when the user types 'exit'
         if user_input.lower() == 'e': break
@@ -115,8 +51,8 @@ def chat(agent:Agent):
         
         # print agent0 response
         PrintStyle(font_color="white",background_color="#1D8348", bold=True, padding=True).print(f"{agent.agent_name}: reponse:")        
-        PrintStyle(font_color="white").print(f"{assistant_response}")
-        if (os.getenv('EDGE_TTS_MODEL')) : tts.speech(assistant_response)
+        PrintStyle(font_color="white").print(f"{assistant_response}")        
+                        
 
 # User intervention during agent streaming
 def intervention():
@@ -132,10 +68,6 @@ def intervention():
         if user_input: Agent.streaming_agent.intervention_message = user_input # set intervention message if non-empty
         Agent.paused = False # continue agent streaming 
     
-def on_press(key):
-    if key == keyboard.Key.alt_gr:
-        # Lancer le speech to text
-        stt.record()
 
 # Capture keyboard input to trigger user intervention
 def capture_keys():
@@ -164,5 +96,6 @@ if __name__ == "__main__":
     # Start the key capture thread for user intervention during agent streaming
     threading.Thread(target=capture_keys, daemon=True).start()
 
-    # Start the chat
-    initialize()
+    # initialize and start the chat
+    agent0 = initialize()
+    chat(agent0)

run_ui.py

@@ -0,0 +1,192 @@
+from functools import wraps
+import os
+from pathlib import Path
+import threading
+from flask import Flask, request, jsonify, Response
+from flask_basicauth import BasicAuth
+from agent import Agent
+from initialize import initialize
+from python.helpers.files import get_abs_path
+from python.helpers.print_style import PrintStyle
+from python.helpers.log import Log
+from dotenv import load_dotenv
+
+#global agent instance
+agent0: Agent|None = None
+
+#initialize the internal Flask server
+app = Flask("app",static_folder=get_abs_path("./webui"),static_url_path="/")
+
+# Set up basic authentication, name and password from .env variables
+app.config['BASIC_AUTH_USERNAME'] = os.environ.get('BASIC_AUTH_USERNAME') or "admin" #default name
+app.config['BASIC_AUTH_PASSWORD'] = os.environ.get('BASIC_AUTH_PASSWORD') or "admin" #default pass
+basic_auth = BasicAuth(app)
+
+# get global agent
+def get_agent(reset: bool = False) -> Agent:
+    global agent0
+    if agent0 is None or reset:
+        agent0 = initialize()
+    return agent0
+
+# Now you can use @requires_auth function decorator to require login on certain pages
+def requires_auth(f):
+    @wraps(f)
+    async def decorated(*args, **kwargs):
+        auth = request.authorization
+        if not auth or not (auth.username == app.config['BASIC_AUTH_USERNAME'] and auth.password == app.config['BASIC_AUTH_PASSWORD']):
+            return Response(
+                'Could not verify your access level for that URL.\n'
+                'You have to login with proper credentials', 401,
+                {'WWW-Authenticate': 'Basic realm="Login Required"'})
+        return await f(*args, **kwargs)
+    return decorated
+
+# handle default address, show demo html page from ./test_form.html
+@app.route('/', methods=['GET'])
+async def test_form():
+    return Path(get_abs_path("./webui/index.html")).read_text()
+
+# simple health check, just return OK to see the server is running
+@app.route('/ok', methods=['GET','POST'])
+async def health_check():
+    return "OK"
+
+# # secret page, requires authentication
+# @app.route('/secret', methods=['GET'])
+# @requires_auth
+# async def secret_page():
+#     return Path("./secret_page.html").read_text()
+
+# send message to agent
+@app.route('/msg', methods=['POST'])
+async def handle_message():
+    try:
+        
+        #agent instance
+        agent = get_agent()
+
+        #data sent to the server
+        input = request.get_json()
+        text = input.get("text", "")
+
+        # print to console and log
+        PrintStyle(background_color="#6C3483", font_color="white", bold=True, padding=True).print(f"User message:")        
+        PrintStyle(font_color="white", padding=False).print(f"> {text}")
+        Log.log(type="user", heading="User message", content=text)
+        
+        #pass the message to the agent
+        threading.Thread(target=agent.communicate, args=(text,)).start()
+        
+        #data from this server    
+        response = {
+            "ok": True,
+            "message": "Message received.",
+        }
+
+    except Exception as e:
+        response = {
+            "ok": False,
+            "message": str(e),
+        }
+
+    #respond with json
+    return jsonify(response)
+
+# pausing/unpausing the agent
+@app.route('/pause', methods=['POST'])
+async def pause():
+    try:
+        
+        #data sent to the server
+        input = request.get_json()
+        paused = input.get("paused", False)
+
+        Agent.paused = paused
+
+        response = {
+            "ok": True,
+            "message": "Agent paused." if paused else "Agent unpaused.",
+            "pause": paused
+        }        
+        
+    except Exception as e:
+        response = {
+            "ok": False,
+            "message": str(e),
+        }
+
+    #respond with json
+    return jsonify(response)
+
+# restarting with new agent0
+@app.route('/reset', methods=['POST'])
+async def reset():
+    try:
+        
+        agent = get_agent(reset=True)
+        Log.reset()
+
+        response = {
+            "ok": True,
+            "message": "Agent restarted.",
+        }        
+        
+    except Exception as e:
+        response = {
+            "ok": False,
+            "message": str(e),
+        }
+
+    #respond with json
+    return jsonify(response)
+
+# Web UI polling
+@app.route('/poll', methods=['POST'])
+async def poll():
+    try:
+        
+        #data sent to the server
+        input = request.get_json()
+        from_no = input.get("log_from", "")
+
+        logs = Log.logs[int(from_no):]
+        to = Log.last_updated #max(0, len(Log.logs)-1)
+        
+        #data from this server    
+        response = {
+            "ok": True,
+            "logs": logs,
+            "log_to": to,
+            "log_guid": Log.guid,
+            "log_version": Log.version,
+            "paused": Agent.paused
+        }
+
+    except Exception as e:
+        response = {
+            "ok": False,
+            "message": str(e),
+        }
+
+    #respond with json
+    return jsonify(response)
+
+
+
+#run the internal server
+if __name__ == "__main__":
+
+    load_dotenv()
+    
+    get_agent() #initialize
+
+    # Suppress only request logs but keep the startup messages
+    from werkzeug.serving import WSGIRequestHandler
+    class NoRequestLoggingWSGIRequestHandler(WSGIRequestHandler):
+        def log_request(self, code='-', size='-'):
+            pass  # Override to suppress request logging
+
+    # run the server on port from .env
+    port = int(os.environ.get("WEB_UI_PORT", 0)) or None
+    app.run(request_handler=NoRequestLoggingWSGIRequestHandler,port=port)