Worker_Node_Template/README.md

Worker Node Setup (For a Single PC)

This folder contains the setup files required to turn any standard Windows or Linux PC into a Worker Node for the local AI cluster.

🧠 How it Works

In a traditional setup, an AI model runs entirely on one computer. However, massive models (like 70B or 120B parameter models) require far more GPU memory (VRAM) than a single consumer PC possesses.

This software uses an RPC (Remote Procedure Call) Cluster Architecture to solve this:

1. The Master Node: The main PC running the IDE. It holds the actual model file (.gguf), processes your chat prompts, and orchestrates the cluster.
2. The Worker Nodes (This PC): These are "dumb appliances." They do not need Python, they do not need the IDE, and they do not even need the .gguf model file.

What happens under the hood: When you start the rpc-server on this PC, it simply opens a network port and waits. When the Master Node loads a massive model, it mathematically splits the model layers into pieces. It looks at the VRAM available on the Worker, sends a portion of the model's layers over the local network to this PC's GPU, and says, "Hold this in your memory."

During generation, when the AI is "thinking," the Master sends mathematical matrix multiplication tasks to the Worker. The Worker's GPU calculates the math on its specific layers and sends the raw numbers back to the Master.

This means your private data, chat logs, and source code are NEVER stored on the Worker PC. The Worker only sees raw, unreadable mathematical tensors.

---

🛠️ Requirements for this PC

Because this PC is just a mathematical calculator for the Master, the setup is incredibly lightweight:

1. llama.cpp (RPC Server):

   • Windows: Download the pre-compiled llama-bin-win-cuda-...-x64.zip release from the official llama.cpp GitHub. Extract it anywhere.
   • Linux: Clone the llama.cpp repository and compile it with GGML_RPC=ON enabled via CMake.
   • Note: The llama.cpp version on the Worker MUST exactly match the version on the Master Node.

2. Network Access: Port 50055 (or whichever port you choose to assign this PC) must be open in the local firewall (e.g., Windows Defender).

---

🚀 How to Run

You only need to run one script to bring this PC online.

For Windows:

1. Open start_worker.ps1 in a text editor (like Notepad).
2. Edit the $LlamaPath to point to the folder where you extracted rpc-server.exe.
3. Right-click start_worker.ps1 and select Run with PowerShell.

For Linux:

1. Open start_worker.sh in a text editor.
2. Edit the LLAMA_DIR to point to the build/bin folder where you compiled llama-server.
3. Open a terminal and run:

   chmod +x start_worker.sh
   ./start_worker.sh
   

Once the script is running, the terminal will say "listening on 0.0.0.0:50055".

That's it! You can now walk away. When the Master Node starts up, it will automatically detect this PC over the network, load the layers onto its GPU, and begin using its compute power.