ohamlab commited on
Commit
bf70ca8
·
verified ·
1 Parent(s): c05b317

Migrated from GitHub

Browse files
Files changed (11) hide show
  1. Dockerfile +73 -0
  2. ORIGINAL_README.md +10 -0
  3. acchite.md +226 -0
  4. app.py +238 -0
  5. app_gradio.py +212 -0
  6. desgin.md +773 -0
  7. entrypoint.sh +16 -0
  8. pygmyclaw.py +241 -0
  9. pygmyclaw_multitool.py +210 -0
  10. requirements.txt +8 -0
  11. ui.py +38 -0
Dockerfile ADDED
@@ -0,0 +1,73 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # ----------------------------
2
+ # PygmyClaw Dockerfile
3
+ # ----------------------------
4
+ FROM ubuntu:22.04
5
+
6
+ ENV DEBIAN_FRONTEND=noninteractive
7
+ # Dockerfile snippet
8
+ ENV MODEL_NAME="hf.co/rahul7star/Qwen3-4B-Thinking-2509-Genius-Coder-AI-Full:Q5_K_M"
9
+ ENV OLLAMA_HOST="0.0.0.0:11434"
10
+ ENV PYTHONUNBUFFERED=1
11
+
12
+ # ----------------------------
13
+ # Install system dependencies
14
+ # ----------------------------
15
+ RUN apt-get update && apt-get install -y \
16
+ build-essential \
17
+ libcurl4-openssl-dev \
18
+ libcjson-dev \
19
+ curl \
20
+ python3 \
21
+ python3-pip \
22
+ git \
23
+ zstd \
24
+ sudo \
25
+ && rm -rf /var/lib/apt/lists/*
26
+
27
+ # ----------------------------
28
+ # Install Ollama
29
+ # ----------------------------
30
+ RUN curl -fsSL https://ollama.com/install.sh | sh
31
+
32
+ # ----------------------------
33
+ # Python dependencies
34
+ # ----------------------------
35
+ RUN pip3 install --upgrade pip \
36
+ && pip3 install \
37
+ streamlit \
38
+ gradio==4.44.0 \
39
+ huggingface_hub==0.23.5 \
40
+ requests \
41
+ redis \
42
+ huggingface_hub \
43
+ torch \
44
+ torchvision \
45
+ torchaudio
46
+
47
+ # ----------------------------
48
+ # Set working directory
49
+ # ----------------------------
50
+ WORKDIR /workspace
51
+
52
+ # ----------------------------
53
+ # Copy the PygmyClaw repo
54
+ # ----------------------------
55
+ COPY . /workspace/
56
+
57
+ # ----------------------------
58
+ # Ensure scripts are executable
59
+ # ----------------------------
60
+ RUN chmod +x /workspace/entrypoint.sh \
61
+ && chmod +x /workspace/pygmyclaw.py \
62
+ && chmod +x /workspace/pygmyclaw_multitool.py \
63
+ && mkdir -p /workspace/data
64
+
65
+ # ----------------------------
66
+ # Expose UI port for Gradio / Streamlit
67
+ # ----------------------------
68
+ EXPOSE 7860
69
+
70
+ # ----------------------------
71
+ # Entrypoint
72
+ # ----------------------------
73
+ CMD ["/workspace/entrypoint.sh"]
ORIGINAL_README.md ADDED
@@ -0,0 +1,10 @@
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ title: Pyclaw
3
+ emoji: 🐠
4
+ colorFrom: pink
5
+ colorTo: blue
6
+ sdk: docker
7
+ pinned: false
8
+ ---
9
+
10
+ Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
acchite.md ADDED
@@ -0,0 +1,226 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+
4
+ # 🦾 PygmyClaw Autonomous Agent — End-to-End Design
5
+
6
+ ## 1. **Overview**
7
+
8
+ PygmyClaw is a compact AI agent framework designed for **dynamic Python tool execution, speculative decoding, and autonomous task handling**.
9
+ With Hugging Face integration, it can now **store persistent memory, code, and artifacts**, evolving toward a fully autonomous agent like Claude.
10
+
11
+ **Key goals:**
12
+
13
+ * Execute Python code dynamically with dependency management.
14
+ * Generate, edit, and run code via UI.
15
+ * Maintain long-term memory and datasets in Hugging Face.
16
+ * Handle autonomous multi-step workflows with multi-instance speculative decoding.
17
+
18
+ ---
19
+
20
+ ## 2. **System Components**
21
+
22
+ ### 2.1 User Interface
23
+
24
+ * Streamlit-based web UI.
25
+ * **Features:**
26
+
27
+ * Prompt input
28
+ * `/HELP` commands
29
+ * Code generation `/WRITE_PY`
30
+ * Code editing + execution (`CPU` or optional `GPU`)
31
+ * Book, story, or poem management
32
+ * Session logs and downloads
33
+
34
+ ---
35
+
36
+ ### 2.2 Agent Core — `pygmyclaw.py`
37
+
38
+ * Handles all **user-agent interaction**:
39
+
40
+ * Receives user prompts.
41
+ * Converts LLM output into **JSON tool calls**.
42
+ * Dynamically loads and executes Python tools.
43
+ * **Speculative decoding**:
44
+
45
+ * 3 drafters + 1 verifier for robust output.
46
+ * **Queue system**:
47
+
48
+ * Redis or JSON-file queue for task scheduling.
49
+ * **Artifact management**:
50
+
51
+ * Stores code, logs, and task outputs in workspace.
52
+ * Supports automatic dependency installation.
53
+
54
+ ---
55
+
56
+ ### 2.3 Python Multitool — `pygmyclaw_multitool.py`
57
+
58
+ * **Tool registry**:
59
+
60
+ * `list_tools_detailed`, `sys_info`, `log_error`, `echo`, etc.
61
+ * **Dynamic tool addition**:
62
+
63
+ * Agents can create new tools that are callable via JSON.
64
+ * **Safe execution sandbox**:
65
+
66
+ * Python subprocess with controlled input/output.
67
+
68
+ ---
69
+
70
+ ### 2.4 LLM Interaction
71
+
72
+ * **Ollama / HF-hosted model** as backend:
73
+
74
+ * Multi-instance support for **parallel drafters**.
75
+ * Token-based speculative decoding.
76
+ * **Workflow**:
77
+
78
+ ```
79
+ User prompt
80
+
81
+ Agent (LLM)
82
+
83
+ JSON tool call
84
+
85
+ Python tool executes
86
+
87
+ Result returned to LLM
88
+
89
+ Final response to user
90
+ ```
91
+ * **Dynamic code generation & execution** integrated:
92
+
93
+ * e.g., user asks for PyTorch demo → agent installs PyTorch → generates editable code → runs it in UI.
94
+
95
+ ---
96
+
97
+ ### 2.5 Persistent Memory — Hugging Face
98
+
99
+ * **Repository:** `rahul7star/pyclaw`
100
+ * **Stores:**
101
+
102
+ * Generated code & scripts
103
+ * Task outputs (`.out`)
104
+ * Logs & session history
105
+ * Dynamic tools and metadata
106
+ * **Mechanism:**
107
+
108
+ * Push artifacts via `huggingface_hub` API
109
+ * Pull existing artifacts for agent memory
110
+ * **Benefits:** Enables **long-term learning**, cross-session continuity, and reproducibility.
111
+
112
+ ---
113
+
114
+ ### 2.6 Autonomous Task Management
115
+
116
+ * **Queue-based execution**:
117
+
118
+ * Tasks added by user or agent itself.
119
+ * Background processor executes tasks in order.
120
+ * **Speculative execution**:
121
+
122
+ * Multi-instance drafters improve code reliability.
123
+ * Verifier ensures correctness of outputs.
124
+ * **Dynamic tools**:
125
+
126
+ * Tools can evolve or new tools can be created on-the-fly.
127
+
128
+ ---
129
+
130
+ ### 2.7 Safety & Resource Management
131
+
132
+ * **Code execution sandbox**:
133
+
134
+ * Controlled Python subprocess.
135
+ * Auto cleanup of temporary files.
136
+ * **CPU/GPU selection**:
137
+
138
+ * Default: CPU
139
+ * Optional: GPU if available and environment variable set.
140
+ * **Dependency management**:
141
+
142
+ * Automatic package installation (e.g., PyTorch for user-requested demos).
143
+
144
+ ---
145
+
146
+ ## 3. **Example User Workflow**
147
+
148
+ **User Prompt:**
149
+ `"Create a neural network demo in Python using PyTorch."`
150
+
151
+ **Agent Actions:**
152
+
153
+ 1. Detects PyTorch requirement → installs on CPU.
154
+ 2. Generates Python code using `/WRITE_PY`.
155
+ 3. Saves code in Hugging Face repo for memory.
156
+ 4. Displays code in UI for editing.
157
+ 5. Runs code → outputs printed and logged.
158
+ 6. Updates agent memory with results and execution logs.
159
+ 7. Optionally creates a new dynamic tool for future NN generation.
160
+
161
+ ---
162
+
163
+ ## 4. **Evolvable Architecture**
164
+
165
+ | Feature | Current Status | Future Evolution |
166
+ | ------------------------- | -------------- | ---------------------------------------------------- |
167
+ | Dynamic Tool Creation | ✅ | Can auto-generate new tools from tasks |
168
+ | Long-Term Memory | ✅ via HF | Add semantic search, embeddings for context |
169
+ | Speculative Decoding | ✅ | Increase drafters, multi-agent cooperation |
170
+ | Autonomous Task Execution | Partial | Recursive task planning, multi-step project handling |
171
+ | Dependency Management | ✅ | Expand to virtual environments per project |
172
+ | Safe Execution | Partial | Containerized execution (Docker/WSL) |
173
+
174
+ ---
175
+
176
+ ## 5. **Roadmap to Claude-Like Autonomy**
177
+
178
+ 1. **Enhance memory:** Semantic embeddings + search in HF repo.
179
+ 2. **Recursive reasoning:** Agent generates subtasks autonomously.
180
+ 3. **Multi-agent collaboration:** Multiple PygmyClaws coordinate on large projects.
181
+ 4. **Learning from outputs:** Store completed tasks + feedback for continuous improvement.
182
+ 5. **Safety & isolation:** Dockerized Python execution with resource limits.
183
+ 6. **Dynamic UI:** Allow live editing, execution, and visualization of code outputs.
184
+
185
+ ---
186
+
187
+ ## 6. **Diagram of End-to-End Flow**
188
+
189
+ ```
190
+ ┌───────────────┐
191
+ │ User Prompt │
192
+ └───────┬───────┘
193
+
194
+
195
+ ┌───────────────┐
196
+ │ Agent (LLM) │
197
+ └───────┬───────┘
198
+ │ JSON Tool Call
199
+
200
+ ┌───────────────┐
201
+ │ Python Tool │
202
+ │ Executes Task │
203
+ └───────┬───────┘
204
+ │ Result
205
+
206
+ ┌───────────────┐
207
+ │ Agent (LLM) │
208
+ │ Processes │
209
+ └───────┬───────┘
210
+
211
+
212
+ ┌───────────────┐
213
+ │ UI Output │
214
+ │ (Code/Result) │
215
+ └───────┬───────┘
216
+
217
+
218
+ ┌───────────────┐
219
+ │ │
220
+ │ Persistent │
221
+ │ Memory Repo │
222
+ └───────────────┘
223
+ ```
224
+
225
+ ---
226
+
app.py ADDED
@@ -0,0 +1,238 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import streamlit as st
2
+ import re
3
+ import uuid
4
+ import subprocess
5
+ import sys
6
+ from datetime import datetime
7
+ from threading import Thread
8
+ from queue import Queue
9
+ from pygmyclaw import PygmyClaw
10
+
11
+ st.set_page_config(page_title="Py", layout="wide")
12
+
13
+ # --------------------------------------------------
14
+ # GLOBAL STATE
15
+ # --------------------------------------------------
16
+
17
+ if "job_store" not in st.session_state:
18
+ st.session_state.job_store = {}
19
+
20
+ if "job_queue" not in st.session_state:
21
+ st.session_state.job_queue = Queue()
22
+
23
+ if "agent" not in st.session_state:
24
+ st.session_state.agent = PygmyClaw()
25
+
26
+ if "worker_started" not in st.session_state:
27
+ st.session_state.worker_started = False
28
+
29
+ job_store = st.session_state.job_store
30
+ job_queue = st.session_state.job_queue
31
+ agent = st.session_state.agent
32
+
33
+
34
+ # --------------------------------------------------
35
+ # BACKGROUND WORKER
36
+ # --------------------------------------------------
37
+
38
+ def worker():
39
+ while True:
40
+ job_id = job_queue.get()
41
+ job = job_store[job_id]
42
+ job["status"] = "running"
43
+ try:
44
+ system_prompt = f"""
45
+ You are a Python coding assistant.
46
+
47
+ Return ONLY runnable Python code.
48
+ Do NOT explain anything.
49
+
50
+ Task:
51
+ {job['prompt']}
52
+ """
53
+ result = agent.submit_prompt(system_prompt, job["tool"])
54
+
55
+ # ===== FIX =====
56
+ match = re.search(r"```python(.*?)```", result, re.S)
57
+ if match:
58
+ code = match.group(1).strip()
59
+ else:
60
+ code = result.strip()
61
+ # ===============
62
+
63
+ job["response"] = result
64
+ job["code"] = code
65
+ job["status"] = "completed"
66
+
67
+ except Exception as e:
68
+ job["status"] = "failed"
69
+ job["response"] = str(e)
70
+
71
+ job_queue.task_done()
72
+
73
+ # --------------------------------------------------
74
+ # START WORKER
75
+ # --------------------------------------------------
76
+
77
+ if not st.session_state.worker_started:
78
+
79
+ Thread(target=worker, daemon=True).start()
80
+
81
+ st.session_state.worker_started = True
82
+
83
+
84
+ # --------------------------------------------------
85
+ # HEADER
86
+ # --------------------------------------------------
87
+
88
+ st.title("🧠 PygmyClaw AI Dev Dashboard")
89
+
90
+ # --------------------------------------------------
91
+ # CREATE TASK
92
+ # --------------------------------------------------
93
+
94
+ with st.expander("🚀 Create AI Task", expanded=True):
95
+
96
+ col1, col2 = st.columns([4,1])
97
+
98
+ prompt = col1.text_area(
99
+ "Prompt",
100
+ height=120,
101
+ placeholder="write a python function to add two numbers",
102
+ value="write a python function to add two numbers"
103
+ )
104
+
105
+ tool = col2.selectbox(
106
+ "Tool",
107
+ ["AI Agent"]
108
+ )
109
+
110
+ if col2.button("Create Job"):
111
+
112
+ job_id = str(uuid.uuid4())[:8]
113
+
114
+ job_store[job_id] = {
115
+ "prompt": prompt,
116
+ "tool": tool,
117
+ "status": "queued",
118
+ "response": "",
119
+ "code": "",
120
+ "created": datetime.now().strftime("%H:%M:%S")
121
+ }
122
+
123
+ job_queue.put(job_id)
124
+
125
+ st.success(f"Job {job_id} queued")
126
+
127
+
128
+ # --------------------------------------------------
129
+ # METRICS
130
+ # --------------------------------------------------
131
+
132
+ queued = sum(1 for j in job_store.values() if j["status"] == "queued")
133
+ running = sum(1 for j in job_store.values() if j["status"] == "running")
134
+ done = sum(1 for j in job_store.values() if j["status"] == "completed")
135
+ failed = sum(1 for j in job_store.values() if j["status"] == "failed")
136
+
137
+ col1, col2, col3, col4 = st.columns(4)
138
+
139
+ col1.metric("Queued", queued)
140
+ col2.metric("Running", running)
141
+ col3.metric("Completed", done)
142
+ col4.metric("Failed", failed)
143
+
144
+ st.divider()
145
+
146
+ # --------------------------------------------------
147
+ # JOB DASHBOARD
148
+ # --------------------------------------------------
149
+
150
+ if not job_store:
151
+ st.info("No jobs yet")
152
+
153
+ for job_id, job in list(job_store.items())[::-1]:
154
+
155
+ with st.container():
156
+
157
+ st.subheader(f"Job {job_id}")
158
+
159
+ st.write("Status:", job["status"])
160
+ st.write("Created:", job["created"])
161
+
162
+ st.code(job["prompt"], language="text")
163
+
164
+ # -----------------------
165
+ # AI RESPONSE
166
+ # -----------------------
167
+
168
+ if job.get("response"):
169
+
170
+ st.markdown("### 🤖 AI Response")
171
+
172
+ st.write(job["response"])
173
+
174
+ # -----------------------
175
+ # CODE EDITOR (ALWAYS VISIBLE)
176
+ # -----------------------
177
+
178
+ st.markdown("### 💻 Generated Code")
179
+
180
+ job["code"] = st.text_area(
181
+ "Edit Code",
182
+ value=job.get("code", ""),
183
+ height=220,
184
+ key=f"code_{job_id}"
185
+ )
186
+
187
+ col1, col2 = st.columns(2)
188
+
189
+ # -----------------------
190
+ # RUN CODE
191
+ # -----------------------
192
+
193
+ if col1.button("▶ Run Code", key=f"run_{job_id}"):
194
+
195
+ try:
196
+
197
+ code = job.get("code", "")
198
+
199
+ # detect imports
200
+ imports = re.findall(
201
+ r"^\s*(?:import|from)\s+([\w_]+)",
202
+ code,
203
+ flags=re.MULTILINE
204
+ )
205
+
206
+ # auto install missing packages
207
+ for pkg in imports:
208
+ try:
209
+ __import__(pkg)
210
+ except ImportError:
211
+ subprocess.run(
212
+ [sys.executable, "-m", "pip", "install", pkg],
213
+ check=True
214
+ )
215
+
216
+ local_vars = {}
217
+
218
+ exec(code, {}, local_vars)
219
+
220
+ st.success("Execution Output")
221
+
222
+ st.json(local_vars)
223
+
224
+ except Exception as e:
225
+
226
+ st.error(str(e))
227
+
228
+ # -----------------------
229
+ # DELETE JOB
230
+ # -----------------------
231
+
232
+ if col2.button("Delete Job", key=f"del_{job_id}"):
233
+
234
+ del job_store[job_id]
235
+
236
+ st.rerun()
237
+
238
+ st.divider()
app_gradio.py ADDED
@@ -0,0 +1,212 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # app_gradio.py
2
+ import gradio as gr
3
+ import re
4
+ import uuid
5
+ import time
6
+ import contextlib
7
+ from io import StringIO
8
+ from threading import Thread
9
+ from queue import Queue
10
+ from pygmyclaw import PygmyClaw
11
+
12
+ # --------------------------------------------------
13
+ # GLOBAL STATE
14
+ # --------------------------------------------------
15
+ job_store = {}
16
+ job_queue = Queue()
17
+ agent = PygmyClaw()
18
+ auto_refresh_enabled = True # toggle for auto-refresh
19
+
20
+ # --------------------------------------------------
21
+ # HELPER FUNCTIONS
22
+ # --------------------------------------------------
23
+ def clean_code(code):
24
+ """Remove markdown fences and normalize code indentation."""
25
+ if not code:
26
+ return ""
27
+ code = re.sub(r"```python", "", code)
28
+ code = re.sub(r"```", "", code)
29
+ return code.strip()
30
+
31
+ def extract_explanation(full_result):
32
+ """Get explanation text from full AI response."""
33
+ text_no_code = re.sub(r"```.*?```", "", full_result, flags=re.S)
34
+ return text_no_code.strip()
35
+
36
+ def extract_code(full_result):
37
+ """Extract Python code blocks from full AI response."""
38
+ code_blocks = re.findall(r"```(?:python)?\s*(.*?)```", full_result, re.S | re.I)
39
+ code = "\n\n".join([c.strip() for c in code_blocks])
40
+ return code
41
+
42
+ # --------------------------------------------------
43
+ # BACKGROUND WORKER
44
+ # --------------------------------------------------
45
+ def worker():
46
+ """Process queued jobs with AI agent, extract code and explanation."""
47
+ while True:
48
+ job_id = job_queue.get()
49
+ job = job_store[job_id]
50
+ job["status"] = "running"
51
+
52
+ try:
53
+ prompt = job["prompt"]
54
+ tool = job["tool"]
55
+
56
+ # System prompt
57
+ system_prompt = f"Write Python code ONLY and explain all lines clearly in comments. Task: {prompt}"
58
+
59
+ # Get fresh AI response
60
+ full_result = agent.generate_with_ssd(system_prompt, timeout=120)
61
+
62
+ # Extract code & explanation from response
63
+ code = extract_code(full_result)
64
+ explanation = extract_explanation(full_result)
65
+
66
+ # Update job
67
+ job["raw_result"] = full_result
68
+ job["code"] = code
69
+ job["response"] = explanation
70
+ job["status"] = "completed"
71
+
72
+ except Exception as e:
73
+ job["status"] = "failed"
74
+ job["response"] = str(e)
75
+ finally:
76
+ job_queue.task_done()
77
+
78
+ # Start background worker
79
+ Thread(target=worker, daemon=True).start()
80
+
81
+ # --------------------------------------------------
82
+ # JOB OPERATIONS
83
+ # --------------------------------------------------
84
+ def create_job(prompt, tool):
85
+ job_id = str(uuid.uuid4())[:8]
86
+ job_store[job_id] = {
87
+ "prompt": prompt,
88
+ "tool": tool,
89
+ "status": "queued",
90
+ "response": "",
91
+ "code": "",
92
+ "raw_result": "",
93
+ "created": time.strftime("%H:%M:%S")
94
+ }
95
+ job_queue.put(job_id)
96
+ return f"Job {job_id} queued", job_id
97
+
98
+ def dashboard():
99
+ rows = []
100
+ for job_id, job in reversed(list(job_store.items())):
101
+ rows.append([job_id, job["status"], job["created"], job["prompt"], job.get("code", "")])
102
+ return rows
103
+
104
+ def load_job(job_id):
105
+ job = job_store.get(job_id)
106
+ if not job:
107
+ return "", "", "", ""
108
+ return (
109
+ job.get("prompt", ""),
110
+ job.get("response", ""),
111
+ job.get("code", ""), # editable code
112
+ job.get("status", "")
113
+ )
114
+
115
+ def run_code(code):
116
+ try:
117
+ code = clean_code(code)
118
+ output_buffer = StringIO()
119
+ with contextlib.redirect_stdout(output_buffer):
120
+ exec(code, {})
121
+ result = output_buffer.getvalue()
122
+ return result if result else "✅ Code executed successfully."
123
+ except Exception as e:
124
+ return str(e)
125
+
126
+ def delete_job(job_id):
127
+ if job_id in job_store:
128
+ del job_store[job_id]
129
+ return dashboard()
130
+
131
+ def toggle_auto_refresh():
132
+ global auto_refresh_enabled
133
+ auto_refresh_enabled = not auto_refresh_enabled
134
+ return "✅ Auto-refresh ON" if auto_refresh_enabled else "❌ Auto-refresh OFF"
135
+
136
+ def explain_code(code):
137
+ """Send the current code to the AI to generate a fresh explanation."""
138
+ code = clean_code(code)
139
+ if not code.strip():
140
+ return "⚠️ No code to explain."
141
+ prompt = f"Explain the following Python code in simple terms:\n```python\n{code}\n```"
142
+ try:
143
+ explanation = agent.generate_with_ssd(prompt, timeout=120)
144
+ # Remove code blocks
145
+ explanation = re.sub(r"```.*?```", "", explanation, flags=re.S).strip()
146
+ return explanation
147
+ except Exception as e:
148
+ return f"⚠️ Error generating explanation: {str(e)}"
149
+
150
+ # --------------------------------------------------
151
+ # GRADIO UI
152
+ # --------------------------------------------------
153
+ with gr.Blocks(title="PygmyClaw AI Dev Dashboard") as demo:
154
+ gr.Markdown("# 🧠 PygmyClaw AI Dev Dashboard")
155
+
156
+ # ---------------- Create Job ----------------
157
+ with gr.Row():
158
+ prompt_input = gr.Textbox(label="Prompt", lines=5, value="write a python function to add two numbers")
159
+ tool_input = gr.Dropdown(["AI Agent"], value="AI Agent", label="Tool")
160
+ create_btn = gr.Button("Create Job")
161
+ create_status = gr.Markdown()
162
+ job_id_input = gr.Textbox(label="Job ID") # auto-filled
163
+ create_btn.click(create_job, inputs=[prompt_input, tool_input], outputs=[create_status, job_id_input])
164
+
165
+ # ---------------- Job Dashboard ----------------
166
+ gr.Markdown("## Job Dashboard")
167
+ job_table = gr.Dataframe(headers=["Job ID", "Status", "Created", "Prompt", "Code"], interactive=False)
168
+ refresh_btn = gr.Button("🔄 Refresh Dashboard")
169
+ refresh_status = gr.Markdown()
170
+ refresh_btn.click(dashboard, inputs=None, outputs=job_table)
171
+
172
+ toggle_btn = gr.Button("⏯ Toggle Auto-Refresh")
173
+ toggle_btn.click(toggle_auto_refresh, inputs=None, outputs=refresh_status)
174
+
175
+ # Auto-refresh dashboard
176
+ def auto_refresh_dashboard():
177
+ if auto_refresh_enabled:
178
+ return dashboard()
179
+ return gr.update()
180
+ refresh_timer = gr.Timer(value=5)
181
+ refresh_timer.tick(auto_refresh_dashboard, inputs=None, outputs=job_table)
182
+
183
+ # ---------------- Selected Job ----------------
184
+ gr.Markdown("## Selected Job Details")
185
+ load_btn = gr.Button("Load Job")
186
+ prompt_box = gr.Textbox(label="Prompt", lines=4)
187
+ response_box = gr.Markdown(label="AI Explanation")
188
+ code_editor = gr.Code(label="Edit Code", language="python", interactive=True)
189
+ status_box = gr.Textbox(label="Status")
190
+ load_btn.click(load_job, inputs=job_id_input, outputs=[prompt_box, response_box, code_editor, status_box])
191
+
192
+ # Auto-refresh selected job
193
+ def auto_refresh_job(job_id):
194
+ if auto_refresh_enabled:
195
+ return load_job(job_id)
196
+ return gr.update(), gr.update(), gr.update(), gr.update()
197
+ detail_timer = gr.Timer(value=40)
198
+ detail_timer.tick(auto_refresh_job, inputs=[job_id_input], outputs=[prompt_box, response_box, code_editor, status_box])
199
+
200
+ # ---------------- Run / Delete / Explain ----------------
201
+ with gr.Row():
202
+ run_btn = gr.Button("▶ Run Code")
203
+ delete_btn = gr.Button("Delete Job")
204
+ explain_btn = gr.Button("💡 Explain Code")
205
+ output_box = gr.Textbox(label="Execution Output", lines=6)
206
+ explanation_box = gr.Markdown(label="Code Explanation")
207
+ run_btn.click(run_code, inputs=code_editor, outputs=output_box)
208
+ delete_btn.click(delete_job, inputs=job_id_input, outputs=job_table)
209
+ explain_btn.click(explain_code, inputs=code_editor, outputs=explanation_box)
210
+
211
+ # ---------------- Launch ----------------
212
+ demo.launch(server_name="0.0.0.0", server_port=7860)
desgin.md ADDED
@@ -0,0 +1,773 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+
3
+ # design.md
4
+
5
+ # PygmyClaw Agent System
6
+
7
+ ```
8
+ app.py → UI
9
+ pygmyclaw.py → AI agent + queue + HF storage
10
+ pygmyclaw_multitool.py → tool execution engine
11
+ tools.json → tool registry stored on HuggingFace
12
+ memory.json → task memory stored on HuggingFace
13
+
14
+
15
+
16
+
17
+ ```
18
+ ┌───────────────┐
19
+ │ Streamlit │
20
+ │ app.py │
21
+ └──────┬────────┘
22
+
23
+
24
+ ┌───────────────┐
25
+ │ PygmyClaw │
26
+ │ pygmyclaw.py │
27
+ └──────┬────────┘
28
+
29
+ Queue + Agent logic
30
+
31
+
32
+ ┌─────────────────────┐
33
+ │ pygmyclaw_multitool│
34
+ │ run_tool() │
35
+ └─────────────────────┘
36
+
37
+
38
+ HF Storage (tools.json / memory.json)
39
+
40
+ ```
41
+
42
+ ```
43
+ ## 1. Overview
44
+
45
+ PygmyClaw is a **local AI agent framework** designed to run in a **Docker Space**.
46
+
47
+ The system combines:
48
+
49
+ * speculative decoding for faster generation
50
+ * tool-calling architecture
51
+ * dynamic tool creation
52
+ * artifact generation (files/code/data)
53
+ * background task execution
54
+ * interactive UI
55
+
56
+ The goal is to create a **self-extensible AI system** where the agent can:
57
+
58
+ * use tools
59
+ * create new tools
60
+ * generate code
61
+ * install dependencies
62
+ * run programs
63
+ * manage tasks
64
+ * interact through a UI
65
+
66
+ ---
67
+
68
+ # 2. High-Level System Architecture
69
+
70
+ ```
71
+ User
72
+
73
+ HF Space UI (Gradio)
74
+
75
+ Agent Engine (pygmyclaw.py)
76
+
77
+ Speculative Decoding Engine
78
+
79
+ Tool Call Parser
80
+
81
+ Tool Executor (subprocess)
82
+
83
+ Dynamic Tool Registry
84
+
85
+ Workspace / Artifacts / Queue
86
+
87
+ Result returned to LLM
88
+ ```
89
+
90
+ ---
91
+
92
+ # 3. System Layers
93
+
94
+ The system is organized into **five layers**.
95
+
96
+ ## Layer 1 — User Interface
97
+
98
+ Provides an interactive interface to the agent.
99
+
100
+ Responsibilities:
101
+
102
+ * display chat interaction
103
+ * show generated artifacts
104
+ * provide code editor
105
+ * allow running generated code
106
+ * show execution output
107
+
108
+ UI components:
109
+
110
+ ```
111
+ Chat Panel
112
+ Artifact Viewer
113
+ Code Editor
114
+ Execution Console
115
+ ```
116
+
117
+ Example layout:
118
+
119
+ ```
120
+ ----------------------------------
121
+ Chat | Code Editor
122
+ |
123
+ | demo_nn.py
124
+ |
125
+ | [Run] [Save]
126
+ ----------------------------------
127
+ Console Output
128
+ ----------------------------------
129
+ ```
130
+
131
+ ---
132
+
133
+ # 4. Layer 2 — Agent Engine
134
+
135
+ Implemented in:
136
+
137
+ ```
138
+ pygmyclaw.py
139
+ ```
140
+
141
+ This is the **central orchestrator** of the system.
142
+
143
+ Responsibilities:
144
+
145
+ * manage LLM interaction
146
+ * run speculative decoding
147
+ * parse tool calls
148
+ * execute tools
149
+ * maintain context
150
+ * handle queues
151
+ * coordinate artifacts
152
+
153
+ The engine runs an **agent execution loop**.
154
+
155
+ ---
156
+
157
+ # 5. Agent Execution Loop
158
+
159
+ The entire system is driven by the following loop.
160
+
161
+ ```
162
+ User Prompt
163
+
164
+ Agent (LLM)
165
+
166
+ LLM outputs JSON tool call
167
+
168
+ Tool executes
169
+
170
+ Result returned to LLM
171
+
172
+ LLM continues reasoning
173
+
174
+ Final response
175
+ ```
176
+
177
+ Expanded loop:
178
+
179
+ ```
180
+ User Prompt
181
+
182
+ LLM reasoning
183
+
184
+ Tool call JSON
185
+
186
+ Tool executor
187
+
188
+ Tool result
189
+
190
+ Context updated
191
+
192
+ LLM reasoning continues
193
+ ```
194
+
195
+ The loop stops when the LLM returns **a final answer instead of a tool call**.
196
+
197
+ ---
198
+
199
+ # 6. Speculative Decoding Engine
200
+
201
+ PygmyClaw speeds up generation using **speculative decoding**.
202
+
203
+ Architecture:
204
+
205
+ ```
206
+ Drafter 1
207
+ Drafter 2
208
+ Drafter 3
209
+
210
+ Verifier
211
+ ```
212
+
213
+ Flow:
214
+
215
+ ```
216
+ User prompt
217
+
218
+ Draft tokens generated
219
+
220
+ Verifier checks tokens
221
+
222
+ Accept or reject
223
+ ```
224
+
225
+ This improves generation speed while maintaining accuracy.
226
+
227
+ Typical configuration:
228
+
229
+ ```
230
+ 3 draft models
231
+ 1 verifier model
232
+ ```
233
+
234
+ Each runs in a separate Ollama instance.
235
+
236
+ ---
237
+
238
+ # 7. Tool System
239
+
240
+ Tools allow the agent to perform actions outside the LLM.
241
+
242
+ Tools run in an isolated subprocess:
243
+
244
+ ```
245
+ pygmyclaw_multitool.py
246
+ ```
247
+
248
+ Execution flow:
249
+
250
+ ```
251
+ Agent
252
+
253
+ Tool call JSON
254
+
255
+ Subprocess execution
256
+
257
+ Tool result JSON
258
+ ```
259
+
260
+ Example tool call:
261
+
262
+ ```
263
+ {
264
+ "tool": "sys_info",
265
+ "parameters": {}
266
+ }
267
+ ```
268
+
269
+ Tool response:
270
+
271
+ ```
272
+ {
273
+ "os": "Linux",
274
+ "python_version": "3.11"
275
+ }
276
+ ```
277
+
278
+ ---
279
+
280
+ # 8. Tool Categories
281
+
282
+ The system supports multiple tool types.
283
+
284
+ ## System Tools
285
+
286
+ ```
287
+ sys_info
288
+ list_files
289
+ read_file
290
+ write_file
291
+ ```
292
+
293
+ ## Environment Tools
294
+
295
+ ```
296
+ install_python_package
297
+ check_package_installed
298
+ execute_shell
299
+ ```
300
+
301
+ ## Code Tools
302
+
303
+ ```
304
+ write_python_code
305
+ run_python_file
306
+ format_code
307
+ ```
308
+
309
+ ## Artifact Tools
310
+
311
+ ```
312
+ create_artifact
313
+ update_artifact
314
+ delete_artifact
315
+ ```
316
+
317
+ ## Agent Tools
318
+
319
+ ```
320
+ create_agent
321
+ list_agents
322
+ run_agent
323
+ ```
324
+
325
+ ---
326
+
327
+ # 9. Dynamic Tool Creation
328
+
329
+ Agents can create new tools dynamically.
330
+
331
+ Example prompt:
332
+
333
+ ```
334
+ create a tool to fetch python documentation
335
+ ```
336
+
337
+ Agent calls:
338
+
339
+ ```
340
+ {
341
+ "tool": "create_tool",
342
+ "parameters": {
343
+ "name": "python_docs",
344
+ "description": "search python docs"
345
+ }
346
+ }
347
+ ```
348
+
349
+ System creates:
350
+
351
+ ```
352
+ tools/python_docs.py
353
+ ```
354
+
355
+ Tool becomes immediately available.
356
+
357
+ This enables **self-extending agents**.
358
+
359
+ ---
360
+
361
+ # 10. Tool Discovery
362
+
363
+ At startup the engine scans the tools directory.
364
+
365
+ ```
366
+ tools/
367
+ echo.py
368
+ sys_info.py
369
+ run_python.py
370
+ ```
371
+
372
+ A tool registry is generated.
373
+
374
+ Example:
375
+
376
+ ```
377
+ TOOLS = {
378
+ echo,
379
+ sys_info,
380
+ run_python
381
+ }
382
+ ```
383
+
384
+ The tool list is provided to the LLM.
385
+
386
+ ---
387
+
388
+ # 11. Artifact System
389
+
390
+ Artifacts are files generated by the agent.
391
+
392
+ Examples:
393
+
394
+ ```
395
+ code
396
+ datasets
397
+ documents
398
+ images
399
+ logs
400
+ ```
401
+
402
+ Directory structure:
403
+
404
+ ```
405
+ artifacts/
406
+ code/
407
+ data/
408
+ documents/
409
+ ```
410
+
411
+ Example artifact:
412
+
413
+ ```
414
+ artifacts/code/addition.py
415
+ ```
416
+
417
+ Artifacts enable **UI interaction**.
418
+
419
+ ---
420
+
421
+ # 12. Artifact UI Interaction
422
+
423
+ When artifacts are created the UI displays controls.
424
+
425
+ Example:
426
+
427
+ ```
428
+ addition.py
429
+
430
+ Edit
431
+ Run
432
+ Download
433
+ ```
434
+
435
+ Artifact metadata may include:
436
+
437
+ ```
438
+ language
439
+ dependencies
440
+ runnable
441
+ created_by
442
+ ```
443
+
444
+ ---
445
+
446
+ # 13. Workspace Environment
447
+
448
+ All agent work occurs in a dedicated workspace.
449
+
450
+ ```
451
+ workspace/
452
+ venv/
453
+ artifacts/
454
+ tools/
455
+ agents/
456
+ ```
457
+
458
+ The workspace provides:
459
+
460
+ * dependency isolation
461
+ * file management
462
+ * persistent agent data
463
+
464
+ ---
465
+
466
+ # 14. Dependency Management
467
+
468
+ Agents may install packages.
469
+
470
+ Example prompt:
471
+
472
+ ```
473
+ create a pytorch neural network demo
474
+ ```
475
+
476
+ Agent detects dependency:
477
+
478
+ ```
479
+ torch
480
+ ```
481
+
482
+ Tool call:
483
+
484
+ ```
485
+ install_python_package("torch")
486
+ ```
487
+
488
+ Installed inside the workspace environment.
489
+
490
+ ```
491
+ workspace/venv/
492
+ ```
493
+
494
+ ---
495
+
496
+ # 15. Code Generation Workflow
497
+
498
+ Example request:
499
+
500
+ ```
501
+ create a neural network code demo in python using pytorch
502
+ ```
503
+
504
+ Execution flow:
505
+
506
+ ```
507
+ User prompt
508
+
509
+ Agent reasoning
510
+
511
+ Install dependency
512
+
513
+ Generate code
514
+
515
+ Save artifact
516
+
517
+ UI displays code
518
+ ```
519
+
520
+ Artifact example:
521
+
522
+ ```
523
+ artifacts/code/pytorch_nn_demo.py
524
+ ```
525
+
526
+ UI shows:
527
+
528
+ ```
529
+ Edit
530
+ Run
531
+ Download
532
+ ```
533
+
534
+ ---
535
+
536
+ # 16. Code Execution Workflow
537
+
538
+ When the user presses **Run**:
539
+
540
+ ```
541
+ UI
542
+
543
+ run_python_file tool
544
+
545
+ workspace python interpreter
546
+
547
+ program execution
548
+
549
+ stdout returned
550
+ ```
551
+
552
+ Console output appears in the UI.
553
+
554
+ ---
555
+
556
+ # 17. Task Queue
557
+
558
+ The system supports background tasks.
559
+
560
+ Queue storage options:
561
+
562
+ ```
563
+ Redis
564
+ JSON file
565
+ ```
566
+
567
+ Example task:
568
+
569
+ ```
570
+ {
571
+ "id": "123",
572
+ "prompt": "generate dataset"
573
+ }
574
+ ```
575
+
576
+ Queue worker processes tasks asynchronously.
577
+
578
+ ---
579
+
580
+ # 18. Agent Registry
581
+
582
+ Agents are stored as configuration files.
583
+
584
+ ```
585
+ agents/
586
+ python_coder.json
587
+ research_agent.json
588
+ ```
589
+
590
+ Example agent definition:
591
+
592
+ ```
593
+ {
594
+ "name": "python_coder",
595
+ "model": "qwen2.5",
596
+ "tools": [
597
+ "write_python_code",
598
+ "run_python_file"
599
+ ]
600
+ }
601
+ ```
602
+
603
+ Agents can be created dynamically.
604
+
605
+ ---
606
+
607
+ # 19. Redis Usage
608
+
609
+ Redis is optional but improves scalability.
610
+
611
+ Uses:
612
+
613
+ ```
614
+ task queue
615
+ agent memory
616
+ caching
617
+ ```
618
+
619
+ Configuration can be stored in Redis or local config.
620
+
621
+ ---
622
+
623
+ # 20. Hugging Face Space Deployment
624
+
625
+ The system runs inside a Docker Space.
626
+
627
+ Components:
628
+
629
+ ```
630
+ Python
631
+ Ollama
632
+ Gradio
633
+ Redis (optional)
634
+ ```
635
+
636
+ Startup sequence:
637
+
638
+ ```
639
+ Start container
640
+
641
+ Start Ollama
642
+
643
+ Load models
644
+
645
+ Start agent engine
646
+
647
+ Launch UI
648
+ ```
649
+
650
+ ---
651
+
652
+ # 21. Security Considerations
653
+
654
+ Important restrictions include:
655
+
656
+ * package allowlist
657
+ * sandboxed tool execution
658
+ * workspace file isolation
659
+ * limited shell access
660
+
661
+ Example allowed packages:
662
+
663
+ ```
664
+ numpy
665
+ pandas
666
+ torch
667
+ scikit-learn
668
+ matplotlib
669
+ ```
670
+
671
+ ---
672
+
673
+ # 22. End-to-End Example
674
+
675
+ User request:
676
+
677
+ ```
678
+ create a neural network code demo in python that uses pytorch
679
+ ```
680
+
681
+ System execution:
682
+
683
+ ```
684
+ User prompt
685
+
686
+ Agent reasoning
687
+
688
+ install_python_package(torch)
689
+
690
+ write_file(pytorch_nn_demo.py)
691
+
692
+ artifact created
693
+
694
+ UI displays code
695
+ ```
696
+
697
+ User interaction:
698
+
699
+ ```
700
+ Edit code
701
+ Run code
702
+ Download file
703
+ ```
704
+
705
+ Execution output appears in the console.
706
+
707
+ ---
708
+
709
+ # 23. Final System Architecture
710
+
711
+ ```
712
+ User
713
+
714
+ HF Space UI
715
+
716
+ Agent Engine
717
+
718
+ Speculative Decoding
719
+
720
+ Tool Parser
721
+
722
+ Tool Executor
723
+
724
+ Dynamic Tools
725
+
726
+ Workspace
727
+
728
+ Artifacts / Agents / Queue
729
+
730
+ Result returned to LLM
731
+ ```
732
+
733
+ ---
734
+
735
+ # 24. Design Principles
736
+
737
+ The system follows several core principles.
738
+
739
+ 1. **Everything is a tool**
740
+
741
+ Actions are performed through tools rather than hardcoded logic.
742
+
743
+ 2. **Agents can extend themselves**
744
+
745
+ Agents may create tools and agents.
746
+
747
+ 3. **Artifacts are first-class outputs**
748
+
749
+ Generated files are accessible through the UI.
750
+
751
+ 4. **Isolation and safety**
752
+
753
+ Tools run in subprocesses.
754
+
755
+ 5. **Local and lightweight**
756
+
757
+ System runs entirely locally using Ollama.
758
+
759
+ ---
760
+
761
+ # 25. Future Extensions
762
+
763
+ Potential future capabilities include:
764
+
765
+ ```
766
+ multi-agent collaboration
767
+ autonomous project generation
768
+ browser automation
769
+ dataset generation
770
+ long-term memory
771
+ ```
772
+
773
+ ---
entrypoint.sh ADDED
@@ -0,0 +1,16 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/bin/bash
2
+ set -e
3
+
4
+ echo "Starting Ollama..."
5
+ ollama serve &
6
+
7
+ sleep 8
8
+
9
+ echo "Pulling model..."
10
+ ollama pull "${MODEL_NAME}" || true
11
+
12
+ mkdir -p /workspace/data
13
+
14
+ echo "Launching Gradio UI..."
15
+ cd /workspace
16
+ python3 app_gradio.py
pygmyclaw.py ADDED
@@ -0,0 +1,241 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ PygmyClaw – Compact AI Agent with async queue, HF + AI tools support.
4
+ Verbose logging added for debugging.
5
+ """
6
+
7
+ import os
8
+ import sys
9
+ import json
10
+ import time
11
+ import queue
12
+ import threading
13
+ import urllib.request
14
+ from pathlib import Path
15
+ import subprocess
16
+ import re
17
+ import textwrap
18
+ from huggingface_hub import hf_hub_download, upload_file
19
+
20
+ # -------------------- Globals --------------------
21
+ SCRIPT_DIR = Path(__file__).parent.resolve()
22
+ DEFAULT_MODEL = os.environ.get("MODEL_NAME", "qwen3.5:0.8b")
23
+ DEFAULT_ENDPOINT = "http://localhost:11434/api/generate"
24
+ HF_TOKEN = os.environ.get("HF_TOKEN")
25
+ HF_REPO = "rahul7star/pyclaw"
26
+ HF_LOCAL_DIR = SCRIPT_DIR / "pyclaw_hf"
27
+ FILES_TO_DOWNLOAD = ["memory.json", "tools.json"]
28
+ TASK_QUEUE = queue.Queue()
29
+ QUEUE_EVENT = threading.Event()
30
+
31
+ print(f"[LOG] PygmyClaw loaded. Default model: {DEFAULT_MODEL}")
32
+
33
+ # -------------------- HF File Download --------------------
34
+ def download_hf_files():
35
+ HF_LOCAL_DIR.mkdir(parents=True, exist_ok=True)
36
+ for file_name in FILES_TO_DOWNLOAD:
37
+ local_path = HF_LOCAL_DIR / file_name
38
+ if not local_path.exists() or local_path.stat().st_size == 0:
39
+ try:
40
+ hf_hub_download(repo_id=HF_REPO, filename=file_name,
41
+ token=HF_TOKEN, local_dir=str(HF_LOCAL_DIR))
42
+ print(f"[LOG] Downloaded {file_name}")
43
+ except Exception as e:
44
+ print(f"[WARN] Failed to download {file_name}: {e}")
45
+ local_path.write_text("{}")
46
+ print(f"[LOG] Created empty {file_name}")
47
+
48
+
49
+ def save_hf_memory():
50
+ mem_file = HF_LOCAL_DIR / "memory.json"
51
+ print("[DEBUG] Saving memory locally...")
52
+ try:
53
+ with open(mem_file, "w") as f:
54
+ print("[DEBUG] Local memory saved:", mem_file, "Size:", mem_file.stat().st_size)
55
+ json.dump(self.memory_data, f, indent=2)
56
+ upload_file(path_or_fileobj=str(mem_file), path_in_repo="memory.json",
57
+ repo_id=HF_REPO, token=HF_TOKEN, repo_type="model")
58
+ print("[LOG] memory.json updated successfully on HF.")
59
+ except Exception as e:
60
+ print(f"[WARN] Failed to push memory to HF: {e}")
61
+
62
+ # -------------------- PygmyClaw Agent --------------------
63
+ class PygmyClaw:
64
+ def __init__(self):
65
+ self.model = DEFAULT_MODEL
66
+ self.endpoint = DEFAULT_ENDPOINT
67
+ self.memory_data = {}
68
+ self.tools_data = {}
69
+ self.python_tools = ["Python Script"]
70
+ download_hf_files()
71
+ self._load_hf_memory()
72
+ self._load_hf_tools()
73
+ self.python_tools += list(self.tools_data.keys())
74
+ self.python_tools.append("AI Agent")
75
+ self._ensure_model_ready()
76
+ self._warmup_model()
77
+ QUEUE_EVENT.set()
78
+ threading.Thread(target=self._process_queue, daemon=True).start()
79
+ self._ssd_backend = "http"
80
+ print("[LOG] PygmyClaw initialization complete.")
81
+
82
+ # -------------------- Memory / Tools --------------------
83
+ def _load_hf_memory(self):
84
+ mem_file = HF_LOCAL_DIR / "memory.json"
85
+ mem_file.parent.mkdir(parents=True, exist_ok=True)
86
+ if not mem_file.exists() or mem_file.stat().st_size == 0:
87
+ mem_file.write_text("{}")
88
+ try:
89
+ with open(mem_file) as f:
90
+ self.memory_data = json.load(f)
91
+ print("[LOG] Loaded memory.json successfully.")
92
+ except json.JSONDecodeError:
93
+ self.memory_data = {}
94
+ print("[WARN] memory.json invalid, initialized empty")
95
+
96
+
97
+
98
+ def _save_hf_memory(self, memory_data=None):
99
+ mem_file = HF_LOCAL_DIR / "memory.json"
100
+ print("[DEBUG] Saving memory locally...")
101
+ try:
102
+ with open(mem_file, "w") as f:
103
+ print("[DEBUG] Local memory saved:", mem_file, "Size:", mem_file.stat().st_size)
104
+ json.dump(self.memory_data, f, indent=2)
105
+ upload_file(path_or_fileobj=str(mem_file), path_in_repo="memory.json",
106
+ repo_id=HF_REPO, token=HF_TOKEN, repo_type="model")
107
+ print("[LOG] memory.json updated successfully on HF.")
108
+ except Exception as e:
109
+ print(f"[WARN] Failed to push memory to HF: {e}")
110
+
111
+ def _load_hf_tools(self):
112
+ tools_file = HF_LOCAL_DIR / "tools.json"
113
+ tools_file.parent.mkdir(parents=True, exist_ok=True)
114
+ if not tools_file.exists() or tools_file.stat().st_size == 0:
115
+ tools_file.write_text("{}")
116
+ try:
117
+ with open(tools_file) as f:
118
+ self.tools_data = json.load(f)
119
+ print("[LOG] Loaded tools.json successfully.")
120
+ except json.JSONDecodeError:
121
+ self.tools_data = {}
122
+ print("[WARN] tools.json invalid, initialized empty")
123
+
124
+ # -------------------- Model Ready --------------------
125
+ def _ensure_model_ready(self):
126
+ print(f"[LOG] Ensuring model '{self.model}' is ready...")
127
+ payload = {"model": self.model, "prompt": "hello", "stream": False, "options": {"num_predict": 1}}
128
+ try:
129
+ req = urllib.request.Request(self.endpoint, data=json.dumps(payload).encode("utf-8"),
130
+ headers={"Content-Type": "application/json"}, method="POST")
131
+ with urllib.request.urlopen(req, timeout=15) as resp:
132
+ resp_data = json.loads(resp.read())
133
+ if "response" in resp_data:
134
+ print("[LOG] Model is ready.")
135
+ except Exception as e:
136
+ print(f"[WARN] HTTP model check failed: {e}. Will use CLI fallback if needed.")
137
+
138
+ def _warmup_model(self):
139
+ try:
140
+ payload = {"model": self.model, "prompt": ".", "stream": False, "options": {"num_predict": 1}}
141
+ req = urllib.request.Request(self.endpoint, data=json.dumps(payload).encode("utf-8"),
142
+ headers={"Content-Type": "application/json"}, method="POST")
143
+ with urllib.request.urlopen(req, timeout=5):
144
+ print("[LOG] Model warmed up.")
145
+ except Exception:
146
+ print("[LOG] Warmup skipped, may use CLI fallback.")
147
+
148
+ # -------------------- Task Queue --------------------
149
+ def add_task(self, prompt, tool="AI Agent", callback=None):
150
+ task_id = str(time.time())
151
+ TASK_QUEUE.put({"id": task_id, "prompt": prompt, "tool": tool, "callback": callback})
152
+ print(f"[LOG] Queued task {task_id} with tool={tool}")
153
+ return task_id
154
+
155
+ def _process_queue(self):
156
+ print("[LOG] Queue processor started...")
157
+ while QUEUE_EVENT.is_set():
158
+ try:
159
+ task = TASK_QUEUE.get(timeout=1)
160
+ except queue.Empty:
161
+ continue
162
+ task_id = task["id"]
163
+ prompt = task["prompt"]
164
+ tool = task.get("tool", "AI Agent")
165
+ callback = task.get("callback", None)
166
+ print(f"[LOG] Processing task {task_id} -> {prompt}")
167
+ try:
168
+ if tool == "Python Script":
169
+ local_vars = {}
170
+ exec(prompt, {}, local_vars)
171
+ result = str(local_vars)
172
+ else:
173
+ result = self.generate_with_ssd(prompt)
174
+ print(f"[LOG] Model output for task {task_id}:\n{result}")
175
+
176
+ # Save memory only after successful response
177
+ self.memory_data[task_id] = {
178
+ "prompt": prompt,
179
+ "response": result,
180
+ "timestamp": time.time(),
181
+ "tool": tool
182
+ }
183
+ self._save_hf_memory(self.memory_data)
184
+ save_hf_memory()
185
+ if callback:
186
+ callback(result)
187
+ print(f"[LOG] Task {task_id} completed successfully.")
188
+
189
+ except Exception as e:
190
+ print(f"[ERROR] Task {task_id} failed: {e}")
191
+ finally:
192
+ TASK_QUEUE.task_done()
193
+
194
+ # -------------------- Unified SSD call with failover --------------------
195
+ def generate_with_ssd(self, prompt, num_predict=600, timeout=120):
196
+ output = ""
197
+ backends_to_try = ["http", "cli"] if self._ssd_backend == "http" else ["cli", "http"]
198
+ for backend in backends_to_try:
199
+ if backend == "http":
200
+ try:
201
+ payload = {"model": self.model, "prompt": prompt, "stream": False,
202
+ "enable_thinking": False, "options": {"num_predict": num_predict, "temperature": 0.2}}
203
+ req = urllib.request.Request(self.endpoint, data=json.dumps(payload).encode("utf-8"),
204
+ headers={"Content-Type": "application/json"}, method="POST")
205
+ with urllib.request.urlopen(req, timeout=timeout) as resp:
206
+ output = resp.read().decode("utf-8")
207
+ self._ssd_backend = "http"
208
+ print("[LOG] HTTP backend succeeded.")
209
+ break
210
+ except Exception as e:
211
+ print(f"[WARN] HTTP backend failed: {e}")
212
+ output = f"❌ HTTP failed: {e}"
213
+ continue
214
+ elif backend == "cli":
215
+ try:
216
+ result = subprocess.run(["ollama", "run", self.model, prompt],
217
+ capture_output=True, text=True, timeout=600)
218
+ output = result.stdout.strip() if result.stdout else result.stderr.strip()
219
+ self._ssd_backend = "cli"
220
+ print("[LOG] CLI backend succeeded.")
221
+ break
222
+ except subprocess.TimeoutExpired:
223
+ output = "⏱️ CLI timed out."
224
+ print("[WARN] CLI backend timed out.")
225
+ except Exception as e:
226
+ output = f"❌ CLI failed: {e}"
227
+ print(f"[WARN] CLI backend failed: {e}")
228
+ continue
229
+
230
+ self.memory_data["last_raw_response"] = output
231
+ try:
232
+ data = json.loads(output)
233
+ except json.JSONDecodeError:
234
+ data = {"response": output}
235
+
236
+ full_text = data.get("response", output)
237
+ code_blocks = re.findall(r"```(?:python)?\s*(.*?)```", full_text, re.S | re.I)
238
+ code = "\n\n".join(code_blocks)
239
+ code = textwrap.dedent(code).replace("\t", " ").strip()
240
+ self.memory_data["last_code"] = code
241
+ return full_text
pygmyclaw_multitool.py ADDED
@@ -0,0 +1,210 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ #!/usr/bin/env python3
2
+ """
3
+ PygmyClaw Multitool – Contains the actual tool implementations.
4
+ Works both as:
5
+ 1) CLI tool (stdin → stdout JSON)
6
+ 2) Python module (import + run_tool())
7
+ """
8
+
9
+ import json
10
+ import sys
11
+ import os
12
+ import platform
13
+ import time
14
+ from pathlib import Path
15
+
16
+ SCRIPT_DIR = Path(__file__).parent.resolve()
17
+
18
+ ERROR_LOG = SCRIPT_DIR / "error_log.json"
19
+
20
+ MAX_LOG_ENTRIES = 1000
21
+
22
+
23
+ # ----------------------------------------------------------------------
24
+ # Tool definitions
25
+
26
+ TOOLS = {
27
+ "list_tools_detailed": {
28
+ "name": "list_tools_detailed",
29
+ "description": "List all available tools with their descriptions and parameters.",
30
+ "parameters": {},
31
+ "func": "do_list_tools"
32
+ },
33
+
34
+ "sys_info": {
35
+ "name": "sys_info",
36
+ "description": "Get system information (OS, Python version, etc.).",
37
+ "parameters": {},
38
+ "func": "do_sys_info"
39
+ },
40
+
41
+ "log_error": {
42
+ "name": "log_error",
43
+ "description": "Log an error message to the error log.",
44
+ "parameters": {
45
+ "msg": "string",
46
+ "trace": "string (optional)"
47
+ },
48
+ "func": "do_log_error"
49
+ },
50
+
51
+ "echo": {
52
+ "name": "echo",
53
+ "description": "Echo the input text (for testing).",
54
+ "parameters": {"text": "string"},
55
+ "func": "do_echo"
56
+ }
57
+ }
58
+
59
+
60
+ # ----------------------------------------------------------------------
61
+ # Tool Implementations
62
+
63
+
64
+ def do_list_tools():
65
+ """Return the list of tools with metadata."""
66
+
67
+ tools_list = []
68
+
69
+ for name, info in TOOLS.items():
70
+
71
+ tools_list.append({
72
+ "name": name,
73
+ "description": info["description"],
74
+ "parameters": info["parameters"]
75
+ })
76
+
77
+ return {"tools": tools_list}
78
+
79
+
80
+ def do_sys_info():
81
+ """Return system information."""
82
+
83
+ return {
84
+ "os": platform.system(),
85
+ "os_release": platform.release(),
86
+ "python_version": platform.python_version(),
87
+ "hostname": platform.node()
88
+ }
89
+
90
+
91
+ def do_log_error(msg, trace=""):
92
+ """Append an error to the error log."""
93
+
94
+ entry = {
95
+ "timestamp": time.time(),
96
+ "msg": msg,
97
+ "trace": trace
98
+ }
99
+
100
+ try:
101
+
102
+ if ERROR_LOG.exists():
103
+
104
+ with open(ERROR_LOG) as f:
105
+ log = json.load(f)
106
+
107
+ else:
108
+
109
+ log = []
110
+
111
+ log.append(entry)
112
+
113
+ if len(log) > MAX_LOG_ENTRIES:
114
+ log = log[-MAX_LOG_ENTRIES:]
115
+
116
+ with open(ERROR_LOG, "w") as f:
117
+ json.dump(log, f, indent=2)
118
+
119
+ return {"status": "logged"}
120
+
121
+ except Exception as e:
122
+
123
+ return {"error": f"Failed to write log: {e}"}
124
+
125
+
126
+ def do_echo(text):
127
+ """Echo input."""
128
+
129
+ return {"echo": text}
130
+
131
+
132
+ # ----------------------------------------------------------------------
133
+ # INTERNAL TOOL EXECUTOR (used by PygmyClaw agent)
134
+
135
+ def run_tool(action, **params):
136
+ """
137
+ Run tool programmatically.
138
+
139
+ Example:
140
+ run_tool("sys_info")
141
+ run_tool("echo", text="hello")
142
+ """
143
+
144
+ tool = TOOLS.get(action)
145
+
146
+ if not tool:
147
+ return {"error": f"Unknown tool '{action}'"}
148
+
149
+ func_name = tool["func"]
150
+
151
+ try:
152
+
153
+ if func_name == "do_list_tools":
154
+ return do_list_tools()
155
+
156
+ elif func_name == "do_sys_info":
157
+ return do_sys_info()
158
+
159
+ elif func_name == "do_log_error":
160
+ return do_log_error(
161
+ params.get("msg"),
162
+ params.get("trace", "")
163
+ )
164
+
165
+ elif func_name == "do_echo":
166
+ return do_echo(params.get("text"))
167
+
168
+ else:
169
+ return {"error": f"Unknown function '{func_name}'"}
170
+
171
+ except Exception as e:
172
+
173
+ return {"error": f"Tool execution failed: {e}"}
174
+
175
+
176
+ # ----------------------------------------------------------------------
177
+ # CLI Dispatcher (existing behavior preserved)
178
+
179
+ def main():
180
+
181
+ try:
182
+
183
+ raw = sys.stdin.read()
184
+
185
+ if not raw:
186
+ print(json.dumps({"error": "No input"}))
187
+ return
188
+
189
+ data = json.loads(raw)
190
+
191
+ action = data.get("action")
192
+
193
+ if not action:
194
+
195
+ print(json.dumps({"error": "No action specified"}))
196
+ return
197
+
198
+ params = {k: v for k, v in data.items() if k != "action"}
199
+
200
+ result = run_tool(action, **params)
201
+
202
+ print(json.dumps(result))
203
+
204
+ except Exception as e:
205
+
206
+ print(json.dumps({"error": f"Multitool exception: {e}"}))
207
+
208
+
209
+ if __name__ == "__main__":
210
+ main()
requirements.txt ADDED
@@ -0,0 +1,8 @@
 
 
 
 
 
 
 
 
 
1
+ streamlit>=1.28.0
2
+ requests>=2.28.0
3
+
4
+ requests>=2.31.0
5
+ redis>=5.0.0
6
+ ollama-python>=0.1.0
7
+ gradio==4.44.0
8
+ huggingface_hub==0.23.5
ui.py ADDED
@@ -0,0 +1,38 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ import streamlit as st
2
+ import subprocess
3
+ from pathlib import Path
4
+ import json
5
+
6
+ st.set_page_config(page_title="PygmyClaw", layout="wide")
7
+ st.title("🦾 PygmyClaw Autonomous Agent")
8
+
9
+ # Workspace
10
+ workspace = Path("/workspace/data")
11
+ workspace.mkdir(exist_ok=True)
12
+
13
+ # User input
14
+ prompt = st.text_area("Enter your prompt:", height=100)
15
+
16
+ if st.button("Submit Prompt"):
17
+ st.info("Processing...")
18
+ # Call pygmyclaw agent
19
+ result = subprocess.run(
20
+ ["python3", "pygmyclaw.py", "generate", prompt],
21
+ capture_output=True, text=True
22
+ )
23
+ st.success("Done!")
24
+ st.code(result.stdout, language="text")
25
+
26
+ # Optional: code editor + run
27
+ code_files = list(workspace.glob("*.py"))
28
+ if code_files:
29
+ st.subheader("Generated Python Scripts")
30
+ for f in code_files:
31
+ code = f.read_text()
32
+ edited_code = st.text_area(f.name, code, height=200)
33
+ if st.button(f"Run {f.name}"):
34
+ exec_result = subprocess.run(
35
+ ["python3", "-c", edited_code],
36
+ capture_output=True, text=True
37
+ )
38
+ st.code(exec_result.stdout + "\n" + exec_result.stderr)