slm_code_interpreter/README.md

# SLM Code Interpreter

A lightweight, CPU-optimized local Python Code Interpreter agent powered by a local Small Language Model (SLM) running via ONNX Runtime GenAI. It is built to support a self-correcting feedback loop in an isolated, sandboxed Python environment, enabling small models (1.5B) to execute Python scripts, catch runtime exceptions, and autonomously correct syntax/semantic errors iteratively.

---

## Features

- **Local Python Execution**: Executes scripts inside a separate, secure subprocess environment with resource bounds (timeout limits).
- **Self-Correcting Loop**: If execution fails, the agent takes the stack trace, traceback, or stderr output, feeds it back into its context history, and attempts to repair the code automatically up to `max_retries`.
- **Ultra Low RAM Footprint**: Runs on standard CPU within ~1.5 GB - 2.0 GB RAM using INT4 quantized Qwen2.5-1.5B-Instruct-ONNX.
- **Claude-style Streaming & Thought process**: Decodes reasoning thoughts inside `<thought>` tags before printing the final output block.

---

## Installation

In your local project environment:
```bash
pip install -e ./slm_code_interpreter
```

Ensure `onnxruntime-genai` is installed. It shares the central monorepo model path cached locally at `models/qwen2.5-1.5b-onnx`.

---

## API Reference

### `SLMCodeInterpreter`

```python
from slm_code_interpreter.code_interpreter import SLMCodeInterpreter

interpreter = SLMCodeInterpreter(
    model_path=None,   # Path to the ONNX model directory (defaults to models/qwen2.5-1.5b-onnx)
    cache_dir=None,    # Alternative HF cache dir
    n_ctx=2048,        # Context length (defaults to 2048)
    n_threads=4        # Number of CPU threads to use for execution
)
```

#### Methods

#### `run(instruction: str, max_retries: int = 3, stream: bool = False)`
Runs the user instruction to write and execute code.
- **Arguments**:
  - `instruction` (str): Task instructions (e.g. "Calculate the 10th Fibonacci number").
  - `max_retries` (int): Number of execution recovery attempts if exceptions occur (default: 3).
  - `stream` (bool): If `True`, returns a generator that yields decoded output tokens in real-time. If `False`, runs the self-correction loop to completion.
- **Returns**:
  - `dict` (when `stream=False`):
    ```python
    {
        "success": True/False,
        "stdout": str,       # Process standard output
        "stderr": str,       # Process errors (or traceback summary if failed)
        "code": str,         # Executed Python source code
        "attempts": int,     # Count of turns taken to complete
        "response": str      # Raw text response generated by model
    }
    ```
  - `Generator` (when `stream=True`): Token yield generator.

---

## Usage Examples

### 1. Basic Generation and Execution
```python
from slm_code_interpreter.code_interpreter import SLMCodeInterpreter

interpreter = SLMCodeInterpreter()

# The interpreter will generate the Python code, execute it, and return output
result = interpreter.run("Write a python script to compute the 10th Fibonacci number and print it.")

print(f"Success: {result['success']}")
print(f"Executed Code:\n{result['code']}")
print(f"Stdout Output: {result['stdout']}")
```

### 2. Sandbox Subprocess Timeout Limits
The interpreter limits runtime scripts (default: 10s timeout) to prevent infinite loops from locking up the system:
```python
# Execute python script that loops infinitely
res = interpreter._execute_sandbox("import time\nwhile True:\n    time.sleep(0.1)", timeout=1.0)
print(res[0]) # Output: -1 (Execution timeout code)
print(res[2]) # Output: Execution Timeout Expired.
```

### 3. Agentic Self-Correction Loop In Action
When an exception occurs (like a NameError or SyntaxError), the agent gets the error traceback back in its prompt history, and fixes it:
```python
# Reference a missing variable manually to trigger correction loop
result = interpreter.run(
    "Write a python script that references an undefined variable `non_existent_var` first, "
    "catches the error, but eventually prints 'Recovered Output'",
    max_retries=3
)

print(f"Attempts: {result['attempts']}")  # Recovered in 1 or more runs depending on model response
print(f"Stdout: {result['stdout']}")       # Output: Recovered Output
```

### 4. Complex Data Processing Example
The interpreter agent can handle robust multi-stage scripts using advanced third-party libraries (e.g. `pandas`, `numpy`, `matplotlib`) for data munging:

```python
from slm_code_interpreter.code_interpreter import SLMCodeInterpreter

interpreter = SLMCodeInterpreter()

query = (
    "Load raw CSV text with employee records containing department, sales, and dates. "
    "Parse the dates, extract the quarter, group by department and quarter, "
    "aggregate total sales, filter out combinations below $40,000, and print "
    "a structured markdown summary table."
)

result = interpreter.run(query)

print("Success Status:", result["success"])
print("Generated Code:\n", result["code"])
print("Execution Output:\n", result["stdout"])
```

#### Generated Python Script:
```python
import pandas as pd
from io import StringIO

csv_data = """date,department,revenue
2026-01-15,Sales,32000
2026-02-10,Marketing,15000
2026-03-01,Sales,45000
2026-04-12,Engineering,60000
2026-05-18,Marketing,45000
2026-06-22,Sales,12000
"""

df = pd.read_csv(StringIO(csv_data))
df['date'] = pd.to_datetime(df['date'])
df['quarter'] = df['date'].dt.to_period('Q')

# Aggregate and group
agg = df.groupby(['department', 'quarter'])['revenue'].sum().reset_index()
filtered = agg[agg['revenue'] >= 40000]

print(filtered.to_markdown(index=False))
```

---

## Configuration (`config.yaml`)

Specify settings inside the project directory:
```yaml
models:
  code_interpreter:
    path: "../../models/qwen2.5-1.5b-onnx"
    repo_id: "tonythethompson/Qwen2.5-1.5B-Instruct-ONNX"
```

---

## 🔌 VS Code Integration Guide

You can integrate the **SLM Code Interpreter** directly inside Visual Studio Code to execute highlighted text prompts or code sections locally on your CPU.

### Step 1: Start the Background Daemon Server
Start the local HTTP JSON API server on port `8085`:
```bash
# Option A: Run directly from python module
python -m slm_code_interpreter.server

# Option B: Run programmatically from code
from slm_code_interpreter import run_server
run_server(port=8085)
```
The server will output:
`[SLMCodeInterpreter] Local VS Code integration server active on http://127.0.0.1:8085`

### Step 2: Install the VS Code Extension Blueprint
The package includes a lightweight, pre-configured VS Code extension folder located at [vscode-extension/](file:///Users/revathysuryaprakash/Documents/SLMAgents/slm_code_interpreter/vscode-extension).

1. Open the [vscode-extension](file:///Users/revathysuryaprakash/Documents/SLMAgents/slm_code_interpreter/vscode-extension) folder in VS Code.
2. Press `F5` to start a new VS Code debug window with the extension activated.
3. In the new window, select any text or code prompt, right-click, and select:
   **"SLM Code Interpreter: Execute Selected Prompt / Code"**
4. Alternately, open the Command Palette (`Cmd+Shift+P` on Mac / `Ctrl+Shift+P` on Windows) and search for:
   **"SLM Code Interpreter: Ask Agent to Write & Run..."**
5. All reasoning traces, code, stdout outputs, and errors will be printed in real-time inside the VS Code **Output Channel** (under the "SLM Code Interpreter" filter).