slm_code_interpreter/README.md

SLM Code Interpreter

A lightweight, CPU-optimized local Python Code Interpreter agent powered by a local Small Language Model (SLM) running via ONNX Runtime GenAI. It is built to support a self-correcting feedback loop in an isolated, sandboxed Python environment, enabling small models (1.5B) to execute Python scripts, catch runtime exceptions, and autonomously correct syntax/semantic errors iteratively.

---

Features

• Local Python Execution: Executes scripts inside a separate, secure subprocess environment with resource bounds (timeout limits).
• Self-Correcting Loop: If execution fails, the agent takes the stack trace, traceback, or stderr output, feeds it back into its context history, and attempts to repair the code automatically up to max_retries.
• Ultra Low RAM Footprint: Runs on standard CPU within ~1.5 GB - 2.0 GB RAM using INT4 quantized Qwen2.5-1.5B-Instruct-ONNX.
• Claude-style Streaming & Thought process: Decodes reasoning thoughts inside <thought> tags before printing the final output block.

---

Installation

In your local project environment:

pip install -e ./slm_code_interpreter

Ensure onnxruntime-genai is installed. It shares the central monorepo model path cached locally at models/qwen2.5-1.5b-onnx.

---

API Reference

SLMCodeInterpreter

from slm_code_interpreter.code_interpreter import SLMCodeInterpreter

interpreter = SLMCodeInterpreter(
    model_path=None,   # Path to the ONNX model directory (defaults to models/qwen2.5-1.5b-onnx)
    cache_dir=None,    # Alternative HF cache dir
    n_ctx=2048,        # Context length (defaults to 2048)
    n_threads=4        # Number of CPU threads to use for execution
)

Methods

run(instruction: str, max_retries: int = 3, stream: bool = False)

Runs the user instruction to write and execute code.

• Arguments:
  • instruction (str): Task instructions (e.g. "Calculate the 10th Fibonacci number").
  • max_retries (int): Number of execution recovery attempts if exceptions occur (default: 3).
  • stream (bool): If True, returns a generator that yields decoded output tokens in real-time. If False, runs the self-correction loop to completion.
• Returns:
  • dict (when stream=False):

    {
        "success": True/False,
        "stdout": str,       # Process standard output
        "stderr": str,       # Process errors (or traceback summary if failed)
        "code": str,         # Executed Python source code
        "attempts": int,     # Count of turns taken to complete
        "response": str      # Raw text response generated by model
    }
    

  • Generator (when stream=True): Token yield generator.

---

Usage Examples

1. Basic Generation and Execution

from slm_code_interpreter.code_interpreter import SLMCodeInterpreter

interpreter = SLMCodeInterpreter()

# The interpreter will generate the Python code, execute it, and return output
result = interpreter.run("Write a python script to compute the 10th Fibonacci number and print it.")

print(f"Success: {result['success']}")
print(f"Executed Code:\n{result['code']}")
print(f"Stdout Output: {result['stdout']}")

2. Sandbox Subprocess Timeout Limits

The interpreter limits runtime scripts (default: 10s timeout) to prevent infinite loops from locking up the system:

# Execute python script that loops infinitely
res = interpreter._execute_sandbox("import time\nwhile True:\n    time.sleep(0.1)", timeout=1.0)
print(res[0]) # Output: -1 (Execution timeout code)
print(res[2]) # Output: Execution Timeout Expired.

3. Agentic Self-Correction Loop In Action

When an exception occurs (like a NameError or SyntaxError), the agent gets the error traceback back in its prompt history, and fixes it:

# Reference a missing variable manually to trigger correction loop
result = interpreter.run(
    "Write a python script that references an undefined variable `non_existent_var` first, "
    "catches the error, but eventually prints 'Recovered Output'",
    max_retries=3
)

print(f"Attempts: {result['attempts']}")  # Recovered in 1 or more runs depending on model response
print(f"Stdout: {result['stdout']}")       # Output: Recovered Output

4. Complex Data Processing Example

The interpreter agent can handle robust multi-stage scripts using advanced third-party libraries (e.g. pandas, numpy, matplotlib) for data munging:

from slm_code_interpreter.code_interpreter import SLMCodeInterpreter

interpreter = SLMCodeInterpreter()

query = (
    "Load raw CSV text with employee records containing department, sales, and dates. "
    "Parse the dates, extract the quarter, group by department and quarter, "
    "aggregate total sales, filter out combinations below $40,000, and print "
    "a structured markdown summary table."
)

result = interpreter.run(query)

print("Success Status:", result["success"])
print("Generated Code:\n", result["code"])
print("Execution Output:\n", result["stdout"])

Generated Python Script:

import pandas as pd
from io import StringIO

csv_data = """date,department,revenue
2026-01-15,Sales,32000
2026-02-10,Marketing,15000
2026-03-01,Sales,45000
2026-04-12,Engineering,60000
2026-05-18,Marketing,45000
2026-06-22,Sales,12000
"""

df = pd.read_csv(StringIO(csv_data))
df['date'] = pd.to_datetime(df['date'])
df['quarter'] = df['date'].dt.to_period('Q')

# Aggregate and group
agg = df.groupby(['department', 'quarter'])['revenue'].sum().reset_index()
filtered = agg[agg['revenue'] >= 40000]

print(filtered.to_markdown(index=False))

---

Configuration (config.yaml)

Specify settings inside the project directory:

models:
  code_interpreter:
    path: "../../models/qwen2.5-1.5b-onnx"
    repo_id: "tonythethompson/Qwen2.5-1.5B-Instruct-ONNX"

---

🔌 VS Code Integration Guide

You can integrate the SLM Code Interpreter directly inside Visual Studio Code to execute highlighted text prompts or code sections locally on your CPU.

Step 1: Start the Background Daemon Server

Start the local HTTP JSON API server on port 8085:

# Option A: Run directly from python module
python -m slm_code_interpreter.server

# Option B: Run programmatically from code
from slm_code_interpreter import run_server
run_server(port=8085)

The server will output: [SLMCodeInterpreter] Local VS Code integration server active on http://127.0.0.1:8085

Step 2: Install the VS Code Extension Blueprint

The package includes a lightweight, pre-configured VS Code extension folder located at vscode-extension/.

1. Open the vscode-extension folder in VS Code.
2. Press F5 to start a new VS Code debug window with the extension activated.
3. In the new window, select any text or code prompt, right-click, and select: "SLM Code Interpreter: Execute Selected Prompt / Code"
4. Alternately, open the Command Palette (Cmd+Shift+P on Mac / Ctrl+Shift+P on Windows) and search for: "SLM Code Interpreter: Ask Agent to Write & Run..."
5. All reasoning traces, code, stdout outputs, and errors will be printed in real-time inside the VS Code Output Channel (under the "SLM Code Interpreter" filter).