src/envs/julia_env/server/README.md

Julia Environment Server

HTTP server for executing Julia code with test result tracking and reward calculation.

Overview

This server provides a Julia code execution environment through OpenEnv's HTTP interface. It executes Julia code, parses test results from the Test module, and calculates rewards based on execution success and test outcomes.

Features

• ✅ Execute Julia code in isolated subprocess
• ✅ Parse Test module output (tests passed/failed)
• ✅ Calculate rewards based on execution results
• ✅ Safety transforms for output truncation
• ✅ Docker support for reproducible execution
• ✅ Compatible with GRPO training

Docker Setup

Prerequisites

First, build the OpenEnv base image (one-time setup):

# From OpenEnv root directory
docker build -t openenv-base:latest -f src/core/containers/images/Dockerfile .

Build Julia Environment Image

# From OpenEnv root directory
docker build -t julia-env:latest -f src/envs/julia_env/server/Dockerfile .

Run the Server

# Run in background with default settings (port 8000, 4 workers)
docker run -d -p 8000:8000 --name julia-env-server julia-env:latest

# OR run in foreground (to see logs)
docker run -p 8000:8000 --name julia-env-server julia-env:latest

# Run with custom port
docker run -d -p 9000:9000 -e PORT=9000 --name julia-env-server julia-env:latest

# Run with custom number of workers (uvicorn workers)
docker run -d -p 8000:8000 -e NUM_WORKER=8 --name julia-env-server julia-env:latest

# Run with custom Julia max workers (for process pool)
docker run -d -p 8000:8000 -e JULIA_MAX_WORKERS=32 --name julia-env-server julia-env:latest

# Run with all custom configurations
docker run -d -p 9000:9000 \
  -e PORT=9000 \
  -e NUM_WORKER=8 \
  -e JULIA_MAX_WORKERS=32 \
  --name julia-env-server julia-env:latest

Test the Server

# Health check
curl http://localhost:8000/health
# Expected: {"status":"healthy"}

# Check Julia version inside container
docker exec julia-env-server julia --version
# Expected: julia version 1.10.0

Docker Management Commands

# View logs
docker logs julia-env-server
docker logs -f julia-env-server  # Follow logs

# Stop/start container
docker stop julia-env-server
docker start julia-env-server

# Remove container
docker rm -f julia-env-server

# Rebuild after code changes
docker build -t julia-env:latest -f src/envs/julia_env/server/Dockerfile .
docker rm -f julia-env-server
docker run -d -p 8000:8000 --name julia-env-server julia-env:latest

# Interactive debugging
docker exec -it julia-env-server /bin/bash

Local Development (Without Docker)

Prerequisites

• Python 3.10+
• Julia 1.10.0+ installed and in PATH
• FastAPI and dependencies

Install Julia

Using juliaup (recommended):

curl -fsSL https://install.julialang.org | sh

Or download from: https://julialang.org/downloads/

Install Python Dependencies

pip install fastapi uvicorn

Run Server Locally

# From OpenEnv root directory
export PYTHONPATH="${PWD}/src:${PYTHONPATH}"
python -m envs.julia_env.server.app

Server will start at: http://localhost:8000

API Endpoints

Health Check

GET /health
Response: {"status": "healthy"}

Reset Environment

POST /reset
Response: {
  "observation": {
    "stdout": "",
    "stderr": "",
    "exit_code": 0,
    "tests_passed": 0,
    "tests_failed": 0,
    "reward": 0.0,
    "execution_time": 0.0
  }
}

Execute Code (Step)

POST /step
Body: {"code": "function add(a,b)\n  a+b\nend\nusing Test\n@test add(2,3)==5"}
Response: {
  "observation": {
    "stdout": "Test Passed",
    "stderr": "",
    "exit_code": 0,
    "tests_passed": 1,
    "tests_failed": 0,
    "reward": 1.0,
    "execution_time": 0.15
  },
  "reward": 1.0,
  "done": false
}

Get State

GET /state
Response: {
  "episode_id": "uuid",
  "step_count": 5,
  "last_exit_code": 0,
  "total_tests_passed": 10,
  "total_tests_failed": 2
}

Reward Structure

The environment calculates rewards based on:

• Failed execution (exit_code != 0): -0.5
• Clean execution (exit_code == 0): +0.2
• Tests passed: +0.3 × (passed/total)
• Tests failed: -0.2 × (failed/total)
• All tests passed bonus: +0.5

Example:

# 3 tests pass, 1 fails → exit_code 1
reward = -0.5  # Failed execution
# Total: -0.5

# 3 tests pass, 0 fail → exit_code 0
reward = 0.2 + 0.3 × 1.0 + 0.5 = 1.0
# Total: 1.0 (perfect score!)

Test Parsing

The environment parses Julia's Test module output:

Method 1: Error Message Pattern

Some tests did not pass: 3 passed, 1 failed, 0 errored, 0 broken.
→ tests_passed=3, tests_failed=1

Method 2: Test Summary Table

Test Summary:      | Pass  Fail  Total  Time
Add function Tests |    3     1      4  0.5s
→ tests_passed=3, tests_failed=1

Example Usage

From Python Client

from envs.julia_env import JuliaEnv, JuliaAction

# Connect to server
env = JuliaEnv(base_url="http://localhost:8000")

# Reset
result = env.reset()

# Execute Julia code with tests
code = """
function fibonacci(n)
    if n <= 1
        return n
    end
    return fibonacci(n-1) + fibonacci(n-2)
end

using Test
@test fibonacci(0) == 0
@test fibonacci(1) == 1
@test fibonacci(5) == 5
@test fibonacci(10) == 55
"""

result = env.step(JuliaAction(code=code))

print(f"Exit code: {result.observation.exit_code}")
print(f"Tests passed: {result.observation.tests_passed}")
print(f"Tests failed: {result.observation.tests_failed}")
print(f"Reward: {result.reward}")

# Close connection
env.close()

Example Script

# From OpenEnv root
python examples/julia_simple.py

GRPO Training Integration

This environment is designed for GRPO (Group Relative Policy Optimization) training:

# In your GRPO training loop
async def play_julia_game(game_idx, game_id, server_url, policy, tokenizer):
    env = JuliaEnv(base_url=server_url)

    # Generate code with LLM
    prompt = format_julia_prompt(task)
    responses = await policy.generate.route(prompt)
    code = extract_julia_code(responses[0].text)

    # Execute in environment
    result = env.step(JuliaAction(code=code))

    # Get reward
    reward = result.observation.reward

    return {
        "prompt": prompt,
        "response": responses[0],
        "reward": reward,
        "tests_passed": result.observation.tests_passed,
        "tests_failed": result.observation.tests_failed
    }

See examples/grpo_blackjack/ for a complete GRPO training example that can be adapted for Julia.

Configuration

Docker Environment Variables

The Docker container accepts the following environment variables:

• PORT: HTTP server port (default: 8000)

  • Controls which port the FastAPI server listens on
  • Must match the port mapping in -p flag (e.g., -p 9000:9000 -e PORT=9000)

• NUM_WORKER: Number of uvicorn worker processes (default: 4)

  • Controls parallel request handling capacity
  • More workers = more concurrent requests but higher memory usage
  • Recommended: 2-8 workers for typical workloads

• JULIA_MAX_WORKERS: Maximum Julia process pool size (default: 16)

  • Controls maximum concurrent Julia code executions
  • Higher values allow more parallel Julia executions
  • Each worker consumes memory; tune based on available resources
  • Recommended: 8-32 workers depending on your workload

Runtime Environment Variables

These can be set when running locally (non-Docker):

• HOST: Server host (default: 0.0.0.0)
• JULIA_TIMEOUT: Julia execution timeout in seconds (default: 60)

Dockerfile Customization

To use a different Julia version:

# In Dockerfile, change the version
RUN curl -fsSL https://install.julialang.org | sh -s -- --yes --default-channel 1.11

Troubleshooting

Julia not found

# Verify Julia is in PATH
julia --version

# In Docker, check installation
docker exec julia-env-server julia --version

Port already in use

# Use different port
docker run -p 8001:8000 --name julia-env-server julia-env:latest

# Update client base_url
env = JuliaEnv(base_url="http://localhost:8001")

Container exits immediately

# Check logs
docker logs julia-env-server

# Run in foreground to see errors
docker run -p 8000:8000 julia-env:latest

Build failures

# Clean build with no cache
docker build --no-cache -t julia-env:latest -f src/envs/julia_env/server/Dockerfile .

# Verbose output
docker build --progress=plain -t julia-env:latest -f src/envs/julia_env/server/Dockerfile .

Architecture

┌─────────────────────────────────────┐
│   Python Client (HTTP)              │
│   JuliaEnv                          │
└────────────┬────────────────────────┘
             │ HTTP POST /step
             │ {"code": "..."}
             ▼
┌─────────────────────────────────────┐
│   FastAPI Server                    │
│   app.py                            │
└────────────┬────────────────────────┘
             │
             ▼
┌─────────────────────────────────────┐
│   JuliaCodeActEnv                   │
│   - Execute code via JuliaExecutor  │
│   - Parse test results              │
│   - Calculate rewards               │
│   - Apply transforms                │
└────────────┬────────────────────────┘
             │
             ▼
┌─────────────────────────────────────┐
│   JuliaExecutor (subprocess)        │
│   - Write code to temp file         │
│   - Run: julia temp_file.jl         │
│   - Capture stdout/stderr           │
│   - Return results                  │
└─────────────────────────────────────┘

Development

Running Tests

# Unit tests
pytest tests/envs/julia_env/

# Integration test
python examples/julia_simple.py

Code Structure

server/
├── Dockerfile              # Docker build instructions
├── README.md              # This file
├── __init__.py            # Package initialization
├── app.py                 # FastAPI server entry point
├── julia_codeact_env.py   # Environment implementation
└── julia_transforms.py    # Output transforms

License

BSD-style license. See LICENSE file in repository root.