README.md

---
title: Space Manufacturing RL
emoji: "🏭"
colorFrom: yellow
colorTo: indigo
sdk: docker
app_port: 8000
base_path: /web
tags:
  - openenv
pinned: false
---
# Space Manufacturing Reinforcement Learning Environment

This repository contains an OpenEnv submission for autonomous space manufacturing operations. Agents must manage a network of orbital manufacturing units that process raw materials, assemble components, and coordinate delivery missions across three deterministic task presets.

## Overview And Motivation

This environment models a complex operations problem: orchestrating a set of orbital manufacturing platforms that must balance raw material intake, production scheduling, energy management, and finished-goods delivery under shifting demand and solar-power availability.

It is intended as a meaningful agent benchmark because good performance requires:
- balancing production throughput against energy and storage constraints
- scheduling assembly and delivery across multiple platforms simultaneously
- avoiding idle waste and resource starvation across long multi-step horizons
- adapting strategy as platform count, product complexity, and delivery windows scale from easy to hard

## What Is Included

- Canonical environment package: `space_manufacturing/`
- Typed Pydantic models for observation, action, reward, and state
- Three built-in tasks: `easy`, `medium`, `hard`
- Deterministic task grader returning scores from `0.0` to `1.0`
- Reward shaping for throughput, efficiency, and penalty signals
- Baseline inference runner at `inference.py`
- Local validator-ready OpenEnv app manifest at `space_manufacturing/openenv.yaml`

## Task Progression

| Task   | Platforms | Max Steps | Products | Main Difficulty |
|--------|-----------|-----------|----------|-----------------|
| Easy   | 2         | 60        | 4 orders | Basic production scheduling |
| Medium | 4         | 120       | 10 mixed orders | Assembly + delivery balancing |
| Hard   | 6         | 240       | 20 mixed orders | Power pressure, multi-platform coordination, strict grading |

The progression is explicit in both configuration and grading:
- `easy` focuses on simple single-product manufacturing and energy health
- `medium` adds multi-stage assembly and delivery missions
- `hard` increases platform count, order variety, solar-power fluctuation, and invalid-action sensitivity

## Canonical API

Use `ManufacturingTaskEnv` for the submission-facing environment API:

```python
from space_manufacturing import ManufacturingAction, ManufacturingTaskEnv

env = ManufacturingTaskEnv(task_name="medium")
observation = env.reset()

observation, reward, done, info = env.step(
    ManufacturingAction(platform_actions={0: "produce", 1: "assemble"})
)

state = env.state()
```

Key methods:
- `reset() -> ManufacturingObservation`
- `step(action) -> (ManufacturingObservation, ManufacturingReward, done, info)`
- `state() -> ManufacturingEnvState`
- `ManufacturingTaskEnv.list_tasks() -> Dict[str, str]`

## Action And Observation Spaces

### Action Space

The action space is a typed `ManufacturingAction` object with one command per platform:

```python
ManufacturingAction(
    platform_actions={
        0: "produce",
        1: "assemble",
        2: "deliver",
        3: "recharge",
    }
)
```

Allowed actions:
- `produce`: process raw materials into components using the platform's fabricator
- `assemble`: combine components into a finished product from the assembly queue
- `deliver`: transmit a completed product to satisfy an open delivery order
- `recharge`: enter low-power recovery mode to restore solar-charged energy reserves

For HTTP `POST /step`, send the action inside the OpenEnv step wrapper:

```json
{
  "action": {
    "platform_actions": {
      "0": "produce",
      "1": "assemble",
      "2": "deliver"
    }
  },
  "timeout_s": 30
}
```

### Observation Space

The observation space is a typed `ManufacturingObservation` object containing:
- `platforms`: per-platform state with `id`, `position`, `energy`, `material_stock`, `component_stock`, `product_stock`, and `last_action`
- `time_step`: current step in the episode
- `delivery_windows`: active delivery order windows (order id, product type, deadline)
- `solar_conditions`: solar irradiance by orbital zone (affects recharge efficiency)
- `pending_orders`: currently visible production orders
- `total_reward`: cumulative reward so far
- `reward`: immediate reward from the latest step
- `done`: whether the episode has ended
- `metadata`: step metadata including reward components and metrics

## Reward Model

Rewards are shaped during the trajectory, not only at the end:
- positive reward for completing production and assembly orders
- additional reward for on-time deliveries within open windows
- moderate reward for timely recharging before energy becomes critical
- penalties for invalid actions (e.g. assemble with no components, deliver with no product)
- penalties for repeated wasteful or no-op actions
- penalties for critically low energy or overfull storage states
- mild penalty for unproductive idling when actionable work is available

## Grading

`ManufacturingTaskGrader` scores episodes deterministically from environment metrics, including:
- completed production runs
- assembled products
- on-time deliveries
- final average energy level
- invalid-action rate

## Local Setup

Create the local virtualenv and install the OpenEnv runtime:

```bash
python3 -m venv .venv
.venv/bin/pip install "openenv-core[core]"
```

## Validate The Environment

The OpenEnv environment root is `space_manufacturing/`, not the repo root.

Use either:

```bash
.venv/bin/openenv validate space_manufacturing
```

or:

```bash
cd space_manufacturing
../.venv/bin/openenv validate .
```

## Baseline Inference

The baseline script evaluates all three tasks and prints:
- per-task score
- per-task reward
- per-task step count
- final aggregate score

Set:

```bash
export HF_TOKEN="your-token"
export MODEL_NAME="your-model"
export API_BASE_URL="https://router.huggingface.co/v1"
python3 inference.py
```

The script uses the OpenAI Python client and reads credentials from `HF_TOKEN`.

### Reproducible Baseline Scores

The repository also supports a deterministic heuristic baseline that can be reproduced locally without a remote model:

```bash
BASELINE_POLICY=heuristic python3 inference.py
```

## Project Structure

```text
SpaceManufacturingRL/
├── inference.py
├── space_manufacturing/
│   ├── __init__.py
│   ├── factory.py
│   ├── env.py
│   ├── graders.py
│   ├── models.py
│   ├── openenv.yaml
│   ├── pyproject.toml
│   ├── tasks.py
│   └── server/
│       ├── __init__.py
│       └── app.py
├── requirements.txt
└── Readme.md
```