README.md

metadata

title: Space Manufacturing RL
emoji: 🏭
colorFrom: yellow
colorTo: indigo
sdk: docker
app_port: 8000
base_path: /web
tags:
  - openenv
pinned: false

Space Manufacturing Reinforcement Learning Environment

This repository contains an OpenEnv submission for autonomous space manufacturing operations. Agents must manage a network of orbital manufacturing units that process raw materials, assemble components, and coordinate delivery missions across three deterministic task presets.

Overview And Motivation

This environment models a complex operations problem: orchestrating a set of orbital manufacturing platforms that must balance raw material intake, production scheduling, energy management, and finished-goods delivery under shifting demand and solar-power availability.

It is intended as a meaningful agent benchmark because good performance requires:

• balancing production throughput against energy and storage constraints
• scheduling assembly and delivery across multiple platforms simultaneously
• avoiding idle waste and resource starvation across long multi-step horizons
• adapting strategy as platform count, product complexity, and delivery windows scale from easy to hard

What Is Included

• Canonical environment package: space_manufacturing/
• Typed Pydantic models for observation, action, reward, and state
• Three built-in tasks: easy, medium, hard
• Deterministic task grader returning scores from 0.0 to 1.0
• Reward shaping for throughput, efficiency, and penalty signals
• Baseline inference runner at inference.py
• Local validator-ready OpenEnv app manifest at space_manufacturing/openenv.yaml

Task Progression

Task	Platforms	Max Steps	Products	Main Difficulty
Easy	2	60	4 orders	Basic production scheduling
Medium	4	120	10 mixed orders	Assembly + delivery balancing
Hard	6	240	20 mixed orders	Power pressure, multi-platform coordination, strict grading

The progression is explicit in both configuration and grading:

• easy focuses on simple single-product manufacturing and energy health
• medium adds multi-stage assembly and delivery missions
• hard increases platform count, order variety, solar-power fluctuation, and invalid-action sensitivity

Canonical API

Use ManufacturingTaskEnv for the submission-facing environment API:

from space_manufacturing import ManufacturingAction, ManufacturingTaskEnv

env = ManufacturingTaskEnv(task_name="medium")
observation = env.reset()

observation, reward, done, info = env.step(
    ManufacturingAction(platform_actions={0: "produce", 1: "assemble"})
)

state = env.state()

Key methods:

• reset() -> ManufacturingObservation
• step(action) -> (ManufacturingObservation, ManufacturingReward, done, info)
• state() -> ManufacturingEnvState
• ManufacturingTaskEnv.list_tasks() -> Dict[str, str]

Action And Observation Spaces

Action Space

The action space is a typed ManufacturingAction object with one command per platform:

ManufacturingAction(
    platform_actions={
        0: "produce",
        1: "assemble",
        2: "deliver",
        3: "recharge",
    }
)

Allowed actions:

• produce: process raw materials into components using the platform's fabricator
• assemble: combine components into a finished product from the assembly queue
• deliver: transmit a completed product to satisfy an open delivery order
• recharge: enter low-power recovery mode to restore solar-charged energy reserves

For HTTP POST /step, send the action inside the OpenEnv step wrapper:

{
  "action": {
    "platform_actions": {
      "0": "produce",
      "1": "assemble",
      "2": "deliver"
    }
  },
  "timeout_s": 30
}

Observation Space

The observation space is a typed ManufacturingObservation object containing:

• platforms: per-platform state with id, position, energy, material_stock, component_stock, product_stock, and last_action
• time_step: current step in the episode
• delivery_windows: active delivery order windows (order id, product type, deadline)
• solar_conditions: solar irradiance by orbital zone (affects recharge efficiency)
• pending_orders: currently visible production orders
• total_reward: cumulative reward so far
• reward: immediate reward from the latest step
• done: whether the episode has ended
• metadata: step metadata including reward components and metrics

Reward Model

Rewards are shaped during the trajectory, not only at the end:

• positive reward for completing production and assembly orders
• additional reward for on-time deliveries within open windows
• moderate reward for timely recharging before energy becomes critical
• penalties for invalid actions (e.g. assemble with no components, deliver with no product)
• penalties for repeated wasteful or no-op actions
• penalties for critically low energy or overfull storage states
• mild penalty for unproductive idling when actionable work is available

Grading

ManufacturingTaskGrader scores episodes deterministically from environment metrics, including:

• completed production runs
• assembled products
• on-time deliveries
• final average energy level
• invalid-action rate

Local Setup

Create the local virtualenv and install the OpenEnv runtime:

python3 -m venv .venv
.venv/bin/pip install "openenv-core[core]"

Validate The Environment

The OpenEnv environment root is space_manufacturing/, not the repo root.

Use either:

.venv/bin/openenv validate space_manufacturing

or:

cd space_manufacturing
../.venv/bin/openenv validate .

Baseline Inference

The baseline script evaluates all three tasks and prints:

• per-task score
• per-task reward
• per-task step count
• final aggregate score

Set:

export HF_TOKEN="your-token"
export MODEL_NAME="your-model"
export API_BASE_URL="https://router.huggingface.co/v1"
python3 inference.py

The script uses the OpenAI Python client and reads credentials from HF_TOKEN.

Reproducible Baseline Scores

The repository also supports a deterministic heuristic baseline that can be reproduced locally without a remote model:

BASELINE_POLICY=heuristic python3 inference.py

Project Structure

SpaceManufacturingRL/
├── inference.py
├── space_manufacturing/
│   ├── __init__.py
│   ├── factory.py
│   ├── env.py
│   ├── graders.py
│   ├── models.py
│   ├── openenv.yaml
│   ├── pyproject.toml
│   ├── tasks.py
│   └── server/
│       ├── __init__.py
│       └── app.py
├── requirements.txt
└── Readme.md