Instructions.md

Overview

Build a deterministic OpenEnv environment for real-world procurement negotiation.

• Must follow OpenEnv API (reset / step / state)
• Must include 3 tasks (easy → medium → hard)
• Must produce deterministic rewards in [0.0, 1.0]
• Must be fully reproducible and deployable

---

Core Requirements

1. Environment

Implement in:

procure_rl/environment.py

• reset(task_id, seed) → initial observation
• step(action) → (observation, reward, done, info)
• state() → internal state

Use typed models from:

procure_rl/models.py

---

2. Tasks (MANDATORY: 3)

Defined in:

procure_rl/environment.py (TASK_CONFIG)

Task	Description
single_issue	price-only negotiation
multi_issue	price + payment tradeoff
adversarial	multi-issue + aggressive opponent

Each must:

• have different difficulty
• run within step limits
• produce score ∈ [0,1]

---

3. Opponent (CRITICAL)

Implemented in:

procure_rl/opponent.py

Requirements:

• deterministic (seeded RNG)
• no LLM usage
• language-sensitive behavior (via keyword detection)

👉 This is what makes LLM useful without breaking reproducibility.

---

4. Reward / Graders

Implemented in:

procure_rl/graders.py

Requirements:

• deterministic

• bounded [0.0, 1.0]

• reflect:

  • deal quality
  • efficiency (rounds)

• no randomness, no LLM

---

5. API Server

Implemented in:

server/app.py

Endpoints:

• /reset
• /step
• /state
• /health

Must return valid JSON and HTTP 200.

---

6. OpenEnv Spec

File:

openenv.yaml

Must define:

• environment name
• tasks (3+)
• reward range
• action/observation description

Validate with:

openenv validate

---

7. Inference Script (MANDATORY)

File:

inference.py

Requirements:

• uses OpenAI client

• reads:

  • API_BASE_URL
  • MODEL_NAME
  • HF_TOKEN

• interacts with env via loop

• prints EXACT format:

[START] ...
[STEP] ...
[END] ...

⚠️ Any formatting deviation → failure

---

8. Docker + Deployment

File:

Dockerfile

Must:

• build successfully
• expose port 7860
• run FastAPI server

Test:

docker build -t procure-rl .
docker run -p 7860:7860 procure-rl

---

9. Hugging Face Space

Must:

• deploy successfully
• respond to /reset with HTTP 200

---

10. README

Must include:

• environment description
• action & observation formats
• task descriptions
• setup instructions
• baseline scores

---

Validation Checklist (ALL REQUIRED)

Run before submission:

openenv validate
docker build .
python inference.py

Ensure:

• all 3 tasks run
• scores ∈ [0,1]
• runtime < 20 minutes
• no crashes

---

Constraints

• No LLM inside environment
• No randomness without seed
• Must run on:
  • 2 vCPU
  • 8GB RAM

---

Key Design Principle

LLM is used for decision-making, not environment logic.

• Environment = deterministic
• Agent (LLM) = intelligent

---

File Reference Summary

procure_rl/
  models.py        # dataclasses
  environment.py   # core logic
  opponent.py      # scripted opponent
  graders.py       # reward functions

server/
  app.py           # API

inference.py       # baseline agent
openenv.yaml       # spec
Dockerfile         # deployment
README.md          # docs

---

Final Rule

If any of these fail:

• Docker build
• openenv validate
• inference script

👉 Submission is disqualified

---

One-line Goal

Build a deterministic, real-world negotiation environment where an LLM agent must make sequential decisions to maximize reward.

---