outline.md

Demo outline

Open these tabs first

Local resources:

• pres/index.html
• pres/training_results.html
• pres/trajectory.html
• pres/training_script.html
• pres/reward_curve.png
• pres/before_after_reward.png

Remote resources:

• HF Space repo: https://huggingface.co/spaces/thomasm6m6/freeciv_env
• HF Space app: https://thomasm6m6-freeciv-env.hf.space

Supporting files:

• reward data: pres/reward_steps.csv
• training script: scripts/train_grpo_fast.py
• env config: openenv.yaml

What we have ready

• real OpenEnv environment for Freeciv
• real live backend on H100 via Freeciv Web
• successful GRPO training run on the live backend
• reward curve PNG
• before/after reward PNG
• live trajectory page with real observations + legal actions
• note: use reward improvement as the before/after story; raw checkpoint-to-checkpoint action examples were too noisy to be worth showing live
• minimal training script page
• HF Space deployed: thomasm6m6/freeciv_env

What not to spend time on

• long architecture explanation
• low-level websocket/runtime debugging
• model internals
• many charts

Use the product demo + reward improvement as the center of the pitch.

---

1 minute YouTube flow

0:00–0:10

Open: pres/trajectory.html

Say:

• We built a real OpenEnv environment for Freeciv, a long-horizon strategy game.
• The model sees text observations and legal actions, and acts turn by turn against a live backend.

0:10–0:22

Stay on pres/trajectory.html

Say:

• This is not a toy prompt task.
• It has delayed reward, persistent world state, multiple units, city-building, and long-horizon planning.
• That maps directly to the hackathon’s long-horizon planning and world-modeling tracks.

0:22–0:38

Switch to pres/training_script.html

Say:

• We also built the minimal RL training loop with Unsloth + TRL GRPO.
• The script collects live Freeciv states, formats them into prompts, and trains a policy on the real environment.

0:38–0:55

Switch to pres/training_results.html

Say:

• We ran training on the H100 against the live Freeciv backend.
• Reward improved from 0.125 at the start to 1.0 by the end of the run.
• This gives observable training progress, which is the key hackathon requirement.

0:55–1:00

Optional final cut to HF Space repo URL

Say:

• The environment is packaged as OpenEnv and deployed to Hugging Face Spaces for submission.

---

3 minute live pitch flow

0:00–0:25 — problem

Open: pres/trajectory.html

Say:

• We wanted a real LLM RL environment for long-horizon strategic planning.
• Freeciv is a strong fit because it has persistent state, delayed reward, many legal actions, and requires planning across turns.

0:25–1:05 — show the environment

Stay on pres/trajectory.html

Point out:

• text-first observation
• legal actions
• units / cities / economy summaries
• live backend on H100

Say:

• The agent does not get a canned benchmark prompt.
• It interacts with a real running world and must choose from legal actions each turn.

1:05–1:35 — show the training loop

Open: pres/training_script.html

Say:

• This is the minimal GRPO loop.
• We use live Freeciv sessions, prepare observations, build prompts, and train with Unsloth + TRL.
• The important thing is that the training loop is small and actually runs on the real backend.

1:35–2:25 — show training improvement

Open: pres/training_results.html

Say:

• This is the core result.
• Reward increases over training steps on real Freeciv states.
• Start: 0.125. End: 1.0.
• This is the evidence that the environment and reward pipeline are coherent enough to drive learning.

If short on time, only show:

• reward curve
• before/after reward bars

2:25–2:50 — why this matters

Stay on pres/training_results.html

Say:

• This fits Statement 2: long-horizon planning.
• It also fits Statement 3.1: world modeling, because the agent interacts with a real dynamic system and must maintain state over time.

2:50–3:00 — close

Open: HF Space repo URL or pres/index.html

Say:

• The environment is packaged in OpenEnv, runs with a real backend, has a minimal RL script, and already shows reward improvement.

---

Likely Q/A answers

Why Freeciv?

• It is long-horizon, strategic, partially observable, and naturally multi-step.
• It is much closer to real planning than one-shot QA.

What exactly is the observation/action interface?

• Observation is text-first: turn summary, economy, units, cities, map, legal actions.
• Actions are structured: end turn, move unit, build city, set city production, set research.

Is the backend real?

• Yes. Training was run against a live Freeciv Web backend on the H100.

What evidence do you have that training worked?

• The reward curve in pres/training_results.html.
• It rises from 0.125 to 1.0 during the live run.

Why not show a bigger model?

• For the hackathon, reliability and observable reward improvement mattered more than model scale.
• A smaller model let us get an end-to-end live run working on the real backend.

What is still incomplete?

• The environment currently exposes a small action subset rather than the full Freeciv action surface.
• The main accomplishment is that live interaction and RL training now work end to end.

---

If something breaks during the pitch

Fallback tab order:

1. pres/training_results.html
2. pres/trajectory.html
3. pres/training_script.html
4. HF Space repo URL

If the live environment demo is flaky, just narrate from the trajectory page and go straight to the reward curve.