Fixing minor issues

index.html

@@ -241,7 +241,7 @@
     <a href="#memory-evolution">Memory gallery</a>
     <a href="#results">Results</a>
     <a href="#engineering">Engineering</a>
-    <a href="<INSERT: HF Space URL>" target="_blank">Live Space &#8599;</a>
+    <a href="https://huggingface.co/spaces/yashu2000/MiniGridEnv" target="_blank">Live Space &#8599;</a>
   </div>
 </nav>
 
@@ -252,13 +252,14 @@
     <h1>MiniGridEnv</h1>
     <p class="subtitle">An OpenEnv-native wrap of Farama <strong>MiniGrid/BabyAI</strong> for text-grounded navigation, extended with <strong>cross-episodic, LLM-rewritten markdown memory</strong> and branch-stable GRPO.</p>
     <div class="badges">
-      <a href="<INSERT: GitHub URL>" target="_blank"><img src="https://img.shields.io/badge/GitHub-Repository-181717?logo=github" alt="GitHub"/></a>
-      <a href="<INSERT: HF Space URL>" target="_blank"><img src="https://img.shields.io/badge/HF%20Space-Live%20Demo-FFD21E?logo=huggingface&logoColor=black" alt="HF Space"/></a>
+      <a href="https://github.com/sharma-yash01/MiniGridEnv" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/badge/MiniGridEnv-GitHub-181717?logo=github" alt="MiniGridEnv on GitHub"/></a>
+      <a href="https://github.com/sharma-yash01/MiniGridPT" target="_blank" rel="noopener noreferrer"><img src="https://img.shields.io/badge/MiniGridPT-GitHub-181717?logo=github" alt="MiniGridPT on GitHub"/></a>
+      <a href="https://huggingface.co/spaces/yashu2000/MiniGridEnv" target="_blank"><img src="https://img.shields.io/badge/HF%20Space-Live%20Demo-FFD21E?logo=huggingface&logoColor=black" alt="HF Space"/></a>
       <img src="https://img.shields.io/badge/OpenEnv-Native-4B8BBE" alt="OpenEnv"/>
       <img src="https://img.shields.io/badge/BabyAI-10%20levels-brightgreen" alt="10 BabyAI levels"/>
       <img src="https://img.shields.io/badge/Training-GRPO%20%2B%20Memory-orange" alt="GRPO + Memory"/>
     </div>
-    <div class="byline">AgentX Phase 2 &middot; OpenEnv Challenge Submission &nbsp;|&nbsp; Yashaswi Sharma (University of Southern California) &nbsp;|&nbsp; Defu Cao (USC) &nbsp;|&nbsp; Muyan Weng (USC)</div>
+    <div class="byline">AgentX Phase 2 &middot; OpenEnv Challenge Submission &nbsp;|&nbsp; Yashaswi Sharma (University of Southern California)&nbsp;|&nbsp; Dongze Ye &nbsp;|&nbsp; Defu Cao (USC) &nbsp;|&nbsp; Muyan Weng (USC)</div>
   </div>
 
   <figure class="banner-figure">
@@ -267,8 +268,9 @@
   </figure>
 
   <div class="btn-group">
-    <a class="btn" href="<INSERT: HF Space URL>" target="_blank">Live Environment Space &rarr;</a>
-    <a class="btn btn-outline" href="<INSERT: GitHub URL>" target="_blank">GitHub Repo</a>
+    <a class="btn" href="https://huggingface.co/spaces/yashu2000/MiniGridEnv" target="_blank">Live Environment Space &rarr;</a>
+    <a class="btn btn-outline" href="https://github.com/sharma-yash01/MiniGridEnv" target="_blank" rel="noopener noreferrer">MiniGridEnv (GitHub)</a>
+    <a class="btn btn-outline" href="https://github.com/sharma-yash01/MiniGridPT" target="_blank" rel="noopener noreferrer">MiniGridPT (GitHub)</a>
   </div>
 
   <!-- Table of Contents -->
@@ -327,7 +329,7 @@
         <thead><tr><th>Prior work bucket</th><th>What it does</th><th>What it does not</th></tr></thead>
         <tbody>
           <tr><td><strong>BabyAI / MiniGrid (base)</strong><br><span style="font-size:.85em;color:var(--muted)">Chevalier-Boisvert et al., <a href="https://arxiv.org/abs/1810.08272" target="_blank" style="color:var(--accent2)">arXiv:1810.08272</a> (ICLR 2019); <a href="https://github.com/Farama-Foundation/Minigrid" target="_blank" style="color:var(--accent2)">Farama-Foundation/Minigrid</a></span></td><td>Compositional language-conditioned navigation as a gym environment with a reference bot and a 10-stage difficulty curriculum</td><td>No OpenEnv/WebSocket contract; no text observation; no LLM post-training pipeline; no memory</td></tr>
-          <tr><td><strong>Memory-augmented LLM agents</strong><br><span style="font-size:.85em;color:var(--muted)">Voyager (<a href="<INSERT: Voyager arXiv URL>" target="_blank" style="color:var(--accent2)">arXiv:2305.16291</a>); Reflexion (<a href="<INSERT: Reflexion arXiv URL>" target="_blank" style="color:var(--accent2)">arXiv:2303.11366</a>); Generative Agents (<a href="<INSERT: Generative Agents arXiv URL>" target="_blank" style="color:var(--accent2)">arXiv:2304.03442</a>)</span></td><td>Cross-episode skill libraries, verbal reflection, structured long-term memory, all <em>prompt-engineered</em> at inference time</td><td>No RL post-training; no branch-stable memory semantics under GRPO; not connected to OpenEnv</td></tr>
+          <tr><td><strong>Memory-augmented LLM agents</strong><br><span style="font-size:.85em;color:var(--muted)">Voyager (<a href="https://arxiv.org/abs/2305.16291" target="_blank" style="color:var(--accent2)">arXiv:2305.16291</a>); Reflexion (<a href="https://arxiv.org/abs/2303.11366" target="_blank" style="color:var(--accent2)">arXiv:2303.11366</a>); Generative Agents (<a href="https://arxiv.org/abs/2304.03442" target="_blank" style="color:var(--accent2)">arXiv:2304.03442</a>)</span></td><td>Cross-episode skill libraries, verbal reflection, structured long-term memory, all <em>prompt-engineered</em> at inference time</td><td>No RL post-training; no branch-stable memory semantics under GRPO; not connected to OpenEnv</td></tr>
           <tr><td><strong>RLVR on language environments</strong><br><span style="font-size:.85em;color:var(--muted)">DeepSeekMath / GRPO (<a href="https://arxiv.org/abs/2402.03300" target="_blank" style="color:var(--accent2)">arXiv:2402.03300</a>); TRL &times; OpenEnv (<a href="https://huggingface.co/docs/trl/en/openenv" target="_blank" style="color:var(--accent2)">TRL docs</a>)</span></td><td>Critic-free RL with verifiable rewards; standard WebSocket env contract and `rollout_func`</td><td>No persistent agent state across episodes; no first-class notion of branch-stable rollout chains</td></tr>
           <tr class="novel"><td><strong>MiniGridEnv + MiniGridPT (ours)</strong></td><td>OpenEnv wrap of MiniGrid/BabyAI + GRPO + <em>cross-episodic LLM-rewritten markdown memory</em> + <em>branch-stable per-chain file naming</em></td><td>Not a human study; memory is text-only (no retrieval index)</td></tr>
         </tbody>
@@ -490,7 +492,7 @@ def _temporary_vllm_max_tokens(trainer, max_tokens: int):
     <h3>Branch-stable file naming (per-chain compaction)</h3>
     <p>GRPO runs <em>G</em> parallel completions per prompt, each with its own advantage and gradient contribution. If every slot writes to a uniquely-named file, there's no continuity across optimizer steps, so each memory chain is one episode long. If every slot writes to one shared file, writes race and the signal is mush.</p>
     <p>The solution: <strong>branch-stable naming</strong> <code>rank{R}_br{k}_{base}.md</code> with <code>k = slot_idx % num_generations</code>. The <em>k</em>-th parallel generation maps to a <strong>stable file across optimizer steps</strong>, so branch <em>k</em> after prompt group P1 is the same file used by branch <em>k</em> after prompt group P2. Each of the <em>G</em> GRPO branches builds its own evolving notebook, which is what gives the model a training signal to <em>compact and summarize</em> episode-to-episode.</p>
-    <p>Requires <code>per_device_train_batch_size == num_generations</code> (otherwise multiple groups in one step hit the same <em>k</em> and a one-time <code>UserWarning</code> fires). A third scheme (a single shared file across all slots and ranks) is documented in <code>impl-context/shared-memory-option.md</code> but not landed; it needs a decision about concurrent-writer races.</p>
+    <p>Requires <code>per_device_train_batch_size == num_generations</code> (otherwise multiple groups in one step hit the same <em>k</em> and a one-time <code>UserWarning</code> fires). A third scheme (a single shared file across all slots and ranks) is sketched but not landed; it needs a decision about concurrent-writer races.</p>
 
     <p>Let $M_e \in \mathcal{M}$ denote the memory file (markdown string) at the start of episode $e$, let $\tau_e$ be the trajectory (observations, parsed actions, outcomes), and let $\pi_\theta^{\mathrm{mem}}$ be the same LLM invoked on the post-episode memory-update prompt. The write is a full rewrite followed by a line-budget projection $\Pi_L(\cdot)$ that keeps the last $L$ lines (here $L = 100$):</p>
     $$M_{e+1} = \Pi_L\!\left( \pi_\theta^{\mathrm{mem}}(M_e,\, \tau_e,\, \mathrm{outcome}_e) \right).$$
@@ -662,7 +664,7 @@ step 4 go forward</code></pre>
           <tr><td><strong>Append + periodic compaction</strong></td><td>Full-episode rewrite cost limits the learning signal</td><td>Requires additional compute</td></tr>
           <tr><td><strong>Hierarchical</strong> (in-episode scratchpad + cross-episode long-term)</td><td>Conflating short- and long-term in one file hurts</td><td>Requires additional compute</td></tr>
           <tr><td><strong>Retrieval-indexed</strong> (embed notes, top-<em>k</em> by observation)</td><td>Linear-file recall fails at scale</td><td>Requires additional compute</td></tr>
-          <tr><td><strong>Shared single-file</strong> across branches / ranks</td><td>Collective memory beats per-branch curation</td><td>See <code>impl-context/shared-memory-option.md</code>; requires additional compute + concurrency design</td></tr>
+          <tr><td><strong>Shared single-file</strong> across branches / ranks</td><td>Collective memory beats per-branch curation</td><td>Shared-memory design TBD; requires additional compute + concurrency design</td></tr>
           <tr><td><strong>Success-gated writes</strong></td><td>Failure episodes poison $M$</td><td>Requires additional compute</td></tr>
           <tr><td><strong>Variable line budget</strong> by level difficulty</td><td>Uniform $L$ is too tight for hardest stages</td><td>Requires additional compute</td></tr>
           <tr><td><strong>Dual-memory</strong> (policy vs. world knowledge)</td><td>Unified $M$ conflates two knowledge types</td><td>Requires additional compute</td></tr>
@@ -756,7 +758,7 @@ quadrantChart
     <p>Combined per-token advantage with mixing weight $\omega \ge 0$:</p>
     $$A_t \;=\; A^{E}_i + \omega\, A^{S}(a_t).$$
     <p>When no anchors are found, $A^{S} = 0$ and GiGPO reduces to GRPO (equivalently $\omega = 0$).</p>
-    <p>Why this fits MiniGrid: all <em>G</em> rollouts share the same initial observation for a given prompt/seed (guaranteed anchor); corridor navigation revisits the same 7&times;7 egocentric view; BabyAI per-seed determinism creates exact hash matches. The full design is in <code>impl-context/gigpo-transition.md</code>.</p>
+    <p>Why this fits MiniGrid: all <em>G</em> rollouts share the same initial observation for a given prompt/seed (guaranteed anchor); corridor navigation revisits the same 7&times;7 egocentric view; BabyAI per-seed determinism creates exact hash matches. The full step-level design is deferred to the GiGPO follow-up (trainer subclass + rollout fields).</p>
 
     <h3>Experimental matrix for the follow-up</h3>
     <div class="table-wrap">
@@ -784,10 +786,10 @@ quadrantChart
           <tr><td><strong>GRPO / DeepSeekMath</strong></td><td>Critic-free group-relative policy optimization; our default trainer via TRL's <code>GRPOTrainer</code></td><td>Shao et al., <a href="https://arxiv.org/abs/2402.03300" target="_blank" style="color:var(--accent2)">arXiv:2402.03300</a></td></tr>
           <tr><td><strong>TRL &times; OpenEnv</strong></td><td><code>rollout_func</code> contract, vLLM colocate/server, <code>loss_type=dapo</code> length-bias handling</td><td><a href="https://huggingface.co/docs/trl/en/openenv" target="_blank" style="color:var(--accent2)">TRL OpenEnv docs</a></td></tr>
           <tr><td><strong>OpenEnv</strong></td><td>Standard WebSocket env contract, per-session state, <code>create_app</code>, HF Space deploy</td><td><a href="https://huggingface.co/blog/openenv" target="_blank" style="color:var(--accent2)">HF Blog: Introducing OpenEnv</a></td></tr>
-          <tr><td><strong>Voyager</strong></td><td>Skill-library / cross-episode knowledge accumulation (closest memory-system analog; ours is RL-trained where Voyager is prompt-engineered)</td><td>Wang et al., <a href="<INSERT: Voyager arXiv URL>" target="_blank" style="color:var(--accent2)">arXiv:2305.16291</a></td></tr>
-          <tr><td><strong>Reflexion</strong></td><td>Verbal reflection after episodes; motivates a post-episode LLM rewrite pass over a persistent buffer</td><td>Shinn et al., <a href="<INSERT: Reflexion arXiv URL>" target="_blank" style="color:var(--accent2)">arXiv:2303.11366</a></td></tr>
-          <tr><td><strong>Generative Agents</strong></td><td>Long-term memory stream with relevance / recency weighting; our line-budgeted rewrite is a deliberately simpler alternative</td><td>Park et al., <a href="<INSERT: Generative Agents arXiv URL>" target="_blank" style="color:var(--accent2)">arXiv:2304.03442</a></td></tr>
-          <tr><td><strong>LotteryElicitationEnv / PT</strong></td><td>Sibling OpenEnv submission; shared structural template for two-repo split, <code>rollout_func</code>, NCCL generate-count padding</td><td>Same monorepo &middot; <a href="<INSERT: Lottery HF Space URL>" target="_blank" style="color:var(--accent2)">HF Space</a></td></tr>
+          <tr><td><strong>Voyager</strong></td><td>Skill-library / cross-episode knowledge accumulation (closest memory-system analog; ours is RL-trained where Voyager is prompt-engineered)</td><td>Wang et al., <a href="https://arxiv.org/abs/2305.16291" target="_blank" style="color:var(--accent2)">arXiv:2305.16291</a></td></tr>
+          <tr><td><strong>Reflexion</strong></td><td>Verbal reflection after episodes; motivates a post-episode LLM rewrite pass over a persistent buffer</td><td>Shinn et al., <a href="https://arxiv.org/abs/2303.11366" target="_blank" style="color:var(--accent2)">arXiv:2303.11366</a></td></tr>
+          <tr><td><strong>Generative Agents</strong></td><td>Long-term memory stream with relevance / recency weighting; our line-budgeted rewrite is a deliberately simpler alternative</td><td>Park et al., <a href="https://arxiv.org/abs/2304.03442" target="_blank" style="color:var(--accent2)">arXiv:2304.03442</a></td></tr>
+          <tr><td><strong>LotteryElicitationEnv / PT</strong></td><td>Sibling OpenEnv submission; shared structural template for two-repo split, <code>rollout_func</code>, NCCL generate-count padding</td><td>Same monorepo &middot; <a href="https://huggingface.co/spaces/yashu2000/LotteryElicitationEnv" target="_blank" style="color:var(--accent2)">LotteryElicitationEnv HF Space</a></td></tr>
           <tr><td><strong>ReasoningEconomicsEnv / PT</strong></td><td>Structural template for <code>_temporary_vllm_max_tokens</code> pattern</td><td>Same monorepo</td></tr>
         </tbody>
       </table>
@@ -797,10 +799,10 @@ quadrantChart
   <!-- 16. QUICK START -->
   <section id="quickstart">
     <h2>Quick start</h2>
-    <p>Single-A100 Lambda recipe (full ops runbook lives at <code>impl-context/v1-prod.md</code>):</p>
-    <pre><code><span class="c"># 0. Clone</span>
-git clone &lt;INSERT: GitHub URL&gt; openenv-minigrid
-cd openenv-minigrid
+    <p>Single-A100 Lambda recipe (use MiniGridEnv Docker + MiniGridPT <code>scripts/</code> as the source of truth for env vars and launch order):</p>
+    <pre><code><span class="c"># 0. Clone both packages (sibling directories)</span>
+git clone https://github.com/sharma-yash01/MiniGridEnv.git
+git clone https://github.com/sharma-yash01/MiniGridPT.git
 
 <span class="c"># 1. Build + start MiniGridEnv (Docker on port 8000)</span>
 cd MiniGridEnv
@@ -854,7 +856,7 @@ bash "$MGPT_ROOT/scripts/launch_curriculum.sh"</code></pre>
     <h2>Future work</h2>
     <ul>
       <li><strong>Run the full A/B/C/D experimental matrix</strong> to publish the memory-vs-stateless and GRPO-vs-GiGPO comparison across the BabyAI curriculum once additional compute is available (measured numbers to be filled in after those runs).</li>
-      <li><strong>Land GiGPO</strong> as a <code>GiGPOTrainer(GRPOTrainer)</code> subclass per <code>impl-context/gigpo-transition.md</code>. Minimum diff: add <code>obs_texts</code> / <code>step_boundaries</code> to the rollout return, compute anchor-state groups, expand step advantages to tokens.</li>
+      <li><strong>Land GiGPO</strong> as a <code>GiGPOTrainer(GRPOTrainer)</code> subclass. Minimum diff: add <code>obs_texts</code> / <code>step_boundaries</code> to the rollout return, compute anchor-state groups, expand step advantages to tokens.</li>
       <li><strong>Close the inference-time gap</strong>: <code>inference/run_episode.py</code> reads memory during play but does not yet mirror training's post-episode LLM memory rewrite. Evaluation should match training end-to-end; add a <strong>post-episode-memory-rewrite eval variant</strong> when more compute is available.</li>
       <li><strong>Baseline harness at scale</strong>: run Random, BabyAI <code>BotAgent</code>, and zero-shot LLM baselines with enough seeds to report completion rates and calibration vs. GRPO / GRPO+memory (deferred for lack of compute).</li>
       <li><strong>Port the NCCL generate-count padding upstream into TRL</strong>: the bug is general, the fix is simple.</li>
@@ -874,8 +876,9 @@ bash "$MGPT_ROOT/scripts/launch_curriculum.sh"</code></pre>
   <div class="footer">
     <p>MiniGridEnv &middot; AgentX OpenEnv Track &middot; UC Berkeley RDI</p>
     <p style="margin-top:.5rem;">
-      <a href="<INSERT: GitHub URL>" target="_blank">GitHub</a> &middot;
-      <a href="<INSERT: HF Space URL>" target="_blank">HF Space</a> &middot;
+      <a href="https://github.com/sharma-yash01/MiniGridEnv" target="_blank" rel="noopener noreferrer">MiniGridEnv</a> &middot;
+      <a href="https://github.com/sharma-yash01/MiniGridPT" target="_blank" rel="noopener noreferrer">MiniGridPT</a> &middot;
+      <a href="https://huggingface.co/spaces/yashu2000/MiniGridEnv" target="_blank">MiniGridEnv HF Space</a> &middot;
       <a href="https://github.com/meta-pytorch/OpenEnv" target="_blank">OpenEnv Framework</a> &middot;
       <a href="https://huggingface.co/docs/trl/en/openenv" target="_blank">TRL x OpenEnv</a> &middot;
       <a href="https://github.com/Farama-Foundation/Minigrid" target="_blank">MiniGrid</a>