tasks/gaps.md

SentinelOps Arena -- Gap Analysis & 4-Hour Action Plan

Generated: Sunday March 8, 2026 ~9:00 AM Deadline: Sunday March 8, 2026 1:00 PM (4 hours remaining) Status: Strong core implementation, missing 3 required submission deliverables

---

EXECUTIVE SUMMARY

The environment implementation is solid: 3 agents, 3 systems, 4 attack types, reward functions, randomized attacker, security metrics, and a polished Gradio UI with cybersecurity theme. The code runs without errors and the trained vs untrained comparison shows meaningful differences (30.0 vs 25.0 worker score).

However, the 3 required hackathon deliverables are NOT done:

1. HuggingFace Spaces deployment -- NOT DEPLOYED
2. Google Colab training notebook -- DOES NOT EXIST
3. Demo video on YouTube -- NOT RECORDED

Without all three, the submission is invalid regardless of code quality.

---

GAP LIST (Prioritized)

BLOCKER -- Must fix or submission fails

#	Gap	Details	Estimated Time
B1	No HuggingFace Spaces deployment	README.md has correct frontmatter (sdk: gradio, sdk_version: 6.9.0, app_file: app.py). No HF remote configured. Need to create Space and push. requirements.txt exists but may need pandas added.	30 min
B2	No Colab training notebook	training/ directory is empty. Submission requires "Minimal Training Script" as Colab notebook. train.py exists at root but is a standalone Python script, not a notebook. Must create training/colab_training.ipynb.	60-90 min
B3	No demo video	Submission requires YouTube demo video. Need to screen record the Gradio app, show episode replay, before/after comparison, and explain the 3-agent dynamic. SETUP.md says 1 minute length.	30 min
B4	No nihal branch	CLAUDE.md says push to nihal but only main exists. All code is on main. Need to create branch and push.	5 min

HIGH -- Significantly improves judging score

#	Gap	Details	Estimated Time
H1	Gradio app not verified to launch	tasks/todo.md shows "Gradio app launches without errors" is UNCHECKED. Must verify python app.py works and the UI renders correctly. Fix any runtime errors.	15 min
H2	requirements.txt missing pandas	requirements.txt has 6 packages but app.py imports pandas (via chart_helpers.py, inspector.py). HF Spaces will fail to install. Must add pandas>=2.0.	2 min
H3	SENTINELOPS_ARENA.md claims "80 ticks" and "OpenEnv 0.4"	Environment actually uses 30 ticks and OpenEnv 0.2.x. Spec doc has aspirational content that doesn't match reality. Judges who read the spec will notice discrepancies. README.md is more accurate but should be cross-checked.	15 min
H4	pyproject.toml version mismatch	pyproject.toml says gradio>=5.0.0, README frontmatter says sdk_version: 6.9.0, requirements.txt says gradio>=6.0.0. Should be consistent.	5 min
H5	train.py uses datasets and trl but these aren't in requirements.txt	train.py has GPU-only dependencies that are correctly optional, but the Colab notebook needs them listed. Just awareness -- Colab notebook handles its own installs.	0 min
H6	No nihal branch for pushing	CLAUDE.md mandates pushing to nihal, but no such branch exists.	2 min

MEDIUM -- Nice to have for judges

#	Gap	Details	Estimated Time
M1	Colab notebook should show real training signal	Even a few training steps with decreasing loss would impress judges (especially Daniel Han from Unsloth and Michael Han, Unsloth CTO). The reward_function in train.py is well-designed for this.	included in B2
M2	About tab could link to Colab notebook and video	Once created, add links to the About tab in app.py for judges to find easily.	10 min
M3	No mcp_x/ gateway demo	SENTINELOPS_ARENA.md describes MCP-X per-agent tool isolation, but it's not implemented. The MCP tools ARE defined in environment.py (19 tools), just no gateway layer. Not critical but was a differentiator in the spec.	SKIP
M4	hackathon_env/ directory is vestigial	Contains old echo environment template. Should be in .gitignore or removed to avoid confusing judges.	5 min
M5	README.md project structure shows files that don't exist	Lists mcp_tools.py separately but MCP tools are inline in environment.py. Minor but sloppy.	10 min

LOW -- Skip for the 4-hour window

#	Gap	Details	Estimated Time
L1	No compound attacks	Spec describes compound attacks (2-3 simultaneous), not implemented	2+ hours
L2	No compliance drift attack type	Spec describes it, not implemented (only 4 of 6 attack types exist)	1+ hours
L3	A2A protocol not implemented	Already marked as "Cut" in spec. Correct decision.	N/A
L4	No Docker support	HF Spaces uses Gradio SDK, Docker was backup option. Not needed.	N/A
L5	SENTINELOPS_ARENA.md has unrealized training dynamics section	Describes episodes 1-50, 50-200, 200-500, 500+ progression that hasn't been trained. This is aspirational/theoretical.	N/A

---

WHAT'S DONE AND WORKING (Assets to leverage)

• Core environment: SentinelOpsArena(MCPEnvironment) with step/reset/state -- WORKING
• 3 enterprise systems (CRM, Billing, Ticketing) with full CRUD -- WORKING
• 4 attack types (schema_drift, policy_drift, social_engineering, rate_limit) -- WORKING
• 3 reward functions matching spec tables exactly -- WORKING
• RandomizedAttacker with budget, probability, seeded RNG -- WORKING
• HeuristicWorker with trained/untrained modes -- WORKING
• HeuristicOversight with violation detection -- WORKING
• 19 MCP tools registered via FastMCP -- WORKING
• HTTP server via create_app() -- WORKING
• Security metrics: ASR, Benign Task Success, FPR, MTTD, Social Eng. Resistance -- WORKING
• Gradio UI with 4 tabs (Run Episode, Untrained vs Trained, Environment Inspector, About) -- EXISTS (needs verification)
• Custom cybersecurity theme (SentinelTheme) -- EXISTS
• Styled HTML replay renderer -- EXISTS
• Chart helpers for LinePlot/BarPlot -- EXISTS
• train.py with GRPO pipeline, env verification, data collection -- EXISTS (GPU-only)
• README.md with correct HF Spaces frontmatter -- EXISTS

---

4-HOUR ACTION PLAN

Phase 1: Verify & Fix (0:00 - 0:45) -- 45 minutes

Goal: Make sure everything that exists actually works

1. [5 min] Create nihal branch and push (B4, H6)

   git checkout -b nihal
   git push origin nihal
   

2. [2 min] Fix requirements.txt (H2)

   • Add pandas>=2.0 to requirements.txt
   • Verify gradio>=6.0.0 (not 5.0.0)

3. [15 min] Verify Gradio app launches (H1)

   cd /Users/nihalnihalani/Desktop/Github/NexusEnv
   python app.py
   

   • Test all 4 tabs: Run Episode, Untrained vs Trained, Environment Inspector, About
   • Fix any import errors, rendering issues, or crashes
   • Take screenshots for the video

4. [10 min] Fix pyproject.toml consistency (H4)

   • Set gradio>=6.0.0 in pyproject.toml
   • Verify requires-python = ">=3.12" matches reality

5. [10 min] Clean up misleading claims (H3, M4, M5)

   • Remove or gitignore hackathon_env/ directory
   • Fix README.md project structure to match reality
   • Do NOT touch SENTINELOPS_ARENA.md (it's a spec doc, acceptable to be aspirational)

6. [3 min] Commit and push everything

Phase 2: HuggingFace Spaces Deployment (0:45 - 1:15) -- 30 minutes

Goal: Get a live public URL

1. [5 min] Create HuggingFace Space

   • Go to huggingface.co/new-space
   • Name: nihalnihalani/sentinelops-arena
   • SDK: Gradio
   • Hardware: CPU Basic (free)

2. [10 min] Configure and push

   git remote add hf https://huggingface.co/spaces/nihalnihalani/sentinelops-arena
   git push hf nihal:main
   

   • If push fails, use HuggingFace Hub Python API:

   from huggingface_hub import HfApi
   api = HfApi()
   api.upload_folder(folder_path=".", repo_id="nihalnihalani/sentinelops-arena", repo_type="space")
   

3. [10 min] Verify Space builds and runs

   • Watch build logs
   • Fix any dependency issues
   • Common issues: missing packages, port mismatch (must be 7860)

4. [5 min] Test live URL

   • Run an episode
   • Run untrained vs trained comparison
   • Verify Environment Inspector works

Phase 3: Colab Training Notebook (1:15 - 2:30) -- 75 minutes

Goal: Create a working Colab notebook that demonstrates GRPO training

1. [45 min] Create training/colab_training.ipynb (B2) Cells:

   • Cell 1: Install dependencies

     !pip install unsloth "trl>=0.15" transformers torch accelerate pydantic datasets
     !pip install openenv-core[core]>=0.2.0 fastmcp>=2.14.5 mcp>=1.26.0 httpx>=0.27
     

   • Cell 2: Clone repo and import environment

     !git clone https://github.com/nihalnihalani/NexusEnv.git
     import sys; sys.path.insert(0, "NexusEnv")
     from sentinelops_arena.environment import SentinelOpsArena
     from sentinelops_arena.models import AgentRole, SentinelAction
     

   • Cell 3: Verify environment works (run 1 episode)
   • Cell 4: Collect training data (reuse build_training_dataset from train.py)
   • Cell 5: Load model with Unsloth
   • Cell 6: Define reward function (reuse from train.py)
   • Cell 7: Configure GRPO and train
   • Cell 8: Show results / save model

   Key decisions:

   • Use Qwen/Qwen2.5-0.5B-Instruct (smallest, fits free Colab T4)
   • Use Unsloth for model loading, vanilla TRL GRPOTrainer for training
   • If openenv-core fails on Colab Python version, inline the minimal env code
   • Even 5-10 training steps is enough to show the pipeline works

2. [15 min] Test notebook runs (at least partially)

   • Upload to Colab
   • Verify cells 1-4 work (env setup + data collection)
   • Cells 5-8 need GPU -- verify they at least don't crash on import

3. [15 min] Polish and save

   • Add markdown cells explaining each step
   • Add the SentinelOps Arena header/description
   • Mention partner tracks (Fleet AI, Patronus AI)
   • Save and get shareable link
   • Commit to repo

Phase 4: Demo Video (2:30 - 3:00) -- 30 minutes

Goal: 1-minute YouTube video demonstrating the environment

1. [5 min] Script the video

   • 0-10s: Title card + what SentinelOps Arena is
   • 10-30s: Run an episode in Gradio, show attack/adapt/flag cycle
   • 30-45s: Show Untrained vs Trained comparison, highlight score difference
   • 45-55s: Show Environment Inspector (databases, task queue)
   • 55-60s: Mention partner tracks, training approach, link to Colab

2. [15 min] Record

   • Screen record the Gradio app (use HF Spaces URL if live, else local)
   • Voice narration or text overlay
   • Keep it to exactly 1 minute

3. [10 min] Upload to YouTube

   • Title: "SentinelOps Arena -- Multi-Agent RL for Enterprise Security | OpenEnv Hackathon"
   • Upload as unlisted
   • Get shareable link

Phase 5: Final Polish & Submit (3:00 - 3:45) -- 45 minutes

1. [10 min] Add links to About tab (M2)

   • HF Spaces URL
   • YouTube demo link
   • Colab notebook link
   • GitHub repo link

2. [10 min] Final push to both remotes

   git add -A
   git commit -m "Final submission: add Colab notebook, update links"
   git push origin nihal
   git push hf nihal:main
   

3. [10 min] Verify everything one last time

   • HF Spaces loads and works
   • Colab notebook link is accessible
   • YouTube video plays
   • All links in About tab work

4. [15 min] Submit

   • Team Name: SentinelOps (or NexusEnv)
   • Project Description: (use draft from SENTINELOPS_ARENA.md)
   • HF Spaces Link: https://huggingface.co/spaces/nihalnihalani/sentinelops-arena
   • Demo Video: YouTube URL
   • Minimal Training Script: Colab link
   • Partner Tracks: Fleet AI (Scalable Oversight), Patronus AI (Schema Drift)

Buffer: 15 minutes (3:45 - 4:00)

For unexpected issues, last-minute fixes, or submission form problems.

---

CRITICAL PATH

The absolute minimum to submit (if everything goes wrong):

1. Fix requirements.txt (2 min)
2. Push to HF Spaces (15 min)
3. Create minimal Colab notebook that at least runs the environment (30 min)
4. Record 60-second screen capture (15 min)
5. Upload video + submit (10 min)

Total critical path: ~72 minutes

This leaves ~2.5 hours for polish, testing, and fixing issues.

---

RISK REGISTER

Risk	Probability	Impact	Mitigation
HF Spaces build fails	Medium	BLOCKER	Test locally first. Have huggingface_hub upload as backup. Check Python version compat.
Colab Python version incompatible with openenv-core	Medium	HIGH	Bundle standalone env code in notebook (no openenv import needed for demo).
Gradio 6 has breaking changes on HF	Low	HIGH	Pin sdk_version in README frontmatter. Test specific version.
Video recording takes too long	Low	BLOCKER	Use simplest tool (QuickTime screen record). Keep to exactly 1 min. No editing.
Unsloth doesn't install on Colab	Medium	MEDIUM	Fall back to vanilla transformers (slower but works). Show pipeline, not convergence.
Submission form has unexpected fields	Low	LOW	Read form early, adapt.

---

WHAT NOT TO DO (Time traps)

• DO NOT try to implement compound attacks, compliance drift, or A2A protocol
• DO NOT try to actually train to convergence -- show the pipeline works, that's enough
• DO NOT refactor the codebase or clean up the spec doc
• DO NOT spend more than 30 min on the video -- 1 minute, simple screen recording
• DO NOT try to add Docker support
• DO NOT spend time on MCP-X gateway -- MCP tools in environment.py are sufficient
• DO NOT worry about the hackathon_env/ directory during final push -- judges won't look at it unless it causes confusion