sql_env / specs /F007-RESEARCH_SUMMARY.md
hjerpe's picture
Upload folder using huggingface_hub
5dd1bb4 verified
|
Raw
History Blame Contribute Delete
5.46 kB
# Research Summary
**Project:** SQLEnv
**Change:** F007 β€” HuggingFace Deployment & Submission
**Date:** 2026-03-27
**Status:** Draft
---
## 1. Change Overview
### What We're Changing
Competition submission package:
1. Validate and push Docker to HF Spaces (`openenv push`)
2. Clean up GitHub repo (README, setup instructions, training notebook)
3. Write HF blog post outline (hook, problem, solution, results, technical)
4. Record/screenshot before-vs-after demo
### Why We're Changing It
This is the deliverable. Judges evaluate: HF Space, GitHub repo, HF blog post. Without this, there's no submission.
### Success Criteria
- Blog tells a compelling story even if training results are modest
- HF Space just works β€” connect, reset, play an episode
- Training notebook runs end-to-end on Colab with one click
---
## 2. System Context
### Current Behavior
- Dockerfile exists at `Dockerfile` (project root) β€” needs validation for HF Spaces
- README.md exists but is minimal
- No blog post, no demo recordings
- `openenv.yaml` may need updating for HF Hub compatibility
### Architecture Context
```
Submission Package:
β”œβ”€β”€ HF Hub Space (Docker)
β”‚ β”œβ”€β”€ Dockerfile β†’ builds server
β”‚ β”œβ”€β”€ openenv.yaml β†’ environment manifest
β”‚ └── SQLEnv server (WebSocket API)
β”œβ”€β”€ GitHub Repo
β”‚ β”œβ”€β”€ README.md (setup, usage, architecture)
β”‚ β”œβ”€β”€ notebooks/train_grpo.ipynb
β”‚ └── Source code
└── HF Blog Post
β”œβ”€β”€ Hook: "Teaching AI to think like a data analyst"
β”œβ”€β”€ Problem: Static benchmarks
β”œβ”€β”€ Solution: SQLEnv
β”œβ”€β”€ Results: Learning curves, comparison
└── Technical: Reward architecture
```
### Entry Points
| Entry Point | Trigger | Current Flow |
|-------------|---------|--------------|
| `openenv push` | CLI command | Validates + pushes to HF Hub |
| `Dockerfile` | Docker build | Builds server container |
| Blog post | Reader visits HF | N/A β€” to be written |
### Data Flow
| Data | Source | Shape/Type | Destination |
|------|--------|------------|-------------|
| Docker image | Build | Container | HF Spaces |
| Training results | F006 | Learning curves, metrics | Blog post |
| Demo recordings | Manual | Screenshots/video | Blog post |
| README | Markdown | Setup instructions | GitHub |
---
## 3. Dependencies
### Code We Depend On
| Dependency | What We Use | Risk if Changed |
|------------|-------------|-----------------|
| F001-F006 | All features complete | **This is the final feature** |
| `openenv` CLI | `openenv push`, `openenv validate` | External tool |
| HuggingFace Hub | Spaces hosting | Must have HF account + token |
### Code That Depends On Us
None β€” this is the terminal feature.
---
## 4. Risks & Edge Cases
### Identified Risks
| Risk | Likelihood | Impact | Mitigation |
|------|------------|--------|------------|
| Docker build fails on HF Spaces | Medium | Can't deploy | Test with `openenv validate` locally first |
| Blog has no compelling results | Medium | Weak submission | Focus on environment design, not just results |
| Notebook has undocumented steps | Medium | Users can't reproduce | Test on fresh Colab |
| HF Spaces resource limits | Low | Server crashes | Keep container lightweight |
### Edge Cases to Handle
| Edge Case | Current Behavior | Required Behavior |
|-----------|------------------|-------------------|
| No GPU on HF Spaces | N/A | Server runs CPU-only (no model inference needed) |
| Large database files | N/A | Include only needed DBs, use .gitattributes for LFS |
---
## 4b. Code Shape & Design Target
### Target Shape
| Component | Purpose | Why This Boundary |
|-----------|---------|-------------------|
| `Dockerfile` | HF Spaces deployment | Must pass `openenv validate` |
| `openenv.yaml` | Environment manifest | Required by OpenEnv |
| `README.md` | GitHub documentation | Setup, usage, architecture |
| `docs/blog-outline.md` | HF blog draft | Submission artifact |
| `notebooks/train_grpo.ipynb` | Training notebook | Submission artifact (from F006) |
### Anti-Patterns to Avoid
- Don't include training weights in Docker image (inference not needed for env server)
- Don't require GPU for HF Space (env server is pure Python + SQLite)
- Don't write the full blog in markdown β€” outline + key sections, polish manually
---
## 5. Constraints
### Technical Constraints
| Constraint | Requirement | Notes |
|------------|-------------|-------|
| HF Spaces | Docker container, WebSocket API | Must pass openenv validate |
| Colab notebook | Must run on free tier | No paid GPU required |
| Blog | HF blog format | Markdown with embedded images |
---
## 6. Open Questions
| Question | Why It Matters | Who Can Answer |
|----------|----------------|----------------|
| HF Space tier? Free or paid? | Resource limits | Recommend free tier (CPU is fine for env server) |
| Include databases in Docker or download at startup? | Image size vs. startup time | Recommend bundle (small SQLite files) |
---
## 7. Context Sources
| Source | Type | Notes |
|--------|------|-------|
| `docs_draft/sql_env_project_brief.md` Phase 5 | Doc | Submission requirements |
| `docs_draft/SQLEnv_Concept_v1.md` Section 1.3-1.4 | Doc | Submission artifacts |
| `Dockerfile` | Code | Existing (needs validation) |
| `openenv.yaml` | Code | Environment manifest |
| OpenEnv Challenge PDF | Doc | Evaluation criteria |