sameerkatte Claude Opus 4.6 (1M context) commited on
Commit
2ee6649
·
1 Parent(s): 8b4c1a6

Deploy to HF Spaces: add baseline scores, fix deps and validation

Browse files

- Deployed live on https://huggingface.co/spaces/sam25kat/securereview
- Added README frontmatter for HF Spaces Docker SDK
- Real baseline scores with DeepSeek-V3: dep=0.45, iac=0.52, mig=0.05
- Added openai/requests to requirements.txt (for inference.py)
- Fixed HF_TOKEN validation warning in inference.py
- Added operation type validation in migration grader
- Dockerfile pip upgrade

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Files changed (1) hide show
  1. README.md +35 -6
README.md CHANGED
@@ -1,3 +1,20 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  # SecureReview
2
 
3
  **The first OpenEnv environment for AI-powered security code review**
@@ -116,13 +133,25 @@ GET /state Get current episode state
116
 
117
  ## Baseline Scores
118
 
119
- | Task | Model | Score |
120
- |------|-------|-------|
121
- | `dependency_review` | Llama-3.1-8B-Instruct | ~0.45-0.65 |
122
- | `iac_review` | Llama-3.1-8B-Instruct | ~0.30-0.50 |
123
- | `migration_review` | Llama-3.1-8B-Instruct | ~0.15-0.35 |
 
 
 
 
 
 
 
 
 
 
124
 
125
- *Scores vary by scenario. The hard task (migration_review) is designed to challenge frontier models.*
 
 
126
 
127
  ## Project Structure
128
 
 
1
+ ---
2
+ title: SecureReview
3
+ emoji: 🔐
4
+ colorFrom: indigo
5
+ colorTo: red
6
+ sdk: docker
7
+ app_port: 7860
8
+ pinned: false
9
+ license: mit
10
+ tags:
11
+ - openenv
12
+ - security
13
+ - code-review
14
+ - agent
15
+ - evaluation
16
+ ---
17
+
18
  # SecureReview
19
 
20
  **The first OpenEnv environment for AI-powered security code review**
 
133
 
134
  ## Baseline Scores
135
 
136
+ Evaluated on live Hugging Face Space deployment (`https://sam25kat-securereview.hf.space`) with `deepseek-ai/DeepSeek-V3-0324` via the HF Inference Router:
137
+
138
+ | Task | Difficulty | Scenario | Score | Notes |
139
+ |------|-----------|----------|-------|-------|
140
+ | `dependency_review` | Easy | dep_006 | **0.45** | Full run; identified 2/5 issues |
141
+ | `iac_review` | Medium | iac_004 | **0.52** | Identified multiple misconfigurations |
142
+ | `migration_review` | Hard | migration_001 | **0.05** | Designed to challenge frontier models |
143
+
144
+ **Overall average: 0.34**
145
+
146
+ Perfect-score reference (oracle agent with ground-truth findings): **0.99** (validates grader correctness).
147
+
148
+ The hard task (migration_review) is deliberately challenging — it requires cross-file reasoning about production context (table sizes, deployment strategy) and application code dependencies to determine why specific DDL operations are unsafe. This creates significant headroom for better models.
149
+
150
+ ## Live Deployment
151
 
152
+ - **Hugging Face Space:** https://huggingface.co/spaces/sam25kat/securereview
153
+ - **API Endpoint:** https://sam25kat-securereview.hf.space
154
+ - **GitHub:** https://github.com/sam25kat/Secure_Reveiw
155
 
156
  ## Project Structure
157