Spaces:
Sleeping
Sleeping
sameerkatte Claude Opus 4.6 (1M context) commited on
Commit ·
2ee6649
1
Parent(s): 8b4c1a6
Deploy to HF Spaces: add baseline scores, fix deps and validation
Browse files- Deployed live on https://huggingface.co/spaces/sam25kat/securereview
- Added README frontmatter for HF Spaces Docker SDK
- Real baseline scores with DeepSeek-V3: dep=0.45, iac=0.52, mig=0.05
- Added openai/requests to requirements.txt (for inference.py)
- Fixed HF_TOKEN validation warning in inference.py
- Added operation type validation in migration grader
- Dockerfile pip upgrade
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
README.md
CHANGED
|
@@ -1,3 +1,20 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
# SecureReview
|
| 2 |
|
| 3 |
**The first OpenEnv environment for AI-powered security code review**
|
|
@@ -116,13 +133,25 @@ GET /state Get current episode state
|
|
| 116 |
|
| 117 |
## Baseline Scores
|
| 118 |
|
| 119 |
-
|
| 120 |
-
|
| 121 |
-
|
|
| 122 |
-
|
|
| 123 |
-
| `
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 124 |
|
| 125 |
-
*
|
|
|
|
|
|
|
| 126 |
|
| 127 |
## Project Structure
|
| 128 |
|
|
|
|
| 1 |
+
---
|
| 2 |
+
title: SecureReview
|
| 3 |
+
emoji: 🔐
|
| 4 |
+
colorFrom: indigo
|
| 5 |
+
colorTo: red
|
| 6 |
+
sdk: docker
|
| 7 |
+
app_port: 7860
|
| 8 |
+
pinned: false
|
| 9 |
+
license: mit
|
| 10 |
+
tags:
|
| 11 |
+
- openenv
|
| 12 |
+
- security
|
| 13 |
+
- code-review
|
| 14 |
+
- agent
|
| 15 |
+
- evaluation
|
| 16 |
+
---
|
| 17 |
+
|
| 18 |
# SecureReview
|
| 19 |
|
| 20 |
**The first OpenEnv environment for AI-powered security code review**
|
|
|
|
| 133 |
|
| 134 |
## Baseline Scores
|
| 135 |
|
| 136 |
+
Evaluated on live Hugging Face Space deployment (`https://sam25kat-securereview.hf.space`) with `deepseek-ai/DeepSeek-V3-0324` via the HF Inference Router:
|
| 137 |
+
|
| 138 |
+
| Task | Difficulty | Scenario | Score | Notes |
|
| 139 |
+
|------|-----------|----------|-------|-------|
|
| 140 |
+
| `dependency_review` | Easy | dep_006 | **0.45** | Full run; identified 2/5 issues |
|
| 141 |
+
| `iac_review` | Medium | iac_004 | **0.52** | Identified multiple misconfigurations |
|
| 142 |
+
| `migration_review` | Hard | migration_001 | **0.05** | Designed to challenge frontier models |
|
| 143 |
+
|
| 144 |
+
**Overall average: 0.34**
|
| 145 |
+
|
| 146 |
+
Perfect-score reference (oracle agent with ground-truth findings): **0.99** (validates grader correctness).
|
| 147 |
+
|
| 148 |
+
The hard task (migration_review) is deliberately challenging — it requires cross-file reasoning about production context (table sizes, deployment strategy) and application code dependencies to determine why specific DDL operations are unsafe. This creates significant headroom for better models.
|
| 149 |
+
|
| 150 |
+
## Live Deployment
|
| 151 |
|
| 152 |
+
- **Hugging Face Space:** https://huggingface.co/spaces/sam25kat/securereview
|
| 153 |
+
- **API Endpoint:** https://sam25kat-securereview.hf.space
|
| 154 |
+
- **GitHub:** https://github.com/sam25kat/Secure_Reveiw
|
| 155 |
|
| 156 |
## Project Structure
|
| 157 |
|