Spaces:
Sleeping
Sleeping
File size: 4,368 Bytes
84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 c9c2d7e 84cf689 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | # Configure Model Options
## Goal Description
Update the agent generation form to include the latest Anthropic models and add support for Hugging Face models. Additionally, implement backend validation, streaming support, security measures, a sandbox executor, and observability.
## User Review Required
>
> [!NOTE]
> I am adding `claude-3-5-haiku-20241022` to the list.
> I am also adding support for Hugging Face models (via `huggingface_hub`).
> I am adding backend validation using Pydantic.
> I am adding a streaming response toggle.
> I am adding security measures (pre-commit hook).
> I am adding a Sandbox Executor for safe agent execution.
> I am adding Prometheus metrics and a `/healthz` endpoint.
## Proposed Changes
### Frontend
#### [MODIFY] [AgentForm.jsx](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/frontend/src/components/AgentForm.jsx)
- Add a "Provider" dropdown (Anthropic, Hugging Face).
- Update "Model" dropdown options based on the selected provider.
- Add `claude-3-5-haiku-20241022` for Anthropic.
- Add `meta-llama/Meta-Llama-3-8B-Instruct` and `mistralai/Mistral-7B-Instruct-v0.3` for Hugging Face.
- Add a "Stream Response" checkbox (default: false).
- Add a "Test Agent" button to execute the generated code in the sandbox.
- Display execution results (output/errors) in the UI.
### Backend
#### [NEW] [models.py](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/server/models.py)
- Define `ProviderEnum` (Anthropic, HuggingFace).
- Define `GenerateRequest` Pydantic model with validation:
- `provider`: ProviderEnum
- `model`: Validated against an allowlist per provider.
- `stream`: bool (default: False)
#### [MODIFY] [main.py](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/server/main.py)
- Import `GenerateRequest` from `models.py`.
- Update `generate_agent` endpoint to use the new validation model.
- Add `/api/execute` endpoint.
- Add `prometheus-fastapi-instrumentator`.
- Add `/metrics` and `/healthz` endpoints.
#### [MODIFY] [agent_builder.py](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/llm_agent_builder/agent_builder.py)
- Update `build_agent` to accept `provider` and `stream` arguments.
- Select the appropriate template (`agent_template.py.j2` or `agent_template_hf.py.j2`) based on the provider.
- Pass `stream` to the template context.
#### [NEW] [agent_template_hf.py.j2](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/llm_agent_builder/templates/agent_template_hf.py.j2)
- Create a new Jinja2 template for agents using `huggingface_hub.InferenceClient`.
- Use `HUGGINGFACEHUB_API_TOKEN` for authentication.
- Implement conditional logic for `stream=True` vs `stream=False`.
#### [MODIFY] [agent_template.py.j2](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/llm_agent_builder/templates/agent_template.py.j2)
- Fix the environment variable name from `GEMINI_API_KEY` to `ANTHROPIC_API_KEY`.
### Sandbox Executor
#### [NEW] [sandbox.py](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/server/sandbox.py)
- Implement `run_in_sandbox(code: str, task: str) -> str`:
- Write code to a temporary file.
- Use `subprocess.Popen` to execute the file.
- Use `preexec_fn` to set `resource.setrlimit`:
- `RLIMIT_CPU`: Limit CPU time (e.g., 30 seconds).
- `RLIMIT_AS`: Limit address space (memory) (e.g., 512MB).
- Capture `stdout` and `stderr`.
- Handle timeouts and errors.
### Security & Misc
#### [NEW] [.env.example](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/.env.example)
- Add `ANTHROPIC_API_KEY` and `HUGGINGFACEHUB_API_TOKEN` placeholders.
#### [NEW] [pre-commit-check.sh](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/pre-commit-check.sh)
- Simple script to grep for potential API keys in staged files.
#### [MODIFY] [requirements.txt](file:///wsl.localhost/Ubuntu/root/LLMAgentBuilder/requirements.txt)
- Add `prometheus-fastapi-instrumentator`.
## Verification Plan
### Automated Tests
- Run `pytest` to ensure no regressions in `agent_builder`.
### Manual Verification
- Start the frontend (`npm run dev` in `frontend/`).
- Verify the new provider and model options.
- Verify the stream toggle works.
- Generate an agent with streaming enabled and check the code.
- Test the "Test Agent" button with a simple task.
- Verify `/metrics` and `/healthz` endpoints.
- Try to commit a file with a fake API key to test the hook.
|