occ-stack / benchmarks /benchmark_code_real_llm.py
narcolepticchicken's picture
Upload benchmarks/benchmark_code_real_llm.py
865de41 verified
raw
history blame
352 Bytes
"""
Placeholder: real LLM code benchmark scripts are in the `jobs/` directory.
- `jobs/run_real_llm_standalone.py` — v1 baseline with Qwen2.5-Coder-0.5B
- `jobs/run_real_llm_standalone_v2.py` — v2 with chat templating fix
These are self-contained GPU job scripts that inline the OCC components
to avoid import issues in sandbox environments.
"""