# Modal Inference Endpoint — Working Config

Fill this in after M3 (deploy) and M4 (smoke test) pass.
This is the contract the app's ModalChatModel adapter targets.

## Active config

| Field | Value |
|---|---|
| model | NousResearch/Hermes-3-Llama-3.1-8B |
| gpu | L4 |
| tool-call-parser | hermes |
| max-model-len | 8192 |
| endpoint_url | https://yannsay--kobo-analyst-inference-serve.modal.run |
| volumes | huggingface-cache, vllm-cache |
| first_cold_start_s | 221 (3m41s — weights cached in volume, vLLM init only) |
| second_cold_start_s | 202 (3m22s — weights cached in volume, vLLM reinit only) |

## 32B swap checklist

When ready to upgrade to Qwen2.5-32B-Instruct:
- [ ] Change `MODEL_NAME = "Qwen/Qwen2.5-32B-Instruct"` in inference_app.py
- [ ] Change `gpu="A100-80GB"` in inference_app.py
- [ ] `modal deploy modal-setup/inference_app.py`
- [ ] Re-run `python modal-setup/smoke_test.py` — must print PASS
- [ ] Update this file with new timings