| # LM Studio memory-safety protocol |
|
|
| The local machine has 64 GiB of unified memory. The observed Qwen3.6 agent model |
| file is approximately 20.4 GB and the embedding model is approximately 0.64 GB; |
| runtime state and KV cache add pressure beyond weight-file sizes. Model |
| co-residency is therefore forbidden by the experiment runner even when both |
| models technically fit. |
|
|
| ## Phase separation |
|
|
| 1. **Retrieval/index phase:** unload `qwen/qwen3.6-35b-a3b`; load only |
| `text-embedding-qwen3-embedding-0.6b`; batch code and query embeddings; store |
| vectors in the content-addressed SQLite cache. |
| 2. **Agent phase:** finish all required embedding calls; unload the embedding |
| instance; load only the Qwen3.6 agent using the pinned 262,144 context; run |
| model/tool interactions. |
| 3. **Iterative dense-query treatments:** alternate residency at explicit phase |
| boundaries. Persist conversation state and query text before unloading. Do |
| not rely on both models being resident for low latency. |
| 4. **Cleanup:** unload the active model after a run block unless the next block |
| needs the same model immediately. |
|
|
| LM Studio's native `POST /api/v1/models/load` and |
| `POST /api/v1/models/unload` endpoints are used for explicit residency control. |
| The E00 runner queries `/api/v1/models` and aborts unless the embedding model is |
| the only resident instance. |
|
|
| ## Pressure observations and stop conditions |
|
|
| The initial audit reported 53% system-wide free memory, but the VM counters also |
| showed substantial historical compression and swap activity. A run block must |
| stop rather than continue when: |
|
|
| - an unexpected model is resident; |
| - available memory enters macOS critical pressure; |
| - swap grows rapidly during a steady-state batch; |
| - embedding requests begin timing out or the OS kills a process; or |
| - index construction exceeds the configured disk or wall-time budget. |
|
|
| Weights, index caches, and result artifacts are not evidence of comparable |
| compute cost. The study records load/unload time, index time, query time, model |
| latency, and resource samples separately. |
|
|
|
|
