File size: 2,063 Bytes
d61821a | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 | # LM Studio memory-safety protocol
The local machine has 64 GiB of unified memory. The observed Qwen3.6 agent model
file is approximately 20.4 GB and the embedding model is approximately 0.64 GB;
runtime state and KV cache add pressure beyond weight-file sizes. Model
co-residency is therefore forbidden by the experiment runner even when both
models technically fit.
## Phase separation
1. **Retrieval/index phase:** unload `qwen/qwen3.6-35b-a3b`; load only
`text-embedding-qwen3-embedding-0.6b`; batch code and query embeddings; store
vectors in the content-addressed SQLite cache.
2. **Agent phase:** finish all required embedding calls; unload the embedding
instance; load only the Qwen3.6 agent using the pinned 262,144 context; run
model/tool interactions.
3. **Iterative dense-query treatments:** alternate residency at explicit phase
boundaries. Persist conversation state and query text before unloading. Do
not rely on both models being resident for low latency.
4. **Cleanup:** unload the active model after a run block unless the next block
needs the same model immediately.
LM Studio's native `POST /api/v1/models/load` and
`POST /api/v1/models/unload` endpoints are used for explicit residency control.
The E00 runner queries `/api/v1/models` and aborts unless the embedding model is
the only resident instance.
## Pressure observations and stop conditions
The initial audit reported 53% system-wide free memory, but the VM counters also
showed substantial historical compression and swap activity. A run block must
stop rather than continue when:
- an unexpected model is resident;
- available memory enters macOS critical pressure;
- swap grows rapidly during a steady-state batch;
- embedding requests begin timing out or the OS kills a process; or
- index construction exceeds the configured disk or wall-time budget.
Weights, index caches, and result artifacts are not evidence of comparable
compute cost. The study records load/unload time, index time, query time, model
latency, and resource samples separately.
|