Elivis
Elivis-AI
AI & ML interests
None yet
Recent Activity
liked a dataset 5 days ago
DKYoon/SlimPajama-6B liked a dataset 3 months ago
ILSVRC/imagenet-1k reacted to alvarobartt's post with 🔥 4 months ago
💥 `hf-mem` v0.4.1 now also estimates KV cache memory requirements for any context length and batch size with the `--experimental` flag!
`uvx hf-mem --model-id ... --experimental` will automatically pull the required information from the Hugging Face Hub to include the KV cache estimation, when applicable.
💡 Alternatively, you can also set the `--max-model-len`, `--batch-size` and `--kv-cache-dtype` arguments (à la vLLM) manually if preferred.Organizations
