| | --- |
| | license: apache-2.0 |
| | base_model: |
| | - Nanbeige/Nanbeige4.1-3B |
| | tags: |
| | - llm-compressor |
| | --- |
| | |
| | <div align="center"> |
| | <img |
| | src="https://cdn-uploads.huggingface.co/production/uploads/64b93e6bd6c468ac7536607e/mj6xac74jHGLqymiovObc.png" |
| | alt="The Kaitchup -- AI on a Budget" |
| | style="width: 100%; max-width: 100%; height: auto; display: inline-block; margin-bottom: 0.5em; margin-top: 0.5em;" |
| | /> |
| | <div style="display: flex; justify-content: center; gap: 0.5em; margin-bottom: 1em;"> |
| | <a href="https://kaitchup.substack.com/subscribe"><strong>Subscribe and Support</strong></a> |
| | </div> |
| | </div> |
| | |
| |
|
| | This is [Nanbeige/Nanbeige4.1-3B](https://huggingface.co/Nanbeige/Nanbeige4.1-3B) quantized with [llm-compressor](https://github.com/vllm-project/llm-compressor) to W8A8 (FP8) . The model is compatible with vLLM (tested: v0.15.1). Tested with an L4 (Google Colab). |
| |
|
| | - **Developed by:** [The Kaitchup](https://kaitchup.substack.com/) |
| |
|
