Text Generation
MLX
Safetensors
English
Chinese
deepseek_v4
deepseek-v4
deepseek
quantized
apple-silicon
mixture-of-experts
Mixture of Experts
affine
conversational
5-bit
Instructions to use osmapi/DeepSeek-V4-Flash-5bit-mlx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use osmapi/DeepSeek-V4-Flash-5bit-mlx with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("osmapi/DeepSeek-V4-Flash-5bit-mlx") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps
- LM Studio
- MLX LM
How to use osmapi/DeepSeek-V4-Flash-5bit-mlx with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "osmapi/DeepSeek-V4-Flash-5bit-mlx"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "osmapi/DeepSeek-V4-Flash-5bit-mlx" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "osmapi/DeepSeek-V4-Flash-5bit-mlx", "messages": [ {"role": "user", "content": "Hello"} ] }'
Add files using upload-large-folder tool
Browse files- model-00001-of-00045.safetensors +3 -0
- model-00002-of-00045.safetensors +3 -0
- model-00003-of-00045.safetensors +3 -0
- model-00006-of-00045.safetensors +3 -0
- model-00007-of-00045.safetensors +3 -0
- model-00008-of-00045.safetensors +3 -0
- model-00009-of-00045.safetensors +3 -0
- model-00010-of-00045.safetensors +3 -0
- model-00011-of-00045.safetensors +3 -0
- model-00014-of-00045.safetensors +3 -0
- model-00015-of-00045.safetensors +3 -0
- model-00022-of-00045.safetensors +3 -0
- model-00023-of-00045.safetensors +3 -0
- model-00027-of-00045.safetensors +3 -0
- model-00028-of-00045.safetensors +3 -0
- model-00029-of-00045.safetensors +3 -0
- model-00030-of-00045.safetensors +3 -0
- model-00031-of-00045.safetensors +3 -0
- model-00034-of-00045.safetensors +3 -0
- model-00035-of-00045.safetensors +3 -0
- model-00039-of-00045.safetensors +3 -0
- model-00040-of-00045.safetensors +3 -0
- model-00043-of-00045.safetensors +3 -0
- model-00045-of-00045.safetensors +3 -0
model-00001-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ffb65efbcef394670ec92a030b0bd884d25f2f0529eadba0c8ec31efc7829e3d
|
| 3 |
+
size 4599988402
|
model-00002-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c29da2b860d7393e9f84db70e7b5a26a4859d40356f4e6b0eb6f89ea82c0bcd6
|
| 3 |
+
size 3571877696
|
model-00003-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:473a209cd0fca51d80f74e9c2ea8931ee2ecf0c6ef9033539cac1873d87c21e5
|
| 3 |
+
size 4429186193
|
model-00006-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:deba4e10bc9eb8d7cf14c9a59454435c1cbbf60ae26b6c8e7efe7a9711d95554
|
| 3 |
+
size 4429186189
|
model-00007-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f43669db3ca1aa5cc83755775c7f1069e6e6d7817246c4d976e481ee3a6e0be1
|
| 3 |
+
size 4429186187
|
model-00008-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:15ae6f2c25d34cf375a1c0a97ef1da3509d7a60433da3f7dc06c80b19ae0c8ba
|
| 3 |
+
size 4429186193
|
model-00009-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:98614a0d9b102f90bd49850c5195c2d178a59ed959a672533b3247db1036786e
|
| 3 |
+
size 4429186193
|
model-00010-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ec6e546bc0c9567c873ec6fc327b388589ec5db8b79656e7a975644462799dbc
|
| 3 |
+
size 4429186189
|
model-00011-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4c2b9576090d70954bf112cac976887700e02379bf3e16293bb7ce7138f8bd9e
|
| 3 |
+
size 4429186189
|
model-00014-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1c684e99b5a52394f0181462b973b32d1acb8ba60bbcabf5636f976146d7b3c5
|
| 3 |
+
size 4429186196
|
model-00015-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9324c0e3d599f20338f8d2e7d9f34c276014ad85638538bcf68e210ee6ed6b7b
|
| 3 |
+
size 4429186198
|
model-00022-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7cf67ccd9d95b71da3ce049788a38306333b7566f5d586aecc91c0141af51fda
|
| 3 |
+
size 4429186192
|
model-00023-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:915c53bace1f5051a15e586ae7923ba5075dc1d4b3448cd3264a1f7e7fdcb3f0
|
| 3 |
+
size 4429186196
|
model-00027-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e17300885ef01ce852de3846e947f92e0c3380f5ab318c46631e2a8060f1aaf7
|
| 3 |
+
size 4429186194
|
model-00028-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c8fef5aa5d207b29160999ce6be6769a5dde95ae80894532947ac1f978119cbf
|
| 3 |
+
size 4429186196
|
model-00029-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:c0402cca0742220094a5a8aa62420ff665a4c51c03fce87ec9471b76092d3a41
|
| 3 |
+
size 4429186196
|
model-00030-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:706d30baa2a600a5c69a48bc501ccb598b5f1bd333db9f93c202c5a71744a071
|
| 3 |
+
size 4429186194
|
model-00031-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:af7247bc1a58237389724307d5a64d8fa96970149127690525751b196f3c5a94
|
| 3 |
+
size 4429186196
|
model-00034-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:94ccff438bf5f440bb76bd041252a2a1eab37fcfd7c1706a80b49dbf9fe33c31
|
| 3 |
+
size 4429186198
|
model-00035-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ddbfcb34e36a62d0f728af1cf0fc1149c33d63c44229faecaa30bf75c99ee7fe
|
| 3 |
+
size 4429186198
|
model-00039-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4475ce070ec438fd60251e21e4a3a568ad1557f96495ed025cf79814112f864a
|
| 3 |
+
size 4429186198
|
model-00040-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1054024007f16c2cf840c8e3cc2ce7e652a1dd413d8d1c5a509871fa975f10f2
|
| 3 |
+
size 4429186196
|
model-00043-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:3eb9833406d9d75d254a63bf990ff061dd8eea29417a3b25ed48cd1dd80ebda4
|
| 3 |
+
size 4429186198
|
model-00045-of-00045.safetensors
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8cf158950645c3ea1551512ac50146a49fdc64e47f00c5f45f74eadf3a19beec
|
| 3 |
+
size 1476395412
|