Image-Text-to-Text
Transformers
Safetensors
deepseek_v4
text-generation
multimodal
vision-language
deepseek-v4
moonvit
nvfp4
blackwell
8-bit precision
fp8
Instructions to use webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4")# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4") model = AutoModelForCausalLM.from_pretrained("webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4
- SGLang
How to use webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - Docker Model Runner
How to use webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4 with Docker Model Runner:
docker model run hf.co/webbrain-one/DeepSeek-V4-Flash-Vision-NVFP4
| import ast | |
| import unittest | |
| from pathlib import Path | |
| ROOT = Path(__file__).resolve().parents[1] | |
| class MoonViTModelSourceTests(unittest.TestCase): | |
| def test_image_features_are_packed_for_sglang_embedding_cache(self): | |
| path = ( | |
| ROOT | |
| / "sglang_ext" | |
| / "deepseek_vision_sglang" | |
| / "models" | |
| / "deepseek_v4_moonvit.py" | |
| ) | |
| module = ast.parse(path.read_text(encoding="utf-8")) | |
| model = next( | |
| node | |
| for node in module.body | |
| if isinstance(node, ast.ClassDef) and node.name == "DeepseekV4ForCausalLM" | |
| ) | |
| method = next( | |
| node | |
| for node in model.body | |
| if isinstance(node, ast.FunctionDef) and node.name == "get_image_feature" | |
| ) | |
| returned = next(node for node in method.body if isinstance(node, ast.Return)) | |
| self.assertEqual(ast.unparse(returned.value.func), "torch.cat") | |
| self.assertEqual( | |
| ast.unparse(returned.value.args[0]), | |
| "mm_projection_auto(self.mm_projector, image_features)", | |
| ) | |
| self.assertEqual( | |
| [(keyword.arg, ast.literal_eval(keyword.value)) for keyword in returned.value.keywords], | |
| [("dim", 0)], | |
| ) | |
| if __name__ == "__main__": | |
| unittest.main() | |