Instructions to use bartowski/DeepSeek-R1-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use bartowski/DeepSeek-R1-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf bartowski/DeepSeek-R1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf bartowski/DeepSeek-R1-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf bartowski/DeepSeek-R1-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf bartowski/DeepSeek-R1-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf bartowski/DeepSeek-R1-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf bartowski/DeepSeek-R1-GGUF:Q4_K_M
Use Docker
docker model run hf.co/bartowski/DeepSeek-R1-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use bartowski/DeepSeek-R1-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "bartowski/DeepSeek-R1-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "bartowski/DeepSeek-R1-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/bartowski/DeepSeek-R1-GGUF:Q4_K_M
- Ollama
How to use bartowski/DeepSeek-R1-GGUF with Ollama:
ollama run hf.co/bartowski/DeepSeek-R1-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use bartowski/DeepSeek-R1-GGUF with Docker Model Runner:
docker model run hf.co/bartowski/DeepSeek-R1-GGUF:Q4_K_M
- Lemonade
How to use bartowski/DeepSeek-R1-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull bartowski/DeepSeek-R1-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.DeepSeek-R1-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Commit History
-
add AIBOM
Verified signatureDear model owner(s), We are a group of researchers investigating the usefulness of sharing AIBOMs (Artificial Intelligence Bill of Materials) to document AI models – AIBOMs are machine-readable structured lists of components (e.g., datasets and models) used to enhance transparency in AI-model supply chains. To pursue the above-mentioned objective, we identified popular models on HuggingFace and, based on your model card (and some configuration information available in HuggingFace), we generated your AIBOM according to the CyclonDX (v1.6) standard (see https://cyclonedx.org/docs/1.6/json/). AIBOMs are generated as JSON files by using the following open-source supporting tool: https://github.com/MSR4SBOM/ALOHA (technical details are available in the research paper: https://github.com/MSR4SBOM/ALOHA/blob/main/ALOHA.pdf). The JSON file in this pull request is your AIBOM (see https://github.com/MSR4SBOM/ALOHA/blob/main/documentation.json for details on its structure). Clearly, the subm…82bd2ba
-
Update README.md
Verified signature2b8aeab -
Update README.md
Verified signature43c6fe4 -
Update README.md
Verified signature5a59e20 -
Update metadata with huggingface_hub
Verified signature1808547 -
Upload README.md with huggingface_hub
Verified signatureef547dc -
Upload DeepSeek-R1.imatrix with huggingface_hub
Verified signaturefec33a1 -
Upload folder using huggingface_hub
Verified signature37df98c -
Upload folder using huggingface_hub
Verified signature356b94a -
Upload folder using huggingface_hub
Verified signaturee697400 -
Upload folder using huggingface_hub
Verified signature630759e -
Upload folder using huggingface_hub
Verified signaturea8e81bc -
Upload folder using huggingface_hub
Verified signature0f2a82f -
Upload folder using huggingface_hub
Verified signature8c5b5ec -
Upload folder using huggingface_hub
Verified signature78b0b82 -
Upload folder using huggingface_hub
Verified signaturea189ce5 -
Upload folder using huggingface_hub
Verified signature7fd7edf -
Upload folder using huggingface_hub
Verified signaturee9dd3f5 -
Upload folder using huggingface_hub
Verified signature1485f60 -
Upload folder using huggingface_hub
Verified signaturecb132e1
-
Upload folder using huggingface_hub
Verified signature5c9e782 -
Upload folder using huggingface_hub
Verified signature744abe3 -
Upload folder using huggingface_hub
Verified signaturea3f1a46 -
Upload folder using huggingface_hub
Verified signature123177a -
Upload folder using huggingface_hub
Verified signature75e1f2e -
Upload folder using huggingface_hub
Verified signature55334d7 -
Upload folder using huggingface_hub
Verified signaturea5750bd -
Upload folder using huggingface_hub
Verified signature7e97928 -
Upload folder using huggingface_hub
Verified signature573c5c6 -
Upload folder using huggingface_hub
Verified signature1dc3719 -
Upload folder using huggingface_hub
Verified signature83e3a1f -
initial commit
Verified signature49991c1