How to use linglingdan/MiniCPM-1b-q4-1 with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="linglingdan/MiniCPM-1b-q4-1", filename="MiniCPM-1B-sft-q4_1.gguf", )
output = llm( "Once upon a time,", max_tokens=512, echo=True ) print(output)
How to use linglingdan/MiniCPM-1b-q4-1 with llama.cpp:
brew install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf linglingdan/MiniCPM-1b-q4-1:Q4_1 # Run inference directly in the terminal: llama-cli -hf linglingdan/MiniCPM-1b-q4-1:Q4_1
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama-server -hf linglingdan/MiniCPM-1b-q4-1:Q4_1 # Run inference directly in the terminal: llama-cli -hf linglingdan/MiniCPM-1b-q4-1:Q4_1
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf linglingdan/MiniCPM-1b-q4-1:Q4_1 # Run inference directly in the terminal: ./llama-cli -hf linglingdan/MiniCPM-1b-q4-1:Q4_1
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf linglingdan/MiniCPM-1b-q4-1:Q4_1 # Run inference directly in the terminal: ./build/bin/llama-cli -hf linglingdan/MiniCPM-1b-q4-1:Q4_1
docker model run hf.co/linglingdan/MiniCPM-1b-q4-1:Q4_1
How to use linglingdan/MiniCPM-1b-q4-1 with Ollama:
ollama run hf.co/linglingdan/MiniCPM-1b-q4-1:Q4_1
How to use linglingdan/MiniCPM-1b-q4-1 with Unsloth Studio:
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for linglingdan/MiniCPM-1b-q4-1 to start chatting
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for linglingdan/MiniCPM-1b-q4-1 to start chatting
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for linglingdan/MiniCPM-1b-q4-1 to start chatting
How to use linglingdan/MiniCPM-1b-q4-1 with Docker Model Runner:
How to use linglingdan/MiniCPM-1b-q4-1 with Lemonade:
# Download Lemonade from https://lemonade-server.ai/ lemonade pull linglingdan/MiniCPM-1b-q4-1:Q4_1
lemonade run user.MiniCPM-1b-q4-1-Q4_1
lemonade list
This repository is publicly accessible, but you have to accept the conditions to access its files and content.
Log in or Sign Up to review the conditions and access this model content.
4-bit
