Instructions to use hongyongjiang/python-reference-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hongyongjiang/python-reference-models with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="hongyongjiang/python-reference-models",
	filename="gemma-3-4b-it_bin/gemma-3-4b-it-f16.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use hongyongjiang/python-reference-models with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
llama-cli -hf hongyongjiang/python-reference-models:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
llama-cli -hf hongyongjiang/python-reference-models:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
./llama-cli -hf hongyongjiang/python-reference-models:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf hongyongjiang/python-reference-models:F16

Use Docker

docker model run hf.co/hongyongjiang/python-reference-models:F16

LM Studio
Jan
Ollama
How to use hongyongjiang/python-reference-models with Ollama:
```
ollama run hf.co/hongyongjiang/python-reference-models:F16
```

Unsloth Studio new

How to use hongyongjiang/python-reference-models with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for hongyongjiang/python-reference-models to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for hongyongjiang/python-reference-models to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for hongyongjiang/python-reference-models to start chatting

Docker Model Runner
How to use hongyongjiang/python-reference-models with Docker Model Runner:
```
docker model run hf.co/hongyongjiang/python-reference-models:F16
```

Lemonade

How to use hongyongjiang/python-reference-models with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull hongyongjiang/python-reference-models:F16

Run and chat with the model

lemonade run user.python-reference-models-F16

List all available models

lemonade list

hongyongjiang commited on about 1 month ago

Commit

e5bb7f1

verified ·

1 Parent(s): 35d2a9c

Upload folder using huggingface_hub

Browse files

Files changed (36) hide show

qwen2.5-vl-3b_bias_bin/layer_0.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_1.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_10.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_11.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_12.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_13.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_14.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_15.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_16.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_17.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_18.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_19.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_2.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_20.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_21.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_22.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_23.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_24.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_25.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_26.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_27.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_28.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_29.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_3.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_30.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_31.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_32.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_33.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_34.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_35.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_4.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_5.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_6.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_7.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_8.bin +3 -0
qwen2.5-vl-3b_bias_bin/layer_9.bin +3 -0

qwen2.5-vl-3b_bias_bin/layer_0.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fe319641190e313fd69915b3f86852798fed39d927993aaabf4a8706a7558fdb
+size 5120

qwen2.5-vl-3b_bias_bin/layer_1.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ca7e1ce330a55152535cda3e7bd329ae5063961b57dbe3ca89d8a14c26e7f220
+size 5120

qwen2.5-vl-3b_bias_bin/layer_10.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:bcfc3e2d7df408747e776c711b2b3ae2c3aff4dc259a916000581bfc5c89b3be
+size 5120

qwen2.5-vl-3b_bias_bin/layer_11.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:859f7dc3ef6c89ed7c902462c63a802e8941012a3e961093398c9d18fa486787
+size 5120

qwen2.5-vl-3b_bias_bin/layer_12.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8d20e3e8ac03493e29711c058da811e913726215794a654fccec6e4a09699081
+size 5120

qwen2.5-vl-3b_bias_bin/layer_13.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:16a7ccc1ae1bf7ef5409beb2df7f657f0f3e03f430584475154e504d7d18d740
+size 5120

qwen2.5-vl-3b_bias_bin/layer_14.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:da65a0747589f4a59085fc95bab589b8ae6ec7eaeee74ae54e9a2036a1227690
+size 5120

qwen2.5-vl-3b_bias_bin/layer_15.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:eb9b47c24d59705ae6b3570eca9651850eaf1fa1ee7df46dbe275f61214b79c1
+size 5120

qwen2.5-vl-3b_bias_bin/layer_16.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:62b749a645ce6403fdc26682e78423eba6442190e471cf0c8ceac40d7356da45
+size 5120

qwen2.5-vl-3b_bias_bin/layer_17.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:433fb0ea477d746b2df73bc7539ca3866c46cef9ef6a52968e902fad4874d78e
+size 5120

qwen2.5-vl-3b_bias_bin/layer_18.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:469cab62c80bc6caaa6626242ae99c2676b6da5b36777114d7d976b6c8f2b502
+size 5120

qwen2.5-vl-3b_bias_bin/layer_19.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:6426c9ad6739dbe60def992ce75d0c7918ec7b9574ea2c9234f190cd3999a77f
+size 5120

qwen2.5-vl-3b_bias_bin/layer_2.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f98c4500b4a6dd3069d7b1e29d19b007a29afca7187c390b840a0ea5d6885d3c
+size 5120

qwen2.5-vl-3b_bias_bin/layer_20.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f51e9daf65a860b990667cba6805f03dac3960ae1b6cf7d5896c330f303bdbe3
+size 5120

qwen2.5-vl-3b_bias_bin/layer_21.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ce12223709a914dd90e634cc4a459a1f77b50ba39c6e7ab7ab09a13da2707c12
+size 5120

qwen2.5-vl-3b_bias_bin/layer_22.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e19fbfd3c625396a09509e94203f0c5bd8c8a62844ce48e24024bf998f3b3f91
+size 5120

qwen2.5-vl-3b_bias_bin/layer_23.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8482e709df310094e45343c6e262976fbc39c5ff45419769a2c58c93a75abdee
+size 5120

qwen2.5-vl-3b_bias_bin/layer_24.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ba1d7b33f1e4355d5221e934fa593ecdd045ea07fed87c65eea4981753d8c2d6
+size 5120

qwen2.5-vl-3b_bias_bin/layer_25.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a6719a0e00f71d3719246abec74b5d7d5d04a0a6bcef2e0e63ec2b5b2a2799d5
+size 5120

qwen2.5-vl-3b_bias_bin/layer_26.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a7be8c16595435c2217e2a9493249763378c80f3451ef3e03252c251cda2bfc6
+size 5120

qwen2.5-vl-3b_bias_bin/layer_27.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:fcbf9bebc7f6c6d69828a3fac08091fa210bb26b944bbcf80c117b3ca26f0a16
+size 5120

qwen2.5-vl-3b_bias_bin/layer_28.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9ad4a12382759099e8c8a788eeb6a6751c61d9cc2cb23e2ede06627256d87962
+size 5120

qwen2.5-vl-3b_bias_bin/layer_29.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:5abb080f9618acf90e74d4030fd026f54e1a6a4b6788657f944e72ddf497fb56
+size 5120

qwen2.5-vl-3b_bias_bin/layer_3.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:916b493fc0ad9c453c7d5e1b22881cb7f6e994ca3a908cc9d4103d51658afa38
+size 5120

qwen2.5-vl-3b_bias_bin/layer_30.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a8c3fb6be92a23e006c31e9ba126b2cb7a2185b2f8941c32f20c002fabf9b77b
+size 5120

qwen2.5-vl-3b_bias_bin/layer_31.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:af7c4d306f2b0bb1a42d878ed0135f18b17c1627ec48e6d80e76b61f8a7271a9
+size 5120

qwen2.5-vl-3b_bias_bin/layer_32.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:ebd506bac9b83ba85cbbf3632482d9bfacac59b3f45b9f53004681937a52b472
+size 5120

qwen2.5-vl-3b_bias_bin/layer_33.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0e8692139974978378030a284214b756358764c15199da3a78a1ab703a3cd6b0
+size 5120

qwen2.5-vl-3b_bias_bin/layer_34.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:e97ba14b519709e0fd2cb7e3d1f2a42f636c2c179332b541729e7dacbd736b31
+size 5120

qwen2.5-vl-3b_bias_bin/layer_35.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:9c14a8fcd8de09a2e2ff1a754eda47e1bb22f07919576ca51f48c143ada2c469
+size 5120

qwen2.5-vl-3b_bias_bin/layer_4.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2a519cb592bad5c5dd3302330a88e5558bf101cbcfa28ef5deceb895b942dd86
+size 5120

qwen2.5-vl-3b_bias_bin/layer_5.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:80ae98db24d511b7306b48b425140f2061685f658744773857caa0d36427f418
+size 5120

qwen2.5-vl-3b_bias_bin/layer_6.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:2b7a2663787caf051921ac6de92855f602245037d82ca5642dc3907c48379a6d
+size 5120

qwen2.5-vl-3b_bias_bin/layer_7.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8f780c038da0b7e121d62f0e2911535b9a6984bfadced2023eb6eafa71fbadc6
+size 5120

qwen2.5-vl-3b_bias_bin/layer_8.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0347b8ed0b3f5643b8f87735035f591e360e3cf063e7c8de87f482f2c99b0ab3
+size 5120

qwen2.5-vl-3b_bias_bin/layer_9.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cb1d6addf1848cdcfecabe34d89728f579bb3f98186087ea74410ff80c7b1a00
+size 5120