Instructions to use hongyongjiang/python-reference-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- llama-cpp-python
How to use hongyongjiang/python-reference-models with llama-cpp-python:
# !pip install llama-cpp-python from llama_cpp import Llama llm = Llama.from_pretrained( repo_id="hongyongjiang/python-reference-models", filename="gemma-3-4b-it_bin/gemma-3-4b-it-f16.gguf", )
llm.create_chat_completion( messages = "No input example has been defined for this model task." )
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use hongyongjiang/python-reference-models with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf hongyongjiang/python-reference-models:F16 # Run inference directly in the terminal: llama cli -hf hongyongjiang/python-reference-models:F16
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf hongyongjiang/python-reference-models:F16 # Run inference directly in the terminal: llama cli -hf hongyongjiang/python-reference-models:F16
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf hongyongjiang/python-reference-models:F16 # Run inference directly in the terminal: ./llama-cli -hf hongyongjiang/python-reference-models:F16
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf hongyongjiang/python-reference-models:F16 # Run inference directly in the terminal: ./build/bin/llama-cli -hf hongyongjiang/python-reference-models:F16
Use Docker
docker model run hf.co/hongyongjiang/python-reference-models:F16
- LM Studio
- Jan
- Ollama
How to use hongyongjiang/python-reference-models with Ollama:
ollama run hf.co/hongyongjiang/python-reference-models:F16
- Unsloth Studio
How to use hongyongjiang/python-reference-models with Unsloth Studio:
Install Unsloth Studio (macOS, Linux, WSL)
curl -fsSL https://unsloth.ai/install.sh | sh # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for hongyongjiang/python-reference-models to start chatting
Install Unsloth Studio (Windows)
irm https://unsloth.ai/install.ps1 | iex # Run unsloth studio unsloth studio -H 0.0.0.0 -p 8888 # Then open http://localhost:8888 in your browser # Search for hongyongjiang/python-reference-models to start chatting
Using HuggingFace Spaces for Unsloth
# No setup required # Open https://huggingface.co/spaces/unsloth/studio in your browser # Search for hongyongjiang/python-reference-models to start chatting
- Atomic Chat new
- Docker Model Runner
How to use hongyongjiang/python-reference-models with Docker Model Runner:
docker model run hf.co/hongyongjiang/python-reference-models:F16
- Lemonade
How to use hongyongjiang/python-reference-models with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull hongyongjiang/python-reference-models:F16
Run and chat with the model
lemonade run user.python-reference-models-F16
List all available models
lemonade list
Upload folder using huggingface_hub
Browse files- qwen2.5-3b_bias_bin/layer_0.bin +3 -0
- qwen2.5-3b_bias_bin/layer_1.bin +3 -0
- qwen2.5-3b_bias_bin/layer_10.bin +3 -0
- qwen2.5-3b_bias_bin/layer_11.bin +3 -0
- qwen2.5-3b_bias_bin/layer_12.bin +3 -0
- qwen2.5-3b_bias_bin/layer_13.bin +3 -0
- qwen2.5-3b_bias_bin/layer_14.bin +3 -0
- qwen2.5-3b_bias_bin/layer_15.bin +3 -0
- qwen2.5-3b_bias_bin/layer_16.bin +3 -0
- qwen2.5-3b_bias_bin/layer_17.bin +3 -0
- qwen2.5-3b_bias_bin/layer_18.bin +3 -0
- qwen2.5-3b_bias_bin/layer_19.bin +3 -0
- qwen2.5-3b_bias_bin/layer_2.bin +3 -0
- qwen2.5-3b_bias_bin/layer_20.bin +3 -0
- qwen2.5-3b_bias_bin/layer_21.bin +3 -0
- qwen2.5-3b_bias_bin/layer_22.bin +3 -0
- qwen2.5-3b_bias_bin/layer_23.bin +3 -0
- qwen2.5-3b_bias_bin/layer_24.bin +3 -0
- qwen2.5-3b_bias_bin/layer_25.bin +3 -0
- qwen2.5-3b_bias_bin/layer_26.bin +3 -0
- qwen2.5-3b_bias_bin/layer_27.bin +3 -0
- qwen2.5-3b_bias_bin/layer_28.bin +3 -0
- qwen2.5-3b_bias_bin/layer_29.bin +3 -0
- qwen2.5-3b_bias_bin/layer_3.bin +3 -0
- qwen2.5-3b_bias_bin/layer_30.bin +3 -0
- qwen2.5-3b_bias_bin/layer_31.bin +3 -0
- qwen2.5-3b_bias_bin/layer_32.bin +3 -0
- qwen2.5-3b_bias_bin/layer_33.bin +3 -0
- qwen2.5-3b_bias_bin/layer_34.bin +3 -0
- qwen2.5-3b_bias_bin/layer_35.bin +3 -0
- qwen2.5-3b_bias_bin/layer_4.bin +3 -0
- qwen2.5-3b_bias_bin/layer_5.bin +3 -0
- qwen2.5-3b_bias_bin/layer_6.bin +3 -0
- qwen2.5-3b_bias_bin/layer_7.bin +3 -0
- qwen2.5-3b_bias_bin/layer_8.bin +3 -0
- qwen2.5-3b_bias_bin/layer_9.bin +3 -0
qwen2.5-3b_bias_bin/layer_0.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:cbc4145dc38490b0f1f80f544a5bad5cd1beaf4b78630a430856c4ddfaf0cac3
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_1.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a174a4055e5fbcf7b1098741e285f3e49fc68111456499c1956bcd8d854693b1
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_10.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:1a9b6b8b7d442ca23f99b7860fba05bbfbd2ddb72c1c9b155bd22d30796dac27
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_11.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5e939cb7b3d0de0cae55b12c6a82bae43fafa47bbda73e356e994147c1aafe73
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_12.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f869d3d153fd993a511df300ab3c96928203f47b75bf381d027697b3005c8d34
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_13.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:5582d56a7eacf31d735107e62be9f6fb496e1eb1e62bf46f93a338259705defb
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_14.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:b6132a45ba9b2ac3af6115b43daf32cf2514235961016720a21b87823b706072
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_15.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:931620eef4bf8df320e364208f5b257821003cbf3c216e79f724b2230c4689b8
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_16.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a3505284ae186cf9dde7b926d417b463ef19855d15a6a34d0ab2c2205e6e5aa9
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_17.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9b0ae643650e9caf0144ded3b27b3274f40a2dfa2d40c0aa866f44df543148d3
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_18.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a751203b2a6e6b3a888498cbead2a1b97849f3b4d883c23e00fa43ad5a0c3cac
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_19.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:aa70edbd069937e378618dca9bcbedb9260df09ded39b0dd1a6c987d4d21e59c
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_2.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:93f97607bb886cb1e5f5eb88e9bb5efba9c9cdc90f76fa2dceb2bd8907ae4882
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_20.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:34ad25a6794385fc3ad3612b806528647a9f178d07d6aafeb3da01d4a88f29b0
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_21.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:991fd02eb62bf39bcabc92718ea8a59b5b71a378e5041a173d92312d076a1ee7
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_22.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:56677b054b9b33253497b4365ececad9a07498c420b44fd8385c6e0eb674071d
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_23.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:02eb65d5928d1c32437079adf7e7e43bdb50b3a99b70f205bb360e240cd89e52
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_24.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8bca80f39c257ffb411306152263d1b2762694a85ec7240db421b20d5d3f59bc
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_25.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:66b248c20abff2e83d587342b06868a6f958c78391adc031b7f7fd80734a9ffa
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_26.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fb951528254926506edebadf2a25aa9256e94c17fc24229b16b0725cfed56f17
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_27.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:4a7fbf989a4ce596afc8807bb2ab090ebb260d8603993b92541ddffd73b36871
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_28.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:fa7a094f4bdbd5547571d5239df79f0f07547b2a4af167c00e2ff0cba453a125
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_29.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:9d4d1b1348dc15e0e7321387018ac98ac949db71accb83cc5f07bf91b626d2ce
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_3.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:505b15e51c0615a942fe24e0c30eacdbe10a3a6037847c5d2e0c5c8a856e6168
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_30.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:8c31dba27a591c4f325040f4859691f3a5178d149b1b246d817df5cb566925d0
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_31.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:77623655447533abda632ae46a22f48bb3b129e5be4079b9f04bc3a9f48a0d6c
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_32.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a4bae45082f435f9f67a2d509b7606fbaa940546c37350879a0444544a607a16
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_33.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:98d939f08c2685df55b0f3110efea84e3eb929d1df8e288db688ee03d6765228
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_34.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f234dc2b6bd0db2183bf3e73965711bffa087d01e98c66d9fb640939f48c3fea
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_35.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ab090e4670192e3d6ac624a9f1191a082b52a08e7414b15aeb32f4e1d091b261
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_4.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:280acc3c83e031e2474ed2c67b3715b3da045aec0e88a6ea39a6ba1f1b76e873
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_5.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ced3206293303d8bb57afa350a3d58e5c249cb936bb1615d1307ca8f3687060b
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_6.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:7b735ec119f88443bdb5c1544a1d95189d02e8b9e44ce9887bebcd307c35070f
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_7.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:37cd02a50230bd7bb21d548bc29e3b89e4ebc602d923fdb0f3c919bfa6280949
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_8.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:ef8c2697684a1de4369e151cc47b75ed5700351a44f288b96b44ce565eb515fb
|
| 3 |
+
size 5120
|
qwen2.5-3b_bias_bin/layer_9.bin
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:e3ba4b3a3a2a0b97c180c33317175f3b55218d580067ca6ada9f3ea3bfc2960f
|
| 3 |
+
size 5120
|