Instructions to use hongyongjiang/python-reference-models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use hongyongjiang/python-reference-models with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="hongyongjiang/python-reference-models",
	filename="gemma-3-4b-it_bin/gemma-3-4b-it-f16.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use hongyongjiang/python-reference-models with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
llama cli -hf hongyongjiang/python-reference-models:F16

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
llama cli -hf hongyongjiang/python-reference-models:F16

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
./llama-cli -hf hongyongjiang/python-reference-models:F16

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf hongyongjiang/python-reference-models:F16
# Run inference directly in the terminal:
./build/bin/llama-cli -hf hongyongjiang/python-reference-models:F16

Use Docker

docker model run hf.co/hongyongjiang/python-reference-models:F16

LM Studio
Jan
Ollama
How to use hongyongjiang/python-reference-models with Ollama:
```
ollama run hf.co/hongyongjiang/python-reference-models:F16
```

Unsloth Studio

How to use hongyongjiang/python-reference-models with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for hongyongjiang/python-reference-models to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for hongyongjiang/python-reference-models to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for hongyongjiang/python-reference-models to start chatting

Atomic Chat new
Docker Model Runner
How to use hongyongjiang/python-reference-models with Docker Model Runner:
```
docker model run hf.co/hongyongjiang/python-reference-models:F16
```

Lemonade

How to use hongyongjiang/python-reference-models with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull hongyongjiang/python-reference-models:F16

Run and chat with the model

lemonade run user.python-reference-models-F16

List all available models

lemonade list

hongyongjiang commited on Mar 11

Commit

30f74c8

verified ·

1 Parent(s): 4c7c5f8

Upload folder using huggingface_hub

Browse files

Files changed (18) hide show

.gitattributes +1 -0
llama3.2-1b_bin/Llama-3.2-1B-Instruct-f16.gguf +3 -0
llama3.2-1b_bin/layer_0.bin +3 -0
llama3.2-1b_bin/layer_1.bin +3 -0
llama3.2-1b_bin/layer_10.bin +3 -0
llama3.2-1b_bin/layer_11.bin +3 -0
llama3.2-1b_bin/layer_12.bin +3 -0
llama3.2-1b_bin/layer_13.bin +3 -0
llama3.2-1b_bin/layer_14.bin +3 -0
llama3.2-1b_bin/layer_15.bin +3 -0
llama3.2-1b_bin/layer_2.bin +3 -0
llama3.2-1b_bin/layer_3.bin +3 -0
llama3.2-1b_bin/layer_4.bin +3 -0
llama3.2-1b_bin/layer_5.bin +3 -0
llama3.2-1b_bin/layer_6.bin +3 -0
llama3.2-1b_bin/layer_7.bin +3 -0
llama3.2-1b_bin/layer_8.bin +3 -0
llama3.2-1b_bin/layer_9.bin +3 -0

.gitattributes CHANGED Viewed

@@ -34,3 +34,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Llama-3.2-1B-Instruct-f16.gguf filter=lfs diff=lfs merge=lfs -text

 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
 Llama-3.2-1B-Instruct-f16.gguf filter=lfs diff=lfs merge=lfs -text
+llama3.2-1b_bin/Llama-3.2-1B-Instruct-f16.gguf filter=lfs diff=lfs merge=lfs -text

llama3.2-1b_bin/Llama-3.2-1B-Instruct-f16.gguf ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1f33ad43d2b85b908ff06fe7002b69806a57359b9b2617ca27d7bdea428ae146
+size 2479595360

llama3.2-1b_bin/layer_0.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a4bc80114f80702f9407de0975dc4e2c3201aa76542923fe95b014b472649fa5
+size 60961024

llama3.2-1b_bin/layer_1.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:8197ca4801922cdbbaa2840205a889e11aba0608b1163471674f4eb2994f6dc6
+size 60961024

llama3.2-1b_bin/layer_10.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a7b3db39d8627cdaa9766073c1a0e6fc486d29050902873fc3669b55b694731d
+size 60961024

llama3.2-1b_bin/layer_11.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0ad23d58eb212ecb29f8b3779b9444527cc4178c570d115de9a04eb2a2aa2fb6
+size 60961024

llama3.2-1b_bin/layer_12.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:33434d90c06688fdc6b91d939c6f2545c4e724f05a2d20abc17a9de26830d4d8
+size 60961024

llama3.2-1b_bin/layer_13.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:d442dd8651229c54a512961e1ac8486925c3b8e588a3436e67f5927e6772e1c7
+size 60961024

llama3.2-1b_bin/layer_14.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:1d082ab099b05109cdf553fc5c4a60b0935434c905b338b4ee2932b750e95037
+size 60961024

llama3.2-1b_bin/layer_15.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:f3261d93f5096e962b9f56efc0c6093d8182e8e82ed89e6a66310e248c27b445
+size 60961024

llama3.2-1b_bin/layer_2.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:009a8d1a7d0455dabe903ac7021ec7cc5b529591b1cbc6495d382cce79cd9f4a
+size 60961024

llama3.2-1b_bin/layer_3.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:b116faebec9bab7a6acd83f065a7812959259857382bc3fc6085f7c8fd79dd6f
+size 60961024

llama3.2-1b_bin/layer_4.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:0ecef27613830f7285dab23653620752ad8365fa413735320c701fd5f8205d6f
+size 60961024

llama3.2-1b_bin/layer_5.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:be52f0f69fa9bef8c7ebb97d5cbba6e2a088ef00abc9a7568a84c73653382238
+size 60961024

llama3.2-1b_bin/layer_6.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:cb598455942e6a359364fe400fdf1ee4584f22373b6347971d0dba187935cf8f
+size 60961024

llama3.2-1b_bin/layer_7.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:abab7cc6c7b6c3565b25502ded62cf17e640658f556ba065cfbe2ed2d5b4fd7a
+size 60961024

llama3.2-1b_bin/layer_8.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:c1e4f41d1c1236c052756d1e2e1dfc5b66b53f5a2f02628f98b4db3cee3e846b
+size 60961024

llama3.2-1b_bin/layer_9.bin ADDED Viewed

	@@ -0,0 +1,3 @@

+version https://git-lfs.github.com/spec/v1
+oid sha256:a69d80368857e58e8b199cd9d637aa4374880bb436e8c98b49f9984f63b8b310
+size 60961024