Instructions to use HelpingAI/HelpingAI-9B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use HelpingAI/HelpingAI-9B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="HelpingAI/HelpingAI-9B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("HelpingAI/HelpingAI-9B")
model = AutoModelForCausalLM.from_pretrained("HelpingAI/HelpingAI-9B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

llama-cpp-python

How to use HelpingAI/HelpingAI-9B with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="HelpingAI/HelpingAI-9B",
	filename="helpingai-9b.Q4_0.gguf",
)

llm.create_chat_completion(
	messages = [
		{
			"role": "user",
			"content": "What is the capital of France?"
		}
	]
)

Notebooks
Google Colab
Kaggle
Local Apps

llama.cpp

How to use HelpingAI/HelpingAI-9B with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf HelpingAI/HelpingAI-9B:Q4_0
# Run inference directly in the terminal:
llama-cli -hf HelpingAI/HelpingAI-9B:Q4_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf HelpingAI/HelpingAI-9B:Q4_0
# Run inference directly in the terminal:
llama-cli -hf HelpingAI/HelpingAI-9B:Q4_0

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf HelpingAI/HelpingAI-9B:Q4_0
# Run inference directly in the terminal:
./llama-cli -hf HelpingAI/HelpingAI-9B:Q4_0

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf HelpingAI/HelpingAI-9B:Q4_0
# Run inference directly in the terminal:
./build/bin/llama-cli -hf HelpingAI/HelpingAI-9B:Q4_0

Use Docker

docker model run hf.co/HelpingAI/HelpingAI-9B:Q4_0

LM Studio
Jan

vLLM

How to use HelpingAI/HelpingAI-9B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "HelpingAI/HelpingAI-9B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HelpingAI/HelpingAI-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/HelpingAI/HelpingAI-9B:Q4_0

SGLang

How to use HelpingAI/HelpingAI-9B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "HelpingAI/HelpingAI-9B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HelpingAI/HelpingAI-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "HelpingAI/HelpingAI-9B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "HelpingAI/HelpingAI-9B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use HelpingAI/HelpingAI-9B with Ollama:
```
ollama run hf.co/HelpingAI/HelpingAI-9B:Q4_0
```

Unsloth Studio new

How to use HelpingAI/HelpingAI-9B with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for HelpingAI/HelpingAI-9B to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for HelpingAI/HelpingAI-9B to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for HelpingAI/HelpingAI-9B to start chatting

Docker Model Runner
How to use HelpingAI/HelpingAI-9B with Docker Model Runner:
```
docker model run hf.co/HelpingAI/HelpingAI-9B:Q4_0
```

Lemonade

How to use HelpingAI/HelpingAI-9B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull HelpingAI/HelpingAI-9B:Q4_0

Run and chat with the model

lemonade run user.HelpingAI-9B-Q4_0

List all available models

lemonade list

Abhaykoul commited on May 29, 2024

Commit

bb1b81c

verified ·

1 Parent(s): c13b485

Upload LICENSE.md

Browse files

Files changed (1) hide show

LICENSE.md +50 -0

LICENSE.md ADDED Viewed

	@@ -0,0 +1,50 @@

+************************************************
+**** HelpingAI License ****
+************************************************
+Version 2.0
+Developed by Abhay Koul
+### Preamble
+The HelpingAI License governs the use of HelpingAI's digital assets, including but not limited to software, scripts, datasets, documents, images, audio recordings, videos. The HelpingAI License aims to provide clear, comprehensive terms for accessing, modifying, and sharing resources, while promoting ethical development practices.
+### Grant of Rights
+Under the HelpingAI License, HelpingAI grants you the rights to copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Content, provided you comply with the terms and conditions outlined in this document.
+### Terms and Conditions
+To exercise the rights granted in the previous section, you must adhere to the following terms and conditions:
+2.1. **Redistribution of Source Code.**
+ If you redistribute the Source Code, you must include the complete HelpingAI License with your distribution. You must also add clear notifications in all modified files stating:
+ > "This Work is released under the HelpingAI License v2.0."
+2.2. **Distribution in Binary Form.**
+ If you distribute Binaries derived from the Source Code, you must include the following statement in your distribution:
+ > "This Work is based on the HelpingAI Licensed Work, under the HelpingAI License v2.0."
+2.3. **Notification of Changes.**
+ You must clearly indicate any modifications you make to the Source Code or Documentation, including detailed comments about the nature and extent of the changes. Include the date and originator of the modifications.
+2.4. **Branding Attribution.**
+ You must not remove or alter any HelpingAI branding, logos, or notices included in the Content without explicit prior consent from HelpingAI.
+2.5. **Disclaimer of Warranty.**
+ The Content is provided "AS IS," without any implied warranties, including but not limited to warranties of merchantability, fitness for a particular purpose, and non-infringement.
+2.6. **Limitation of Liability.**
+ To the maximum extent permitted by law, neither HelpingAI nor any contributor shall be liable for any loss, personal injury, property damage, or any indirect, special, incidental, or consequential damages arising from or related to the use of the Content.
+2.7. **Governing Law.**
+ This HelpingAI License shall be governed and construed in accordance with the laws of the jurisdiction where HelpingAI primarily operates.
+### Definitions
+3.1. **"Source Code"** refers to the preferred form for making modifications to the Content, typically represented by human-readable programming languages, scripts, or documentation formats.
+3.2. **"Binaries"** refers to compiled forms of the Source Code, such as executables, libraries, or similar artifacts produced from the Source Code.